Use sunset as your turnaround alarm.
These are not natural caves. They were dug into the Navajo sandstone as a sand mine, cut for fine glassmaking sand around the World War II era, and the connected chambers and tunnels are what the mining left behind. The walk in is flat and quick, and then the last stretch is a slickrock ramp that usually has a fixed rope on it. That ramp is the whole difficulty of the hike, and it is why people who dislike heights turn around there.
Man-made caves in a cliff band - short scramble, big photos.
Heat note: desert country - in summer start at dawn and carry more water than feels reasonable.
Water bottle, sunscreen, hat, snacks, comfortable shoes.
Make it a weekend: South Campground (Zion) (Camping, 26 mi) · Watchman Campground (Camping, 26 mi) · Duck Creek Campground (Camping, 30 mi) · Navajo Lake Campground (Camping, 31 mi)
More hikes nearby: Observation Point (East Mesa) (Hike, 24 mi) · Canyon Overlook Trail (Hike, 24 mi) · Emerald Pools (Hike, 26 mi) · Angels Landing (Hike, 26 mi)