Diving: Go to Malaysia. I used to drive 3 hours to a tiny village and then take a 20 minute boat ride to it. God diving, and a great crew. It's called the Rimba Resort on Sibu island.
(yes, I searched it with a detctor - found nothing but a few modern coins).
Now, there is a tiny group of islands just south of Sentosa that have been occupied for hundreds of years, and those beaches may be really cool to search. There is only one boat per day, and you need camp out there (rough it). You can get
permits to camp at Sentosa. The Islands are called the Sisters. These beaches and warm blue waters make snorkeling a favourite activity, and the islands are also popular with (few) picnickers and (fewer) campers.
The islands are home to some of Singapore's richest reefs. A wide variety of corals can be found in the waters surrounding the islands. Big Sister's Island is home to some long-tailed macaques.
There are no overnight facilities. Here's a
Google map.