Kep, Cambodia has a dozen small islands just off the coast. The main island, Koh Tonsay (or Rabbit Island) has a few small restaurants and a half dozen places to stay for $5 to $15 a night, and is currently undergoing development.
You can swim, fish, snorkel, scuba dive or just take a boat ride around the islands.
One island just off Kep’s coast is not in Cambodia. Phu Quoc Island is in Vietnam. And although it looks like it’s in Cambodia, you can not go there (or near there) without a passport. It is a few times larger (and more developed) than all of Cambodia’s islands combined. If you go by boat from Kampot at the Kampot International Tourist Port, you can go with just a passport for up to 30 days. No visa needed. For all countries.
