Islands are a great place for a small colony. But you don't know if the virus is there too. You can easily fish, with the right know-how, but there's no telling if the fish are infected from any possible zombies in the water. Water is obviously abundant, as long as you distill it. You also could get it from a stream or pool ( if there are any on the island), but you will have to filter and boil it before you drink. The farther away the island is from the shore the better, because lots of other people in an outbreak will be looking for an island and some will be willing to take the island by force.
However, if you wipe out any zombies there, you will have almost no threat from the undead horde. A problem that might happen that you may have to deal with hordes of refugees swarming for a safe haven.
Some islands may be safe should it have a large military pressence or a completely abandoned military complex. For example, Hawaii and Guam would be ideal because of it's large military presence, but would be swarmed by refugees and zombies walking underwater. Additionally, some islands are restricted areas harboring military facilities, such as the joint United States-United Kingdom military base in Diego Garcia in the British Indian Ocean Territory, Midway Island, Wake Island, etc. Reaching this would be impossible since one might get shot upon arrival. Additionally, some islands are uninhabited, such as Navassa Island, Johnston Atoll, etc. The problem with these islands though is that freshwater may not be readily available.
Another problem will be storms. If a large storm or hurricane does come to your island it could wipe out your entire settlement, and it could wash waterborne zombies up your shore. A way to prevent this is to choose an island out side of hurricane waters( like the Caribbean sea).
The Zombie Survival Guide: Recorded Attacks
St. Thomas, U.S. Virgin Islands. In 2002, a single zombie believed to have walked all the way underwater from Morrocco ended up here but was quickly dispatched.
In 2002, years before World War Z, a single waterlogged zombie washed ashore on the northeast coast of St. Thomas, U.S. Virgin Islands. Fortunately, the zombie was dispatched before it could bite and possibly infect more. It is possible this was one of the zombie dumped off the coast of Morocco in 2001 and is also the first case of zombies walking long distances underwater.
World War Z
Islands in World War Z are something of a mixed bag: their isolated nature makes them an obvious first choice for those seeking to avoid the mega-hordes on the mainland. Correspondingly, this means that human refugees will flood into islands - sometimes bringing infectees with them. Even if no one is infected, like any other well-sited survival base, they become a prime target for marauders.
As part of the Redeker Plan, the United States capital was moved to Honolulu, Hawaii, using the isolated island chain as the heart of the US Safe Zone. Ireland - a particularly large island - became the Safe Zone for the British Isles and other European powers such as the Vatican (along with other minor outlying islands like the Isle of Man).
The island situation was best exemplified, however, by how events played out in the Caribbean. Cuba reaped all the benefits of being an island nation with a disproportionately large military, that didn't have a high level of traffic with the outside world in the early stages of the outbreak. Other islands in the Caribbean were not so lucky: several that didn't have large military presences already were overwhelmed, either by zombies, or refugees that brought the infected with them, or marauders (or failed marauder attacks that weakened their defenses so much that they were subsequently overwhelmed by zombies). Barbados was one of the islands overwhelmed in such fashion, leaving it overrun with zombies.
Iceland saw this pattern play out on an even larger scale, albeit in frozen sub-arctic territory: with an initially low pre-outbreak population and cold climate (which slows down zombies in winter), it was seen as an ideal destination for waves of refugees from the rest of Europe. They flooded into the island, bringing many infectees with them; combined with Iceland's small standing military, it was soon totally overrun by zombies. Even years after the "official" end of the war (with victory declared in the continental United States, Europe, and China), Iceland remains one of the worst "White Zones" on the planet.