A visit to a firearms store would not only have to contend with the swarms of infected/undead, but also the considerably more challenging horde of desperate people wanting access to their supplies (or the owner holing up to defend their cache). Likewise, the constant commotion would likely make the whole area a denser hunting and breeding ground for zombies. All in all, a trip to a gun store at almost any phases of the outbreak are very high on both risk and reward. Options should be weighed carefully on a case by case basis, and advance info, or scouting of the area may be crucial. If at all possible, procurement of a secret personal stockpile of both guns, ammo, and other tools and accessories before/in case of a zombie outbreak is the best option- in accordance with the laws in your state, of course. The last thing you want is unwanted attention from the police.
If one does manage to enter a gun store, make sure to check whether or not it has a firing range. If you can manage to take the targets used for practice off of their mount, these can be useful for armoring a base or vehicle against both humans and the undead. They can also be used as blast shields, although one should avoid the need to have one in the first place.
Plentiful supply of weapons and ammunition. If one has access/permission to use it, it can be a great resource for a safe-zone. If the situation permits gun stores may be used to produce homemade weapons and ammunition to be distributed.
They do not contain much food beyond vending machines and whatever is left in a possible employee fridge. They typically are not built well as a defensable location and should be looted rather than used as refuge, unless you are in Tennessee where it is the law that all privately owned gun shops must have permanent bars placed over their windows and doors. Check your local laws for the security requirements of local gun stores. Also looters would try to get in there.
Lastly, if already in use as a base, it is a very difficult building to assault, break into, or persuade one's own entry, as its occupants will likely have already seen the worst in humanity before your arrival, and will be extra defensive. Or they will barricade themselves on the roof with a hunting rifle, some binoculars, and a large amount of ammo to play a game of whack a looter.