Southeast of the tourist town there is a small port with a fish market. If you have an apartment with kitchen, or even better a barbecue, this is a good place to buy a fish to cook yourself. Fish is sold per kilo, and sellers do not seem to be very interested in bargaining, especially when dealing with foreigners.
If you do not have time to cook yourself, in local restaurants you will find plenty of fresh fish and seafood dishes. Even if you do not want to buy fish, I would recommend a walk around the old port. The fish market is less than five minutes' walk from Porto Seguro's main avenue. In this main avenue you will also find banks with ATM's and a big supermarket.