The George Washington bridge links Manhattan with New Jersey. If you enjoy sports, there is a bike lane that goes from Battery Park, where ferries leave for Liberty Island, up the west side of Manhattan to the George Washington Bridge. I did this circuit several times on a bike and, believe me, it is great. Pick up one of the New York Cycling Maps for more info on bike lanes.
I also recommend crossing the bridge and going to New Jersey, (or you can also go to New Jersey with the Path subway.) Late afternoon is the time to go. Do not miss a night view of Manhattan from Newport!