When we go to the city we love to take the ferry to the south street seaport (Pier 11), check out the street entertainment there and have a bite at an outdoor cafe, then walk up to Ground Zero. Then we like to putz around Greenwich Village or Chinatown.
If you go in the summer, here is another idea. On a really hot day maybe you would want to get out of Manhattan for a few hours and go to the beach. You can take a ferry to Sandy Hook, NJ (35 minutes) right to the beach, and there is even a water taxi to take you to a few waterfront seafood restaurants right there. And on the ferry ride, you get a great view of the Manhattan skyline and the Statue of Liberty. Then you can spend the evening in New York when it has cooled down.
This website has alot of information:
www.ExperienceNJ.com