You can catch a New Jersey Transit train from Penn Station. There are three different lines that stop in Perth Amboy:
Northeast Corridor Line (NEC)
North Jersey Coast Line (NJCL)
North Jersey Coast Line (Long Branch) (NJCLL)
Check out
www.njtransit.com for the schedule. The cost is:
Adult One Way 9.25
Child/Senior/Disabled One Way 4.00
Adult Off-Peak Round Trip* 15.75
Child Off-Peak Round Trip* 7.75
Hope that helps.