From all the castle gardens in Czech republic, the most beautiful ones are the Gardens and Castle at Kromeriz in Moravia, and the Wallenstein Palace Gardens in Prague (located not far from St Vitus Cathedral). Both gardens are nicely kept and don't draw hordes of tourist crowds which is good if you're looking to relax for a few hours.
One of the most beautiful Castles in Czech Republic is the one in Karlstejn. You can get there by train from Prague Smichovske or Hlavni Nadrazi. If you can, make sure you do the tour in Karlstejn Castle to see closely the different rooms and levels.