This paper introduces baby Verma modules for symplectic reflection algebras of complex reflection groups at parameter $t=0$ (the so-called rational Cherednik algebras at parameter $t=0$), and presents their most basic properties. Baby Verma modules are then used to answer several problems posed by Etingof and Ginzburg, and to give an elementary proof of a theorem of Finkelberg and Ginzburg.