The purpose of this article is to develop the theory of differential modular forms introduced by A. Buium. The main points are the construction of many isogeny covariant differential modular forms and some auxiliary (nonisogeny covariant) forms and an extension of the ‘classical theory’ of Serre differential operators on modular forms to a theory of ‘δ-Serre differential operators’ on differential modular forms. As an application, we shall give a geometric realization of the space of elliptic curves up to isogeny.