A (folklore?) conjecture states that no holomorphic modular form $F(\tau )=\sum _{n=1}^{\infty } a_nq^n\in q\mathbb Z[[q]]$ exists, where $q=e^{2\pi i\tau }$ , such that its anti-derivative $\sum _{n=1}^{\infty } a_nq^n/n$ has integral coefficients in the q-expansion. A recent observation of Broadhurst and Zudilin, rigorously accomplished by Li and Neururer, led to examples of meromorphic modular forms possessing the integrality property. In this note, we investigate the arithmetic phenomenon from a systematic perspective and discuss related transcendental extensions of the differentially closed ring of quasi-modular forms.