Let $C$ be a smooth, separated and geometrically connected curve over a finitely generated field $k$ of characteristic $p\geqslant 0$ , $\unicode[STIX]{x1D702}$ the generic point of $C$ and $\unicode[STIX]{x1D70B}_{1}(C)$ its étale fundamental group. Let $f:X\rightarrow C$ be a smooth proper morphism, and $i\geqslant 0$ , $j$ integers. To the family of continuous $\mathbb{F}_{\ell }$ -linear representations $\unicode[STIX]{x1D70B}_{1}(C)\rightarrow \text{GL}(R^{i}f_{\ast }\mathbb{F}_{\ell }(j)_{\overline{\unicode[STIX]{x1D702}}})$ (where $\ell$ runs over primes $\neq p$ ), one can attach families of abstract modular curves $C_{0}(\ell )$ and $C_{1}(\ell )$ , which, in this setting, are the analogues of the usual modular curves $Y_{0}(\ell )$ and $Y_{1}(\ell )$ . If $i\not =2j$ , it is conjectured that the geometric and arithmetic gonalities of these abstract modular curves go to infinity with $\ell$ (for the geometric gonality, under a certain necessary condition). We prove the conjecture for the arithmetic gonality of the abstract modular curves $C_{1}(\ell )$ . We also obtain partial results for the growth of the geometric gonality of $C_{0}(\ell )$ and $C_{1}(\ell )$ . The common strategy underlying these results consists in reducing by specialization theory to the case where the base field $k$ is finite in order to apply techniques of counting rational points.