We establish a Schwarz lemma for $V$-harmonic maps of generalised dilatation between Riemannian manifolds. We apply the result to obtain corresponding results for Weyl harmonic maps of generalised dilatation from conformal Weyl manifolds to Riemannian manifolds and holomorphic maps from almost Hermitian manifolds to quasi-Kähler and almost Kähler manifolds.