Groups having exactly one normaliser are well known. They are the Dedekind groups. All finite groups having exactly two normalisers were classified by Pérez-Ramos [‘Groups with two normalizers’, Arch. Math.50 (1988), 199–203], and Camp-Mora [‘Locally finite groups with two normalizers’, Comm. Algebra28 (2000), 5475–5480] generalised that result to locally finite groups. Then Tota [‘Groups with a finite number of normalizer subgroups’, Comm. Algebra32 (2004), 4667–4674] investigated properties (such as solubility) of arbitrary groups with two, three and four normalisers. In this paper we prove that every finite group with at most 20 normalisers is soluble. Also we characterise all nonabelian simple (not necessarily finite) groups with at most 57 normalisers.