A new skills-based immigration system, with a preference for the highly-skilled, is central to UK policy debates in the Brexit context, arguably responding to majority public opinion on migration. Through qualitative fieldwork with British, Polish and Romanian citizens living in two local authorities in England, this paper shows what participants understand by ‘low-skilled’ and how there is broad support of those who ‘contribute’, but are ‘controlled’ at the same time. Migrants’ narratives of downskilling also illustrate why the category of ‘low-skilled’ migration needs to be seen through a more critical lens in research and policymaking.