Evidence on the association between maternal Hb concentration and preterm birth (PTB) risk is inconclusive. This paper aimed to explore whether women with anaemia or high Hb level before pregnancy would be at higher risk of PTB. We conducted a population-based cohort study with 2 722 274 women aged 20–49 years, who participated in National Free Pre-Pregnancy Checkups Project between 2013 and 2015 and delivered a singleton before 2016 in rural China. Logistic models were used to estimate OR and 95 % CI after adjusting for confounding variables. Restricted cubic spline models were applied to evaluate the dose–response relationships. A total of 192 819 (7·08 %) women had preterm deliveries. Compared with women with Hb of 110–149 g/l, the multivariable-adjusted OR for PTB was 1·19 (95 % CI 0·98, 1·44) for women with Hb<70 g/l, 1·01 (95 % CI 0·97, 1·03) for 70–99 g/l, 0·96 (95 % CI 0·95, 0·98) for 100–109 g/l, 1·04 (95 % CI 1·01, 1·06) for 150–159 g/l, 1·11 (95 % CI 1·05, 1·17) for 160–169 g/l and 1·19 (95 % CI 1·11, 1·27) for ≥170 g/l, respectively. The multivariable-adjusted OR for very PTB (VPTB) was 1·07 (95 % CI 1·03, 1·12) and 1·06 (95 % CI 1·01, 1·12) for women with Hb <110 and ≥150 g/l, compared with those with Hb of 110–149 g/l, respectively. Our study identified a U-shaped relationship between maternal preconception Hb concentration and PTB risk. Both preconception anaemia and high Hb level can significantly increase VPTB risk. Appropriate intervention for women with abnormal Hb levels before pregnancy is very necessary.