Maternal prenatal and postnatal psychological distress, including depression and anxiety, may affect children’s cognitive development. However, the findings have been inconsistent. We aimed to use the dataset from the Japan Environment and Children’s Study, a nationwide prospective birth cohort study, to examine this association. We evaluated the relationship between the maternal six-item version of the Kessler Psychological Distress Scale (K6) scores and cognitive development among children aged 4 years. K6 was administered twice during pregnancy (M-T1; first half of pregnancy, M-T2; second half of pregnancy) and 1 year postpartum (C-1y). Cognitive development was assessed by trained testers, using the Kyoto Scale of Psychological Development 2001. Multiple regression analysis was performed with the group with a K6 score ≤ 4 for both M-T1 and M-T2 and C-1y as a reference. Records from 1,630 boys and 1,657 girls were analyzed. In the group with K6 scores ≥ 5 in both M-T1 and M-T2 and C-1Y groups, boys had significantly lower developmental quotients (DQ) in the language-social developmental (L-S) area (partial regression coefficient: −4.09, 95% confidence interval: −6.88 – −1.31), while girls did not differ significantly in DQ for the L-S area. Among boys and girls, those with K6 scores ≤ 4 at any one or two periods during M-T1, M-T2, or C-1y did not have significantly lower DQ for the L-S area. Persistent maternal psychological distress from the first half of pregnancy to 1 year postpartum had a disadvantageous association with verbal cognitive development in boys, but not in girls aged 4 years.