Introducing cover crops (CC) in annual cropping systems can promote nutrient cycling and improve soil health. However, impacts of CC on soil health indicators vary and depend on the duration of CC, cropping systems, and other environmental conditions. We performed an on-farm assessment of cover cropping impacts on soil health indicators including C and N pools, enzyme activities, and microbial community structure under different no-till maize-based cropping systems (maize (Zea mays L.)–soybean (Glycine max L.) [CS], CS-winter wheat (Triticum aestivum L.) [CSWw], and maize-oats (Avena sativa L.) [CO]). At five farms, fields with different durations of CC were compared to adjacent no CC (NCC) fields. In general, long-term CC enhanced the soil health parameters compared to NCC. Long-term (20-year) winter rye CC had higher water-extractable C and N content, enzyme activities (β-glucosidase (1.2 times greater), urease (5.5 times greater), acid (1.5 times greater) and alkaline (4 times greater) phosphatase, arylsulfatase (0.8 times greater) and fluorescein diacetate (FDA) (0.7 times greater)) and soil bacterial community abundance (1.2 times greater). Short-term (3–6 years) legume and grass CC mixtures increased β-glucosidase (0.9 times), acid (0.7 times) and alkaline (1.5 times) phosphatase, arylsulfatase (3 times), FDA (0.8 times) activities and total phospholipid fatty acid (1.6 times) concentration. However, short-term (3–6 years) winter rye, legume and brassica mixtures did not significantly alter soil microbial community structure. This study showed that implementation of CC for >6 years promoted C, N, S, and P cycling that are beneficial to soil health in maize-based cropping systems.