Soil conservation on the Loess Plateau is important not only for local residents but also for reducing sediment downstream in the Yellow River. In this paper, we report a decrease in soil erosion from 2000 to 2010 as a result of the ‘Grain for Green' (GFG) Project. By using the Revised Universal Soil Loss Equation and data on land cover, climate and sediment yield, we found that soil erosion decreased from 6579.55tkm–2yr–1 in 2000 to 1986.66tkm–2yr–1 in 2010. During this period, there was a major land cover change from farmland to grassland in response to the GFG. The area of low vegetation coverage with severe erosion decreased dramatically, whereas the area of high vegetation coverage with slight erosion increased. Our study demonstrates that the reduction in soil erosion on the Loess Plateau contributed to the decrease in the sediment concentration in the Yellow River.