Selenium (Se) is an essential micronutrient for human health, and Se concentration of wheat grain in China has no significant relationships with selenium concentration of wheat and with soil organic matter, nitrogen, phosphorus, potassium in the 0–20 cm soil layer. However, a significant indigenous positive correlation was found with soil Se concentration. Field experiments were conducted from 2018 to 2020 to clarify the differences in the Se accumulation in wheat plants grown in Se-rich areas. We used two common wheat (ZM-175, SN-20), two purple wheat (JZ-496, ZM-8555), and two black wheat (YH-161, LH-131) cultivars to investigate changes in Se build-up and transportation in plant organs. The grain Se concentration of six wheat genotypes in Se-rich areas varied between 178 and 179 μg Se kg−1, with organic Se accounting for 87 to 91%. All genotypes had more than 150 μg Se kg−1, the standard Se concentration in grains. Purple grain wheat had the highest total and organic Se concentrations. Purple wheat also exhibited significantly higher Se transfer coefficient in roots, stem and leaves, and glumes, when compared to common wheat. Moreover, purple wheat had the highest Se uptake efficiency (e.g., JZ-496 with 31%) when compared to common wheat and black wheat. Regardless of the color, wheat grains met the Se-enriched criteria (150 μg Se kg−1) when grown in a natural Se-enriched area. Due to higher Se uptake and accumulation, purple wheat grain genotypes, such as JZ-496, are recommended for wheat breeding programs aiming for high Se functional foods.