Our study presents experimentally based results on how large herbivore species affect savanna vegetation and thus murid rodents in the Hluhluwe-iMfolozi Park in KwaZulu-Natal, South Africa. We permanently excluded groups of large herbivore guilds of various body sizes (ranging from white rhino to hares) from sixteen 40 × 40-m plots of vegetation by using different fence types. We determined grass species composition and vegetation height and collected capture–mark–recapture data on murid rodents. Nutrient concentrations of the dominant grass species and rodent diet compositions were analysed. We found that herbivore species of different body sizes had different effects on murid rodents. The exclusion of medium-sized herbivores, such as warthog, impala and nyala increased the abundance of high-quality grass species, especially Panicum maximum. However, the dominant rodent species Lemniscomys rosalia preferred the most abundant grass species, rather than high-quality grasses. The absence of large bulk feeders, such as zebra, buffalo and white rhino led to an increase in vegetation height. In response, tall vegetation promoted both rodent abundance and species diversity and altered rodent species composition. Ultimately, our results indicate that the greatest effect on murid rodents came from the reduction of vegetation cover by large bulk feeders, which likely increased rodent predation risk.