Recent astrophysical findings suggest that the era during which the Universe is habitable has just begun. This raises the question whether the entire Universe may at some point in the future be filled with intelligent life. Hanson et al. (2021, The Astrophysical Journal 922, 182) argued that we can be confident that the Universe will, by cosmic standards, soon be dominated by imperialist civilizations which expand rapidly, persist long and make drastic changes to the volumes they control. The main motivation for this ‘grabby civilizations’ hypothesis is that it supposedly provides a good explanation of why we are so early in cosmic history. In this paper, we criticize this motivation and suggest that it fails, for reasons analogous to why the notorious Doomsday argument fails. In the last part of the paper we broaden our discussion and argue that it may be rational to assign a rather low prior probability to the grabby civilizations hypothesis. For instance, if there are any civilizations that expand rapidly and indefinitely, they may well not make any drastic changes to the volumes they inhabit, potentially for strategic reasons. Hence, we call for epistemic caution and humility regarding the question of the long-term evolution of intelligence in the Universe.