The entire agricultural supply chain, from crop production to food consumption, is expected to suffer significant damages from climate change. This paper empirically investigates the effects of warming on agricultural labor supply through variation in dietary intake in rural Uganda. We examine labor supply, food consumption, and overall social welfare under various climate change scenarios. First, we combine nationally representative longitudinal survey data with high-resolution climatic data using an instrumental variable approach. Controlling for calorie intake, our study shows that warming has a non-linear impact on agricultural labor supply, with the number of hours worked being optimized at an optimal temperature of 21.3°C. Using these econometric estimates to parametrize an overlapping generations model, we find that under RCP8.5, output per adult decreases by 20 per cent by the end of the century due to the combined effect of climate change on food consumption and labor supply.