Overcoming vaccine hesitancy is critical to containing the COVID-19 pandemic in the United States. To increase vaccination rates, the State of Ohio launched a million dollar lottery in May 2021. Following a pre-registered analysis, we estimate the effects of Ohio’s lottery program Vax-a-Million on COVID-19 vaccination rates by comparing it to a “synthetic control” composed of eight other states. We find a statistically insignificant 1.3% decrease in the full vaccination rate in Ohio at the end of the lottery period. We investigate the robustness of our conclusion to model specifications through a multiverse analysis of 216 possible models, including longer time periods and alternative vaccination measures. The majority (88%) find small negative effects in line with the results of our pre-registered model. While our results are most consistent with a decrease in vaccination rate, they do not allow a firm conclusion on whether the lottery increased or decreased vaccine uptake.