This paper argues, first, that recent studies of experimentation, most notably by Deborah Mayo, provide the conceptual resources to describe scientific discovery's early stages as error-probing processes. Second, it shows that this description yields greater understanding of those early stages, including the challenges that they pose, the research strategies associated with them, and their influence on the rest of the discovery process. Throughout, the paper examines the phenomenon of “chemical hormesis” (i.e., anomalous low-dose effects from toxic chemicals) as a case study that is important not only for the biological sciences but also for contemporary public policy. The resulting analysis is significant for at least two reasons. First, by elucidating the importance of discovery's earliest stages, it expands previous accounts by philosophers such as William Wimsatt and Lindley Darden. Second, it identifies the discovery process as yet another philosophical topic on which the detailed studies of the “new experimentalists” can shed new light.