There is evidence in Kant of the idea that concepts of particular numbers, such as the number 5, are derived from the representation of units, and in particular pure units, that is, units that are qualitatively indistinguishable. Frege, in contrast, rejects any attempt to derive concepts of number from the representation of units. In the Foundations of Arithmetic, he softens up his reader for his groundbreaking and unintuitive analysis of number by attacking alternative views, and he devotes the majority of this attack to the units view, with particular attention to pure units. Since Frege, the units view has been all but abandoned. Nevertheless, the idea that concepts of number are derived from the representation of units has a long history, beginning with the ancient Greeks, and was prevalent among Frege's contemporaries. I am not interested in resurrecting the units view or in righting wrongs in Frege's criticisms of his contemporaries. Rather, I am interested in the program of deriving concepts of number from pure units and its history from Kant to Frege. An examination of that history helps us understand the units view in a way that Frege's criticisms do not, and in the process uncovers important features of both Kant's and Frege's views. I will argue that, although they had deep differences, Kant and Frege share assumptions about what such a view would require and about the limits of conceptual representation. I will also argue that they would have rejected the accounts given by some of Frege's contemporaries for the same reasons. Despite these agreements, however, there is evidence that Kant thinks that space and time play a role in overcoming the limitations of conceptual representation, while Frege argues that they do not.