In this review, we explore the origin of the rudimentary embryo, its relationship to other kinds of plant embryos and its role in the diversification of angiosperms. Rudimentary embryos have a length:width ratio of ≤2.0, and they have organs, including cotyledon(s) and a primary root. A literature survey failed to reveal rudimentary embryos in the pre-angiosperms, suggesting that this kind of embryo is an angiosperm invention. Although proembryos of some gymnosperms and angiosperms have a length:width ratio of ≤2.0, they have not formed meristems or organs. Thus, rudimentary embryos are not proembryos. During the development of rudimentary embryos in monocots and dicots (all non-monocots), the growth pattern of the epicotyledonary cells differs, resulting in differences in the placement of the shoot meristem and in one versus two cotyledons, respectively, but the embryo size is similar. Rudimentary embryos grow inside the seed prior to germination, which is true for linear-underdeveloped embryos, including those in some gymnosperms. Rudimentary embryos served as the starting point for the great diversification of embryos, and ultimately of seeds, in angiosperms, and they are still present in many families of extant angiosperms. The rudimentary embryo is part of the syndrome of changes, including increased speed of pollen germination and pollen tube growth, simplification of the female gametophyte, development of endosperm and elimination of multiple embryo production from each zygote, that distinguish angiosperm seed production from that of gymnosperms. We conclude that the rudimentary embryo was one of many new developments of angiosperms that contributed to their great success on earth.