We give a graph-theoretic definition for the number of ends of Cayley digraphs for finitely generated semigroups and monoids. For semigroups and monoids, left Cayley digraphs can be very different from right Cayley digraphs. In either case, the number of ends for the Cayley digraph does not depend upon which finite set of generators is used for the semigroup or monoid. For natural numbers m and n, we exhibit finitely generated monoids for which the left Cayley digraphs have m ends while the right Cayley digraphs have n ends. For direct products and for many other semidirect products of a pair of finitely generated infinite monoids, the right Cayley digraph of the semidirect product has only one end. A finitely generated subsemigroup of a free semigroup has either one end or else has infinitely many ends.