Dynamical interactions that take place between objects in dense stellar systems lead to frequent formation of exotic stellar objects, unusual binaries, and systems of higher multiplicity. They are most important for the formation of binaries with neutron stars and black holes, which are usually observationally revealed in mass-transferring binaries. Here we review the current understanding of compact object's retention, of the metallicity dependence on the formation of low-mass X-ray binaries with neutron stars, and how mass-transferring binaries with a black hole and a white dwarf can be formed. We discuss as well one old unsolved puzzle and two new puzzles posed by recent observations: what descendants do ultra-compact X-ray binaries produce, how are very compact triples formed, and how can black hole low-mass X-ray binaries acquire non-degenerate companions?