The SARS-CoV-2 is transmitted not only through coughing, but also through breathing, speaking or singing. We perform direct numerical simulations of the turbulent transport of potentially infectious aerosols in short conversations, involving repetitive phrases separated by quiescent intervals. We estimate that buoyancy effects due to droplet evaporation are small, and neglect them. A two-way conversation is shown to significantly reduce the aerosol exposure compared with a relative monologue by one person and relative silence of the other. This is because of the ‘cancelling’ effect produced by the two interacting speech jets. Unequal conversation is shown to significantly increase the infection risk to the person who talks less. Interestingly, a small height difference is worse for infection spread, due to reduced interference between the speech jets, than two faces at the same level. For small axial separation, speech jets show large oscillations and reach the other person intermittently. We suggest a range of lateral separations between two people to minimize transmission risk. A realistic estimate of the infection probability is provided by including exposure through the eyes and mouth, in addition to the more common method of using inhaled virions alone. We expect that our results will provide useful inputs to epidemiological models and to disease management.