Probabilistic quasi-metric spaces are introduced and used to define ordered probabilistic metric spaces. The latter spaces arise naturally in the study of probability and statistics; they closely resemble the uniform ordered spaces of L. Nachbin. A procedure is described for constructing ordered probabilistic metric spaces from quasi-simple spaces, and a completions theory is developed simultaneously for probabilistic quasi-metric spaces and ordered probabilistic metric spaces.