We take the first steps towards a better understanding of continuous orbit equivalence, i.e., topological orbit equivalence with continuous cocycles. First, we characterize continuous orbit equivalence in terms of isomorphisms of $C^{\ast }$-crossed products preserving Cartan subalgebras. This is the topological analogue of the classical result by Singer and Feldman-Moore in the measurable setting. Second, we turn to continuous orbit equivalence rigidity, i.e., the question whether for certain classes of topological dynamical systems, continuous orbit equivalence implies conjugacy. We show that this is not always the case by constructing topological dynamical systems (actions of free abelian groups and also non-abelian free groups) that are continuously orbit equivalent but not conjugate. Furthermore, we prove positive rigidity results. For instance, for solvable duality groups, general topological Bernoulli actions and certain subshifts of full shifts over finite alphabets are rigid.