We compare two cohomological Hall algebras (CoHA). The first one is the preprojective CoHA introduced in [19] associated with each quiver Q, and each algebraic oriented cohomology theory A. It is defined as the A-homology of the moduli of representations of the preprojective algebra of Q, generalizing the K-theoretic Hall algebra of commuting varieties of Schiffmann-Vasserot [15]. The other one is the critical CoHA defined by Kontsevich-Soibelman associated with each quiver with potentials. It is defined using the equivariant cohomology with compact support with coefficients in the sheaf of vanishing cycles. In the present paper, we show that the critical CoHA, for the quiver with potential of Ginzburg, is isomorphic to the preprojective CoHA as algebras. As applications, we obtain an algebra homomorphism from the positive part of the Yangian to the critical CoHA.