This article presents an innovative adaptive-observer-based scaled four-channel (4-CH) control approach applying damping injection for nonlinear teleoperation systems, which unify the study of robotic dynamic uncertainties, operator/environment force acquirements and asymmetric time-varying delays in the same framework. First, a scaled 4-CH scheme with damping injection is developed to handle time-varying delay while guaranteeing the passivity of communication channels. Then, the improved extended active observer (IEAOB) is deployed to derive the operator/environment force while addressing the issues of measurement noise and model uncertainties. Furthermore, the system stability is analyzed by choosing Lyapunov functional. Finally, the proposed method is validated through simulation.