The asymptotic standard errors of the correlation residuals and Bentler's standardized residuals in covariance structures are derived based on the asymptotic covariance matrix of raw covariance residuals. Using these results, approximations of the asymptotic standard errors of the root mean square residuals for unstandardized or standardized residuals are derived by the delta method. Further, in mean structures, approximations of the asymptotic standard errors of residuals, standardized residuals and their summary statistics are derived in a similar manner. Simulations are carried out, which show that the asymptotic standard errors of the various types of residuals and the root mean square residuals in covariance, correlation and mean structures are close to actual ones.