In many problems, one wishes to solve the Helmholtz equation with variable coefficients within the Laplacian-like term and use a high order accurate method (e.g., fourth order accurate) to alleviate the points-per-wavelength constraint by reducing the dispersion errors. The variation of coefficients in the equation may be due to an inhomogeneous medium and/or non-Cartesian coordinates. This renders existing fourth order finite difference methods inapplicable. We develop a new compact scheme that is provably fourth order accurate even for these problems. We present numerical results that corroborate the fourth order convergence rate for several model problems.