We calculate asymptotic estimates for the conjugacy growth function of finitely generated class 2 nilpotent groups whose derived subgroups are infinite cyclic, including the so-called higher Heisenberg groups. We prove that these asymptotics are stable when passing to commensurable groups, by understanding their twisted conjugacy growth. We also use these estimates to prove that, in certain cases, the conjugacy growth series cannot be a holonomic function.