The reliable change index (RCI) was introduced approximately 30 decades ago in
order to provide an empirical, statistically grounded technique for determining
whether improvement after a therapeutic intervention was real or due to
measurement error. Since the definitions of the properties and limitations of
scales of measurement described by Stevens in 1947, there has been vigorous
controversy about whether it is permissible to analyse ordinal data with
parametric statistics. Specifically, are parameters and statistics such as means
and standard deviations meaningful in the context of ordinal data? These are
important concerns because many of the scales used to measure outcomes in
behavioural research and clinical settings yield ordinal-scale measures. Given
that the standard deviation is used in the computation of the RCI, the question
as to whether or not the RCI is reliable when used with ordinal-scale data is
explored. Data from the SPRS-2 was used to calculate minimum reliable difference
criteria in terms of both (ordinal) Total Raw Scores (MRDRS) and logit scores (MRDLS) derived from Rasch analysis. Test–retest differences across
the Total Raw Score range were evaluated using each criterion. At both extremes
of the range, small changes in Total Raw Score not deemed to be reliable
according to the MRDRS criterion were classified as reliable according to the MRDLS criterion. Conversely, test–retest changes in the centre of
the range deemed to be reliable according to the MRDRS criterion were classified as unreliable according to the MRDLS criterion.
It is suggested that while MRDRS can determine numerically reliable differences, MRDLS can determine reliable differences that are meaningful in terms of the
underlying construct being measured.
