EPA-0837 - An Empirical Assessment of Methodological Quality Related to Allegiance Bias Hypothesis in Randomised Controlled Trials

Abstract

Introduction:

Previous studies have found an association between allegiance bias and methodological quality of psychological treatment comparisons.

Objective:

To test the hypothesis whether poorer designed randomised controlled trials (RCTs) of psychotherapeutic interventions could be yield increased allegiance effects.

Aims:

Using a systematic approach with meta-analysis we aimed to assess the relationships between allegiance effect and outcomes with respect to treatment integrity, quality of treatment and overall quality of RCTs.

Methods:

We searched meta-analyses of RCTs of different types of psychotherapies in the Cochrane Database of Systematic Reviews. Eligible articles included were meta-analyses of RCTs with at least 1 study with allegiance of the experimenter. Effect sizes in allegiance trials were compared with non-allegiance by sensitivity analysis using random and fixed models and a summary relative odds ratio was calculated. Heterogeneity was quantified with the I² metric.

Results:

A total of 30 meta-analyses including 240 RCTs were analyzed. The analysis addressing the quality of studies did not show any significant association between allegiance and non-allegiance studies, whereas significant differences were observed in cases where treatment integrity were not evaluated (sROR=1.54 [95% CI 1.01 to 2.35], P=0.047 and I²=74). Regarding the overall quality of administered psychotherapy the magnitude of the effect was stronger only in the case of low quality (sROR=1.61 [95% CI 1.00 to 2.59], P=0.049 and I²=76).

Conclusions:

The results support the view that experimenter's allegiance acts as a systematic bias and can be explained through low quality and integrity of delivered psychotherapy.