Hostname: page-component-78c5997874-m6dg7 Total loading time: 0 Render date: 2024-11-13T01:52:07.823Z Has data issue: false hasContentIssue false
  • English
  • Français

Biases in research: risk factors for non-replicability in psychotherapy and pharmacotherapy research

Published online by Cambridge University Press:  13 December 2016

F. Leichsenring ,
A. Abbass ,
M. J. Hilsenroth ,
F. Leweke ,
P. Luyten ,
J. R. Keefe ,
N. Midgley ,
S. Rabung ,
S. Salzer  and
C. Steinert
F. Leichsenring*
Affiliation:
Department of Psychosomatics and Psychotherapy, Justus-Liebig-University Giessen, Giessen, Germany
A. Abbass
Affiliation:
Department of Psychiatry, Dalhousie University, Centre for Emotions and Health, Halifax, NS, Canada
M. J. Hilsenroth
Affiliation:
The Derner Institute of Advanced Psychological Studies, Adelphi University, NY, USA
F. Leweke
Affiliation:
Department of Psychosomatics and Psychotherapy, Justus-Liebig-University Giessen, Giessen, Germany
P. Luyten
Affiliation:
Faculty of Psychology and Educational Sciences, University of Leuven, Klinische Psychologie (OE), Leuven, Belgium Research Department of Clinical, Educational and Health Psychology, University College London, London, UK
J. R. Keefe
Affiliation:
Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA
N. Midgley
Affiliation:
The Anna Freud Centre, London, UK Research Department of Clinical, Educational and Health Psychology, UCL, London, UK
S. Rabung
Affiliation:
Department of Psychology, Alpen-Adria-Universität Klagenfurt, Universitätsstr, Klagenfurt, Austria Department of Medical Psychology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
S. Salzer
Affiliation:
Clinic of Psychosomatic Medicine and Psychotherapy, Georg-August-Universität Goettingen, Göttingen, Germany International Psychoanalytic University (IPU), Berlin, Germany
C. Steinert
Affiliation:
Department of Psychosomatics and Psychotherapy, Justus-Liebig-University Giessen, Giessen, Germany
*
*Address for correspondence: F. Leichsenring, DSc, Department of Psychosomatics and Psychotherapy, Justus-Liebig-University Giessen, Ludwigstr. 76, Giessen, Germany. (Email: falk.leichsenring@psycho.med.uni-giessen.de)
Get access Rights & Permissions [Opens in a new window]

Abstract

Replicability of findings is an essential prerequisite of research. For both basic and clinical research, however, low replicability of findings has recently been reported. Replicability may be affected by research biases not sufficiently controlled for by the existing research standards. Several biases such as researcher allegiance or selective reporting are well-known for affecting results. For psychotherapy and pharmacotherapy research, specific additional biases may affect outcome (e.g. therapist allegiance, therapist effects or impairments in treatment implementation). For meta-analyses further specific biases are relevant. In psychotherapy and pharmacotherapy research these biases have not yet been systematically discussed in the context of replicability. Using a list of 13 biases as a starting point, we discuss each bias's impact on replicability. We illustrate each bias by selective findings of recent research, showing that (1) several biases are not yet sufficiently controlled for by the presently applied research standards, (2) these biases have a pernicious effect on replicability of findings. For the sake of research credibility, it is critical to avoid these biases in future research. To control for biases and to improve replicability, we propose to systematically implement several measures in psychotherapy and pharmacotherapy research, such as adversarial collaboration (inviting academic rivals to collaborate), reviewing study design prior to knowing the results, triple-blind data analysis (including subjects, investigators and data managers/statisticians), data analysis by other research teams (crowdsourcing), and, last not least, updating reporting standards such as CONSORT or the Template for Intervention Description and Replication (TIDieR).

Keywords

Efficacyevidence-based medicinepsychotherapy researchreplicabilityrisk factors
Type
Review Article
Information
Psychological Medicine , Volume 47 , Issue 6 , April 2017 , pp. 1000 - 1011
Copyright
Copyright © Cambridge University Press 2016 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

† This paper is dedicated to my late teacher and friend Willi Hager.

References

Antman, EM, Lau, J, Kupelnick, B, Mosteller, F, Chalmers, TC (1992). A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. Journal of the American Medical Association 268, 240–248.Google Scholar
Asendorpf, J, Conner, M, De Fruyt, F, De Houwer, J, Denissen, J, Fiedler, K, Fiedler, S, Funder, DC, Kliegel, R, Nosek, BA, Perugini, M, Roberts, BW, Schmitt, M, van Aken, MAG, Weber, H, Wicherts, JM (2016). Recommendations for increasing replicability in psycholgy. In Methodological Issues and Strategies in Clinical Research, 4th edn (ed. Kazdin, A.), pp. 607–622. American Psychological Association Washington: DC, US.CrossRefGoogle Scholar
Baardseth, TP, Goldberg, SB, Pace, BT, Wislocki, AP, Frost, ND, Siddiqui, JR, Lindemann, AM, Kivlighan, DM III, Laska, KM, Del Re, AC, Minami, T, Wampold, BE (2013). Cognitive-behavioral therapy versus other therapies: redux. Clinical Psychology Review 33, 395–405.Google Scholar
Barkham, M, Shapiro, DA, Hardy, GE, Rees, A (1999). Psychotherapy in two-plus-one sessions: outcomes of a randomized controlled trial of cognitive-behavioral and psychodynamic-interpersonal therapy for subsyndromal depression. Journal of Consulting and Clinical Psychology 67 201–211.Google Scholar
Borenstein, M, Hedges, LV, Higgins, JPT, Rothstein, HR (2011). Introduction to Meta-analysis. Wiley: Chichester, UK.Google Scholar
Brunswik, E (1955). Representative design and probabilistic theory in a functional psychology. Psychological Review 62, 193–217.Google Scholar
Carey, B (2015). Psychology's fears confirmed: rechecked studies don't hold up. New York Times, 27 August 2015.Google Scholar
Chan, AW, Hrobjartsson, A, Haahr, MT, Gotzsche, PC, Altman, DG (2004). Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. Journal of the American Medical Association 291, 2457–2465.Google Scholar
Clark, DM, Salkovskis, PM, Hackmann, A, Middleton, H, Anastasiades, P, Gelder, M (1994). A comparison of cognitive therapy, applied relaxation and imipramine in the treatment of panic disorder. British Journal of Psychiatry 164, 759–769.Google Scholar
Cohen, J (1988). Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum: Hillsdale.Google Scholar
Cohen, J (1990). Things I have learned (so far). American Psychologist 45, 1304–1312.Google Scholar
Cottraux, J, Note, ID, Boutitie, F, Milliery, M, Genouihlac, V, Yao, SN, Note, B, Mollard, E, Bonasse, F, Gaillard, S, Djamoussian, D, Guillard Cde, M, Culem, A, Gueyffier, F (2009). Cognitive therapy versus Rogerian supportive therapy in borderline personality disorder. Two-year follow-up of a controlled pilot study. Psychotherapy and Psychosomatics 78, 307–316.Google Scholar
Cuijpers, P (2016). Are all psychotherapies equally effective in the treatment of adult depression? The lack of statistical power of comparative outcome studies. Evidence-Based Mental Health 19, 39–42.Google Scholar
Cuijpers, P, Berking, M, Andersson, G, Quigley, L, Kleiboer, A, Dobson, KS (2013 a). A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Canadian Journal of Psychiatry 58, 376–385.Google Scholar
Cuijpers, P, Huibers, M, Ebert, DD, Koole, SL, Andersson, G (2013 b). How much psychotherapy is needed to treat depression? A metaregression analysis. Journal of Affective Disorders 149, 1–13.Google Scholar
Cuijpers, P, Karyotaki, E, Andersson, G, Li, J, Mergl, R, Hegerl, U (2015). The effects of blinding on the outcomes of psychotherapy and pharmacotherapy for adult depression: a meta-analysis. European Psychiatry 30, 685–693.Google Scholar
Cuijpers, P, Sijbrandij, M, Koole, SL, Andersson, G, Beekman, AT, Reynolds, CF III (2013 c). The efficacy of psychotherapy and pharmacotherapy in treating depressive and anxiety disorders: a meta-analysis of direct comparisons. World Psychiatry 12, 137–148.Google Scholar
Cuijpers, P, Smit, F, Bohlmeijer, E, Hollon, SD, Andersson, G (2010). Efficacy of cognitive-behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias. British Journal of Psychiatry 196, 173–178.Google Scholar
Davidson, JR, Foa, EB, Huppert, JD, Keefe, FJ, Franklin, ME, Compton, JS, Zhao, N, Connor, KM, Lynch, TR, Gadde, KM (2004). Fluoxetine, comprehensive cognitive behavioral therapy, and placebo in generalized social phobia. Archives of General Psychiatry 61, 1005–1013.Google Scholar
Doshi, P, Dickersin, K, Healy, D, Vedula, SS, Jefferson, T (2013). Restoring invisible and abandoned trials: a call for people to publish the findings. BMJ (Clinical Research Edition) 346, f2865.Google Scholar
Dragioti, E, Dimoliatis, I, Evangelou, E (2015). Disclosure of researcher allegiance in meta-analyses and randomised controlled trials of psychotherapy: a systematic appraisal. BMJ Open 5, e007206.Google Scholar
Driessen, E, Hollon, SD, Bockting, CL, Cuijpers, P, Turner, EH (2015). Does publication bias inflate the apparent efficacy of psychological treatment for major depressive disorder? a systematic review and meta-analysis of US National Institutes of Health-Funded Trials. PLoS ONE 10, e0137864.Google Scholar
Dush, DM, Hirt, ML, Schroeder, H (1983). Self-statement modification with adults: a meta-analysis. Psychological Bulletin 94, 408–422.Google Scholar
Elkin, I, Parloff, MB, Hadley, SW, Autry, JH (1985). NIMH treatment of depression collaborative research program. Archives of General Psychiatry 42, 305–316.Google Scholar
Elkin, I, Shea, MT, Watkins, JT, Imber, SD, Sotsky, SM, Collins, JF, Glass, DR, Pilkonis, PA, Leber, WR, Docherty, JP, et al. (1989). National institute of mental health treatment of depression collaborative research program. General effectiveness of treatments. Archives of General Psychiatry 46, 971–983.Google Scholar
Eysenck, HJ (1978). An exercise in mega-silliness. American Psychologist 33, 517.Google Scholar
Falkenström, F, Markowitz, JC, Jonker, H, Philips, B, Holmqvist, R (2013). Can psychotherapists function as their own controls? Meta-analysis of the crossed therapist design in comparative psychotherapy trials. Journal of Clinical Psychiatry 74, 482–491.Google Scholar
Foa, EB, Rothbaum, BO, Riggs, DS, Murdock, TB (1991). Treatment of posttraumatic stress disorder in rape victims: a comparison between cognitive-behavioral procedures and counseling. Journal of Consulting and Clinical Psychology 59, 715–723.Google Scholar
Fugelsang, JA, Stein, CB, Green, AE, Dunbar, KN (2004). Theory and data interactions of the scientific mind: evidence from the molecular and the cognitive laboratory. Canadian Journal of Experimental Psychology 58, 86–95.Google Scholar
Gerber, AJ, Kocsis, JH, Milrod, BL, Roose, SP, Barber, JP, Thase, ME, Perkins, P, Leon, AC (2011). A quality-based review of randomized controlled trials of psychodynamic psychotherapy. American Journal of Psychiatry 168, 19–28.Google Scholar
Giesen-Bloo, J, Arntz, A (2007). Questions concerning the randomized trial of schema-focused therapy vs transference-focused psychotherapy – Reply. Archives of General Psychiatry 64, 610–611.Google Scholar
Giesen-Bloo, J, van Dyck, R, Spinhoven, P, van Tilburg, W, Dirksen, C, van Asselt, T, Kremers, I, Nadort, M, Arntz, A (2006). Outpatient psychotherapy for borderline personality disorder: randomized trial of schema-focused therapy vs transference-focused psychotherapy. Archives of General Psychiatry 63, 649–658.Google Scholar
Gilboa-Schechtman, E, Foa, EB, Shafran, N, Aderka, IM, Powers, MB, Rachamim, L, Rosenbach, L, Yadin, E, Apter, A (2010). Prolonged exposure versus dynamic therapy for adolescent PTSD: a pilot randomized controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry 49, 1034–1042.Google Scholar
Higgins, JP, Altman, DG, Goetzsche, PC, Juni, P, Moher, D, Oxman, AD (2011). The cochrane statistical methods group. The Cochrane Collaboration's tool for assesingrisk of bias in randomized trials. British Medical Journal 343, d5928.Google Scholar
Hoffmann, TC, Glasziou, PP, Boutron, I, Milne, R, Perera, R, Moher, D, Altman, DG, Barbour, V, Macdonald, H, Johnston, M, Lamb, SE, Dixon-Woods, M, McCulloch, P, Wyatt, JC, Chan, AW, Michie, S (2014). Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ (Clinical Research Ed.) 348, g1687.Google Scholar
Horowitz, M, Kaltreider, N (1979). Brief therapy of the stress response syndrome. Psychiatric Clinics of North America 2, 365–377.Google Scholar
Hsu, L (1989). Random sampling, randomization, and equivalence of contrasted groups in psychotherapy outcome research. Journal of Consulting and Clinical Psychology 57, 131–137.Google Scholar
Ioannidis, JP (2005 a). Contradicted and initially stronger effects in highly cited clinical research. JAMA 294, 218–228.Google Scholar
Ioannidis, JP (2005 b). Why most published research findings are false. PLoS Medicine 2, e124.Google Scholar
Ioannidis, JP, Allison, DB, Ball, CA, Coulibaly, I, Cui, X, Culhane, AC, Falchi, M, Furlanello, C, Game, L, Jurman, G, Mangion, J, Mehta, T, Nitzberg, M, Page, GP, Petretto, E, van Noort, V (2009). Repeatability of published microarray gene expression analyses. Nature Genetics 41, 149–155.Google Scholar
Jacobson, NS (1991). To be or not to be behavioral when working with couples. What does it mean? Journal of Familiy Psychology 4, 436–445.Google Scholar
Kazdin, AE (1994). Methodology, design, and evaluation in psychotherapy research. In Handbook of Psychotherapy and Behavior Change (ed. Bergin, A. E. and Garfield, S. L.), pp. 19–71. Wiley: New York.Google Scholar
Keller, MB, Ryan, ND, Strober, M, Klein, RG, Kutcher, SP, Birmaher, B, Hagino, OR, Koplewicz, H, Carlson, GA, Clarke, GN, Emslie, GJ, Feinberg, D, Geller, B, Kusumakar, V, Papatheodorou, G, Sack, WH, Sweeney, M, Wagner, KD, Weller, EB, Winters, NC, Oakes, R, McCafferty, JP (2001). Efficacy of paroxetine in the treatment of adolescent major depression: a randomized, controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry 40, 762–772.Google Scholar
Leichsenring, F, Leibing, E (2003). The effectiveness of psychodynamic therapy and cognitive behavior therapy in the treatment of personality disorders: a meta-analysis. American Journal of Psychiatry 160, 1223–1232.Google Scholar
Leichsenring, F, Leweke, F, Klein, S, Steinert, C (2015 a). The empirical status of psychodynamic psychotherapy – an update: Bambi's alive and kicking. Psychotherapy and Psychosomatics 84, 129–148.Google Scholar
Leichsenring, F, Luyten, P, Hilsenroth, MJ, Abbass, A, Barber, JP, Keefe, JR, Leweke, F, Rabung, S, Steinert, C (2015 b). Psychodynamic therapy meets evidence-based medicine: a systematic review using updated criteria. Lancet Psychiatry 2, 648–660.Google Scholar
Leichsenring, F, Salzer, S, Beutel, ME, Herpertz, S, Hiller, W, Hoyer, J, Huesing, J, Joraschky, P, Nolting, B, Poehlmann, K, Ritter, V, Stangier, U, Strauss, B, Stuhldreher, N, Tefikow, S, Teismann, T, Willutzki, U, Wiltink, J, Leibing, E (2013). Psychodynamic therapy and cognitive-behavioral therapy in social anxiety disorder: a multicenter randomized controlled trial. American Journal of Psychiatry 170, 759–767.Google Scholar
Leichsenring, F, Steinert, C, Hoyer, J (2016). Psychotherapy versus pharmacotherapy of depression: What's the evidence? Zeitschrift fur Psychosomatische Medizin und Psychotherapie 62, 190–195.Google Scholar
Le Noury, J, Nardo, JM, Healy, D, Jureidini, J, Raven, M, Tufanaru, C, Abi-Jaoude, E (2015). Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. BMJ (Clinical Research Ed.) 351, h4320.Google Scholar
Lieb, K, Osten-Sacken, J, Stoffers-Winterling, J, Reiss, N, Barth, J (2016). Conflicts of interest and spin in reviews of psychological therapies: a systematic review. BMJ Open 6, e010606.Google Scholar
Luborsky, L, Diguer, L, Seligman, D, Rosenthal, R, Krause, E, Johnson, S, Halperin, G, Bishop, M, Berman, J, Schweizer, E (1999). The researcher's own allegiances: a ‘wild’ card in comparison of treatment efficacy. Clinical Psychology: Science and Practice 6, 95–106.Google Scholar
MacCoun, R, Perlmutter, S (2015). Hide resuls to seek the truth. Nature 526, 187–189.Google Scholar
Marcus, DK, O'Connell, D, Norris, AL, Sawaqdeh, A (2014). Is the Dodo bird endangered in the 21st century? A meta-analysis of treatment comparison studies. Clinical Psychology Review 34, 519–530.Google Scholar
McKay, KM, Imel, ZE, Wampold, BE (2006). Psychiatrist effects in the psychopharmacological treatment of depression. Journal of Affective Disorders 92, 287–290.Google Scholar
Meehl, PE (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald and the slow progress of soft psycholgy. Journal of Consulting and Clinical Psychology 46, 806–834.Google Scholar
Mellers, B, Hertwig, R, Kahneman, D (2001). Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration. Psychological Science 12, 269–275.Google Scholar
Miller, LE, Stewart, ME (2011). The blind leading the blind: use and misuse of blinding in randomized controlled trials. Contemporary Clinical Trials 32, 240–243.Google Scholar
Miller, S, Wampold, B, Varhely, K (2008). Direct comparisons of treatment modalities for youth disorders: a meta-analysis. Psychotherapy Research 18, 5–14.Google Scholar
Milrod, B, Chambless, DL, Gallop, R, Busch, FN, Schwalberg, M, McCarthy, KS, Gross, C, Sharpless, BA, Leon, AC, Barber, JP (2016). Psychotherapies for panic disorder: a tale of two sites. Journal of Clinical Psychiatry 77, 927–935.Google Scholar
Moher, D, Hopewell, S, Schulz, KF, Montori, V, Gotzsche, PC, Devereaux, PJ, Elbourne, D, Egger, M, Altman, DG (2010). CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. British Medical Journal (Clinical Research Edition) 340, c869.Google Scholar
Moher, D, Shamseer, L, Clarke, M, Ghersi, D, Liberati, A, Petticrew, M, Shekelle, P, Stewart, LA (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews 4, 1.Google Scholar
Munder, T, Brutsch, O, Leonhart, R, Gerger, H, Barth, J (2013). Researcher allegiance in psychotherapy outcome research: an overview of reviews. Clinical Psychology Review 33, 501–511.Google Scholar
Munder, T, Fluckiger, C, Gerger, H, Wampold, BE, Barth, J (2012). Is the allegiance effect an epiphenomenon of true efficacy differences between treatments? a meta-analysis. Journal of Counseling Psychology 59, 631–637.Google Scholar
Munder, T, Gerger, H, Trelle, S, Barth, J (2011). Testing the allegiance bias hypothesis: a meta-analysis. Psychotherapy Research 21, 670–684.Google Scholar
Nuzzo, R (2015). How scientists fool themselves – and how they can stop. Nature 526, 182–185.Google Scholar
Open Science Collaboration (2015). Psychology. Estimating the reproducibility of psychological science. Science 349, aac4716.Google Scholar
Owen, J, Drinane, JM, Idigo, KC, Valentine, JC (2015). Psychotherapist effects in meta-analyses: how accurate are treatment effects? Psychotherapy (Chic) 52, 321–328.Google Scholar
Perepletchikova, F, Treat, AT, Kazdin, AE (2007). Treatment integrity in psychotherapy research: analysis of studies and examination of the associated factors. Journal of Consulting and Clinical Psychology 75, 829–841.Google Scholar
Popper, KR (1959). The Logic of Scientific Discovery. Basic Books: New York.Google Scholar
Rothstein, HR, Sutton, AJ, Borenstein, M (2005). Publication bias. In Publication Bias in Meta-analysis: Prevention, Assessment and Adjustment (ed. Rothstein, H. R., Sutton, A. J. and Borenstein, M.), pp. 277–302. Wiley & Sons: New York.Google Scholar
Silberzahn, R, Uhlmann, EL (2015). Many hands make tight work. Nature 526, 189–191.Google Scholar
Smith, ML, Glass, GV, Miller, TI (1980). The Benefits of Psychotherapy. John Hopkins University Press: Baltimore.Google Scholar
Snyder, DK, Wills, RM (1989). Behavioral vs. insight-oriented marital therapy: effects on individual and interspousal functioning. Journal of Consulting and Clinical Psychology 57, 39–46.Google Scholar
Snyder, DK, Wills, RM, Grady-Fletcher, A (1991). Long-term effectiveness of behavioral versus insight-oriented marital therapy: a 4-year follow-up study. Journal of Consulting and Clinical Psychology 59, 138–141.Google Scholar
Spiwak, M (2016). Nothing but reviews in mind [Nichts als Gutachten im Kopf]. Die Zeit 32, 31–32.Google Scholar
Stangier, U, Schramm, E, Heidenreich, T, Berger, M, Clark, DM (2011). Cognitive therapy vs interpersonal psychotherapy in social anxiety disorder: a randomized controlled trial. Archives of General Psychiatry 68, 692–700.Google Scholar
Tajika, A, Ogawa, Y, Takeshima, N, Hayasaka, Y, Furukawa, TA (2015). Replication and contradiction of highly cited research papers in psychiatry: 10-year follow-up. British Journal of Psychiatry 207, 357–362.Google Scholar
Thoma, NC, McKay, D, Gerber, AJ, Milrod, BL, Edwards, AR, Kocsis, JH (2012). A quality-based review of randomized controlled trials of cognitive-behavioral therapy for depression: an assessment and metaregression. American Journal of Psychiatry 169, 22–30.Google Scholar
Tolin, DF (2010). Is cognitive-behavioral therapy more effective than other therapies? A meta-analytic review. Clinical Psychology Review 30, 710–720.Google Scholar
Turner, EH, Matthews, AM, Linardatos, E, Tell, RA, Rosenthal, R (2008). Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine 358, 252–260.Google Scholar
Walker, E, Nowacki, AS (2011). Understanding equivalence and noninferiority testing. Journal of General Internal Medicine 26, 192–196.Google Scholar
Wampold, BE, Imel, ZE (2015). The Great Psychotherapy Debate: the Evidence for what Makes Psychotherapy Work. Routledge: New York.Google Scholar
Wampold, BE, Mondin, GW, Moody, M, Stich, F, Benson, K, Ahn, H (1997). A meta-analysis of outcome studies comparing bona fide psychotherapies: empirically, ‘All must have prizes’. Psychological Bulletin 122, 203–215.Google Scholar
Watzke, B, Rüddel, H, Jürgensen, R, Koch, U, Kriston, L, Grothgar, B, Schulz, H (2012). Longer term outcome of cognitive-behavioural and psychodynamic psychotherapy in routine mental health care: randomised controlled trial. Behaviour Research and Therapy 50, 580–587.Google Scholar
Woeller, W, Leichsenring, F, Leweke, F, Kruse, J (2012). Psychodynamic psychotherapy for posttraumatic stress disorder related to childhood abuse. Principles for a treatment manual. Bulletin of the Menninger Clinic 76, 69–93.Google Scholar
Yeaton, WH, Sechrest, L (1981). Critical dimensions in the choice and maintenance of successful treatments: strength, integrity, and effectiveness. Journal of Consulting and Clinical Psychology 49, 156–167.Google Scholar
Yeomans, F (2007). Questions concerning the randomized trial of schema-focused therapy vs transference-focused psychotherapy. Archives of General Psychiatry 64, 610–611.Google Scholar