Hostname: page-component-745bb68f8f-g4j75 Total loading time: 0 Render date: 2025-01-14T12:00:30.592Z Has data issue: false hasContentIssue false

Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study

Published online by Cambridge University Press:  20 December 2024

Myrthe van den Broek
Affiliation:
Research and Development, War Child Alliance, Amsterdam, The Netherlands Amsterdam Institute for Social Science Research, University of Amsterdam, Amsterdam, The Netherlands
M. Claire Greene
Affiliation:
Program on Forced Migration and Health, Columbia University Mailman School of Public Health, NY, USA
Anthony F. Guevara
Affiliation:
Research and Development, War Child Alliance, Amsterdam, The Netherlands
Sandra Agondeze
Affiliation:
Research and Development, War Child Alliance, Kampala, Uganda
Erimiah Kyanjo
Affiliation:
Transcultural Psychosocial Organization Uganda, Kampala, Uganda
Olivier Irakoze
Affiliation:
Transcultural Psychosocial Organization Uganda, Kampala, Uganda
Rosco Kasujja
Affiliation:
Department of Mental Health, School of Psychology, College of Humanities and Social Sciences, Makerere University, Kampala, Uganda
Brandon A. Kohrt
Affiliation:
Center for Global Mental Health Equity, Department of Psychiatry and Behavioral Health, George Washington University, DC, USA
Mark J. D. Jordans*
Affiliation:
Research and Development, War Child Alliance, Amsterdam, The Netherlands Amsterdam Institute for Social Science Research, University of Amsterdam, Amsterdam, The Netherlands
*
Corresponding author: Mark J.D. Jordans; Email: mark.jordans@warchild.net
Rights & Permissions [Opens in a new window]

Abstract

This proof-of-concept study evaluated an optimization strategy for the Community Case Detection Tool (CCDT) aimed at improving community-level mental health detection and help-seeking among children aged 6–18 years. The optimization strategy, CCDT+, combined data-driven supervision with motivational interviewing techniques and behavioural nudges for community gatekeepers using the CCDT. This mixed-methods study was conducted from January to May 2023 in Palorinya refugee settlement in Uganda. We evaluated (1) the added value of the CCDT+ in improving the accuracy of detection and mental health service utilization compared to standard CCDT, and (2) implementation outcomes of the CCDT+. Of the 1026 children detected, 801 (78%) sought help, with 656 needing mental health care (PPV = 0.82; 95% CI: 0.79, 0.84). The CCDT+ significantly increased detection accuracy, with 2.34 times higher odds compared to standard CCDT (95% CI: 1.41, 3.83). Additionally, areas using the CCDT+ had a 2.05-fold increase in mental health service utilization (95% CI: 1.09, 3.83). The CCDT+ shows promise as an embedded quality-optimization process for the detection of mental health problems among children and enhance help-seeking, potentially leading to more efficient use of mental health care resources.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0), which permits non-commercial re-use, distribution, and reproduction in any medium, provided that no alterations are made and the original article is properly cited. The written permission of Cambridge University Press must be obtained prior to any commercial use and/or adaptation of the article.
Copyright
© The Author(s), 2024. Published by Cambridge University Press

Impact statement

Globally, nearly a quarter of all years lived with disability due to mental disorders occurring before the age of 25 (Kieling et al., Reference Kieling, Buchweitz, Caye, Silvani, Ameis, Brunoni, Cost, Courtney, Georgiades, Merikangas, Henderson, Polanczyk, Rohde, Salum and Szatmari2024). Yet, help-seeking rates for mental health problems among children and adolescents remain low (Reardon et al., Reference Reardon, Harvey, Baranowska, O’Brien, Smith and Creswell2017). The Community Case Detection Tool (CCDT) is an evidence-based tool developed for trusted and respected community members to facilitate community-level proactive detection of mental health needs and promote help-seeking at available care (van den Broek et al., Reference van den Broek, Agondeze, Greene, Kasujja, Guevara, Tukahiirwa, Kohrt and Jordans2024).

This proof-of-concept study evaluates an optimization strategy of the CCDT, called CCDT+, designed to enhance the quality of detection and effectiveness in promoting help-seeking. The CCDT+ consists of a dashboard that presents actionable outcomes for data-driven supervision and integrates motivational interviewing techniques, along with behavioural nudges, into the training of community members using the CCDT to encourage help-seeking.

The CCDT+ significantly improved detection accuracy, with 2.34 times higher odds compared to standard CCDT. Additionally, areas using the CCDT+ saw a 2.05-fold increase in mental health service utilization. Qualitative findings showed that the CCDT+ was perceived to improve work efficiency, effectiveness, quality and boosted motivation. Access issues to real-time data for supervisors and gaps in coordination between service providers and gatekeepers were the main reported barriers.

The CCDT+ introduces an embedded quality-improvement process for mental health detection tools and shows promise in enhancing the accuracy of referrals over time and in real time. Optimization strategies like the CCDT+ can contribute to the more effective use of scarce resources, which is especially important given the limited availability of mental health services in most low- and middle-income countries (LMICs) (Patel et al., Reference Patel, Saxena, Lund, Kohrt, Kieling, Sunkel, Kola, Chang, Charlson, O’Neill and Herrman2023).

Introduction

Globally, nearly a quarter (24.85%) of all years lived with disability caused by mental disorders occur before the age of 25 (Kieling et al., Reference Kieling, Buchweitz, Caye, Silvani, Ameis, Brunoni, Cost, Courtney, Georgiades, Merikangas, Henderson, Polanczyk, Rohde, Salum and Szatmari2024). Despite this important window for detection and access to care, rates of help-seeking for mental health problems among children and adolescents remain low (Reardon et al., Reference Reardon, Harvey, Baranowska, O’Brien, Smith and Creswell2017). Children often rely on others to identify mental health problems, access services and continue the use of care (Godoy et al., Reference Godoy, Mian, Eisenhower and Carter2015). Children in low- and middle-income countries (LMICs) are disproportionately affected in terms of access to mental health care due to limited financial and human resources, lack of policies and services focusing specifically on child and adolescent mental health, and a paucity of accurate tools to support identification and screening of mental health conditions among children (Babatunde et al., Reference Babatunde, van Rensburg, Bhana and Petersen2019). Despite the growing availability of effective mental health interventions for children in LMICs (Venturo-Conerly et al., Reference Venturo-Conerly, Eisenman, Wasil, Singla and Weisz2023), only a limited number have been brought to scale (Jordans and Kohrt, Reference Jordans and Kohrt2020). Even where services are available, demand-side barriers – such as a low perceived need for care, under-detection, stigma and a preference to handle the problem by oneself – further hinder help-seeking for mental health problems (Andrade et al., Reference Andrade, Alonso, Mneimneh, Wells, Al-Hamzawi, Borges, Bromet, Bruffaerts, De Girolamo, De Graaf, Florescu, Gureje, Hinkov, Hu, Huang, Hwang, Jin, Karam, Kovess-Masfety, Levinson, Matschinger, O’Neill, Posada-Villa, Sagar, Sampson, Sasu, Stein, Takeshima, Viana, Xavier and Kessler2014; Kazdin, Reference Kazdin2019). In children and adolescents, detecting mental health problems is particularly challenging due to varying developmental stages and a wide range of normal behaviours throughout these stages, which make it difficult for caregivers to identify behaviours that indicate a need for care (Kazdin, Reference Kazdin2019). These challenges are exacerbated in conflict-affected and low-resourced settings, where daily disruptions and the burden on gatekeepers may hinder early identification.

The community case detection tool (CCDT; also known as ReachNow) has been developed to address demand-side barriers to mental health care for children and adolescents by facilitating community-level proactive detection of mental health care needs and promoting help-seeking. The tool was developed with and for community gatekeepers – trusted and respected community members without specialized training in mental health – and can be used in daily routine activities (Jordans et al., Reference Jordans, Kohrt, Luitel, Komproe and Lund2015, Reference Jordans, Luitel, Lund and Kohrt2020; van den Broek et al., Reference van den Broek, Ponniah, Jeyakumar, Koppenol-Gonzalez, Kommu, Kohrt and Jordans2021, Reference van den Broek, Hegazi, Ghazal, Hamayel, Barrett, Kohrt and Jordans2023). It presents common symptoms of childhood psychological distress through contextualized easy-to-understand illustrated vignettes. Previous studies have demonstrated the accuracy and effectiveness of the tool: the positive predictive value (PPV) of the tool found was 0.67 in Sri Lanka, 0.69 in Uganda and 0.77 in occupied Palestinian territories (van den Broek et al., Reference van den Broek, Ponniah, Jeyakumar, Koppenol-Gonzalez, Kommu, Kohrt and Jordans2021, Reference van den Broek, Hegazi, Ghazal, Hamayel, Barrett, Kohrt and Jordans2023, Reference van den Broek, Agondeze, Greene, Kasujja, Guevara, Tukahiirwa, Kohrt and Jordans2024). Furthermore, in the locations where the CCDT was used, a significant 17-fold increase in the utilization rate of mental health care services among children aged 6–18 years was found, compared to routine detection and mental health awareness-raising activities (van den Broek et al., Reference van den Broek, Agondeze, Greene, Kasujja, Guevara, Tukahiirwa, Kohrt and Jordans2024).

Given the limited availability of mental health care services in most LMICs (Patel et al., Reference Patel, Saxena, Lund, Kohrt, Kieling, Sunkel, Kola, Chang, Charlson, O’Neill and Herrman2023), it is important to ensure that tools to detect children in need of those services have a low false positive rate so that scarce resources can be used most optimally. Establishing the accuracy of tools to detect mental health problems in new contexts is a resource-intensive process. Even after validation, standardized tools like the PHQ-9 still often yield high rates of false positives, with PPVs ranging from 0.17 to 0.37 in South Africa, 0.23 in Kenya and 0.31 in Nepal (Luitel et al., Reference Luitel, Rimal, Eleftheriou, Rose-Clarke, Nayaju, Gautam, Pant, Devkota, Rana, Chaudhary, Gurung, Åhs, Carvajal-Velez and Kohrt2024; Marlow et al., Reference Marlow, Skeen, Grieve, Carvajal-Velez, Åhs, Kohrt, Requejo, Stewart, Henry, Goldstone, Kara and Tomlinson2023; Tele et al., Reference Tele, Carvajal-Velez, Nyongesa, Ahs, Mwaniga, Kathono, Yator, Njuguna, Kanyanya, Amin, Kohrt, Wambua and Kumar2023). Furthermore, without leveraging routine data, the accuracy levels of these instruments remain the same. High rates of false positives can cause unnecessary discomfort for children and risk overburdening available services.

Digital dashboards have emerged as increasingly common tools for monitoring service quality and optimizing outcomes (Bickman, Reference Bickman2008; Randell et al., Reference Randell, Alvarado, Elshehaly, McVey, West, Doherty, Dowding, Farrin, Feltbower, Gale, Greenhalgh, Lake, Mamas, Walwyn and Ruddle2022). These dashboards use data visualisation techniques to summarize data and provide insight into key metrics in an easy-to-understand format. Furthermore, these key metrics can be used to inform supervision and enhance supervision effectiveness (Randell et al., Reference Randell, Alvarado, Elshehaly, McVey, West, Doherty, Dowding, Farrin, Feltbower, Gale, Greenhalgh, Lake, Mamas, Walwyn and Ruddle2022).

This study is a proof-of-concept study of an optimisation strategy for the CCDT, the CCDT+, developed to monitor and improve the quality of detection and effectiveness of help-seeking promotion. The CCDT+ includes a dashboard presenting actionable outcomes for data-driven supervision and integrates motivational interviewing (MI) techniques and behavioural nudges in the community gatekeeper training and supervision sessions to promote help-seeking. The objectives of this study are to (1) assess the added value of the CCDT+ in improving accuracy and service utilization outcomes compared to the standard CCDT, and (2) evaluate implementation outcomes of the CCDT+.

Methods

Study design

This mixed-methods study was conducted from January to May 2023 in Palorinya refugee settlement located in Obongi District in the West Nile region in Uganda. Uganda accommodates over 1.5 million refugees and asylum seekers and is one of the world’s leading hosts for refugees (UNHCR, 2023). There are 14 formal refugee settlements in Uganda, each sub-divided into administrative units called ‘zones’. Despite being entitled to several services – such as education, healthcare and employment – refugees often face a multiplicity of risk factors for adverse mental health outcomes, including social isolation and loss of livelihoods (Stark et al., Reference Stark, Meinhart, Hermosilla, Kajungu, Cohen, Agaba, Obalim, Knox and Onyango Mangen2024). The prevalence of mental health problems among children and adolescents has been reported to reach 23% (Opio et al., Reference Opio, Munn and Aromataris2022). Palorinya refugee settlement was established in 2016 and is divided into five zones with a total population of 127,000 during the time of this study, an estimated 43% of whom are aged between 5 and 17 years. Majority of refugees are from South Sudan (UNHCR, 2022).

The CCDT+ was integrated into ongoing programs of an international humanitarian organization, War Child, and a national mental health care provider, the Transcultural Psychosocial Organization (TPO) Uganda. This study was conducted in all five zones. Two neighbouring zones were combined as one. The median zone population size was 36434.5 (IQR 25828.5, 37671).

This proof-of-concept study comes after a stepped wedge cluster randomized trial (SW-CRT) that evaluated the effectiveness of the standard CCDT in Uganda from January till November 2022 (van den Broek et al., Reference van den Broek, Agondeze, Greene, Kasujja, Guevara, Tukahiirwa, Kohrt and Jordans2024). During the SW-CRT, the CCDT was sequentially rolled out across 28 zones in five refugee settlements over a period of nine months. These settlements encompassed Bidi Bidi, Kyaka II, Kyangwali, Omugo and Rhino. The proof-of-concept study presented here follows the same procedures in a similar setting and population, and the comparative data used in this study is drawn from the SW-CRT conducted immediately prior to this study.

Participants

Participants included trusted and respected community gatekeepers trained in using the CCDT+, children and adolescents detected by these gatekeepers, and one clinical psychologist and two social workers contracted by TPO. Similar to the SW-CRT, the number of gatekeepers per zone was based on the total zone population size, applying a ratio of one gatekeeper for every 3000 residents. Gatekeepers were selected by War Child through their established networks and existing working relationships, taking into account their roles and positions within the community. Inclusion criteria for gatekeepers were individuals aged 18 years or older who were trusted and respected members of the community, actively involved in child wellbeing, and with access to families. Examples of such gatekeepers included youth club leaders, village health team members and intervention facilitators. Children and adolescents participating in this study included all children aged 6–18 who were detected by gatekeepers as matching with the CCDT. Only those who subsequently sought help at TPO were included in our sub-sample for analysing the main outcomes related to the accuracy of detected cases and service utilization.

Procedures

Standard CCDT

The CCDT was developed based on the adult Community Informant Detection Tool (CIDT) (Jordans et al., Reference Jordans, Kohrt, Luitel, Komproe and Lund2015). The tool consists of two illustrated vignettes printed on a single sheet of paper. Each vignette presents a case story and six illustrations of a child experiencing common internalizing or externalizing problems, including symptoms related to depression, anxiety and oppositional defiant disorder. At the end of each vignette, a short decision algorithm supports gatekeepers to gauge the resemblance, frequency and intensity of symptoms observed, and to determine the follow-up action. See Figure 1. In case of a match with the tool, the gatekeeper is advised to engage in a dialogue with the caregivers to encourage help-seeking to a known and available mental health service. The vignettes are culturally adapted through input from potential gatekeepers and national mental health care workers, blind back-translations and focus group discussions (FGDs) with potential gatekeepers to assess appropriateness and acceptability. The tool uses colloquial language and non-stigmatizing local idioms of distress to support proactive detection of symptoms by people without specialized training in mental health, and by using daily observations.

Figure 1. The Community Case Detection Tool.

The standard CCDT training is two days and focuses on the basics of child and adolescent mental health, use of the tool, child safeguarding and ethical considerations. Gatekeepers (n = 177) in the SW-CRT participated in the standard CCDT training delivered by a clinical psychologist (n = 4 in total) and a project officer based in each settlement. Gatekeepers used the tool during their daily routine activities and promoted help-seeking for children and adolescents matching with one of the vignettes. They provided information about how to access mental health care services, assigned a study ID and recorded de-identified detection data in a logbook (i.e., date of detection, age, gender, vignette used and location). Upon accessing the mental health care services, routine intake data was collected (i.e., date of intake, age, gender, mental health assessment outcome and location). Monthly supervision sessions were organized by the psychologist and a project officer based in their settlement.

Optimisation strategy: CCDT+

The CCDT+ is an enhanced version of the standard CCDT. It combines the standard CCDT (i.e., the tool for gatekeepers to support proactive community-level detection and help-seeking promotion) with an optimisation strategy consisting of: (i) MI techniques combined with behavioural nudges used by gatekeepers to promote help-seeking; and (ii) a digital dashboard for supervisors with key metrics around help-seeking and the accuracy of detection. Gatekeepers received a 2.5 day training by a trained supervisor in the standard CCDT training, plus an additional half-day session focusing on the MI techniques and behavioural nudges. MI is used as a collaborative conversation technique to enhance an individual’s own motivation and commitment to change and was originally developed as a treatment for individuals with substance use disorders (Miller and Rollnick, Reference Miller and Rollnick2013). MI has been extended to treat other mental health problems and health behaviours such as medication adherence for chronic illness. Furthermore, MI has also been effectively used as a pre-treatment intervention to increase motivation to seek help and engage in further assistance (Lawrence et al., Reference Lawrence, Fulbrook, Somerset and Schulz2017). Three MI techniques were integrated in the gatekeeper training: (i) asking open questions, (ii) affirming, and (iii) reflective listening. In addition, gatekeepers were trained in delivering in-person reminder messages as behavioural nudges to further encourage help-seeking among those that were detected. Nudges are based on behavioural economic theory and are used as strategies to alter an individual’s behaviour in a predictable manner without prohibiting any choices or significantly altering their economic incentives (Thaler and Sunstein, Reference Thaler and Sunstein2008). Reminders are an example of a low-cost behavioural nudge and have been effectively applied to promote other health-related decisions such as vaccination uptake (Dai et al., Reference Dai, Saccardo, Han, Roh, Raja, Vangala, Modi, Pandya, Sloyan and Croymans2021). This combination of MI and behavioural nudges aims to first increase motivation and intentions to seek help among those detected, followed by targeted reminders to support the transition from intentions to action.

Gatekeepers used the CCDT, MI and behavioural nudges during their daily routine activities to detect children and promote help-seeking. Caregivers of children detected were encouraged to seek help and received a referral card from the gatekeeper with information about how to contact and reach TPO. Mental health services provided by TPO included group interventions such as Journey of Life, Cognitive Behavioural Therapy, individual specialized care or referral to other service providers.

Fortnightly data-driven supervision meetings led by two social workers were organized for gatekeepers by a project officer. These social workers, supervised by a clinical psychologist, were each responsible for gatekeepers in two zones. The supervisors (two social workers and a clinical psychologist) had access to the CCDT+ dashboard on a tablet or laptop. This dashboard combines detection data collected by gatekeepers and routine intake data collected by the mental health service providers (TPO) and provides the following actionable insights: (1) the number and location of CCDT-detected cases, (2) which CCDT-detected cases sought help and accessed care using a client ID, and (3) the accuracy of the CCDT-detected cases that sought help. A supportive supervision approach was followed, which is a collaborative and non-hierarchical approach to supervision. It fosters open communication, joint problem-solving and skill-building, allowing gatekeepers to discuss challenges, and receive constructive feedback based on the data presented on the dashboard (McBride and Travers, Reference McBride and Travers2021). The supervisors were trained by the research team in the gatekeeper training materials and received two days of training in using the CCDT+ dashboard to supervise gatekeepers. See Figure 2 for a screenshot of the dashboard.

Figure 2. Screenshot of the CCDT+ dashboard – overview page (mobile and desktop version).

The dashboard enables data-driven supervision and was used by supervisors to identify areas for quality improvement and to strengthen the capacity of gatekeepers in terms of accuracy of detection and effectiveness in help-seeking promotion. Prior to each supervision meeting, supervisors accessed the dashboard to record key observations based on the trends in the data. With data linked to individual gatekeeper IDs, supervisors provided feedback to groups of gatekeepers as well as more targeted support to individual gatekeepers. The following outcome metrics were shown on the dashboard for quality improvement and capacity strengthening:

  1. 1) Service utilization. Calculated as the proportion of children and adolescents detected by gatekeepers that utilized TPO’s mental health care services. If detected cases had not sought help within four to eight weeks after being detected, supervisors would share the client IDs with individual gatekeepers and revisit the MI techniques and reminder methods with the gatekeeper. The four- to eight-week window was chosen to provide enough time to seek help (four weeks after detection) while also respecting the right not to seek help (beyond eight weeks after detection).

  2. 2) Accuracy expressed as the PPV. PPV was calculated as the proportion of children and adolescents detected through the CCDT who were considered as needing mental health care services. The need for services was based on the information gathered during the clinical interview conducted by TPO using structured mental health symptom checklists. A PPV below 75% served as a prompt for supervisors to provide additional capacity strengthening with (individual) gatekeepers by revisiting the content of the vignettes. This PPV threshold was chosen because a PPV lower than 75% indicates that more than one in four children did not meet the criteria to receive services, and therefore potentially overburdening the health system and causing discomfort among children.

The dashboard was developed through three steps including (1) a hackathon with data scientists to develop a minimum viable product; (2) the development of proof-of-concept version based on multiple feedback rounds with the research team; and (3) two rounds of online user testing in Uganda and adaptations with three clinical supervisors and a coordinator from TPO as potential end-users of the dashboard.

Consent procedures

Gatekeepers, social workers and the clinical psychologist provided written informed consent for participating in the research activities. Children and adolescents under the age of 18 provided written assent, and their caregivers provided written informed consent to share data on mental health service utilization with the research team for study purposes.

Outcomes and measures

The outcomes used to assess the added value of the CCDT+ compared to the standard CCDT included: (1) the PPV of the CCDT+, and (2) mental health care services utilization during the implementation of the CCDT+. Both outcomes were operationalized and measured the same way as in the SW-CRT evaluating the effectiveness of the CCDT. The PPV was defined as the proportion of children and adolescents detected who were considered as needing mental health care services (i.e., true positive). The primary reference criterion for a true positive was an indication for treatment as assessed by a mental health care provider. The secondary reference criterion was the presence of any mental health condition matching the CCDT or severe distress as assessed by a mental health care provider. Mental health care utilization was defined as: (i) the count of new cases, that is, children and adolescents aged 6–18 years, who are seeking mental health care services for the first time, and (ii) the count of re-entry cases, seeking mental health care services after a lapse of at least six months, assuming the CCDT facilitated their re-entry to care. These data were extracted and tabulated monthly using TPO’s routine mental health case registration form.

The implementation outcomes included the perceived acceptability, appropriateness, feasibility and usability of the CCDT+ by gatekeepers and supervisors. Acceptability was defined as the perception of whether various elements of CCDT+ were agreeable, palatable or satisfactory (Proctor et al., Reference Proctor, Silmere, Raghavan, Hovmand, Aarons, Bunger, Griffey and Hensley2011). This was assessed using the 4-item Acceptability of Intervention Measure (AIM) (Weiner et al., Reference Weiner, Lewis, Stanick, Powell, Dorsey, Clary, Boynton and Halko2017). Appropriateness was defined as the perceived fit, relevance or compatibility of the CCDT+ (Proctor et al., Reference Proctor, Silmere, Raghavan, Hovmand, Aarons, Bunger, Griffey and Hensley2011) and assessed using the 4-item Intervention Appropriateness Measure (IAM) (Weiner et al., Reference Weiner, Lewis, Stanick, Powell, Dorsey, Clary, Boynton and Halko2017). Feasibility was defined as the extent to which various elements of CCDT+ can be successfully used (Proctor et al., Reference Proctor, Silmere, Raghavan, Hovmand, Aarons, Bunger, Griffey and Hensley2011) and assessed using the 4-item Feasibility of Intervention Measure (FIM) (Weiner et al., Reference Weiner, Lewis, Stanick, Powell, Dorsey, Clary, Boynton and Halko2017). Usability was defined as the extent to which various elements of the CCDT+ could be used by gatekeepers and supervisors to achieve specified goals with effectiveness, efficiency and satisfaction and was assessed using the 10-item Intervention Usability Scale (IUS) (Lyon et al., Reference Lyon, Pullmann, Jacobson, Osterhage, Achkar, Renn, Munson and Arean2021). These implementation science measures were adapted for use in Uganda and administered in English, Juba Arabic and Bari. The adaptation process included an initial review of the items, forward and blind back-translation, cognitive interviewing and pilot testing. These surveys were administered post-implementation with the clinical psychologist (n = 1), social workers (n = 2) and all gatekeepers (n = 45).

Qualitative feedback regarding these implementation outcomes was gathered post-implementation, through key-informant interviews (KIIs) with the clinical psychologist (n = 1), social workers (n = 2), gatekeepers (n = 8) and three FGDs with gatekeepers (n = 27 in total). Gatekeepers for the FGDs were purposively selected based on their level of participation (e.g., active and less active in using the tool and in supervision meetings). These were conducted in a central place in the community, by the trained project officer coordinating the training and supervision sessions. Topics included experiences in using the dashboard, organizing and participating in supervision sessions, using the MI techniques and reminders, and challenges and recommendations. See Supplementary Material S1 for the sample characteristics and topic guides.

Analyses

Statistical analyses

We estimated the added value of the CCDT+ on improving the PPV and mental health care service utilization outcomes compared to the standard CCDT. This involved comparing the PPV and mental health service utilization rates in Palorinya during CCDT+ implementation with those of five other refugee settlements in Uganda where standard CCDT was in place, using data from the SW-CRT for the latter.

We compared the PPV of detected cases between the SW-CRT and current study data over four months post-CCDT implementation period using logistic regression accounting for clustering within zones using a sandwich estimator. We compared the mental health care service utilization between the SW-CRT and current study data using a negative binomial regression model with a population size offset.

For both, the comparison data was restricted to the data collected during the same post-CCDT implementation timeline as the CCDT+ implementation period in Palorinya (i.e., four months post-CCDT+ implementation data in Palorinya were compared to the first four months of post-CCDT implementation data in the comparison settlements).

The distribution of usability, feasibility, acceptability and appropriateness indicators collected during post-interviews are presented as descriptive analyses. We explored whether these indicators varied by gatekeeper type using Kruskal–Wallis tests.

Qualitative analyses

A pragmatic approach to analysing the qualitative data was used, in line with the applied nature and aim of this study to gather experiences and feedback about the CCDT+ as an optimization strategy. We used a modified framework method (Ramanadhan et al., Reference Ramanadhan, Revette, Lee and Aveling2021; Ritchie and Spencer, Reference Ritchie and Spencer2002), with a hybrid inductive and deductive approach to the analysis. The process included familiarization, open-coding and thematic framework development. All transcripts were indexed based on the framework, charted in NVivo version 12 and interpreted per theme. A more detailed description of the process can be found in Supplementary Material S1, and the completed COREQ (consolidated criteria for reporting qualitative research) checklist can be found in Supplementary Material S2 (Tong et al., Reference Tong, Sainsbury and Craig2007).

Results

During the proof-of-concept period, 45 gatekeepers (33% female) were trained in the five zones in Palorinya. Gatekeepers detected 1026 children and adolescents as matching with the CCDT. On average, detected children and adolescents were 12.18 years of age (SD = 3.63) and 58.38% were male. Of the 1026 detected cases, 801 (78.1%) utilized TPO’s mental health care services for the first time or re-entered after not having sought help for at least six months. Among the group that sought help (n = 801), 656 children and adolescents were indicated to be in need of mental health care based on the clinical interview (PPV = 0.82; 95% CI: 0.79, 0.84), and 670 were diagnosed with a mental health condition corresponding to the CCDT or experienced severe distress (PPV = 0.84; 95% CI: 0.81, 0.86). The odds of accurate case detection (among children who utilized care for the first time or re-entered) was significantly higher in zones where the CCDT+ was implemented when compared to zones using standard CCDT. More specifically, there was a 2.34-fold increase in the odds of accurate case detection among children who utilized treatment based on the indication for treatment criterion (95% CI: 1.41, 3.83). Similarly, there was a 5.53-fold increase in the odds of accurate case detection among children who utilized treatment based on the diagnostic outcome criterion (95% CI: 3.94, 7.76). See Table 1.

Table 1. Positive predictive value of the CCDT+ vs. CCDT

PPV = positive predictive value

1 One observation is missing information on the indication for treatment.

There was a 2.05-fold increase in the rate of mental health services utilization over time in the CCDT+ zones as compared to the zones that implemented the standard CCDT (95% CI: 1.09, 3.83). We observed a significant decline in utilization over time, which did not appear to differ across study conditions (IRR = 1.06, 95% CI: 0.70, 1.60). Similarly, case detection also declined over time in both conditions (IRR = 0.80, 95% CI: 0.59, 1.08). The rate of detection over time is 1.54 times higher in CCDT+ zones, however, this difference was not significant (95% CI: 0.62, 3.81). Settlement-specific utilization rates can be found in Supplementary Table S1.

The levels of acceptability, appropriateness, feasibility and usability of the CCDT+ as reported by gatekeepers and supervisors were high, see Supplementary Table S2. There were no significant differences in implementation outcomes by gatekeeper type.

Qualitative findings regarding the implementation of the CCDT+ were around; (1) work efficiency and effectiveness, (2) professional development, (3) perceived impact on work quality, and (4) role and expectations. The main findings, themes and key quotes are presented in Table 2.

Table 2. Key themes regarding the implementation of the CCDT+

Theme 1. Work efficiency and effectiveness

Supervisors found the dashboard useful for daily tasks, particularly for guiding community outreach efforts, monitoring gatekeepers’ performance and identifying areas needing attention during supervision. The insights presented in the dashboard combined with feedback provided by gatekeepers – such as reasons for individuals not seeking help – allowed for more efficient outreach scheduling by the supervisors. Furthermore, supervisors observed an increase in help-seeking during the period of implementation, which was a motivating factor for supervisors. The main challenges supervisors experienced were related to the technological aspects of the dashboard. Issues such as data errors and limited access to the dashboard due to license issues impacted follow-ups and outreach planning. Gatekeepers perceived the MI techniques and reminders as enhancing their effectiveness in promoting help-seeking. Additionally, the information shared by supervisors enabled gatekeepers to plan their mobilization efforts more precisely. One related key recommendation from gatekeepers was to improve coordination between gatekeepers and service providers to ensure that gatekeepers can share up-to-date information about when and where services will be available.

Theme 2. Professional development

Supervisors and gatekeepers both valued the feedback loops from supervisor to gatekeeper and gatekeeper to supervisor as key motivators in their work. It was seen as confirming the positive outcomes of their efforts and enhancing their sense of accomplishment and effectiveness. Supervisors appreciated the use of the dashboard as a new skill they learned, which enhanced their supervision capabilities. In addition, having access to this type of data was seen as unique for teams implementing projects. Gatekeepers valued both positive and negative feedback, this boosted their confidence, kept them motivated and minimized mistakes. Ongoing capacity strengthening during the supervision meetings helped gatekeepers recall forgotten aspects of the training and addressed new questions that came up from practical implementation. The supervision meetings provided a supportive environment where challenges were openly discussed and practical solutions were developed. This opportunity to receive and provide peer support was another important element for gatekeepers.

Theme 3. Work quality

The dashboard enabled supervisors to identify trends and inconsistencies in the data nearly in real time. Supervisors used this to continue capacity-strengthening activities with gatekeepers in a group and allowed for more precise and individual training if certain areas had to be improved by specific gatekeepers. After conducting these sessions, supervisors noticed increases in true positive rates. Gatekeepers played an active role in setting the agenda for the supervision meetings. The additional training during the supervision sessions was appreciated by gatekeepers, not only to correct mistakes but also to refresh certain skills and practice.

Theme 4. Role and expectations

The dashboard aligned well with the work of supervisors. For gatekeepers, the activities aligned particularly well with those who were already conducting household visits. The main challenge with reminding people to seek help and the more frequent interaction between gatekeepers and families was that families often asked for details regarding the care that was provided, which gatekeepers did not know due to confidentiality measures. Gatekeepers therefore sometimes struggled to provide satisfactory answers. Additionally, families sometimes expected material goods and questioned gatekeepers when these were not provided, which posed a challenge for the gatekeepers and affected their status within the community. Despite the role as a gatekeeper being voluntary, gatekeepers appreciated the small transportation refunds and breakfast provided. This minimal compensation was crucial for their motivation and ability to support their own families. It was recommended to increase the transport refund based on distance, provide relevant material goods and organize more frequent meetings in central locations.

Discussion

The gap between the need for mental health care among children and adolescents and its provision is a global issue. Given the scarcity of mental health resources in most LMICs, optimization strategies are essential to monitor and improve the quality of evidence-based detection tools. These strategies can contribute to a more efficient use of limited resources. In this proof-of-concept study, we evaluated the CCDT+, an optimization strategy for a tool developed to detect children in need of mental health care and promote help-seeking.

In areas where the CCDT+ was implemented, the PPVs were high and consistent across both reference criteria: needing mental health services (PPV=0.82) and the presence of any mental health condition matching the CCDT or severe distress (PPV=0.84). Furthermore, the odds of accurate detection were significantly higher, in fact, more than two times as high in zones using the CCDT+ compared to those using the standard CCDT. This suggests that the CCDT+ reduces false positives and alleviates unnecessary burden on mental health services and discomfort for children. A key element of the optimization strategy was the data-driven supervision which included ongoing feedback for (individual) gatekeepers about the percentage of children they detected who met criteria for mental health services out of the total number detected. If more than one in four children did not meet the criteria to receive services, individual gatekeepers received extra training during supervision. This ongoing feedback could have improved the accuracy of detection and reduced the number of false positives.

Comparing the PPV found in this study with that of traditional mental health screening tools suggests that the CCDT+ may be more accurate in detecting mental health conditions. The PPV of the PHQ-9 for instance was reported as 0.23 in Kenya and 0.17–0.37 in South Africa (Marlow et al., Reference Marlow, Skeen, Grieve, Carvajal-Velez, Åhs, Kohrt, Requejo, Stewart, Henry, Goldstone, Kara and Tomlinson2023; Tele et al., Reference Tele, Carvajal-Velez, Nyongesa, Ahs, Mwaniga, Kathono, Yator, Njuguna, Kanyanya, Amin, Kohrt, Wambua and Kumar2023). However, caution is needed in this comparison, as we are comparing the accuracy against a broad range of diagnoses, whereas symptom checklists are often evaluated against specific diagnoses. Furthermore, existing tools require validation to establish local cutoffs – a time-consuming process, and after validation, the false positive rate often does not change with ongoing use. The optimization strategy presented here is an embedded quality-improvement process for mental health detection tools which has the potential to enhance the accuracy of referral over time and in real-time. The quality-improvement aspect was also appreciated by supervisors and gatekeepers. For supervisors, the CCDT+ not only allowed them to monitor the performance of specific gatekeepers but also facilitated more precise, individualised training, potentially an important factor in boosting the accuracy results discussed above. According to gatekeepers, feedback on performance, creating ongoing learning opportunities, having access to a supportive group of peers and receiving regular updates on their work served as key motivators.

We observed an overall 2-fold increase in the rate of mental health services utilization, while no significant difference in the case detection rates was observed between study conditions. This is an important finding, as other existing tools only focus on the identification of symptoms and lack an integrated help-seeking component. Our results suggest that the combination of data-driven supervision, the use of MI techniques and behavioural nudges by gatekeepers may have facilitated the transition from intentions to actual help-seeking behaviours among those detected. While this proof-of-concept demonstrates the promise of the use of MI techniques by key community members (Lawrence et al., Reference Lawrence, Fulbrook, Somerset and Schulz2017; Naar-King et al., Reference Naar-King, Outlaw, Green-Jones, Wright and Parsons2009), the effectiveness of the CCDT+ and which components are active or which dose leads to the best outcomes, will need to be evaluated using more rigorous research designs.

An important consideration in the design of the dashboard was to avoid over-detection and we therefore did not assign a threshold or target for the number of children detected. A steady rate of detection with improved accuracy and help-seeking rate in this study was therefore regarded as a positive, expected finding. Another anticipated outcome of the optimization strategy was to find a sustained or even improved impact of the CCDT over time. While this held true for accuracy outcomes, we noted a decline in mental health utilization over time, like the standard CCDT. The observation of this decline in both conditions suggests that after a certain period, the majority of cases in a given area may have been identified and sought assistance.

The qualitative findings indicated several areas for strengthening the CCDT+. First, close collaboration between gatekeepers, who mobilize families and service providers, who organize outreach services, became increasingly important with the implementation of behavioural nudges. Gatekeepers emphasized the need for up-to-date information on when and where services would be available. Second, families frequently requested information about the care provided, which gatekeepers were unable to share due to confidentiality protocols. To address this need, we recommend future initiatives that aim to promote help-seeking to include a feature enabling gatekeepers to give families broad, non-confidential updates on care progress. Finally, supervisors stressed the importance of having continuous, real-time access to detection and utilization data. Replacing paper-based detection data with digitally collected data could be one way to improve access to real-time information.

Several limitations merit attention when interpreting the results of this study. Although the comparison data was drawn from the same project, from a similar setting in Uganda, following similar procedures, the data were technically collected separately, using a different study design and at a different time point (up to 12 months earlier). Additionally, the CCDT+ gatekeeper training was a half day longer compared to the standard CCDT training. The accuracy findings relied on routinely collected data and included only children who sought help. Furthermore, the supervisors using the dashboard were also responsible for assessing mental health outcomes used for accuracy testing, potentially introducing confirmation bias. Another limitation is that we could only report the accuracy of cases that sought help; thus, false positives might have self-selected themselves out of this study. Finally, proactive case detection needs to be accompanied by accessible, quality mental health services. In this study, a partnership with TPO Uganda, a national mental health care provider, was established to support service provision; however, assessing the quality of care delivered was beyond the scope of this study.

Conclusions

Implementing optimization strategies that monitor and improve the quality of evidence-based detection tools can contribute to more efficient use of mental health care resources. The CCDT+ shows promise as an embedded quality-optimization process that integrates data-driven supervision with MI techniques and behavioural nudges to enhance the detection of mental health problems among children and promote help-seeking. This proof-of-concept study indicates that the CCDT+ may not only improve the accuracy of detection but also enhance the effectiveness of help-seeking promotion among children compared to the standard CCDT. Furthermore, it highlights some important areas for improvement. Further research is needed to evaluate the effectiveness of the different elements of the CCDT+ and the techniques used.

Abbreviations

AIM

Acceptability of Intervention Measure

CCDT

Community Case Detection Tool

CCDT+

Community Case Detection Tool+ (optimization strategy)

CIDT

Community Informant Detection Tool

FGD

Focus group discussion

FIM

Feasibility of Intervention Measure

IAM

Intervention Appropriateness Measure

IUS

Intervention Usability Scale

KII

Key-informant interview

PPV

Positive Predictive Value

TPO Uganda

Transcultural Psychosocial Organization Uganda

Open peer review

To view the open peer review materials for this article, please visit http://doi.org/10.1017/gmh.2024.150.

Supplementary material

The supplementary material for this article can be found at http://doi.org/10.1017/gmh.2024.150.

Data availability statement

The data will be available after article publication from the principal investigator at: . Data sharing requests will be assessed by a data use team, comprising of the principal investigators Prof Mark Jordans and Dr Rosco Kasujja, and investigators Dr M. Claire Greene, Myrthe van den Broek and Sandra Agondeze and shared after a data sharing agreement has been signed.

Acknowledgements

We thank all gatekeepers, children and their families for spending time in this study and acknowledge all contributions of staff from War Child Uganda and TPO Uganda. War Child and TPO Uganda were supported by Sint Antonius Stichting Projects (SAS-P) [Grant number: SAS-P-21103-UG]. The foundation had no role in the study design and implementation. We would also like to thank Deloitte for their support in developing the initial version of the dashboard through Deloitte Impact Foundation.

Author contribution

MJ and MvdB were responsible for the funding acquisition. Conceptualization of the study was done by MvdB and MJ and all authors were involved in designing and conducting the research, including (procedures related to) data collection. Management and coordination were done by SA, AFG, OI, MvdB and MJ. MCG was responsible for all formal analysis, preparation and creation of data presentation. SA, AFG, MvdB, MCG and MJ accessed and verified the data. All authors were responsible for the decision to submit the manuscript. The writing and preparation of the draft manuscript was done by MvdB and all authors were part of the reviewing and editing process. Overall supervision was done by MJ.

Financial support

War Child was supported by Sint Antonius Stichting Projects (SAS-P) [Grant number: SAS-P-21103-UG]. The foundation had no role in the study design and implementation.

Competing interest

None declared.

Ethics statement

This study was reviewed and approved by Makerere University School of Health Sciences Research and Ethics Committee (MAKSHSREC-2022-416) and Uganda’s National Council for Science and Technology (HS2609ES). Gatekeepers, social workers and the clinical psychologist participating in this study provided written informed consent. Service level consent and assent to document and release de-identified routine mental health service utilization data with the research team was obtained for study purposes from caregivers, adolescents and children detected by the CCDT that sought help.

References

Andrade, LH, Alonso, J, Mneimneh, Z, Wells, JE, Al-Hamzawi, A, Borges, G, Bromet, E, Bruffaerts, R, De Girolamo, G, De Graaf, R, Florescu, S, Gureje, O, Hinkov, HR, Hu, C, Huang, Y, Hwang, I, Jin, R, Karam, EG, Kovess-Masfety, V, Levinson, D, Matschinger, H, O’Neill, S, Posada-Villa, J, Sagar, R, Sampson, NA, Sasu, C, Stein, DJ, Takeshima, T, Viana, MC, Xavier, M and Kessler, RC (2014) Barriers to mental health treatment: Results from the WHO world mental health surveys. Psychological Medicine 44(6), 13031317. https://doi.org/10.1017/S0033291713001943.Google Scholar
Babatunde, GB, van Rensburg, AJ, Bhana, A and Petersen, I (2019) Barriers and facilitators to child and adolescent mental health services in low-and-middle-income countries: A scoping review. Global Social Welfare. https://doi.org/10.1007/s40609-019-00158-z.Google Scholar
Bickman, L (2008) A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry 47(10), 11141119. https://doi.org/10.1097/CHI.0b013e3181825af8.Google Scholar
Dai, H, Saccardo, S, Han, MA, Roh, L, Raja, N, Vangala, S, Modi, H, Pandya, S, Sloyan, M and Croymans, DM (2021) Behavioural nudges increase COVID-19 vaccinations. Nature 597(7876), 404409. https://doi.org/10.1038/s41586-021-03843-2.Google Scholar
Godoy, L, Mian, ND, Eisenhower, AS and Carter, AS (2015) Pathways to service receipt: Modeling parent help-seeking for childhood mental health problems. Administration and Policy in Mental Health 345(6195), 455459. https://doi.org/10.1126/science.1249749.Ribosome.Google Scholar
Jordans, MJD and Kohrt, BA (2020) Scaling up mental health care and psychosocial support in low-resource settings: A roadmap to impact. Epidemiology and Psychiatric Sciences. https://doi.org/10.1017/S2045796020001018.Google Scholar
Jordans, MJD, Kohrt, BA, Luitel, NP, Komproe, IH and Lund, C (2015) Accuracy of proactive case finding for mental disorders by community informants in Nepal. British Journal of Psychiatry 207(6), 501506. https://doi.org/10.1192/bjp.bp.113.141077.Google Scholar
Jordans, MJD, Luitel, NP, Lund, C and Kohrt, BA (2020) Evaluation of proactive community case detection to increase help seeking for mental health care: A pragmatic randomized controlled trial. Psychiatric Services 71(8), 810815. https://doi.org/10.1176/appi.ps.201900377.Google Scholar
Kazdin, AE (2019) Annual research review: Expanding mental health services through novel models of intervention delivery. Journal of Child Psychology and Psychiatry and Allied Disciplines 60(4), 455472. https://doi.org/10.1111/jcpp.12937.Google Scholar
Kieling, C, Buchweitz, C, Caye, A, Silvani, J, Ameis, SH, Brunoni, AR, Cost, KT, Courtney, DB, Georgiades, K, Merikangas, KR, Henderson, JL, Polanczyk, G V., Rohde, LA, Salum, GA and Szatmari, P (2024) Worldwide prevalence and disability from mental disorders across childhood and adolescence evidence from the global burden of disease study. JAMA Psychiatry. https://doi.org/10.1001/jamapsychiatry.2023.5051.Google Scholar
Lawrence, P, Fulbrook, P, Somerset, S and Schulz, P (2017, November 1) Motivational interviewing to enhance treatment attendance in mental health settings: A systematic review and meta-analysis. Journal of Psychiatric and Mental Health Nursing. 699718. https://doi.org/10.1111/jpm.12420.Google Scholar
Luitel, NP, Rimal, D, Eleftheriou, G, Rose-Clarke, K, Nayaju, S, Gautam, K, Pant, SB, Devkota, N, Rana, S, Chaudhary, JM, Gurung, BS, Åhs, JW, Carvajal-Velez, L and Kohrt, BA (2024) Translation, cultural adaptation and validation of patient health questionnaire and generalized anxiety disorder among adolescents in Nepal. Child and Adolescent Psychiatry and Mental Health 18(1). https://doi.org/10.1186/s13034-024-00763-7.Google Scholar
Lyon, AR, Pullmann, MD, Jacobson, J, Osterhage, K, Achkar, MA, Renn, BN, Munson, SA and Arean, PA (2021) Assessing the usability of complex psychosocial interventions: The intervention usability scale. Implementation Research and Practice 2. https://doi.org/10.1177/2633489520987828.Google Scholar
Marlow, M, Skeen, S, Grieve, CM, Carvajal-Velez, L, Åhs, JW, Kohrt, BA, Requejo, J, Stewart, J, Henry, J, Goldstone, D, Kara, T and Tomlinson, M (2023) Detecting depression and anxiety among adolescents in South Africa: Validity of the isiXhosa patient health questionnaire-9 and generalized anxiety disorder-7. Journal of Adolescent Health 72(1), S52S60. https://doi.org/10.1016/j.jadohealth.2022.09.013.Google Scholar
McBride, K and Travers, Á (2021) Integrated Model for Supervision for Mental Health and Psychosocial Support. Retrieved from www.tcd.ie/medicine/global-healthGoogle Scholar
Miller, WR and Rollnick, S (2013) Motivational Interviewing: Helping People Change, 3rd Edn. New York: The Guilford Press. https://doi.org/10.1093/alcalc/agt010.Google Scholar
Naar-King, S, Outlaw, A, Green-Jones, M, Wright, K and Parsons, JT (2009) Motivational interviewing by peer outreach workers: A pilot randomized clinical trial to retain adolescents and young adults in HIV care. AIDS Care – Psychological and Socio-Medical Aspects of AIDS/HIV 21(7), 868873. https://doi.org/10.1080/09540120802612824.Google Scholar
Opio, JN, Munn, Z and Aromataris, E (2022) Prevalence of mental disorders in Uganda: A systematic review and meta-analysis. Psychiatric Quarterly. https://doi.org/10.1007/s11126-021-09941-8.Google Scholar
Patel, V, Saxena, S, Lund, C, Kohrt, B, Kieling, C, Sunkel, C, Kola, L, Chang, O, Charlson, F, O’Neill, K and Herrman, H (2023, August 19) Transforming mental health systems globally: Principles and policy recommendations. The Lancet. 656666. https://doi.org/10.1016/S0140-6736(23)00918-2.Google Scholar
Proctor, E, Silmere, H, Raghavan, R, Hovmand, P, Aarons, G, Bunger, A, Griffey, R and Hensley, M (2011) Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health 38(2), 6576. https://doi.org/10.1007/s10488-010-0319-7.Google Scholar
Ramanadhan, S, Revette, AC, Lee, RM and Aveling, EL (2021) Pragmatic approaches to analyzing qualitative data for implementation science: An introduction. Implementation Science Communications 2(1), 110. https://doi.org/10.1186/S43058-021-00174-1.Google Scholar
Randell, R, Alvarado, N, Elshehaly, M, McVey, L, West, RM, Doherty, P, Dowding, D, Farrin, AJ, Feltbower, RG, Gale, CP, Greenhalgh, J, Lake, J, Mamas, M, Walwyn, R and Ruddle, RA (2022) Design and evaluation of an interactive quality dashboard for national clinical audit data: A realist evaluation. Health and Social Care Delivery Research 10(12), VII–122. https://doi.org/10.3310/WBKW4927.Google Scholar
Reardon, T, Harvey, K, Baranowska, M, O’Brien, D, Smith, L and Creswell, C (2017) What do parents perceive are the barriers and facilitators to accessing psychological treatment for mental health problems in children and adolescents? A systematic review of qualitative and quantitative studies. European Child and Adolescent Psychiatry. https://doi.org/10.1007/s00787-016-0930-6.Google Scholar
Ritchie, J and Spencer, L (2002) Qualitative data analysis for applied policy research. In Analyzing Qualitative Data. Routledge, pp. 173194.Google Scholar
Stark, L, Meinhart, M, Hermosilla, S, Kajungu, R, Cohen, F, Agaba, GS, Obalim, G, Knox, J and Onyango Mangen, P (2024) Improving psychosocial well-being and parenting practices among refugees in Uganda: Results of the journey of life effectiveness trial. Cambridge Prisms: Global Mental Health 11, e42. https://doi.org/10.1017/gmh.2024.38.Google Scholar
Tele, AK, Carvajal-Velez, L, Nyongesa, V, Ahs, JW, Mwaniga, S, Kathono, J, Yator, O, Njuguna, S, Kanyanya, I, Amin, N, Kohrt, B, Wambua, GN and Kumar, M (2023) Validation of the English and Swahili adaptation of the patient health questionnaire–9 for use among adolescents in Kenya. Journal of Adolescent Health 72(1), S61S70. https://doi.org/10.1016/j.jadohealth.2022.10.003.Google Scholar
Thaler, RH and Sunstein, CR (2008) Nudge: Improving Decisions about Health, Wealth, and Happiness. London: Yale University Press.Google Scholar
Tong, A, Sainsbury, P and Craig, J (2007) Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care 19(6), 349357. https://doi.org/10.1093/intqhc/mzm042.Google Scholar
UNHCR (2022) Uganda – Refugee Statistics January 2022 – Palorinya.Google Scholar
UNHCR (2023) Uganda: Refugees and Asylum-Seekers Map. Retrieved from https://reporting.unhcr.org/uganda-refugees-and-asylum-seekers-map-5605Google Scholar
van den Broek, M, Ponniah, P, Jeyakumar, PJR, Koppenol-Gonzalez, G V, Kommu, JVS, Kohrt, BA and Jordans, MJD (2021) Proactive detection of people in need of mental healthcare: Accuracy of the community case detection tool among children, adolescents and families in Sri Lanka. Child and Adolescent Psychiatry and Mental Health 15(1), 57. https://doi.org/10.1186/s13034-021-00405-2.Google Scholar
van den Broek, M, Hegazi, L, Ghazal, N, Hamayel, L, Barrett, A, Kohrt, BA and Jordans, MJD (2023) Accuracy of a proactive case detection tool for internalizing and externalizing problems among children and adolescents. Journal of Adolescent Health 72(1), S88S95. https://doi.org/10.1016/j.jadohealth.2021.03.011.Google Scholar
van den Broek, M, Agondeze, S, Greene, C, Kasujja, R, Guevara, AF, Tukahiirwa, RK, Kohrt, BA and Jordans, MJD (2024) A community case detection tool to promote help-seeking for mental health care among children and adolescents in Ugandan refugee settlements: A stepped wedge cluster randomised trial. The Lancet Child & Adolescent Health 8, 571579. https://doi.org/10.1186/ISRCTN19056780.Google Scholar
Venturo-Conerly, KE, Eisenman, D, Wasil, AR, Singla, DR and Weisz, JR (2023) Meta-analysis: The effectiveness of youth psychotherapy interventions in low- and middle-income countries. Journal of the American Academy of Child and Adolescent Psychiatry 62(8), 859873. https://doi.org/10.1016/j.jaac.2022.12.005.Google Scholar
Weiner, BJ, Lewis, CC, Stanick, C, Powell, BJ, Dorsey, CN, Clary, AS, Boynton, MH and Halko, H (2017) Psychometric assessment of three newly developed implementation outcome measures. Implementation Science 12(1), 112. https://doi.org/10.1186/S13012-017-0635-3/TABLES/3.Google Scholar
Figure 0

Figure 1. The Community Case Detection Tool.

Figure 1

Figure 2. Screenshot of the CCDT+ dashboard – overview page (mobile and desktop version).

Figure 2

Table 1. Positive predictive value of the CCDT+ vs. CCDT

Figure 3

Table 2. Key themes regarding the implementation of the CCDT+

Supplementary material: File

Van Den Broek et al. supplementary material

Van Den Broek et al. supplementary material
Download Van Den Broek et al. supplementary material(File)
File 157 KB

Author comment: Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study — R0/PR1

Comments

Subject: Manuscript Submission for Publication

September 9, 2024

Dear Professor Judy Bass and Professor Dixon Chibanda,

On behalf of the author group, I would like to submit our manuscript titled ‘Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study’ for publication in Cambridge Prisms: Global Mental Health.

In our manuscript we present the findings of a proof-of-concept study of the CCDT+, an enhanced version of the Community Case Detection Tool (CCDT), a tool developed for key community members to support proactive community-level detection and enhance help-seeking among children in need of mental health care. The CCDT+ includes a dashboard presenting actionable outcomes used for data-driven supervision and integrates Motivational Interviewing techniques and behavioural nudges in the training of community members using the CCDT to promote help-seeking. This mixed-methods study, conducted from January to May 2023 in the Palorinya refugee settlement in Uganda, assessed the added value of the CCDT+ in improving detection accuracy and mental health service utilization compared to the standard CCDT, which was recently evaluated in a stepped wedge cluster randomized trial (van den Broek et al. 2024).

The results demonstrate that among the group that sought help (n=801), 656 children and adolescents were indicated to be in need of mental health care based on the outcomes of a clinical interview (PPV=0.82; 95% CI: 0.79, 0.84). Furthermore, the CCDT+ significantly enhanced detection accuracy, with a 2.34-fold increase in the odds of accurate detection and a 2.05-fold increase in the rate of mental health service utilization over time, compared to the standard CCDT.

The CCDT+ introduces an embedded quality-improvement process for mental health detection tools and shows promise in enhancing the accuracy of referral over time and in real-time. Optimization strategies like the CCDT+ can contribute to the more effective use of scarce resources, which is especially important given the limited availability of mental health services in most low- and middle-income countries (LMICs) and the growing global mental health crisis (Patel et al., 2023).

We believe that our study seamlessly aligns with the focus of Cambridge Prisms: Global Mental Health and contributes a valuable and promising innovation to the mental health care gap, a matter of worldwide concern for policymakers, clinicians, and researchers.

All named authors have approved the manuscript for submission. The content of this manuscript has not been published elsewhere and will not be submitted to any other journal while under consideration by Cambridge Prisms: Global Mental Health. We hope you will find our manuscript interesting and suitable for publication.

Yours sincerely,

Prof. Dr. Mark J. D. Jordans

Director Research and Development, War Child Alliance

Amsterdam Institute for Social Science Research, University of Amsterdam

Mark.jordans@warchild.net

References

van den Broek M, Agondeze S, Greene C, Kasujja R, Guevara AF, Tukahiirwa RK, Kohrt BA and Jordans MJD (2024) A community case detection tool to promote help-seeking for mental health care among children and adolescents in Ugandan refugee settlements: a stepped wedge cluster randomised trial. The Lancet Child & Adolescent Health 8, 571–579. https://doi.org/10.1186/ISRCTN19056780.

Review: Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study — R0/PR2

Conflict of interest statement

I have no competing interests.

Comments

Thank you for this important paper. LMIC have a scarcity of mental health resources, therefore, identification of mental health needs in children and adolescents is an important step. The CCDT+ as pointed out in your paper, improves accuracy in mental health detection. Well done!

Review: Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study — R0/PR3

Conflict of interest statement

Reviewer declares none.

Comments

Thank you for the invitation to review this interesting manuscript describing an innovative tool aimed at increasing proactive case detection for mental healthcare among children. This concept is need of the hour and the paper is very well thought-out and written.

Please see few minor points for consideration:

1. Title: Data-driven supervision to optimize the effectiveness of proactive case

detection for mental health care among children: a proof-of-concept study. More details about supervision meetings (format, how the feedback in provided and the inputs given) since the paper is on supervision than on CCDT+ would be useful.

2. The number of gatekeepers is mentioned as a ratio of one gatekeeper for every 3,000 residents. Does that include all residents, not just the children and adolescents. Could the authors provide an approximate number of children and adolescents per gatekeeper.

3. A suggestion for future implementation would be to include a format in which some (non-confidential) broad update could be provided to family members by the gatekeepers.

4. It would be good to have details about how was individualised training, based on data-driven feedback, done.

5. The authors have emphasised on how the MI techniques and behavioural nudges are likely to have helped increase help-seeking behaviour. However, as this is a Proof-Of-Concept study, there doesn’t appear to be evidence for this and needs to be tested.

6. The authors have acknowledged the limitations of their study, specially not including indicators assessing the political commitment or assessing the quality of care.

Review: Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study — R0/PR4

Conflict of interest statement

Reviewer declares none.

Comments

A very well-written MS which seeks to demonstrate the utility of an enhanced mental health community detection tool for children and adolescents that seeks to improve the quality of detection and effectiveness of help-seeking promotion.

Page 7: Lines 132-134 It would be helpful if the authors could further explain how the CCDT+ was integrated in relation to War Child and TPO as it’s unclear if this was administrative or integrated with its programs.

Page 8: Lines 154-157: Further clarification is needed about recruitment of children and adolescents into the study. For inclusion into the study, was it necessary to be identified both by CCDT and help seeking at TPO? What instructions were given to the gatekeepers about recruitment of this population?

Could the authors comment on how adequately the research design was able to distinguish the impact of CCDT+ relative to supervision activity related to the CCDT+? In part, this question is motivated by the decline in utilization and detection over time suggesting that the increased monitoring was a function of supervisory activity and the substantive comments of the findings eloquently captured in the Themes 1-4 of the Results section. The function of the dashboard, beyond MI was to identify individuals “lost-to follow up” whereas no similar follow-up was part of the original CCDT data. The authors' allude to confirmation bias in describing the weaknesses of the study. If anything, this study demonstrates that well-trained supervisors are essential in the use of detection tools among lay health providers (“as an embedded quality-optimization process”).

Nevertheless, this study is an important and useful contribution to literature in the detection and help-seeking behavior of children and adolescents with internalizing and externalizing mental health problems in low and middle-income countries.

Review: Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study — R0/PR5

Conflict of interest statement

None

Comments

It was a pleasure to review this article! Thank you to the authors for their hard work in testing such an important tool that has the potential to transform how cases are detected and enhance help-seeking among children and adolescents. I have added a few small comments for your consideration.

Line 65: Could you briefly specify which mental health problems you are referring to?

Introduction: In the introduction, could you mention why child and adolescent mental health problems may go undetected? Is this the same for adults?

Line 70: Would it be worthwhile to note that mental health services are simply not available for children and adolescents in most settings and when they are, they often are not evidence based or informed and tend to be very specialized and medicalized? This statement could be strengthened to emphasize the lack of services.

Line 73: While mental health interventions for children in LMICs exist, they often are not brought to scale or made available. Is it worth elaborating on why they are not brought to scale or accessible? This could strengthen the statement.

Line 111: I suggest clarifying the difference between the CCDT tool and the CCDT+.

Line 152: How was it determined that community gatekeepers were trusted and respected members of the community? Was this established through an interview process? I recommend explaining this further.

Line 212: Could you provide more detail on what the supervision involved?

Line 272: Can you elaborate on what mental health care services were sought, perhaps by providing an example or two?

Line 339: Could you share more about TPO’s mental health care services? While this may not be the primary focus of the paper, providing this context could be important.

Recommendation: Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study — R0/PR6

Comments

Dear authors,

Your study “Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study” has now been reviewed.

Decision: Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study — R0/PR7

Comments

No accompanying comment.

Author comment: Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study — R1/PR8

Comments

Subject: Revised Manuscript [GMH-2024-0144] November 8, 2024

Dear Dr Catherine Abbo and Prof. Dixon Chibanda,

Thank you for giving us the opportunity to submit a revised version of the manuscript for publication in Cambridge Prisms: Global Mental Health.

We appreciate the valuable feedback from the reviewers and editor, which has allowed us to improve the manuscript. In our response letter, which you will find uploaded alongside this document, we have copied the comments and have written our reply and explained the changes that we have made in the manuscript. The manuscript has been revised accordingly to address the comments.

We thank you for considering our revised manuscript and are looking forward to hearing from you.

On behalf of my colleagues and myself, thank you for considering our manuscript.

Yours sincerely,

Prof. Dr Mark J. D. Jordans

War Child Alliance and University of Amsterdam

Mark.Jordans@warchild.nl

Review: Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study — R1/PR9

Conflict of interest statement

Reviewer declares none.

Comments

All my comments have been addressed

Review: Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study — R1/PR10

Conflict of interest statement

Reviewer declares none.

Comments

There are syntax errors in the MS - for example page6: line 128

Page 6: Lines 141-142: Unclear what is meant by the sentence

Page 9: Lines 200-202: Suggest rephrase sentence - “...detected with the CCDT by gatekeepers”

Recommendation: Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study — R1/PR11

Comments

Dear Authors,

Your revised manuscript titled: ‘Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study’ has now been reviewed

Decision: Data-driven supervision to optimize the effectiveness of proactive case detection for mental health care among children: a proof-of-concept study — R1/PR12

Comments

No accompanying comment.