Introduction
Adolescence is a key period of human development that corresponds with greater vulnerability to symptoms of depression and anxiety when compared with all other age groups (Weinberger et al., Reference Weinberger, Gbedemah, Martinez, Nash, Galea and Goodwin2018). Worldwide, the current prevalence rates of clinically elevated depressive and anxiety symptoms among adolescents are 25% and 20%, respectively (Racine et al., Reference Racine, McArthur, Cooke, Eirich, Zhu and Madigan2021). In addition to emotional upheaval, symptoms of depression and anxiety reduce a young person’s social and academic functioning, hindering their potential for a fulfilling and productive life (Lawrence et al., Reference Lawrence, Johnson, Hafekost, Boterhoven de Haan, Sawyer, Ainley and Zubrick2015; McGorry and Goldstone, Reference McGorry and Goldstone2011; Patel et al., Reference Patel, Flisher, Hetrick and McGorry2007). Access to effective and high-quality early intervention and treatment is critical to reduce symptoms, prevent the exacerbation of illness, and expediate recovery. Cognitive behavioural therapy (CBT) is a structured, skills-based psychological intervention that alters the unhelpful patterns of thinking, feeling and behaving that characterise depression and anxiety (Southam-Gerow and Kendall, Reference Southam-Gerow and Kendall2000). Traditionally delivered in-person by trained professionals, CBT is the gold-standard approach for symptom reduction for these illnesses and is supported by a significant evidence-base (Compton et al., Reference Compton, March, Brent, Albano, Weersing and Curry2004; Curry, Reference Curry2014; Southam-Gerow and Kendall, Reference Southam-Gerow and Kendall2000). However, a full course (i.e. 12–18 sessions) of CBT is intensive and requires significant commitment (Barlow et al., Reference Barlow, Farchione, Fairholme, Ellard, Boisseau, Allen and May2010; Zinbarg et al., Reference Zinbarg, Craske and Barlow2006). It has been estimated that more than 60% of youth with poor mental health are reluctant to engage with professional treatment (Mission Australia, 2014). Barriers related to accessibility, stigma, negative beliefs about treatment and a preference for autonomy significantly impact young people’s abilities to seek help for their mental health (Aguirre Velasco et al., Reference Aguirre Velasco, Cruz, Billings, Jimenez and Rowe2020; Gulliver et al., Reference Gulliver, Griffiths and Christensen2010; Radez et al., Reference Radez, Reardon, Creswell, Lawrence, Evdoka-Burton and Waite2021).
Due to increased computer ownership, in-person CBT has been adapted to computerised delivery to overcome many treatment barriers by providing accessible, anonymous, cost-effective, flexible, and less stigmatising access to treatment (Clarke et al., Reference Clarke, Kuosmanen and Barry2015; Hollis et al., Reference Hollis, Livingstone and Sonuga‐Barke2020; Kauer et al., Reference Kauer, Mangan and Sanci2014). Computerised CBT typically consists of automated, self-directed sessions on core therapeutic elements, such as psychoeducation, cognitive restructuring and behavioural activation, delivered sequentially via web-based programs at weekly intervals. Computerised CBT for symptoms of depression and anxiety in children and adolescents has been shown to be effective (Clarke et al., Reference Clarke, Kuosmanen and Barry2015; Ebert et al., Reference Ebert, Zarski, Christensen, Stikkelbroek, Cuijpers, Berking and Riper2015; Grist et al., Reference Grist, Croker, Denne and Stallard2019; Hollis et al., Reference Hollis, Falconer, Martin, Whittington, Stockton, Glazebrook and Davies2017), with clinical guidelines now recommending computerised CBT as a feasible first-line intervention for mild illness in this age group (Wise, 2019). However, widespread uptake of computerised CBT has been low. Premature disengagement and non-completion are common problems (Calear et al., Reference Calear, Christensen, Mackinnon, Griffiths and O’Kearney2009; Lillevoll et al., Reference Lillevoll, Vangberg, Griffiths and Eisemann2014), with many users unlikely to receive a sufficient therapeutic dose (Hollis et al., Reference Hollis, Falconer, Martin, Whittington, Stockton, Glazebrook and Davies2017). While use supported by a parent, coach or clinician may benefit treatment adherence (Bennett et al., Reference Bennett, Cuijpers, Ebert, Smith, Coughtrey, Heyman, Manzotti and Shafran2019), computerised CBT remains under-utilised for addressing the worldwide treatment gaps in youth mental health.
Notably, poor engagement in computerised CBT does not appear to be due to insufficient motivation in young people to improve their mental wellbeing. Young people have indicated that many computerised CBT programs developed by academics do not reflect the same appeal as newer commercially developed products, such as smartphone applications (apps). Specifically, young people have described some of the existing evidence-based computerised CBT programs as boring and laborious, of low technical quality, unappealing aesthetically, lacking personalisation, and containing content that is not age-appropriate (Garrido et al., Reference Garrido, Millington, Cheers, Boydell, Schubert, Meade and Nguyen2019b). In contrast, commercially developed mental health smartphone apps attract millions of downloads and several thousand monthly active users (Carlo et al., Reference Carlo, Ghomi, Renn, Strong and Areán2020). Users of two popular meditation apps, Headspace and Calm, account for more than half of all users of depression and anxiety apps (Wasil et al., Reference Wasil, Venturo-Conerly, Shingleton and Weisz2019). Although commercially developed apps appear to generate higher levels of uptake and use, most of these apps rarely contain all core elements of CBT, such as exposure and cognitive restructuring (Wasil et al., Reference Wasil, Venturo-Conerly, Shingleton and Weisz2019). Several of these apps also lack sufficient evidence of effectiveness, particularly in symptomatic youth (Bry et al., Reference Bry, Chou, Miguel and Comer2018; Grist et al., Reference Grist, Porter and Stallard2017). A recent systematic review found only 12 studies that evaluated smartphone-based interventions for internalising disorders in adolescents. Only three of these interventions were based on CBT and none were apps for depression or anxiety disorders or symptoms (Buttazzoni et al., Reference Buttazzoni, Brar and Minaker2021). Smartphones provide unceasing access to software, internet and multimedia functionality. This attribute, coupled with almost universal ownership (Anderson and Jiang, Reference Anderson and Jiang2018) and increasing integration into daily life (Bhattacharya et al., Reference Bhattacharya, Bashar, Srivastava and Singh2019) position smartphones as a promising, and largely unexplored, avenue to deliver CBT to adolescents.
The importance of co-design and co-creation in digital mental health
One method to promote the acceptability of new interventions is to engage users in the design process. Co-design aims to establish end-users’ needs and preferences and ensures that intervention content, features and aesthetics align to these. For young people, this involves ensuring the intervention is appealing, age-appropriate, relatable and suitable to their lifestyle (Garrido et al., Reference Garrido, Millington, Cheers, Boydell, Schubert, Meade and Nguyen2019b; Liverpool et al., Reference Liverpool, Mota, Sales, Čuš, Carletto, Hancheva, Sousa, Cerón, Moreno-Peral, Pietrabissa, Moltrecht, Ulberg, Ferreira and Edbrooke-Childs2020; Thabrew et al., Reference Thabrew, Fleming, Hetrick and Merry2018). Co-design is common in commercial technology (Steen et al., Reference Steen, Manschot and De Koning2011) and is associated with greater end-user engagement in final products (e.g. WeClick and iBobbly; O’Dea et al., Reference O’Dea, Achilles, Werner-Seidler, Batterham, Calear, Perry, Shand and Christensen2018; Tighe et al., Reference Tighe, Shand, Ridani, Mackinnon, De La Mata and Christensen2017). However, a recent review indicated that more than 70% of preventative digital mental interventions for young people did not report on users’ involvement in their design and development (Bergin et al., Reference Bergin, Vallejos, Davies, Daley, Ford, Harold, Hetrick, Kidner, Long, Merry, Morriss, Sayal, Sonuga-Barke, Robinson, Torous and Hollis2020). Inadequate collaboration with youth and other end-users may produce inferior and less appealing products.
Interventions that have resulted from co-design processes show some degree of consistency regarding the needs and preferences of youth (Kenny et al., Reference Kenny, Dooley and Fitzgerald2016; Stoyanov et al., Reference Stoyanov, Zelenko, Staneva, Kavanagh, Smith, Sade, Cheers and Hides2021; Werner-Seidler et al., Reference Werner-Seidler, O’Dea, Shand, Johnston, Frayne, Fogarty and Christensen2017). Specifically, young people’s feedback on digital mental health prototypes has consistently indicated that they endorse aspects such as trustworthy design and content, engaging and interactive features, gamification, ease of use, personalisation and customisability, security and privacy, ability to access and use independently, social interaction, and visual appeal (Kenny et al., Reference Kenny, Dooley and Fitzgerald2016; Stoyanov et al., Reference Stoyanov, Zelenko, Staneva, Kavanagh, Smith, Sade, Cheers and Hides2021; Werner-Seidler et al., Reference Werner-Seidler, O’Dea, Shand, Johnston, Frayne, Fogarty and Christensen2017). However, to capitalise on the potential of digital health interventions in addressing treatment gaps, it is critical to also include those supporting adolescent care in the co-design process. For example, in-person CBT programs commonly involve parents suggesting a CBT smartphone intervention may benefit from parental involvement (Cardy et al., Reference Cardy, Waite, Cocks and Creswell2020). Furthermore, involving parents in the design of a smartphone app is particularly relevant for addressing concerns around excessive screen time and its potential negative impacts on mental health (Twenge and Campbell, Reference Twenge and Campbell2019). It is also critical to include mental health professionals. While many mental health professionals do not consider computerised CBT to be a suitable replacement for in-person therapy, they have shown an increasingly positive attitude towards using technology as a therapy adjunct (Cliffe et al., Reference Cliffe, Croker, Denne and Stallard2020). Smartphone apps are viewed as particularly beneficial to support and consolidate in-session learning and CBT skill development (Gindidis et al., Reference Gindidis, Stewart and Roodenburg2020). Yet, many mental health professionals are uncertain about the safety and reliability of digital interventions and lack the knowledge to effectively integrate them into their practice (Cliffe et al., Reference Cliffe, Croker, Denne and Stallard2020; Donovan et al., Reference Donovan, Poole, Boyes, Redgate and March2015). Involving mental health professionals in the co-design process may address these limitations and help to improve acceptability and uptake of digital interventions in clinical care for youth mental health. To this end, adolescents, parents and mental health professionals must all be involved in the co-design process.
A CBT smartphone app that is co-designed with all relevant end-users and of equivalent quality to commercial products may improve uptake and engagement in this gold-standard treatment and transform access to CBT in an increasingly digital world. However, we are unaware of such a product. Furthermore, while although many papers report that young people endorse interactivity in digital programs, there is little guidance or deeper exploration of what this means for intervention design and development. Given young people’s ever-evolving digital needs and preferences, and the paucity of CBT smartphone apps developed for and evaluated among adolescents and other key stakeholders (Buttazzoni et al., Reference Buttazzoni, Brar and Minaker2021; Grist et al., Reference Grist, Croker, Denne and Stallard2019), we aimed to create a new smartphone app (called ClearlyMe) that delivers CBT to reduce elevated symptoms of depression and anxiety in adolescents aged 12 to 17 years.
Aims
The purpose of this paper is to outline the co-design process for the creation of ClearlyMe. Our approach was informed by the co-design literature (Bevan Jones et al., Reference Bevan Jones, Thapar, Rice, Beeching, Cichosz, Mars, Smith, Merry, Stallard, Jones, Thapar and Simpson2018; Hagen et al., Reference Hagen, Collin, Metcalf, Nicholas, Rahilly and Swainston2012; Thabrew et al., Reference Thabrew, Fleming, Hetrick and Merry2018). We incorporated methods employed routinely in digital product design while maintaining the over-arching objective of delivering CBT via one’s smartphone. The outcomes of our co-design process will guide researchers and clinicians in the co-design of digital mental health interventions and provide CBT practitioners with insights into the rigorous process underlying the design of ClearlyMe.
Method
Research plan
Our approach adhered to the principles of co-design using recommendations and methodologies described by Thabrew et al. (Reference Thabrew, Fleming, Hetrick and Merry2018) and Bevan Jones et al. (Reference Bevan Jones, Thapar, Rice, Beeching, Cichosz, Mars, Smith, Merry, Stallard, Jones, Thapar and Simpson2018). The goal was to understand and accommodate the needs and preferences of users at every stage of the design process. This ensured an iterative design approach that continually modified the program according to the user group’s ideas, preferences and feedback. Our approach consisted of four steps: (1) Discover – focus groups with adolescents, parents/guardiansand mental health professionals, (2) Define – feature development, feature prioritisation and site mapping, (3) Design – visual design and content design, and (4) Test – prototyping and user testing. Users were actively involved in each step. Figure 1 presents the co-design process and associated activities. The project team consisted of mental health researchers, mental health clinicians, user experience (UX) designers, visual and educational designers, an illustrator and a copy editor.
Figure 1. The four steps of the co-design process and their associated activities and outcomes.
Participants
Adolescent participants included English-speaking youth aged between 12 and 17 years old with access to the internet. Parents or guardians included English-speaking primary carers of any young person aged 12–17 years, with access to the internet. No criteria related to current mental health status or treatment experience determined inclusion for adolescents or parents/guardians. While the intended users of ClearlyMe are young people with elevated symptoms, the co-design framework used in this study advises against restrictive recruitment to ensure that a broad range of potential user views contribute to the design process. As such, we recruited a community sample of adolescents and parents to ensure individuals with a range of lived experience were involved. This ensured we included adolescents who did not self-identify as having depressive or anxiety symptoms or a current need for mental healthcare (despite being potentially symptomatic) and adolescents and parents with indirect mental health experiences such as caring for siblings or friends experiencing symptoms. Mental health professionals included any registered, currently practising health professional supporting the mental health of young people aged 12–17 years, English speaking, with access to the internet, and no criteria related to their current or past use of digital mental health products or services.
A total of 66 adolescents, 43 parents/guardians and 40 mental health professionals consented to participate. Of these, 36 (55%) adolescents, 15 (35%) parents/guardians and 32 (80%) mental health professionals took part. A total of 47 participants failed to select a session (reasons not sought) and 19 did not attend their scheduled session due to a change of mind or unavailability. Table 1 outlines participant characteristics. There was a higher number of females than males in all participant groups. In the adolescent sample, the majority were 15 years or older with a current or previous diagnosis of a mental illness. Two-thirds had moderate to high levels of psychological distress.
Table 1. Participant characteristics
a K6, Kessler 6 Psychological Distress.
Procedures
Participants were recruited from Australia between May and December 2020 using social media advertisements, the Black Dog Institute ‘Participate in research’ webpage, and by emailing recruitment notices to existing stakeholder networks, including youth and mental health professional organisations. Potential participants were invited to ‘help us to design a digital mental health and wellbeing program’ and instructed to visit the study webpage. The study webpage provided a summary of the study activities (i.e. completion of an online survey, focus groups, online sessions and prototype testing) and the inclusion criteria for the participant groups. Eligibility to participate and demographic information were assessed by an online Qualtrics survey. The Kessler 6 scale (K6), a validated measure of psychological distress in adolescent samples (Ferro, Reference Ferro2019) and predictor of probable mental illness (Kessler et al., Reference Kessler, Barker, Colpe, Epstein, Gfroerer, Hiripi, Howes, Normand, Manderscheid and Walters2003), was also administered to adolescent participants. Participants selected their preferred session time from a list of options, which was confirmed by the research team via email. Email reminders were sent a day prior to the scheduled session and contained session information, teleconference instructions, and a guide to group etiquette. SMS reminders containing the teleconference link were sent one hour prior to the session. Consistent with co-design procedures (Hagen et al., Reference Hagen, Collin, Metcalf, Nicholas, Rahilly and Swainston2012; Thabrew et al., Reference Thabrew, Fleming, Hetrick and Merry2018), it was intended that participants would complete all co-design sessions. However, due to attrition we instigated rolling recruitment to ensure that an adequate number of participants took part in each co-design session. As such, not all participants took part in every co-design activity. Participants were reimbursed 30AUD for each session attended.
Step 1: Discover
A series of focus groups was conducted to establish the foundations for the app development and test initial assumptions about the suitability of this approach for delivering CBT. All focus group sessions were conducted using Zoom videoconferencing software. Sessions were conducted separately for each participant group. Each session contained up to six participants, was 60–90 min in duration and facilitated by two team members, including one clinical psychologist (S.H.L.). When two or less participants attended, the session was attended by only one team member. A semi-structured focus group guide was developed collaboratively with the project team to ensure key research and design objectives were addressed (see Supplementary material, item 1). The interview schedule was dynamic and iterative such that feedback from initial focus groups informed the approach used in subsequent groups. The guide included prompts to elicit discussion about adolescents’ general technology use, problem areas and mental health experiences, their health-related and mental health help-seeking behaviours, views and opinions regarding current mental health support options, and views on currently available digital mental health tools. Adolescents were specifically asked about their preferences regarding pathways to accessing mental health support, and digital mental health program preferences. Mental health professionals were specifically asked about digital tools currently used in the delivery of therapy, the limitations of these, and dissemination barriers. Audio was recorded and automatically transcribed by Zoom. A total of 23 concurrent focus groups were conducted, six with adolescents, nine with parents/guardian and eight with mental health professionals.
A thematic analysis approach was used to analyse the data (Braun and Clarke, Reference Braun and Clarke2006). This analysis was predominantly deductive as the interpretation was focused on understanding users’ needs and preferences and was informed by the research team’s previous research and related theoretical frameworks (Garrido et al., Reference Garrido, Millington, Cheers, Boydell, Schubert, Meade and Nguyen2019b; Jeminiwa et al., Reference Jeminiwa, Hohmann and Fox2019; O’Dea et al., Reference O’Dea, Achilles, Werner-Seidler, Batterham, Calear, Perry, Shand and Christensen2018; Werner-Seidler et al., Reference Werner-Seidler, O’Dea, Shand, Johnston, Frayne, Fogarty and Christensen2017). However, an inductive analysis was also employed to provide a richer analysis of the data (Braun and Clarke, Reference Braun and Clarke2006). Researchers (M.R.A. and S.S.) listened to the dataset of audio to perform an accuracy check of the transcripts. Incorrect words were marked and replaced on the transcript as the researchers listened to the audio. Transcript errors included misinterpreted words and grammar. These errors were influenced by poor quality audio, technical issues, misinterpretation of uncommon words, and participants speaking over each other. Changes were made to less than 5% of the transcripts. The accuracy check also contributed to the first phase of the thematic analysis as researchers began their familiarisation of the dataset and compiled initial ideas for themes and codes. In parallel to the thematic analysis, the UX designers analysed the entire audio dataset using an affinity diagram method (Dam and Siang, Reference Dam and Siang2020). This method organised participants’ responses into natural groupings and design ideas that could be translated into program features (Dam and Siang, Reference Dam and Siang2020). Together, the team then discussed and generated a set of codes that represented the data. This collaborative approach helped support the validity of the analysis (Patton, Reference Patton1999). The adolescent, parent, and mental health professional data were then coded systematically by one researcher (M.R.A.) who first applied the codes to the data extracts and then synthesised these into broader themes. Following this, two researchers (M.R.A. and S.S.) discussed and confirmed the high-level themes. To further support the coding reliability, 50% of the adolescent transcripts were independently double coded (S.S.). This process was repeated for the parent and mental health professional datasets (S.H.). For the parent data set, a deductive approach was used to triangulate the themes with those identified in the adolescent interviews and identify additional relevant themes. As such, the adolescent and parent results are reported together. Discrepancies were discussed (M.R.A., S.H.L., S.H. and S.S.) and final themes were confirmed. The themes were then used to generate design principles, personas and storyboards to guide the design process. Personas serve as examples of key end-users’ behaviours, goals, motivations and attitudes, and help to ensure that end-users remain central in the design. Storyboards display users’ experience with a product and are a tool that is used to highlight which features are most important.
To validate the identified themes and ensure new themes were not identified, additional sessions were conducted with adolescents using a set of UX activities called ‘gamestorming’. Sessions were recorded and transcripts were reviewed applying the code generation in the initial thematic analysis. In the first activity, Product Pinocchio, adolescents were asked to personify the product by imagining it had come to life and had a fully formed character (Gray, Reference Gray2011a). They were then asked to respond with adjectives and phrases to the following five questions in relation to the product’s character: (1) what am I like?; (2) what are my values?; (3) what is my community?; (4) what makes me different?; and (5) what is my fight (i.e. mission and motivation)? In the second activity, Speedboat, adolescents were asked to identify obstacles to their wellbeing and barriers preventing goal attainment (Gray, Reference Gray2011b).
Step 2: Define
The focus of this step was app feature development, prioritisation and site mapping. The gamestorming sessions were followed by a team ideation workshop that aimed to develop app features based on the insights from the focus groups and gamestorming activities. The Crazy Eights method (Levey, Reference Levey2016) was used, whereby the team were asked to sketch eight distinct ideas in eight minutes (see Fig. 2). The goal was to generate a wide variety of features that were subsequently voted on by the project team. The most popular features within the team were developed into 2D concept sketches (or ‘paper prototypes’) that were presented to the co-design participants. The concept sketches were refined based on participant feedback. Following the feedback, two methods were used to prioritise features for further refinement and ultimately to determine the features selected for inclusion in the final product. The first was a comparison of the feature’s value (i.e. contribution to achieving the objectives of the intervention) with the complexity of the feature’s build, whereby features with higher value and lower complexity were prioritised. The MoSCoW method was then used to reach consensus. MoSCoW is an acronym for must-have, should-have, could-have, and will not-have right now, with each denoting a category of prioritisation (ProductPlan, n.d.). Prioritisation was determined based on the value and complexity of the feature, budgetary constraints, and IT development capacity. These two prioritisation methods were also used in circumstances where a feature was endorsed by one user group but not another to either identify the need for feature refinement or to facilitate decision making regarding the inclusion or exclusion of that particular feature. A responsive sitemap (or ‘master flow’) was constructed to present the selected features. The master flow was refined throughout the remaining co-design activities.
Figure 2. Crazy Eights activity during the concept ideation workshop.
Step 3: Design
The last step consisted of further refinement of the selected app features through the design and incorporation of the CBT content and the finalisation of the visual identity. Content was initially developed by the team’s clinical psychologist (S.H.L.) based on Barlow’s Unified protocol for the treatment of emotional disorders (Barlow et al., Reference Barlow, Farchione, Fairholme, Ellard, Boisseau, Allen and May2010). A transdiagnostic treatment approach ensured that the program would be appropriate for a range of different emotional disorders and did not require the adolescent to have an established diagnosis or set of identified symptoms. A team workshop was then used to determine how to present the CBT content in a way that adhered to the established design principles. The content was then reviewed by a panel of experts and an educational designer to validate the therapeutic quality and ensure learning objectives were met. A copy editor also reviewed the content to ensure the tone was consistent with the design principles. To develop the visual identity, a mood-boarding workshop was held with the project team. Three visual concepts were then developed and presented to adolescents for feedback. The visual concepts were: (1) collaged, vibrant photography, (2) illustrative colourful portraits, and (3) calming, scenic illustrations. Based on feedback, one visual concept was selected to refine, with the illustrations presented to adolescents in a small survey to determine preferences and gain further feedback. A second small survey was conducted to select the product name. In the survey, users were presented with mock app images and short descriptions of the app’s purpose. They were then asked to rank a short list of potential app names or suggest their own.
Step 4: Test
Finally, a low-fidelity clickable prototype was built. This simulated the final product and provided a visual representation of the app interface. It allowed for an assessment of the concept design, usability and intuitive navigation. The clickable prototype was then presented to adolescents for user testing. This involved individual interviews using a ‘think out loud’ approach. For the low-fidelity prototype, the goal was to identify non-intuitive UX/UI and assess the concept design. Feedback was used to iterate features, illustrations and content and were used to develop the high-fidelity prototype. The high-fidelity clickable prototype incorporated more detailed visual and interactive features. The same user testing processes were used to examine the high-fidelity prototype. Only adolescent participants took part in this final step as they were deemed to be the end-users of the product, with the content and functionality specifically designed for their individual use.
Results
Step 1: Discover
Adolescents and parents’ thematic analysis outcomes
Overall, five key themes were identified from the focus group session. These included: (1) drivers of digital behaviours and attitudes, (2) online safety and risks, (3) program preferences, (4) support to use the program, and (5) help-seeking barrier and facilitators. The themes, subthemes and concepts, including exemplar quotes, and their implications on product design are outlined in Table 2. The themes identified from the gamestorming activities were consistent with the themes identified from the focus groups. No new themes were identified during gamestorming.
Table 2. Thematic analysis themes, example quotes and implications for design derived from the adolescent and parent focus groups
Mental health professionals’ thematic analysis outcomes
Overall, eight key themes were identified from the focus group sessions. These included: (1) the main issues facing young people, (2) barriers to help-seeking, (3) application design, (4) program preferences, (5) clinical content preferences, (6) support to use the program, (7) dissemination via mental health professionals, and (8) barriers to app use. These key themes and the subthemes that emerged from the interviews and groups, along with their implications for product design, are summarised in Table 3.
Table 3. Thematic analysis themes, example quotes and implications for design derived from the mental health professional focus groups
Key design principles
The key design principles devised by the design team are presented in Fig. 3.
Figure 3. The design principles generated from the focus group themes.
Personas and story boarding
Two adolescent, two parent/carer and one mental health professional personas were created (Supplementary material, item 2). Two adolescent storyboards with potential app features were also created (Supplementary material, item 3). These represented two types of adolescent users that were identified during the focus groups: first, younger adolescents (12–13 years) who tended to have less experience with mental health concerns and services and who were more likely to seek help from parents or a trusted adult; second, older adolescents (16–17 years) who may have been experiencing deteriorating mental health for a longer period and preferred self-reliance and seeking help from friends rather than parents or adults.
Step 2: Define
The Crazy Eights activity and subsequent vote by the project team resulted in the following seven features being developed into 2D paper prototypes for testing: audio content and video tutorials, calming home screen graphics, swipe cards for questions and quizzes, an online mental health question and answer forum, a chatbot, and the ability to share content with a mental health professional. These can be viewed in Supplementary material item 4. Based on user feedback, the following features were validated: dashboard layout for app homepage, swipe cards for quizzes/questionnaires, using text, video and audio content to maintain attention, displaying the Black Dog Institute logo for credibility, personalisation of content to ensure relevance, mood tracking, and including relatable stories from other young people. The chatbot feature was not endorsed by adolescents, whereas the ability to directly contact a mental health professional was not endorsed by mental health professionals. Prioritisation of features resulted in the following master flow of key features: registration, onboarding, dashboard, upper navigation bar (including sections on how to Get Help, Information about the Black Dog Institute), and lower navigation bar (including navigation tabs to dashboard, mood check feature, full content library, progress, and personal profile).
Step 3: Design
Content development
The most frequently preferred product name in the short survey was ClearlyMe, which was selected as the app name. Based on the workshop, the transdiagnostic CBT content was adapted to be displayed using three templates: (1) quizzes, (2) multiple choice and (3) free text activities. Each CBT skill was delivered via a single ‘Activity’ using one of the three templates. Activities were the smallest unit of content, designed to be completed within no more than 10 minutes. Users could provide feedback to the designers (like/dislike) at the end of each activity. Activities were then curated into nine separate ‘Collections’. Collections contain three to five activities with a specific focus (e.g. targeting depression symptoms) and can be accessed via the Explore tab. To promote access to the therapeutic content, three pathways were added to the master flow. MindHacks are a series of quick tips and strategies that lead to single activity. MoodCheck is a mood tracking feature that allows users to rate their current mood and view mood fluctuations on a mood chart. Finally, Stories are a series of short TikTok-style videos created by young people with lived experience of mental health challenges presenting their experiences and mental health management strategies. MoodCheck and Stories lead to a specified collection. The following features were added to support use as a therapy adjunct: (1) a ‘Saved’ section, allowing activities to be marked for later completion (e.g. as between session homework), (2) activity summary pages at the end of each activity that can be shared with a clinician to show progress or as a talking point in the session, and (3) an ‘Activities’ tab, where activities categorised into Emotion, Thought and Behaviour modules and listed as to be easily viewed by the therapist and selected for client completion.
Visual identity
In relation to the app imagery, adolescents showed an equal preference for concepts 1 (collaged vibrant photography) and 2 (illustrative colour portraits) and unanimously preferred a vibrant colour palette. Due to adolescents’ desire for the imagery to be relatable and be respectful of diverse representation, a suite of five characters were developed. Characters varied in cultural and gender identity and diverse abilities were represented. Three colour ways with variations in skin tone and fashion were developed: (1) vibrant, high-contrasting fashion with unnatural skin tone, (2) monochromatic fashion with unnatural skin tone, and (3) natural skin tones, vibrant clothing. The survey showed that adolescents preferred natural skin tones and endorsed the inclusivity of the proposed character illustrations, but reflected that a ‘normal teenager’ was missing. In response, the images were toned down.
Step 4: Test
Upon review of the low-fidelity prototype, adolescents endorsed the following features: onboarding, swipe functionality, MindHacks, videos and MoodCheck. Adolescents reflected that the usability of the explore feature (designed to navigate the content) was poor. Adolescents also requested a temporal visualisation of their response to the mood check. Upon review of the high-fidelity prototype, adolescents endorsed the app layout, colour palette, style and usability of the app. Character imagery was found to be relatable and appealing. The tone of the app was described by adolescents as empathic and relatable, and the variety of content was endorsed. More guidance was requested to complete activities, so instructional language and illustrations supporting the concepts were added (for an example, see Fig. 4). Adolescents also wanted to be able to revisit summary pages, so this functionality was added. They also wanted transparency on the purpose of feedback at the end of each activity, so copy was added to clarify how this.
Quotes from user testing:
‘Oooh that’s cool … everything … the stories spark interest immediately ’ (F, 16)
‘I think what’s written there is really good, it’s short and sweet. I like it … it’s to the point and lets me know what I’m about to do’ (M, 16)
‘There’s big variety and lots of diversity, which gives me a positive feeling’ (NB, 16)
Figure 4. An example of an illustration matching the content to support learning.
Product features
The final selection of product features and a description of each are summarised in Table 4.
Table 4. Summary of the features included in the final ClearlyMe design
Discussion
Principal findings
This paper describes the comprehensive, rigorous co-design process employed to design ClearlyMe, a new CBT smartphone app that targets elevated symptoms of depression and anxiety among adolescents. Our co-design process adhered to recommendations from the co-design literature (Bevan Jones et al., Reference Bevan Jones, Thapar, Rice, Beeching, Cichosz, Mars, Smith, Merry, Stallard, Jones, Thapar and Simpson2018; Hagen et al., Reference Hagen, Collin, Metcalf, Nicholas, Rahilly and Swainston2012; Thabrew et al., Reference Thabrew, Fleming, Hetrick and Merry2018) and those arising from the field of digital mental health research (Bakker et al., Reference Bakker, Kazantzis, Rickwood and Rickard2018; Garrido et al., Reference Garrido, Cheers, Boydell, Nguyen, Schubert, Dunne and Meade2019a). The initial co-design sessions (Step 1) indicated that young people, parents and mental health professionals were supportive of a CBT smartphone app for adolescents with symptoms of depression and anxiety. However, the likelihood of uptake, use and adult endorsement was dependent on several factors. Specifically, this related to the credibility and transparency of the app’s purpose and privacy policy, ease of use, aesthetics, accessibility and capacity for social interaction. Youth participants wanted the app to contain interactive, readily accessible elements, be relatable and relevant to their current concerns and delivered in a youth-friendly and positive tone. Parents wanted an app that was from a credible source and acknowledged that parental involvement in the intervention may be a deterrent. Mental health professionals specifically required the app to be a ‘safe’ (i.e. not cause distress or exacerbate symptoms) adjunct to in-person treatment that promoted CBT skill acquisition.
The findings from our co-design process are largely consistent with what has been found by other research groups regarding youth design preferences (Jeminiwa et al., Reference Jeminiwa, Hohmann and Fox2019; Kenny et al., Reference Kenny, Dooley and Fitzgerald2016; Stoyanov et al., Reference Stoyanov, Zelenko, Staneva, Kavanagh, Smith, Sade, Cheers and Hides2021; Werner-Seidler et al., Reference Werner-Seidler, O’Dea, Shand, Johnston, Frayne, Fogarty and Christensen2017). However, in contrast to the design of other adolescent CBT interventions that utilise smartphone technology, such as Sleep Ninja (Werner-Seidler et al., Reference Werner-Seidler, O’Dea, Shand, Johnston, Frayne, Fogarty and Christensen2017) and Pesky gNATs (Chapman et al., Reference Chapman, Loades, O’Reilly, Coyle, Patterson and Salkovskis2016), adolescents in the current study did not endorse gamification, a chatbot feature, or calming images and colours. Differences in sample characteristics may account for these contrasting findings. Youth preferences have been found to be influenced by age, mental health condition, symptom severity and gender (Fleming et al., Reference Fleming, Merry, Stasiak, Hopkins, Patolo, Ruru, Latu, Shepherd, Christie and Goodyear-Smith2019). For example, younger, asymptomatic adolescents have been found to endorse ‘fun’ gamified interventions, whereas symptomatic, older adolescents have reported that such an approach trivialises their concerns and instead preferred content that was ‘straight to the point’ (Fleming et al., Reference Fleming, Merry, Stasiak, Hopkins, Patolo, Ruru, Latu, Shepherd, Christie and Goodyear-Smith2019). Adolescent preferences also appear to change over time given the rapid evolution of technology (Munsch, Reference Munsch2021; Turner, Reference Turner2015) and the diverse mental health needs of young people (Li et al., Reference Li, Beames, Newby, Maston, Christensen and Werner-Seidler2021). As such, their design preferences are also likely to change. Most importantly, young people’s initial preferences were not always endorsed during the subsequent steps in the co-design process. For example, the project team ideated a chatbot feature based on adolescents’ initial preference for interactive features and relatability. However, adolescents almost unanimously invalidated this feature when presented with a 2D sketch (i.e. paper prototype). This highlights the importance of the iterative approach of our co-design process as it allowed for continual collaboration and refinement of ideas with end-users. Therefore, differences between our findings and past studies may be due to the degree of co-design employed by intervention developers. Our findings endorse a key recommendation from a recent review of digital mental health interventions – that end-users should be involved at all stages of design, from problem identification to evaluation to testing (Garrido et al., Reference Garrido, Cheers, Boydell, Nguyen, Schubert, Dunne and Meade2019a).
Differences in preferences between adolescents, parents and mental health professionals
While preferences between the three user groups generally aligned, there were points of difference. One difference related to the level of support to use the app. Adolescents identified a preference for independent symptom management and self-reliance. While they were not opposed to clinician support, adolescent participants requested that this was optional. In contrast, mental health professionals were almost unanimous in their belief that a CBT smartphone app should be supported by a professional providing in-person treatment. These findings validate an existing body of literature showing that mental health professionals view technology as a tool to support in-person treatment rather than a replacement (Cliffe et al., Reference Cliffe, Croker, Denne and Stallard2020; Donovan et al., Reference Donovan, Poole, Boyes, Redgate and March2015; Gindidis et al., Reference Gindidis, Stewart and Roodenburg2020). A second point of difference was the nature of the therapeutic content. Adolescent participants preferred practical coping strategies that could be used in the moment when needed. However, mental health professionals emphasised the need for CBT skill development and mastery through practice. Some CBT skills, such as cognitive restructuring and exposure, are less appealing to individuals because these skills are more effortful and require significantly more practice to master when compared with strategies that provide immediate symptom relief (Wasil et al., Reference Wasil, Gillespie, Patel, Petre, Venturo-Conerly, Shingleton, Weisz and DeRubeis2020). Differences in preferences for therapeutic content among our study participants suggest that there may be a disconnect between what mental health practitioners aim to deliver and what adolescents hope to gain from an intervention. While there are indisputable benefits in prioritising the delivery of evidence-based practice, especially CBT skill development, over ‘quick fixes’ (Compton et al., Reference Compton, March, Brent, Albano, Weersing and Curry2004), incorporating strategies that require less skill and provide rapid distress relief may improve adolescents’ engagement with the ClearlyMe app and ultimately lead to greater treatment effects.
A final point of difference in user preferences was in relation to parental involvement. Parents and mental health professionals believed that involving parents in younger adolescents’ use of the ClearlyMe app would be superior to self-directed use for CBT skill development. However, it was also acknowledged that parental involvement may inhibit use, particularly in older adolescents. Our youth participants concurred with this notion. While a recent meta-analysis was unable to determine whether parental involvement enhanced treatment outcomes for adolescent anxiety (Cardy et al., Reference Cardy, Waite, Cocks and Creswell2020), parental supervision and support has been shown to significantly improve treatment adherence in youth (Wahlund et al., Reference Wahlund, Wallhem, Serlachius and Engberg2021). In past interventions (Cardy et al., Reference Cardy, Waite, Cocks and Creswell2020), parental involvement has ranged from individual treatment sessions for parents, parental participation in adolescent sessions, or hard copy workbooks for parental completion. To our knowledge, no studies have examined parental involvement in smartphone-delivered CBT interventions for adolescents. Based on the lack of evidence in the field and our inconclusive findings, it is not yet possible for this study to describe the optimal nature of parental involvement in ClearlyMe. We argue that an additional co-design process is required to explore this concept to ensure harmony between parental involvement, adolescent autonomy, and treatment effectiveness.
Comparison with other digital CBT programs
An obvious difference between ClearlyMe and computerised CBT is the device used for delivery. Smartphone delivery supports unrestricted access, wherever and whenever suitable, plus the inclusion of brief, interactive features designed to be completed in vivo. These aspects may improve treatment adherence, but this requires confirmation through appropriate empirical investigations. Another difference between the proposed design of ClearlyMe and many of the available computerised CBT programs (e.g. MoodGYM, Brave Online) is the modular, rather than sequential, nature of the therapeutic content. Adolescents expressed a strong preference for self-navigation that allowed them to select content most relevant to the situation without requiring the support of a professional. This is consistent with the literature showing that help-seeking hesitancy is related to concerns regarding the therapeutic relationship with professionals, particularly regarding confidentiality and trust (Radez et al., Reference Radez, Reardon, Creswell, Lawrence, Evdoka-Burton and Waite2021). However, a modular, self-guided CBT smartphone app that does not include professional input may be used inappropriately. For example, the modules a CBT practitioner would consider most relevant are not selected by the adolescent user. Such products could consider ways to provide subtle guidance through the content that does not impinge on the user’s freedom to consume the content in any order they choose.
One defining feature that emerged through our co-design process was the need to accommodate the different ways that adolescents interact with smartphones. This process indicated that young people prefer an app that presents information in a variety of formats, keeps text to a minimum and relies heavily on illustrations to demonstrate concepts. Compared with existing smartphone apps designed for adults, ClearlyMe contains less text, uses illustrations to support learning and presents content in a variety of formats (video, audio, illustration and text) (Bakker et al., Reference Bakker, Kazantzis, Rickwood and Rickard2018; Paul and Fleming, Reference Paul and Fleming2019). The co-design process also highlighted key differences in young people’s perspectives of positive visual imagery and an interactive chatbot feature. Many of the current mental health apps use subtle pastel colours alongside scenery photographs (Paul and Fleming, Reference Paul and Fleming2019; Werner-Seidler et al., Reference Werner-Seidler, O’Dea, Shand, Johnston, Frayne, Fogarty and Christensen2017) and two popular CBT apps, Woebot and Wysa (Fitzpatrick et al., 2017; Inkster et al., Reference Inkster, Sarda and Subramanian2018), use a chatbot feature to deliver content and interact with the user. In contrast, adolescents in the current study requested a vibrant and cheerful colour scheme and invalidated preferences for a chatbot feature. These preferences are reflected in the design of ClearlyMe. Finally, unlike other CBT apps (Werner-Seidler et al., Reference Werner-Seidler, O’Dea, Shand, Johnston, Frayne, Fogarty and Christensen2017), ClearlyMe incorporates features to specifically facilitate its use as an adjunct to in-person CBT. While ClearlyMe may have the potential to augment treatment outcomes, or even improve engagement and client retention, these possibilities require empirical investigation.
Next steps for ClearlyMe
The critical next step is to evaluate ClearlyMe. This includes determining the effectiveness of ClearlyMe in reducing symptoms of depression and anxiety in adolescents, but also whether our co-design process is associated with improved uptake and use of digital CBT. The measurement of engagement, or adherence, to ClearlyMe is an important consideration. To date, engagement in digital mental health interventions has been poorly defined and inconsistently measured (Ng et al., Reference Ng, Firth, Minen and Torous2019; Perski et al., Reference Perski, Blandford, West and Michie2017). Engagement with ClearlyMe requires operationalisation to ensure a meaningful measure of the degree to which individuals’ use aligns with how it was intended to be used can be devised. A fully powered randomised controlled trial (RCT) has been planned to determine the acceptability, engagement and effectiveness of ClearlyMe when used by adolescents with depressive symptoms autonomously or with non-clinical support. Future studies will also be required to examine the use and effectiveness of ClearlyMe when initiated entirely by mental health professionals (i.e. therapist-guided use), in contrast to self-directed uptake. These findings will be key to determining the optimal treatment recommendations for use of ClearlyMe among young people with elevated symptoms of depression and anxiety. These findings will be key to determining the optimal treatment recommendations for use of ClearlyMe among young people with elevated symptoms of depression and anxiety.
Several features that were popular with all user groups were not included in the final version of the app. These included an interactive social feature, an online mental health question and answer forum, intuitive tailoring of content based on previous preferences or inputs and novel content (e.g. similar to new content in social media feeds). These features were deprioritised based on feasibility. For example, inadequate resources were available to moderate a peer-interaction forum or the ongoing development of new content. Given the value of these features to users, they have been placed on the ‘roadmap’ and will be considered in future iterations of ClearlyMe, pending outcomes of the RCT. Finally, careful consideration is required regarding how the app will be disseminated and promoted. Adolescents reported that they prefer self-reliance, and while younger adolescents seek help from parents, many older adolescents will use friends or the internet to access help or information. The development of a marketing and communications plan, and dissemination strategy is currently underway taking these insights into consideration.
Limitations
The current study had several limitations that warrant mention. The use of social media for recruitment may have attracted participants with higher levels of openness towards smartphone apps when compared with samples recruited via other methods. Similarly, there was an over-representation of girls and older adolescents in our sample. Although this is consistent with other co-design studies in mental health (Bevan Jones et al., Reference Bevan Jones, Thapar, Rice, Beeching, Cichosz, Mars, Smith, Merry, Stallard, Jones, Thapar and Simpson2018), it is also a pattern in studies using online recruitment (Whitaker et al., Reference Whitaker, Stevelink and Fear2017) and may be due to the higher prevalence of depression and anxiety in females and their positive attitudes to seeking help (Riecher-Rössler, Reference Riecher-Rössler2017). As such, the acceptability and effectiveness of ClearlyMe remains to be verified in a broad and representative sample of adolescents in a clinical trial. Future trials may need to adopt additional recruitment strategies to target male participants to ensure that the ClearlyMe app has not been designed with a gender bias. This is particularly important given the lower rates of help-seeking among adolescent males (Möller-Leimkühler, Reference Möller-Leimkühler2002). There was also an over-representation of female parents in this study. This is consistent with past research (Bevan Jones et al., Reference Bevan Jones, Thapar, Rice, Beeching, Cichosz, Mars, Smith, Merry, Stallard, Jones, Thapar and Simpson2018) and likely reflective of the primary caregiver role of mothers. Future research would be strengthened by actively targeting fathers to measure their perceptions of smartphone-delivered mental healthcare and their likely endorsement for their children. The co-design process was labour and time intensive and the capacity to evaluate the value of the process is only achievable once the app build has been completed. However, the high cost of building digital products means there is less room for error. Engaging in a co-design process prior to development is both cost and time effective in comparison with extensive iteration and refinement of the product post-build. Additionally, future work would benefit from testing the clickable prototype or final product among parents or mental health professionals to measure their acceptability on the app content, features and functionality.
Conclusions
Symptoms of depression and anxiety during adolescence are highly prevalent and have a significant impact on functioning across social, emotional and academic domains. Our co-design process validated the need for a CBT smartphone app designed with and for adolescents that utilises the ubiquitous, ‘in-the-pocket’ nature of smartphones in the adolescent population. While some adolescent preferences and feedback were consistent with those found in previous studies, we also found some critical differences. Including parents and mental health professionals in the co-design process also resulted in novel insights that guided the design process, including ClearlyMe as a therapy adjunct to facilitate mastery of CBT skills. Despite being a time-consuming and resource-heavy process, co-design is more likely to produce an application that is relevant, engaging and suitable to the needs of all the product users. However, this requires be assessment by an appropriately powered RCT.
Key practice points
-
(1) CBT has been adapted to computerised delivery and been showed to be effective; however, engagement and uptake is suboptimal.
-
(2) The next stage in CBT delivery evolution is via mobile technology (smartphones), which allows access to in vivo strategies anywhere and anytime.
-
(3) During the co-design of ClearlyMe – a new CBT smartphone app – one major difference between stakeholders was that adolescents preferred a program that could be used autonomously with optional support from a professional, whereas mental health professionals desired a program that could be used alongside in-person treatment.
-
(4) We outline the co-design process of ClearlyMe to demonstrate how these divergent preferences were addressed to produce a product that has the flexibility to be used autonomously and as a therapy adjunct to augment CBT skill development in traditional in-person CBT.
-
(5) Future research is required to evaluate ClearlyMe in terms of effectiveness as a standalone CBT program and as a therapy adjunct.
Supplementary material
To view supplementary material for this article, please visit: https://doi.org/10.1017/S1754470X22000095
Acknowledgements
The authors would like to thank Jill Newby for her advice and support in the development of this manuscript.
Author contributions
Sophie Li: Conceptualization (lead), Data curation (equal), Formal analysis (supporting), Investigation (lead), Methodology (lead), Project administration (supporting), Writing – original draft (lead); Melinda Achilles: Conceptualization (supporting), Data curation (equal), Formal analysis (lead), Methodology (supporting), Project administration (equal), Writing – review & editing (supporting); Samantha Spanos: Data curation (supporting), Formal analysis (supporting), Writing – review & editing (supporting); Stephanie Hakak: Data curation (supporting), Formal analysis (supporting), Writing – review & editing (supporting); Aliza Werner-Seidler: Conceptualization (supporting), Investigation (supporting), Project administration (supporting), Supervision (supporting), Writing – review & editing (supporting); Bridianne O’Dea: Conceptualization (equal), Data curation (supporting), Formal analysis (supporting), Funding acquisition (lead), Investigation (supporting), Methodology (supporting), Supervision (lead), Writing – review & editing (lead).
Financial support
This work has been supported by the Goodman Foundation; National Health and Medical Research Council (NHMRC) Investigator Grant (to A.W.-S., grant number GNT1197074); and a National Health and Medical Research Council (NHMRC) Medical Research Future Fund (MRFF) Fellowship (to B.O’D., grant number 1197249).
Conflicts of interest
The authors declare no conflicts of interest.
Ethics statements
All procedures were carried out in accordance with the Declaration of Helsinki and approved by the UNSW Human Research Ethics Committee (approval number: HC200152). All authors abided by the Ethical Principles of Psychologists and Code of Conduct as set out by the BABCP and BPS. Written consent was gained for all participants and corresponding parental/guardian consent was obtained for those under 16 years.
Data availability statement
Transcripts are not publicly available due to the sensitive nature of the data and ethical guidelines.
Open access
Comments
No Comments have been published for this article.