Formative evaluation of practice changes for managing depression within a Shared Care model in primary care

Abstract

Aim

To investigate the implementation and initial impact of the Physician Integrated Network (PIN) mental health indicators, which are specific to screening and managing follow-up for depression, in three primary care practices with Shared Mental Health Care in Manitoba.

Background

Manitoba Health undertook a primary care renewal initiative in 2006 called the PIN, which included the development of mental health indicators specific to screening and managing follow-up for depression. These indicators were implemented in three PIN group practice sites in Manitoba, which are also part of Shared Mental Health Care.

Methods

The design was a non-experimental longitudinal design. A formative evaluation investigated the implementation and initial impact of the mental health indicators using mixed methods (document review, survey, and interview). Quantitative data was explored using descriptive and comparative statistics and a content and theme analysis of the qualitative interviews was conducted. Survey responses were received from 32 out of 36 physicians from the three sites. Interviews were conducted with 15 providers.

Findings

This evaluation illustrated providers’ perceived attitudes, knowledge, skills, and behaviours related to recognizing and treating depression and expanded our understanding of primary care processes related to managing depression related to the implementation of a new initiative. Depression is viewed as an important problem in primary care practice that is time consuming to diagnose, manage and treat and requires further investigation. Implementation of the PIN mental health indicators was variable across sites and providers. There was an increase in use of the indicators across time and a general sentiment that benefits of screening outweigh the costs; however, the benefit of screening for depression remains unclear. Consistent with current guidelines, a question the findings of this evaluation suggests is whether there are more effective ways of having an impact on depression within primary care than screening.

Depression is a common problem and the majority of cases are identified and managed in primary care. Currently, however, a significant portion of cases go undetected and even more untreated (Scott, 2006; Vermani et al., 2011; Addiction and Mental Health Collaborative Project Steering Committee, 2014). Moreover, depression commonly co-exists with other chronic conditions (eg, diabetes, heart disease) and inadequate treatment of depression is associated with poorer outcomes for such comorbid conditions (Katon et al., 2005).

In 2006, Manitoba Health undertook a new primary care renewal initiative, the Physician Integrated Network (PIN). An overarching objective of PIN is to demonstrate high-quality primary care, focussing on chronic disease management, with the support of a quality-based incentive funding approach within a blended funding model. The funding model is based on pay-for-performance that compensates a group practice based on their achievement on quality indicators. This compensation represented <10% of their total funding, the majority of which came from fee-for-service billings. In supporting the demonstration of quality primary care practice, several primary care indicators (Canadian Institute of Health Information, 2006) were adopted as part of phase 1 of PIN. Lacking in the selection of these indicators were appropriate mental health indicators. In response to this issue, one important component of Manitoba’s PIN was the development of potential mental health indicators specific to screening and managing follow-up for depression. An expert group including a panel of physicians representing PIN clinics and Manitoba Health staff identified possible indicators by drawing on existing and emerging evidence and clinical practice guidelines. Indicators adopted were required to be supported by the evidence, aligned with best clinical practice, and be measurable.

Overall, evidence on the benefits of screening for depression is mixed. At the time of implementing this initiative, both the Canadian Medical Association and US Preventive Services Task Force recommended screening for depression but only in situations where screening was targeted, such as on high risk groups, and when practice also included accurate diagnosis and access to effective treatments (Pignone et al., 2002; MacMillan et al., 2005). Recently, the Canadian Task Force on Preventive Health Care (2013) updated their position on screening for depression in adults. It continues to recommend against screening for depression within the general population. It further recommends against screening even if targeted for high risk groups. These recommendations were listed as weak given the limited evidence demonstrating benefits and/or harms of screening for depression. Some have argued more generally that evidence does not support the practice of screening for depression (Thombs and Ziegelstein, 2014; Thombs et al., 2014). Where evidence for screening does exist, it is in support of integrated care systems for depression, such as collaborative care. The guidelines also highlighted the importance of clinicians identifying depression in patients presenting with relevant symptoms (eg, insomnia, low mood).

With the 2005 guidelines in mind, and the need to balance thorough assessment with the realities of busy primary care practice, Manitoba Health targeted the implementation of mental health indicators in select primary care sites that have access to Shared Mental Health Care (Shared Care), as part of Phase 2 of PIN (Manitoba Health). The Manitoba Shared Care Program was introduced into primary care in 2003 and involves a mix of regionally funded primary care centres and fee-for-service sites. The programme developed as a result of mounting evidence demonstrating collaborative care as being associated with better outcomes for patients (eg, symptom improvement, functional improvement, fewer disability days, increased workplace tenure, increased quality-adjusted life years) and improved cost-effectiveness (due to improved efficiency of medication and medical service use and increase rates of return to work; Kates et al., 2011). A key focus of Canadian primary health care reform more generally has been the integration of other providers into primary care (Health Canada, 2004), towards more interprofessional care including improved access and delivery of patient-centred comprehensive care (Jeffries et al., 2013).

The Manitoba Shared Care Program initially involved the integration of Bachelor or Master’s level prepared counsellors into primary care and the availability of consulting psychiatrists. In 2009, a psychologist was hired. Models of collaborative care vary considerably from those in which team members work at separate sites or periodically visit a common work site (consultation-liaison model) to a model where team members share a common work site and have common systems such as scheduling and charting (integrated model; Gagne, 2005; see Goodrich et al., 2013 for further discussion on collaborative mental health care in primary care). There is no one best model; however, there are common elements that have been identified as important across the different models of care such as the use of electronic health records to facilitate sharing of information, the use of a stepped care approach, inclusion of diverse health professionals, and designated and dedicated funding (Peachey et al., 2013). For the Manitoba model, the counsellors are co-located within the primary care clinic while the psychologist and psychiatrists rotate across the sites playing more of a consultative role. The WHO’s global strategy on people-centred and integrated health services 2016–2026 calls for significant changes to health services with the goal of continuous, integrated, comprehensive, and people-centred care (World Health Organization, 2015). Both the Manitoba Shared Care Program and the PIN initiative are consistent with this strategy and evidence informed.

Two mental health indicators were implemented as part of the PIN initiative. The first indicator was a screening indicator and included the Patient Health Questionnaire – 2 (PHQ-2), which includes the following two questions: (1) During the past month, have you had little interest or pleasure in doing things; and (2) During the past month, have you been feeling down, depressed, or hopeless (Kroenke et al., 2003). The PHQ-2 was selected as it has been well researched and can be quickly administered. If patients respond yes to one of the two questions, the second indicator was then to be implemented, which involved a follow-up assessment. The separation of these indicators into two visits, if needed, was in response to clinician feedback for a two-step process; the first step to identify those who may be depressed, followed by a more in-depth assessment, either during the same visit, or during a follow-up appointment. The follow-up assessment was to be conducted within four weeks of the initial screening to confirm diagnosis and treatment option. Patients screened as possibly depressed after the first indicator were then to be identified according to one of the following five options: (1) not depressed; (2) depressed and referred (eg, Shared Care counsellor, community treatment option); (3) depressed, treatment prescribed and managed by primary care provider; (4) depressed, patient refused treatment; or (5) depressed, treatment prescribed and managed by primary care provider and referred. These indicators were targeted to patients viewed as highest-risk for depression by a group of experts and primary care providers. These patients included those between the ages of 18 and 69 years of age without an active depression diagnosis and who have a diagnosis of diabetes, congestive heart failure, and/or coronary artery disease, and/or women who have given birth within the past 12 months. This target population was easily identified through the other primary care quality indicators and the electronic medical record system.

Formative evaluation of depression screening within Shared Care

A formative evaluation focussed on the implementation of these new mental health indicators on primary care practice within collaborative care settings. Specifically, the evaluation investigated the implementation and initial impact of the PIN mental health indicators in three PIN practices where Shared Care was available in Manitoba (one rural, two urban); one of the urban sites (site 3) was a quasi-control site in that it did not have incentive funding and/or Electronic Medical Record (EMR) functionality for the PHQ-2 prompts. Shared Care had only been recently implemented in two of the sites before Time 1 (site 2 and site 3). The evaluation sought to document the barriers and facilitators to the programme implementation, satisfaction, and perceived impacts (on staff, patients, and system) of implementing PIN mental health indicators along the following key questions:

1. (1) What are providers’ attitudes (including satisfaction), knowledge, skills, and behaviours related to recognizing and treating depression?

2. (2) As implementing the PIN mental health indicators, what is each clinic’s process related to the management of depression with the target high risk groups?

3. (3) What are the barriers and facilitators to the implementation of the PIN mental health indicators?

4. (4) What is the initial impact of the PIN mental health indicators on the provider, patients, and collaborative practice?

Method

This evaluation investigated the trial implementation of the mental health indicators in the three trial sites (see Appendix A for evaluation timelines). It involved a non-experimental design using a mixed methods approach, including data extraction, survey, and qualitative interview methods.

Document review

Data from the Shared Care Program was extracted such as the number of referrals. EMR data extracted related to depression screening items at the de-identified provider level and included: (1) Date of last PHQ-2 Administration; (2) Response to the PHQ-2 questions; (3) The date a depression screening follow-up assessment had occurred; (4) The depression screening follow-up outcome selected; and (5) Date of active depression diagnosis. For Time 1 of the evaluation, the data extraction period captured included 1 December 2010 through to 30 June 2011. For Time 2 of the evaluation, the data extraction period captured included 1 July 2011 through to 31 March 2012.

Survey

A survey on providers’ attitudes, skills, behaviours, and satisfaction related to recognizing and treating depression was completed, either online through Survey Gizmo or in paper-and-pencil format (provider choice), by all interested providers. Two versions of the survey were developed, one for physicians and a slightly modified one for counsellors. Many of the questions were adapted from Upshur and Weinreb’s (2008) published but non-validated survey of primary care provider attitudes and behaviours regarding treatment of adult depression. Most of the questions were rated on a scale of 1–4, with 1 being strongly disagree, very uncertain, or very unlikely, to 4 being strongly agree, very certain, very likely. An example attitude question included: ‘treating depression is time consuming’. Respondents perceived skills and behaviours were assessed through such items as ‘effectively treat by counselling’ and ‘conduct a differential diagnosis’, respectively. Specific questions related to the implementation of the PIN mental health indicators and Shared Care were also asked using a four-point Likert scale such as ‘The benefits of screening for depression outweigh the costs’ and ‘It would be feasible for me to continue to screen and treat depressed patients without access to Shared Care’. The survey took ~10 min to complete and providers did not receive any compensation for this portion of their participation. Of 36 physicians, 32 across the three sites responded; 28 physicians responded to the survey at both Time 1 and Time 2, for a response rate of 78% (6/7 physicians from site 1, 16/16 physicians from site 2, 6/13 physicians from site 3). Of the physicians at site 2, six did not have access to Shared Care. All three Shared Care counsellors also responded.

Individual interviews

Key informant semi-structured qualitative interviews were conducted by the lead investigator and two psychology residents with five providers per site (total of 15). The providers interviewed at each site included four physicians (clinic lead, PIN lead, and two others) and one counsellor. Example questions included ‘What is your current practice of screening and providing follow-up care for depression?’ and ‘How, if at all, has implementing the PIN depression screening and follow-up indicators changed your practice?’ The interview followed an interview guide and lasted ~20 to 30 min. It was digitally recorded and transcribed verbatim. Physicians received an honorarium of $100 for their participation and counsellors were interviewed during work hours.

Analysis

The first analysis involved descriptive statistics of the three sites for EMR, Shared Care, and survey data. Comparative statistics using multiple t-tests or analyses of variance (ANOVAs) were then used to assess for differences between the three sites and across providers (eg, number of referrals to Share Care, number of cases of depression identified, attitudes of providers). A content and theme analysis of the qualitative interviews was conducted, following Thomas’ (2006) general inductive approach. A coding scheme was developed through consensus on an initial review of four of the interviews by multiple reviewers. Transcripts were subsequently coded by one reviewer who remained open to new themes.

Findings

Findings are presented in response to the four evaluation questions.

Providers’ attitudes, knowledge, skills, and behaviours related to recognizing and treating depression

Survey data was used to respond to this question. In terms of providers’ attitudes related to recognizing and treating depression, all agreed that depression is a frequent problem, most agreed that treating depression is time consuming (23/30) and many disagreed that patients are better off treated by mental health specialists (21/29). All providers viewed medication treatment and counselling as very effective, while many also indicated self-help approaches as also very effective (22/29). Most expressed the benefits for screening for depression as outweighing the costs (25/28). All were certain that they could effectively treat with medications, while only half of the physicians were certain that they could effectively treat by counselling (as compared with all counsellors) or by encouraging self-management; counsellors were mixed on their attitudes of self-management. Providers stated that they would be likely to start patients on antidepressant medications (all but one) and/or give supportive counselling. Satisfaction related to managing depression was mixed overall (see Table 4); however, most viewed treating depression as economically viable (25/28). Satisfaction related to the mental health indicators was also mixed and yet almost all stated that they would continue to screen for depression beyond the trial period (23/25; it is possible that providers intend to continue to screen using other tools). All providers reported satisfaction with Shared Care and about half viewed it as an essential resource for continuing to manage depression in primary care (11/24; see Tables 1–4 for more details).

Table 1 Providers’ attitudes related to recognizing and treating depression

Total physician sample=32 (total counsellor sample=3).

Table 2 Providers’ perceived skills related to recognizing and treating depression

Total physician sample=32 (total counsellor sample=3).

Table 3 Providers’ perceived behaviour related to recognizing and treating depression

Total physician sample=32 (total counsellor sample=3).

Table 4 Providers’ satisfaction related to recognizing and treating depression, PIN, and Shared Care

PIN=Physician Integrated Network; PHQ-2=Patient Health Questionnaire.

Total physician sample=32 (total counsellor sample=3).

Comparative statistics using ANOVAs were then conducted to assess for differences across time periods and across sites. To avoid alpha inflation due to multiple ANOVAs, a P-value of 0.025 was used to determine significant difference. No significant changes were identified across time for counsellors. There was a trend for physicians (n=28) towards seeing treating depression as less time consuming across time (ie, at Time 2 as compared with Time 1). Providers were also less likely to report training and educational materials provided by Shared Care as useful in managing depressed patients at Time 2 (M=2.38, SD=1.63) as compared with Time 1 (M=3.48, SD=0.75); t(20)=2.77, P=0.012. Across sites, site 3 (ie, the site without incentive funding and/or EMR functionality for the PHQ-2 prompts) came out with some significant differences. Although all sites tended to disagree with the statement that ‘Patients are better off treated by mental health specialists’; site 3 was significantly more likely to disagree with this statement, as compared with site 2 [F(2,23)=5.73, P=0.01]. Site 3 was also significantly more likely to disagree with the statement that ‘I will continue to screen beyond trial’, as compared with sites 1 and 2 [F(2,24)=5.94, P=0.008]; however, as only one physician was screening with the PHQ-2 at the time of this survey, this finding could also be interpreted as not planning on screening with the PHQ-2. Similarly, site 3 was significantly more likely to disagree with the statement that ‘Benefits of screening for depression outweigh the costs’, as compared with sites 1 and 2 [F(2,23)=7.60, P=0.003].

Clinic process related to depression management

Depression screening

Survey, interview, and document review findings were consistent in finding a range of implementation of the PIN mental health indicators across clinics and providers. On the survey, 20 out of 27 physicians reported to be screening with the PHQ-2 at Time 2. Using an α of 0.025 due to heterogeneous variances and significant differences between groups, an ANOVA comparing use at Time 2 between sites revealed a significant difference. Site 3 was significantly less likely to screen with the PHQ-2 (1/5 physicians) as compared with sites 1 (4/6 physicians) and 2 (15/15 physicians): F(2,24)=16.72, P=0.000. Those physicians using the PHQ-2 tended to report using it with <50% of target group patients (see Table 5). They also tended to report not using the PHQ-2 outside of the target groups (see Table 6); however, there was a significant increase in reported frequency of PHQ-2 use outside of the target groups between Time 1 (M=1.67, SD=0.82) and Time 2 (M=2.13, SD=0.74); t(23)=−2.70, P=0.013.

Table 5 Percentage of target groups screened with PHQ-2

PHQ-2=Patient Health Questionnaire.

Table 6 PHQ-2 used outside of target groups

PHQ-2=Patient Health Questionnaire.

A descriptive analysis of EMR depression screening data was conducted; this data was available for only two sites (site 1, site 2) as the third site was not part of the original trial (see Table 7). Data were then compared across the two sites and two time periods by way of multiple t-tests. The t-tests showed a trend towards greater screening (ie, total PHQ-2 screenings and follow-up count) at Time 2 for site 2 as compared with site 1 but these differences were not significant. t-Tests investigating the same variables across time identified a significant increase in total PHQ-2 screenings from Time 1 to Time 2.

Table 7 EMR data

EMR=Electronic Medical Record.

Phase 1 Data extraction captures 1 December 2010 to 30 June 2011; Phase 2 captures 1 July 2011 to 31 March 2012.

In terms of physician’s self-reporting of their use of the PHQ-2, overall practice was reported as mixed with some reporting occasional use of the PHQ-2 or suggestions that they were using it loosely in practice, while others expressed to being unaware of the indicators. A general sentiment of the implementation being physician dependent was reported. Of those physicians who had implemented the indicators, many described them as simple and reasonable. When the question of the appropriateness of PHQ-2 as a screening tool was explored, physician opinion varied. Some described the tool as appropriate, such as articulated by one physician: ‘As a screening tool it’s both quick and relatively effective so I think you know it’s hard to improve on something that’s as simple as asking two questions’ (Interview 2). In contrast, others disagreed, one physician reporting that ‘I don’t tend to use tools like checklists, I just find them very impersonal and these are people that I’ve known for years’ (Interview 11).

While counsellors screened most patients on depression symptoms, screening practice was variable and tended to be informal for physicians, one expressing this common sentiment by stating ‘I can usually tell if something’s not quite right, then I will ask about it’ (Interview 12). Many physicians described using other screening methods such as open-ended questions, conversation, or the SIGECAPS acronym (ie, Sleep, Interest, Guilt, Energy, Concentration, Appetite, Psychomotor, Suicidal). The general sentiment expressed was that they relied on information coming from patients and trusted their clinical judgement and/or intuition when deciding on whether to screen a patient for depression.

Treatment follow-up

In terms of treatment follow-up, EMR data showed that physicians tended to follow-up with most patients screened as possibly depressed (see Table 7). About half were identified as not depressed upon further assessment. Of those identified as depressed, a mix of treatment options were recorded (referred, treated by physician, referred and treated by physician, treatment refused). Physicians surveyed on type of follow-up care provided to depressed patients, on average, indicated that they treated 32% of cases with medication only, 28% of cases with counselling only, 14% of cases with self-help/self-management approach only, and 43% of cases with a combined approach (eg, medication and counselling; note that percentages do not add up to 100).

Upon interview, physicians described the purpose of providing follow-up care as to monitor symptoms, assess for improvement and side effects of medications, and to provide support. The treatment option pursued and timelines followed were reported to be based on patient preferences and need; physicians expressed to being more directive with increasing patient level of impairment.

Consistent with survey findings, interviews also found a preference for a combined as compared with a single treatment approach. All physicians expressed confidence in managing medication but some described limitations of this treatment option for depression. Counselling was viewed by physicians as effective but not a viable option for all depending on patient motivation and cognitive abilities. Physicians described some limited past training in counselling and to providing supportive present-focussed counselling; overall, however, they expressed to having limited knowledge and experience in this area and to tending to refer to Shared Cared for counselling. Counsellors treatment with depressed patients was described as multi-orientation, solution-focussed, and incorporating patient education. Consistent with other findings presented, counsellors expressed that they commonly promoted self-management while reported use of this approach by physicians was limited.

In terms of Shared Care referral numbers across sites and time periods, referral numbers appeared to be greater at Time 2 for site 1 and site 2 but relatively unchanged for site 3 (see Table 8).

Table 8 Shared Care referrals

Phase 1: March–May 2011; Phase 2: January–March 2012.

Treatment collaboration

A key theme identified in the interviews was that of collaboration. Shared Care was viewed to facilitate patient treatment (including bridging to other resources), physician practice, and reduce physician burden through collaboration on treatment. Physicians expressed significant appreciation for Shared Care and confidence in the counsellors. They also reported significant improvement in access to mental health services as the implementation of Shared Care within their clinics. Outside of Shared Care, access to such services was described to be limited. Moreover, physicians expressed a preference to refer to Shared Care over community mental health due to such factors as time constraints, easier access, and stigma in accessing services outside of primary care. Some concern was stated regarding the six-session limit for counselling within Shared Care. The following description by one physician articulates the importance of collaboration and access to mental health care: ‘one of the reasons why the Shared Care Counsellor is so important is because very many people don’t have “…” coverage “…” for counselling and, and can’t afford private counselling. And so therefore most people went without. So therefore it was left on our shoulders to “…” provide most of the counselling, and of course we weren’t trained for that…’ (Interview 4).

Physicians and counsellors also emphasized the importance of collaboration with patients. Both described involving patients in a discussion of treatment options; counsellors also indicated that they focus treatment on patient goals. The physician–patient relationship was highlighted as important as was ‘knowing your patients’.

Barriers and facilitators to the implementation of the PIN mental health indicators

Interviews highlighted a number of factors that likely contributed to the pattern of implementation observed. A key factor appeared to be physician level of comfort in managing depression, which seemed to vary greatly. Some physicians described apprehension and/or fear of identifying depression, such as expressed by one physician saying ‘When you’re talking about their diabetes and sugars and their cholesterol and blood pressure then sometimes it doesn’t flow nicely just to all of a sudden ask them if they’re depressed’ (Interview 8). Other physicians expressed to being comfortable managing depression; this appeared to facilitate ease and demand of screening. A commonly expressed barrier to implementing the indicators was the issue of high workload demand as compared to time available, as expressed by one physician: it is ‘…not because we don’t want to, just there’s a lot coming at us that we’re trying to sort through in a fifteen minute visit’ (Interview 1). At the heart of this issue, seemed to be remuneration policy and agreements not specific to the PIN initiative. This issue was captured by the following quotation ‘And that’s problematic because Manitoba Health says, well yeah okay so yeah we’ll pay you to look at his toe, but you did what, you treated him for his depression, we’re not paying for that in addition’ (Interview 10).

The methods of implementing practice processes to track the indicators, such as workflow, the process of administering the PHQ-2, and the presence of EMR reminders, were reported to influence implementation. For instance, some physicians described the use of patient-administered screening questionnaires as facilitative. Technology literacy related to EMR optimization was another factor mentioned. Access to resources, such as Shared Care, effective treatment options, and information on PIN, were further expressed to facilitate implementation.

Finally, the perceived need for screening was mixed. Most physicians were convinced of the need but not all. The doubt regarding screening importance was articulated by one physician who stated: ‘A question to put out there, like we know that if we lower haemoglobin A1C in a diabetic we’re reducing mortality. We know that “…” if we reduced our LDL we’re reducing mortality, but if we screen for depression are we changing anything in the long-run…’ (Interview 11). Some physicians expressed resentment towards implementing the mental health indicators, such as expressed by one physician saying ‘I’ve used it but I was um slightly resentful thinking this is you know this is like kindergarten, put on your, put on your shoes before you go outside’ (Interview 7).

Initial impact of the PIN Mental Health Indicators

The interview data suggested that overall it is too soon to speak to impacts of the PIN mental health indicators on patient care and practice. This sentiment was validated by one physician’s response to the question regarding impacts: ‘Not too much, I mean I found that a lot of the times, the people who had a lot of signs of it had already been diagnosed and um I don’t know that it’s caught too many new cases of depression, but that being said it’s still early so maybe they just haven’t popped up yet’ (Interview 2). When asked about potential impacts, of the providers who had implemented the indicators, many suggested that it had the impact of cueing to assess for depression and increasing their awareness of the possibility of depression for high risk patient groups. For some, a change in attitude around screening for depression was noticed from skeptic to proponent. Further, some physicians suggested an increase in detection of depression and mild mental health problems, one indicating ‘I can think of people in my mind where I was surprised they answered positively, I was, if I would answer ahead of time for each patient beforehand and, and make my own prediction, I’d say oh I think this guy’s, this guy’s okay and, and then they’re not. So, so impact is that I think I’ve picked up depression that I would not have otherwise’ (Interview 5).

Some physicians felt the PIN follow-up indicator, which was deliberately implemented as the second step of the process to ease physician burden at the initial visit, was more impactful than the screening indicator (ie, the PHQ-2), in that it assisted or forced treatment planning. One physician expressed ‘It makes me commit to a plan of action. So I’m going to have to put down something, either this person’s not really depressed or this person is depressed but I can manage it myself or I’m going to refer on’ (Interview 7). Some physicians also reported benefits of the initiative for the patient–physician relationship, in that care and concern were shown to patients beyond the initial chronic disease being discussed, one physician expressing ‘…not that we medicated her or not that she needs a counselor but, but it was appropriate just to come touch on it and I know she appreciated it cause it’s kind of like wow you never really asked me that before or for three years, so it’s kind of almost a little embarrassing on my part to go wow, I should have asked that’ (Interview 1). An additional impact suggested was that screening provided a segue to discussing mental health more generally. Patient response was described to be positive overall and sometimes surprised. Significant concerns regarding impacts on workload and referrals were not reported.

Discussion

Implementation of the PIN mental health indicators was variable across sites and providers. The findings suggest that the lack of incentive funding and EMR capability may have impacted implementation at one of the sites. Technology, and the capacity to share information such as through EMR, has been identified as a key factor related to capacity for collaboration (Addiction and Mental Health Collaborative Project Steering Committee, 2014). Key barriers and facilitators to implementation were physician level of comfort in managing depression, perceived need for screening, issues related to remuneration, workflow, clinician/clinic process of administering the PHQ-2, the presence of EMR reminders, access to resources (eg, Shared Care), and provider level of information on PIN. These factors suggest ways to enhance participation in this initiative. The issue of remuneration was not surprising and has been identified as an important area to be addressed in primary care reform internationally (The Commonwealth Fund, 2011). Although many physicians reported that they would continue to screen for depression beyond the trial period, findings suggested that screening for some would not necessarily be according to the PIN measures.

Notwithstanding these findings, this evaluation showed that depression is viewed as an important problem in primary care that is time consuming to diagnose, manage, and treat. Interestingly, over the study period, there was a trend towards physicians seeing treating depression as less time consuming. Two possible contributors to this trend are the presence of Shared Care and increased exposure to talking to patients about depression, which may have reduced physicians’ belief that managing depression is time consuming. In other words, the mental health indicators and/or the evaluation of the initiative may have served to promote capacity. Although physicians reported adequate knowledge and skills in managing depression they also expressed concern for their limited training in counselling and non-pharmaceutical approaches, some concerns regarding the limitations of medication, and unanimously reported Shared Care as important in assisting in the management and care of depressed patients. An implication of this later finding is that implementation of this initiative in sites without Shared Care or access to other mental health services and supports would likely be challenging. This finding is consistent with the current evidence highlighting the need for adequate resources and multi-sectoral collaboration when seeking to address mental health care needs (Addiction and Mental Health Collaborative Project Steering Committee, 2014). Screening is not a useful stand-alone component of enhancing care for depression and guidelines recommend screening only in the presence of appropriate comprehensive care (including access to treatment; Canadian Task Force on Preventive Health Care, 2013; O’Connor et al., 2016). Models of collaborative care that have been demonstrated to be beneficial for depression outcomes, such as the Improved Access to Psychological Therapies (IAPT) programme in the United Kingdom (Centre HSCI, 2015; Fonagy and Clark, 2015) or the Better Access Plan (Pirkis et al., 2011; Littlefield, 2014) in Australia offer an integrated, comprehensive approach to responding to mental health (Peachey et al., 2013). The Better Access Plan has been reported as cost-effective (Pirkis et al., 2011; Littlefield, 2014); this issue appears more controversial in the literature for IAPT (McCrone, 2013; Mukuria et al., 2013). The significant reduction in reporting of training and educational materials provided by Shared Care as useful in managing depressed patients between time periods was unexpected. This finding could have been the result of training needs being met in the year prior but warrants follow-up within Shared Care.

Consistent with trends in clinical practice, physicians expressed a preference for combined treatment (likely often medication and counselling). However, a combined approach is not necessarily recommended over monotherapy for depression and medication is not first-line treatment for mild or moderate depression (Kennedy et al., 2009; NICE, 2009). This finding highlights the need for further exploration on interventions to foster clinical practice that is more consistent with treatment guidelines on effective treatment and management of depression. As increasing attention is being paid to interprofessional education and collaborative practice (Bainbridge, 2010; Canadian Interprofessional Health Collaborative, 2010), these findings also highlight the need to further enhance Shared Care approaches in primary care. The business case for integrating psychological services into primary care is strong and government policy and programming in Canada has not followed with the mounting evidence for this need (Peachey et al., 2013).

The underutilization of self-management for depression is consistent with the literature (den Boer et al., 2004) and is another important area for intervention within primary care, as this low intensity treatment approach is low cost and has high potential reach, particularly within the primary care context (Blisker et al., 2012; Houle et al., 2013). Collaborative care models implemented in the United Kingdom, Australia, and elsewhere (eg, IAPT programme in the United Kingdom) are more consistent with the best practice evidence in that they explicitly include self-management as a step in the care delivery plan, which also facilitates expanding access to mental health services at a broader scale. There have been some regional efforts in Canada (eg, Alberta) towards a stepped care approach that includes self-management, however, no national programme. The Manitoba Model of Shared Care does not follow an explicit stepped care pathway and such could be considered as a means of further enhancing access to depression care.

Despite an increase in use of the indicators across time and the general sentiment that benefits of screening outweigh the costs, the benefit of screening for depression remains unclear and the guidelines for screening for depression in adults recently recommended against screening (Canadian Task Force on Preventive Health Care, 2013). This evaluation suggested some possible impacts in terms of an increase in screening and detection, change in attitude towards screening, and benefits to patient–physician relationship. Some providers questioned the usefulness of screening and when a possible impact was suggested, physicians felt that primarily cases of mild depression were detected, implying that these patients may not have needed treatment. Research has suggested that undetected cases of depression tend to be milder and resolve without treatment; it has also highlighted potential harms in identifying milder cases through screening initiatives such as false positives, poor cost-benefit ratio, and unnecessary labelling of patients (Gilbody et al., 2006; Thombs et al., 2012). The finding of an increase in referral rates across time periods for site 2 could be explained by the fact that Shared Care had only recently been implemented within the site before this evaluation.

Limitations of these findings include the self-report nature of the survey and interview data, in addition to the absence of an evaluation for the period before the implementation of Shared Care or the PIN initiative. Although multiple data sources were used, findings do not speak to the impact of screening in terms of objective patient outcomes, nor do they speak to the patient perspective. A limitation of the EMR data is that multiple tests within a patient record may not have been captured. Moreover, the interviews themselves may have served as an intervention, as some counsellors commented on increased use of the PHQ-2 in their clinics after the interviews. In addition, many physicians made comments during the interviews suggesting that they would think more about screening or implementing the indicators.

Conclusion

This evaluation illustrated providers’ perceived attitudes, knowledge, skills, and behaviours related to recognizing and treating depression and expanded our understanding of primary care process related to managing depression related to the implementation of a new initiative. Consistent with current guidelines, a question the findings of this evaluation suggests is whether there are more effective ways of having an impact on depression within primary care than screening. Collaborative care has been demonstrated as an effective model of care for improving depression outcomes and costs (Archer et al., 2012). Screening, however, is just one component of a comprehensive, interprofessional, and integrated system of care that is needed for depression care. This study also highlights some key barriers and facilitators to implementing practice changes for depression management within primary care, such as clinic process, access to resources, remuneration, and provider information.