Article Text
PDF
Abstract
Background Psychotherapy implementation must contend with the task of preparing a mental health workforce to provide the highest quality services to as much of a service population as possible, in high-income as well as low-to-middle income countries.
Objective We outline general challenges and solutions and investigate how well various implementation strategies would fit a clinical population.
Methods Using a data set from a prior cluster randomised trial with a clinically diverse population and 33 intervention practices, we presented multiple illustrations comparing the ability of different implementation strategies to serve youth and families with procedures in which service providers were trained.
Findings A series of survival functions demonstrated that many common implementation strategies are unlikely to create a prepared workforce, given the large and diverse number of practices needed to be mastered by providers.
Clinical implications ‘Benchmark’ solutions that afforded superior coverage of the service population could be supported through paced learning approaches (ie, training interventions a little at a time) using extensible, modular intervention designs.
- child & adolescent psychiatry
Statistics from Altmetric.com
Background
Despite wide availability of evidence-based treatments (EBTs), the size of the global mental health workforce is widely recognised as insufficient1 with few direct service providers having doctoral-level training and many without graduate training at all.2 3 Despite a plethora of EBTs, 70%–85% of individuals with mental health needs do not receive any services, and thus we contend that a vision of the future of psychotherapy will require more systematic attention to methodologies and strategies to create a prepared workforce. By ‘prepared workforce’, we mean one that can serve as many individuals as possible with interventions that are as effective as possible. By ‘strategies’, we refer to plans for distributing competencies to perform up to a given standard across the workforce,4 and for now we set aside questions about the specific tactics involved (eg, workshops vs supervision).
Challenges
If the task is to prepare a workforce to have as many effective skills as are relevant to the population being served, one must contend with a variety of challenges. Principal among these is workforce baseline competency, or the repertoire of skills already mastered at a given point in time (eg, the skill of teaching a caregiver to use a ‘time out’ procedure). Because baseline capacity differs across contexts and across individuals, and one challenge is determining what is known in order to efficiently build on those competencies to create a fully prepared workforce. A second challenge is workforce development potential, or the rates at which workers can increase their competency over time. It is widely recognised that not everything can be learnt at once, and thus questions arise about what skills to train in what order, working within the limits of workforce development potential. In the management literature, this notion has been referred to as absorptive capacity.5 This capacity is always limited, and an emerging trend toward simpler, ‘scalable’ interventions in the literature is based on this premise that mastery of more than a handful skills in a given time period is often too challenging for the already overburdened mental health provider.6 7 Although the precise limits of this capacity are an empirical question,8 it is clear that one must carefully consider strategies to organise training opportunities, paying particular attention to how much can be trained over what period of time.
Other challenges include turnover and the sheer growth and size of the evidence base. Turnover rates range from 30% to 60% in public mental health systems,9 which impose a continual drain on the collective developed skillset of those workers. And for all mental health workers, the large and ever-growing evidence base also represents a challenge to master the latest skills needed. Because science continually produces important new information to better inform or guide therapy, the ‘state of the art’ is always moving away from the state of the workforce. Collectively, these challenges represent the task of preparing a given number of providers with diverse baseline competencies and different capacities for professional growth to possess and use a suitable amount of skills informed by a large and evolving evidence base, while working against turnover. In this context, we explore the strengths and limitations of various arrangements of strategies.
Discovery or pacing
We focus on one widely used approach to manage a wealth of information in the context of limited developmental and attentional capacity, known as ‘discovery’ in some contexts; that is, not learning everything all at once. The discovery concept has demonstrated high utility in other intensive content-management contexts such as software engineering (cf. ‘lazy loading’10), and with respect to organisational change, this notion has been referred to as ‘pacing’.11 In the current context, discovery implies not mastering all competencies at once, but instead having an awareness of what could be learnt, how to find and learn it at the time it is needed. An analogy is the difference between simply knowing how to use a dictionary and having to memorise its entire contents at once.
Objectives
Solutions to workforce preparedness challenges can be facilitated by modular, extensible intervention designs that directly support partial decomposability (ie, the ability to break something large into smaller independent subcomponents) and paced workforce development strategies (ie, intentionally training only a little at a time). To consider this explicitly, we illustrate training scenarios, showing the portion of youth that could be served by workers who are trained in the skills as they are delivered. We impose the following assumptions: (a) the upper limit of competencies representing the evidence base is 33 skills, corresponding to practices used in the clinical trial (see table 1)12; (b) no practices were mastered at baseline; (c) population characteristics (problems, age, gender) are limited to those found in the same clinical trial data set; (d) the ability to connect indicated procedures to population characteristics is assumed (eg, once a provider masters a practice for depression, that provider knows when to use it) and (e) each practice becomes useable the day it is trained. Despite these mostly generous assumptions, we posited that illustrating the connection between various workforce development strategies and their ability to serve youth with mastered actions (ie, meeting a standard of competency) would highlight particular limitations and yielding important insights about the future of psychotherapy design and implementation.
Practices used in the illustrations as a function of their timing (phase) and target problem focus
Methods
Participants
Youth participants
Youth participants were a subset of those from a cluster randomised trial comparing the Modular Approach to Therapy for Children (MATCH) to community implemented EBT.12 For the purposes of this investigation, we used data only from 78 participants whose therapists were randomised to the MATCH condition (ie, for whom we had detailed practice implementation data). The primary treatment focus was determined using an expert consensus procedure (Youth Top Problems7). As described in Chorpita et al,12 top ranked problems ranked for each case were: conduct problems (n=29; 37%), anxiety (n=19; 24%), depression (n=29; 37%) and traumatic stress (n=1; 1%). Baseline prevalence was much higher when not restricted to the top problem, as follows: conduct problems (n=62; 79%), anxiety (n=69; 88%), depression (n=58; 74%) and traumatic stress (n=10; 13%). Nearly all cases (N=75, 96%) reported elevated concerns in at least two different areas, and the majority (N=56, 73%) reported elevated concerns in three or more areas. Because of the limited number of cases with traumatic stress as the primary treatment focus, we limited illustrations to practices relevant to conduct problems, anxiety and depression only.
All participants were naturally occurring referrals to their local public mental health agency who showed clinical elevations on screening measures as described by Chorpita et al 12 (see 37 for inclusion and exclusion criteria). Youth were aged 5–15 (N=78; M=9.7, SD=2.8), with grade levels ranging from pre-kindergarten through 11th grade. Slightly more than half (55.1%; n=43) were boys, and the sample was predominantly Latinx (83% Latinx, 9% African American, 5% Multiethnic and 3% Caucasian). Families frequently reported living in poverty, with 96% of the sample reporting income of $39 000 or less and 77.0% reporting income of $19 000 or less, supporting an average number of 3.9 dependents (SD=1.5). Families were typically led by a single parent (60.5%), 44% of primary caregivers had not finished high school and 62% were born outside the USA.
Therapist participants
Twenty-nine therapists in three different community agencies in Los Angeles County were included in this study. Therapists saw an average of 2.7 study cases (SD=1.8; range=1–7). Therapists were 32.2 years old on average (SD=3.8), and 96.6% identified as female. They had an average of 3.1 years clinical experience post degree (SD=2.6), and 45% described their professional specialty as marriage and family therapy, 38% as social work and 17% as counselling or clinical psychology. For graduate training, 17% reported a doctoral degree (ie, PhD or PsyD) and 79% reported a master’s degree as their highest education obtained, with 24% holding a state license.
Treatment protocol
All therapists used MATCH,13 which features a library of 33 modules, with 31 corresponding to three EBTs targeting youth anxiety, depression and conduct problems.14 A therapist typically uses only a subset of the procedures with a given youth or family (ie, in this study, therapists used between 1 and 19 MATCH procedures with any single study case; M=8.0; SD=4.2).
Measures
Youth status
Some survival analyses examined cases only up to the point of clinical response (ie, the therapist only needed to be trained in skills used up until an initial observation of improvement), rather than to the end of the case. To model improvement, we used youth and caregiver scales collected to yield eight score arrays for the Brief Problem Checklist (1) total, (2) externalising, (3) internalising and Youth Top Problems (4) total scores from (a) youth and (b) caregivers, respectively. Using procedures described by Sayers et al,15 a minimally improved difference (MID) indicator was calculated for each score to describe whether the youth had improved as of the date of each treatment session. We considered an improvement of at least 1 SD from baseline on at least one measure sufficient to indicate minimal change.
Clinical events
The MATCH consultation record was used as the measure of clinical practices used.16 This form was completed by the therapist for each client contact and indicated which practice was used.
Analysis
Analyses illustrated the extent to which practices used for each case were mastered or trained at the time of delivery. As the current analyses were simulations, the sets of skills were fixed based on the training strategy. For example, to simulate a strategy of training in a single EBT for conduct problems, the set of mastered skills was defined as the set of practices that were relevant to conduct problems (see table 1).
Findings
Single-EBT strategies
Figure 1 shows the percentage of youth for whom all skills were trained according to a single EBT for conduct problems, anxiety or depression. The upper panel represents youth coverable up until the point of the final treatment session, and the lower panel represents those youth coverable only up until the point of an MID. Across both models, training in a single EBT afford minimal coverage of the service population, dropping to approximately 50% by day 30. By day 60, percentages ranged from 28% to 39%. Although 99% of this service population had a primary clinical focus of conduct problems, depression or anxiety, we observed that the sum of cases covered across all three strategies by day 150 was 50% without modelling improvement, and 60% modelling improvement, reflecting the fact that approximately half of the cases required practices for comorbid problems.
Survival functions comparing a limited procedure, transdiagnostic strategy with three single-EBT training strategies on the coverability of youth cases over time.
Limited-practice strategies
A limited-practice strategy is a plan to train a workforce in a chosen subset of practices. For this illustration, we selected skills by treatment phase across problems, to prepare the therapist with at least one skill for connecting with clients, one for cultivating a solution targeted to each problem and one skill for helping the client consolidate gains (see table 1). This strategy was modelled due to calls for increasingly streamlined, pragmatic, ‘transdiagnostic’ or ‘scalable’ treatment programmes to facilitate implementation.
Figure 1 shows that with or without modelling improvement, the limited-protocol approach covered a larger portion of the service population. Despite positive comparisons to single-EBT models, however, this approach did not cover most of the youth in the service population by day 150, even when modelling improvement. Thus, it seems that within a few months, the workforce used a wider variety of skills with clients.
Pacing single-EBT strategies
One solution to address absorptive capacity of the workforce is to distribute learning experiences over time. In this example, a paced-EBT scenario trains the workforce in practices for multiple target problems in series. For these models, we narrowed our focus on practices that fit the two most common primary problems in the service population (conduct problems and anxiety, respectively), only to make the graphs more interpretable. We modelled training the first set of practices at day 0, followed by the complement set of practices at day 60. Solid lines in figure 2 represent the coverage for the first EBT, with dashed lines modelling the advantage in coverage when the second EBT was trained at day 60. Overall, paced trainings in single EBTs did not yield more than minimal enhancement of population coverage.
Survival functions illustrating the coverability of youth cases over time for anxiety-focused or conduct-focused training strategies, followed by the addition of the complement strategy at day 60.
Pacing according to treatment phase
We modelled one final paced solution that organised practices according to whether they were expected in an earlier or later phase of treatment. Specifically, we organised the practices into the ‘connect, cultivate, consolidate’ framework that is central to the Managing and Adapting Practice system on which MATCH was built.17 For this illustration, we focused on the first two phases, ‘connect’ and ‘cultivate’, whose component MATCH practices are outlined in table 1. Figure 3 shows paced trainings in these phased sets of practices produced the most favourable long-term coverage (other than the strategy of training all practices on day 0, yielding 100% coverage). Whether improvement was modelled or not, there was a large drop in coverage from days 30 to 60, suggesting that many cases in the trial dataset were administered ‘cultivate’ practices before day 60. Following day 60, almost no additional cases were lost due to not being covered, regardless of whether improvement was accounted for in the models.
Survival functions illustrating the coverability of youth cases over time training ‘connect’ phase practices with and without the addition of the ‘cultivate’ phase practices at day 60.
Discussion
The collective results suggest that even under some highly favourable assumptions, each strategy examined had clear limitations in preparing a workforce to engage in competent practice for a large portion of its service population. Specifically, in nearly every strategy illustrated, roughly half of the youth service population was unable to receive trained practices. We found the sheer diversity of strategic failures to be somewhat surprising. Single-EBT implementation strategies are the presumed standard of implementation but produced some of the most discouraging results in terms of coverage, even for a set of practices that targeted the most common problem focus in the sample (ie, conduct problems). Even more surprising, pacing single-EBT strategies yielded literally no improvement when starting with the intervention targeting the most prevalent problem in the population, and only minimal improvement when the order was reversed.
Development strategies that aimed at breadth first, rather than depth first, improved coverage. Specifically, a ‘limited’ intervention model produced roughly double the coverage of any single-EBT strategy, with a comparable amount of practices to master. Nevertheless, most youth were still not coverable over time. Prioritising skill training by treatment phase (‘connect’ practices, ‘cultivate’ practices) as opposed to by problem type yielded better coverage, but again was inadequate to cover more than half of the service population by day 60. Thus, these limited and paced strategies ‘buy a little time’ for additional development and help a subset of youth to achieve early gains (ie, MIDs), but appear insufficient as comprehensive strategies.
Of note, we did not directly model three strategies that would yield 100% coverage, which include: (1) selection, which requires all providers to have all competencies in place at the time of hire; (2) all at once, which requires immediate training in all competencies and (3) just in time, which involves paced development and mastery of all skills just prior to their use in practice. Because of their comparative potential, a vision of the future of psychotherapies should consider the factors that could facilitate or hinder their adoption.
Clinical implications and the future of psychotherapy
A variety of insights emerge from these illustrations. One broad conclusion at this point is that future research and implementation endeavours must more fully consider workforce development and should articulate and evaluate system-wide implementation strategies that go beyond specific tactics.18 Given that systems and the contexts in which they operate are diverse, we expect there will not be one ‘optimal’ strategy, and that many applications will involve combinations of strategies articulated above (eg, combinations of selection, ‘limited’ all-at-once training and pacing).
Another insight is that the knowledge delivery burden of creating a prepared workforce is larger than one might expect, and it will only increase as new evidence emerges. Hence, solutions merely aimed at simplification (eg, through reducing the number of practices or competencies) are unlikely to be comprehensive. We are sceptical that limited as opposed to paced strategies will be the preferred solutions of the future, given an evidence base already that has far more than 33 skills whose mastery could contribute to the overall productive capacity of service systems. We hope and expect both that the number of practices will continue to grow and that the preferred execution of these strategies will evolve as research produces new insights into when and how they work best.
These considerations suggest two pragmatic recommendations relevant to the future of psychotherapy. Regardless of how the more promising workforce development strategies emerge and unfold, their success is likely to be afforded by (1) well-specified and comprehensive competency models with established instrumentation and metrics19 20 and (2) extensible and developmental intervention architectures that allow new practices to incrementally contribute to intervention capability21 and allow interventions themselves to develop the competencies of the user.22
Regarding competency models and metrics, we envision resources that go beyond broadly defined aptitudes and operationalise specific skills whose competent application can be measured directly.23 24 These models can enhance any of the ‘benchmark’ strategies noted above, for example, with precise metrics to inform workforce selection, or to customise curricular content in paced learning scenarios to match different diverse baseline competencies and absorptive capacities. We envision that the real-world feasibility of such resources, which has long been a barrier,25 will be enhanced through technological innovations (eg, automated coding of therapeutic interactions) that enable rapid scoring and feedback of in vivo skill application, for example. These innovations and their evaluation should be considered key enabling technologies to realising vastly improved workforce development strategies.
Regarding intervention architecture, we see all these strategies potentially enhanced by structures that allow aggregation and accumulated applications of the evidence base. At least two properties of modular design are relevant here. ‘Partial decomposability’, in the context of intervention, refers to the ability to break the evidence base into meaningful sub-units with highly similar form (eg, discrete practices, or logic models for their (re)assembly into working protocols). ‘Standardised interface’ in this context, refers to the ability of these components to interact with one another in a structured fashion. As an example, one can imagine learning a single procedure for behavioural activation, which can subsequently be coordinated within one’s existing set of practice competencies and woven into a therapeutic episode, without the need to learn and master the entire intervention from which that procedure was drawn. Rather than defining a fixed set of competencies, a modular intervention architecture can operate as an expandable library of masterable skills, and enables discovery (ie, just in time mastery) by structures that catalogue what can be mastered by whom for what anticipated purpose.
Such intervention designs need not separate their ‘indexing’ functions from their contents (eg, the way books are separate from card catalogues in libraries); rather, their ability to facilitate workforce development can be enhanced even further when the components themselves are organised in linked hierarchies. A simple example is a relaxation procedure whose initial presentation is simplified to be appropriate for a novice provider, but then becomes increasingly elaborate over time, introducing more features, exercises or ‘special cases’ as the provider’s experience increases. In this way, interventions can be sensitive to individual differences in absorptive capacity and quite literally ‘teach themselves’ to providers based on evolving patterns of use and an established index of relationships among the various intervention components.
Summary
In our vision of the future of psychotherapy, we see great potential and many possible solutions to the challenge of creating a prepared workforce. We offer these illustrations as a point of reflection and to heighten our collective attention to the need for well-articulated strategies and parameters relevant to these challenges. Given the computational demands of measuring multidimensional competencies to trigger specific learning experiences in the context of an evolving literature and diverse service populations, we expect the future will see a continued escalation of machines to enable such solutions. We offer these illustrations not as definitive endpoints, but rather merely as guideposts. Thus, although they cannot tell us precisely where we will end up, we hope that they at least point our field in a clear direction and that they credibly illustrate the need for us to continue down that path.
References
Footnotes
Contributors BFC implemented the trial and oversaw all operations, training and data collection. BFC and ED planned the analyses and set up the survival models. ED, BFC and JDV worked on the data analysis. All authors contributed to the writing.
Funding This work was supported by the John D. and Catherine T. MacArthur Foundation, award number 12-103104-000-USP.
Competing interests Dr Chorpita and Dr Daleiden are owners and partners in PracticeWise, LLC, which publishes a version of the protocol used in the illustrations in this study.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon reasonable request. Data are available upon reasonable request to the first author. Permission for re-use is granted on a case by case basis.