You are here

Article Text

Article menu

Download PDFPDF

Pharmacological treatments
Estimating the smallest worthwhile difference of antidepressants: a cross-sectional survey
  1. Ethan Sahker1,2,
  2. Toshi A Furukawa1,
  3. Yan Luo1,2,
  4. Manuela L Ferreira3,
  5. Kaori Okazaki4,
  6. Astrid Chevance5,6,
  7. Sarah Markham7,8,
  8. Roger Ede7,
  9. Stefan Leucht9,
  10. Andrea Cipriani10,11,12,
  11. Georgia Salanti13
  1. 1 Department of Health Promotion and Human Behavior, Graduate School of Medicine / School of Public Health, Kyoto University, Kyoto, Japan
  2. 2 Population Health and Policy Research Unit, Medical Education Center, Graduate School of Medicine, Kyoto University, Kyoto, Japan
  3. 3 The University of Sydney, Sydney Musculoskeletal Health and The Kolling Institute, School of Health Sciences, Faculty of Medicine and Health, Sydney, New South Wales, Australia
  4. 4 School of Medicine, Kyoto University, Kyoto, Japan
  5. 5 Université Paris Cité and Université Sorbonne Paris Nord, Inserm, INRAE, Center for Research in Epidemiology and StatisticS (CRESS), Paris, France
  6. 6 Centre d’Epidémiologie Clinique, AP-HP, Hôpital Hôtel Dieu, Paris, France
  7. 7 Patient and Public Involvement (PPI) Group, Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, University of Oxford, Oxford, UK
  8. 8 Department of Biostatistics & Health Informatics, Institute of Psychiatry, Psychology & Neuroscience (IoPPN), King's College, London, UK
  9. 9 Department of Psychiatry and Psychotherapy, Technical University of Munich, School of Medicine, München, Germany
  10. 10 Department of Psychiatry, University of Oxford, Oxford, UK
  11. 11 Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, Oxford, UK
  12. 12 Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
  13. 13 Institute of Social and Preventive Medicine and Clinical Trials, University of Bern, Bern, Switzerland
  1. Correspondence to Professor Toshi A Furukawa, Department of Health Promotion and Human Behavior, Kyoto University, Kyoto, Japan; furukawa{at}kuhp.kyoto-u.ac.jp

Abstract

Background Approximately 30% of patients experience substantial improvement in depression after 2 months without treatment, and 45% with antidepressants. The smallest worthwhile difference (SWD) refers to an intervention’s smallest beneficial effect over a comparison patients deem worthwhile given treatment burdens (harms, expenses and inconveniences), but is undetermined for antidepressants.

Objective Estimating the SWD of commonly prescribed antidepressants for depression compared to no treatment.

Methods The SWD was estimated as a patient-required difference in response rates between antidepressants and no treatment after 2 months. An online cross-sectional survey using Prolific, MQ Mental Health and Amazon Mechanical Turk crowdsourcing services in the UK and USA between October 2022 and January 2023 garnered participants (N=935) that were a mean age of 44.1 (SD=13.9) and 66% women (n=617).

Findings Of 935 participants, 124 reported moderate-to-severe depressive symptoms but were not in treatment, 390 were in treatment and 495 reported absent-to-mild symptoms with or without treatment experiences. The median SWD was a 20% (IQR=10–30%) difference in response rates for people with moderate-to-severe depressive symptoms, not in treatment, and willing to consider antidepressants, and 25% (IQR=10–35%) for the full sample.

Conclusions Our observed SWDs mean that the current 15% antidepressant benefit over no treatment was sufficient for one in three people to accept antidepressants given the burdens, but two in three expected greater treatment benefits.

Implications While a minority may be satisfied with the best currently available antidepressants, more effective and/or less burdensome medications are needed, with more attention given to patient perspectives.

  • depression
  • data interpretation, statistical

Data availability statement

Data are available upon reasonable request. Data collected for this study and the code used to analyse these data is available upon request from the corresponding author, for purposes of reproducing the results. The lead author affirms that the manuscript is an honest, accurate and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as originally planned have been explained.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjment-2023-300919

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    WHAT IS ALREADY KNOWN ON THIS TOPIC

    • Acute treatment of depression with antidepressant medications can benefit patients with an increased response over placebo, with ORs ranging between 1.5 and 1.9.

    • Antidepressant treatments are associated with burdens including side effects, expenses and may involve other inconveniences.

    • Patient preferences and trade-offs between the benefits and harms of pharmacological interventions for treating depression have never been estimated.

    WHAT THIS STUDY ADDS

    • People with moderate-to-severe depressive symptoms who are not in treatment considered a 20 percentage point (IQR=10–30%) difference in treatment response rates, for antidepressant treatments over no treatment, to be worthwhile for initiating antidepressant treatment.

    • This study showed that one in three people would consider the efficacy of the antidepressant treatment worthwhile despite its burdens. However, the other two out of three would need to see greater efficacy or fewer burdens to consider the treatment.

    • There was large variability in expectations of worthwhile differences, but demographic and clinical variables were not materially associated with these estimates.

    HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

    • The smallest worthwhile difference (SWD), not the minimum important change, should inform the between-group difference to be aimed for in new antidepressant developments and to be detected in antidepressant trials.

    • We should consider the SWD in balancing the benefits and harms of antidepressant treatments both in individual decision-making and in guideline or policy developments.

    • We need more research estimating SWD for various treatments for depression (eg, psychotherapies, electroconvulsive therapies), for other mental disorders and for other diseases.

    Background

    Depression is the second leading cause of disability worldwide1 with millions seeking treatment through antidepressant medications.2 Antidepressants are proven efficacious, backed by hundreds of randomised controlled trials and rigorous meta-analyses.3 4 For example, a network meta-analysis including 21 commonly prescribed antidepressants demonstrated response odds ratios (ORs) ranging between 1.49 and 1.85 favouring antidepressants over placebo.3 However, effect size measures such as ORs miss the patient’s perspective on the significance of intervention benefits.5 The patient-deemed worthiness of antidepressant treatments, given the symptom improvement benefits on the one hand and the burdens (harms, expenses and inconveniences) on the other, remains controversial.6 7

    The minimum important change (MIC) can help determine whether changes in health outcomes over time are important from the patient viewpoint. The MIC, also known as the minimal important difference or minimal important clinical difference, is the smallest change after treatment in a health outcome perceived as important.8 Defining the MIC is a useful way to interpret patient-reported outcome measures.9 By definition, the MIC is specific to a particular instrument or outcome measure,10 11 generally lacks association with an intervention12 and does not explicitly account for burdens and benefits relative to an alternative.10 12 The MIC for depression scales is estimated to be a 6-point reduction for the Beck Depression Inventory-II13 and a 7-point to 8-point reduction for the Hamilton Depression Rating Scale 17-item.14 Estimates of the MIC concern intraindividual change and the mean change seen in participants in antidepressant trials (both on placebo and antidepressants) is usually larger than the MIC.13 14

    A conceptually different approach to facilitate interpretation of patient importance in the context of an intervention is to estimate the smallest worthwhile difference (SWD). The SWD is ‘the smallest beneficial effect of an intervention that justifies the costs, risks and inconveniences of that intervention’ over a treatment alternative.10 The SWD represents a between-treatments assessment reflecting a trade-off of the benefits and burdens of two treatment options. It is patient-derived, intervention-specific, control-specific and expressed as an absolute difference between treatment options.10 12 Two methods have been proposed to estimate the SWD. The discrete choice experiment asks individuals their preferences for hypothetical scenarios where benefits and costs vary. Regression models determine the preference threshold for one treatment over another.15 The benefit-harm trade-off method (BHTM) asks individuals to state their preferences for scenarios, but benefits vary, while burdens remain constant. The BHTM ascertains how many benefits people are willing to trade-off for the expected burdens of one intervention over another.16 The BHTM is easy to understand and has been used to estimate the SWD in treatments for respiratory disease,12 fall prevention15 and pain reduction therapies.17 Despite the debate about whether the effect of antidepressants is large enough to justify their burdens, the SWD of antidepressants has never been estimated.3 18

    Objective

    The present study aims to estimate the SWD of commonly prescribed antidepressants for depressive disorders compared with no treatment/natural course using the BHTM. We hypothesise that the SWD estimates may vary by participants’ clinical needs and current/past-treatment experiences. Our primary focus was on people who were currently depressed but are not receiving treatment. We also surveyed people in treatment and those without depressive symptoms. This study aimed to help doctors understand patient expectations and establish evidence-based benchmarks for clinical trials testing new antidepressant therapies.

    Methods

    Online supplemental 1 presents the prespecified protocol approved by the Kyoto University Graduate School of Medicine Ethics Committee on 9 September 2022 (R3574-1, changes in online supplemental 2), and all participants provided e-consent.

    Supplemental material

    Study design

    We conducted a cross-sectional survey using three research participant crowdsourcing services (RPCSs): Prolific, and MQ Mental Health (MQ) and Amazon Mechanical Turk (MTurk) between October 2022 and January 2023. We invited participation through RPCSs and linked participants fitting the inclusion criteria to an online survey. While Prolific and MTurk represent a general internet population, the MQ participant pool includes mainly people with lived experiences in mental health and healthcare professionals who volunteer to improve research representation. RPCS participants generally demonstrate high test–retest reliability and high convergent and concurrent validity in psychological tests,19 although there remain concerns of careless or fraudulent responses.20 Following previous methods to increase data quality,19 20 we restricted MTurk participants to those in the USA rated by researchers as a 95%+approval and graded at the ‘masters’ level; the highest vetting. Prolific participants were limited to the UK or USA. MQ participants were limited to the UK only. These RPCSs provide monetary compensation for participation from their established participant pools. The compensation was £1.20 for Prolific, US$1 for MTurk and voluntary for MQ. Remuneration was commensurate with similar length RPCS studies, based on time-to-completion, and above the average amount as evaluated by Prolific.

    Primary outcome and its measurement

    Our primary outcome was the SWD, representing the patient-preferred efficacy of depression treatment with antidepressants that would be deemed worthwhile compared with no treatment, given the treatment burdens (harms, expenses and inconveniences).

    We estimated the individual participant preference through the BHTM. We presented a major depressive episode symptoms summary and explained the benefits and burdens of antidepressant treatment and the no treatment/natural course alternative based on The Diagnostic and Statistical Manual of Mental Disorders, 5th edition (DSM-5) and US Federal Drug Administration (FDA) descriptors (online supplemental 3). We used selective serotonin reuptake inhibitors (SSRIs) and serotonin-norepinephrine reuptake inhibitors (SNRIs) in the antidepressants description because they are most frequently prescribed21 and have similar tolerability/efficacy profiles.3 We estimated that response, defined as 50% or greater reduction in depression severity, occurs in approximately 30 out of 100 people after 2 months without any treatment.22–24

    Next, we asked participants if they believed antidepressant treatment was worthwhile given variable hypothetical response rates for antidepressants compared with the response rate for no treatment (ie, 30%) after 2 months. We asked them to weigh the benefits and burdens, then decide if they would accept the drug. Based on their yes/no response, we next asked the participants about lower or higher response rates, respectively. To eschew a default to heuristics and laziness, we randomly assigned participants to two different response algorithms, requiring different attention and concentration levels (online supplemental 4). As we observed no difference in SWD by algorithm, in the following we report the combined results. Then, the difference between 30% and the minimum antidepressant response the participant would consider taking defines the individual’s SWD.

    Demographic and clinical variables

    We collected demographic information including gender, race, education, employment, country of residence and insurance status. Insurance was categorised as Affordable Care Act (USA only), Medicare/Medicaid (USA only), national healthcare insurance, private health insurance, other and uninsured.

    Clinical variables included lifetime depression prevalence, family history of depression, lifetime antidepressant treatment, lifetime psychotherapy, current antidepressant treatment, current psychotherapy, treatment preference (antidepressants vs psychotherapy) and current depression symptoms. Current depression symptom severity was assessed with the Patient Health Questionnaire-9 (PHQ-9).

    Participants

    We included participants aged 18 or older, residing in the UK or USA, who were fluent in English. We were interested in a general population, but we were most concerned with people experiencing depressive symptoms of at least moderate severity (PHQ-9≥10) but not currently engaged in any treatment. This group most closely resembles potential treatment seekers, and we can expect them to provide more accurate estimates of the SWD for a major depressive episode as depicted in the provided clinical scenario because of their current experiences and potential treatment needs. This group would represent the best estimate of a clinical sample taken from a general internet population. Thus, to explore how different experiences with depression and treatment could be associated with treatment-seeking judgements and SWD estimates, we included participants with four differing profiles, (1) moderate-to-severe depressive symptoms but not in treatment: PHQ-9≥10 and not receiving any treatment, the primary interest group for SWD estimation, (2) currently in treatment: ongoing antidepressant therapy or psychotherapy, (3) absent-to-mild depressive symptoms with treatment experiences: zero-to-mild depression symptoms (PHQ-9<10), no current treatment, but previous antidepressant treatment and (4) absent-to-mild depressive symptoms without treatment experiences: zero-to-mild depression symptoms and no current or previous antidepressant treatment.

    Sample size

    We set the sample size to achieve the expected precision in the estimate of the SWD. We assumed that the SD of the SWD would resemble that estimated in a pain study (SD=22 for SWDs of 20%)17 because it is an SWD study investigating a subjective outcome, and no depression SWD study exists from which to base a power analysis. To obtain a 95% CI within 10 percentage points, we needed approximately 80 participants with at least moderate symptoms but not in treatment. We also anticipated similar precision for the three other groups as well. Estimates of depression incidence in RPCSs vary but are demonstrably higher than in the general population.19 We therefore assumed approximately 20% of participants would present with moderate-to-severe depressive symptoms on the PHQ-9. Depending on these subgroup populations, approximately 800 participants may be necessary to reach n=80 in the groups with the smallest populations (online supplemental 5). Recruitment was stopped after all four groups included 80 participants. We accepted responses with no missing outcome variables.

    Data analysis

    We first presented the SWD with its distribution and percentile ranking for people with moderate-to-severe depressive symptoms but not in treatment and estimated its median and IQR. We then examined the SWD by participant groups for comparison using box-violin plots. Finally, we analysed the entire sample’s SWD using demographic and clinical independent variables in 11 univariable regressions and a single multivariable regression investigating SWD predictors using the least absolute shrinkage and selection operator (LASSO) method.25 Participants who reported they would not accept antidepressant treatment, even if the response were 100%, were removed from the primary analysis because they would not be real-world candidates for antidepressant treatment.12 They may be philosophically opposed to taking mood-altering drugs, or any drugs. To examine the effect of this decision, we conducted sensitivity analyses by assigning them an SWD=71, which is an impossible value representing an antidepressant response rate over 100%. We used SAS V.9.4 (Cary, North Carolina, USA, SAS Institute) and R V.4.2.2 (R Core Team, 2022) for all statistical analyses.

    Patient and public involvement

    We piloted the BHTM script with two members of the patient and public involvement (PPI) group at the Oxford Precision Psychiatry Lab of the University of Oxford. PPI members (SM and RE) reviewed descriptions of a DSM-5 major depressive episode, and the benefits and burdens of antidepressant treatments. These experts by experience provided feedback on clarity, inclusivity and accuracy of patient experiences, and we modified the scripts accordingly. They further collaborated in the interpretation and write-up of the manuscript.

    Findings

    Participants

    The total sample included 935 participants from three RPCSs: MTurk (n=255), Prolific (n=395) and MQ (n=285). Participants had a mean age of 44.1 (SD=13.9), were mostly women (66%) and Caucasian (84%). Table 1 lists the demographic and clinical characteristics for those with moderate-to-severe symptoms but not in treatment and for the full sample.

    View this table:
    Table 1

    Demographic and clinical characteristics of participants

    Ninety-five participants (10.2%; n=20 with moderate-to-severe symptoms but not in treatment, n=20 currently in treatment, n=14 with absent-to-mild symptoms with treatment experiences and n=41 with absent-to-mild symptoms without treatment experiences) reported that they would not consider taking antidepressants even if these drugs achieved 100% response.

    SWD for people with moderate-to-severe depressive symptoms but not in treatment

    Figure 1 shows the distribution and percentile rank of the SWD as reported by participants with moderate-to-severe symptoms but not in treatment. The median was 20% (IQR=10–30%, n=104).

    Figure 1
    Figure 1

    Frequency distribution of the smallest worthwhile difference (SWD) for people with moderate-to-severe depressive symptoms but not in treatment (n=104). The top row shows the cumulative percentiles of the distribution. The stripped bar represents the median SWD.

    SWD for participant groups according to depression symptoms and treatment experience

    Table 2 shows the average and dispersion data of the SWD for the four groups and the full sample. Distributions of individual estimates overlapped to a great degree across the groups, but dispersion varied considerably (figure 2). The participants currently in treatment comprised the largest proportion of the sample (44%) and showed the largest dispersion.

    View this table:
    Table 2

    The SWD for participant groups according to depression symptoms and treatment experience

    Figure 2
    Figure 2

    Box-violin plots of the SWD distributions for four participant groups. Boxes represent the second and third quartiles with the median line in the middle. Whiskers represent the range. Violins represent the frequency distribution, enhanced with raw data points. The sample used to estimate the figure does not include 95 participants who reported they would not take antidepressants for depression even if response were 100%. SWD, smallest worthwhile difference.

    To further investigate potential correlates of the SWD, we conducted univariable regressions and a single multivariable linear regression between all baseline covariates and the SWD (online supplemental 6). LASSO indicated that only treatment preference could be an important predictor of the SWD. Participants who preferred antidepressants at outset reported a lower SWD (median=20%, IQR=10–35%) than those who preferred psychotherapy (median=25%, IQR=15–35%).

    As a sensitivity analysis, we assigned an SWD of 71% to participants unwilling to ever accept antidepressants, which increased the median SWD for the people with moderate-to-severe symptoms but not in treatment (median=25%, IQR=15–42.5%, n=124) and for the entire sample (median=30%, IQR=15–40%, n=935).

    Discussion

    We used the BHTM to estimate the SWD for initiation of antidepressant treatment for depression in an online cross-sectional survey. The median SWD among participants with moderate-to-severe depressive symptoms but not in treatment was an additional 20 percentage points (IQR: 10–30%) over the assumed natural response rate of 30% for no treatment. Other groups showed a median of 25% with considerable dispersion: People currently in treatment showed the largest variability (IQR=10–40%), and people with absent-to-mild symptoms without treatment experiences showed the smallest variability (IQR=20–30%). About 10% of the participants reported they would not take antidepressants even if these drugs achieved 100% response. Treatment preference, either for drug therapy or for psychotherapy, was the only important predictor of the SWD (median of 20% vs 25%, respectively). These findings, and their implications for the relevance with existing antidepressants, should be interpreted considering the natural response rate of no treatment (assumed to be 30% in our study) and the average greater response rate of antidepressants (assumed to be about 45% according to the literature, see below).

    We queried the expected difference in antidepressants response rates versus the no treatment natural course because placebo is not a real-world treatment alternative. However, there are a very limited number of studies examining natural response rates among depressed seeking-treatment patients. The survival curve of 393 incident depressive episodes observed in a naturalistic community cohort study suggests a response rate between 10% and 40% after 2 months.23 Compared with placebo, a systematic review comprising 252 randomized controlled trials (RCTs) showed the average response rate was 37%.22 Some evidence suggests that placebo produces greater response than no treatment, due to the placebo effect. For example, the OR for pill placebo response over no treatment was 1.38 (0.75–2.55) in a network meta-analysis in depression psychotherapy trials.26 One large pragmatic trial compared watchful waiting versus antidepressants among primary care patients and found watchful waiting response rate to be 29%.24 Taken together, we conservatively set the no treatment response rate to be 30%.

    A network meta-analysis of 522 randomised trials of antidepressants showed that the response ORs for SSRIs and SNRIs over placebo ranged between 1.5 and 1.9.3 Assuming placebo response rates of 30 or 40%, these ORs would translate into antidepressants response rates of 39–45% or 50–56%, constituting risk differences between 9% and 16%. However, it is generally believed that more responsive patients are selectively enrolled in RCTs, and antidepressant real-world effectiveness may be smaller than these 9–16% risk differences. Alternatively, a greater probability of receiving placebo in RCTs is associated with lesser response rates for the same antidepressant, with an observed response rate difference up to 10%.27 In practice we have no placebo condition, and the antidepressant response rate may be somewhat higher than the above estimations. A recent individual participant data meta-analysis of 73 388 patients from 232 placebo controlled RCTs reported an overall 15% risk difference for antidepressants over placebo when individual response patterns were considered.28 Everything considered, we assume 15% as the realistic response rate difference between the currently most efficacious antidepressants and no treatment/natural course in the following discussion.

    The estimated SWD of 20% (IQR: 10–30%) for people with moderate-to-severe symptoms but not in treatment, or 25% (IQR: 10–35%) for the entire sample, are greater than the 15% greater response rate to be expected on currently available best antidepressant drugs. However, there was wide variability in individual SWDs, both for the participants with moderate-to-severe symptoms but not in treatment and for the entire sample. Approximately one in three (40/104, or 276/840) would be willing to take antidepressants for a depressive episode at the currently expected response rate (ie, 15 percentage points greater than no treatment), in exchange for the potential burdens. Another one-third would need double the current antidepressant effect (ie, 30%) before they initiate antidepressant treatment. The remaining third would need to see greater response rates or fewer burdens.

    To explore potential sources of variability in the SWD values, we examined all demographic and clinical variables. Only treatment preference demonstrated a robust association. Preconceived notions about antidepressants can affect confidence in therapies and motivation to seek psychiatric treatment. Depressed but not yet treated patients would likely be seeking treatment in the real world. Our findings suggest that these people may show the smallest average SWD, but the group SWD distributions largely overlapped. The fact that the SWD estimates did not substantively differ among those with or without clinical needs, and with or without lived experiences, corroborate the appropriateness of our method co-produced by people with lived experiences. The observed substantial variability in the participants’ requirements to accept antidepressants, coupled with the increasing prescription rates,22 highlights the need for an explanation of the high SWD observed in the current study. We can only speculate, but perhaps there is a lack of understanding from patients and a lack of communication from doctors (understating burdens or overstating efficacy) that factor into hasty decision-making in prescription acceptance. Much more research is needed in this area about actual transactions in the real world and their appropriateness.

    The SWD may provide a benchmark for efficacy to be expected for future antidepressant drugs over placebo. The median SWD of a 20–25 percentage points greater response rate than the 30% for no treatment would correspond with an OR between 2.3 and 2.9 and an SMD between 0.47 and 0.5829 (online supplemental 7 for calculations). Unfortunately, a systematic review of currently approved general medicine and psychiatric drugs shows that only a minority of drugs have evidence for this magnitude of efficacy.30 It should also be noted that this SMD of 0.5–0.6 would translate into a mean difference of 3–4 points on the Hamilton Rating Scale for Depression, because the average SD of this scale is around 7 points.3 The MIC of the same rating scale is reported to be 7–8,14 and is therefore much larger than the SWD. Thus, the MIC is both conceptually and pragmatically distinct from the SWD. We must use the SWD, not the MIC, as a benchmark to evaluate the observed group differences in RCTs and to estimate the sample size of future trials.

    Limitations

    There were some noteworthy limitations. Foremost is our use of an RPCS sample, which may not represent real-world populations.19 While the SWD from a clinical sample is important and may potentially produce a different SWD, evaluation of a general population serves as foundation for further research in the perception of people with lived experiences. Our study focused on people observed in an RPCS sample with moderate-to-severe depression symptoms and represents a group who may or may not access psychiatric services. Participants recruited through RPCSs tend to be younger, more educated and report greater psychopathology, when compared with the general population.20 However, RPCSs produce greater generalisability than convenience samples.20 We balanced our population by including two RPCSs based in two different countries and we also sampled from MQ Mental Health, a UK-based charity specifically aimed at improving mental health outcomes research. By sampling the RPCSs with performance restrictions, we were able to focus on those with more pressing clinical needs and evaluate other groups with potentially different needs.20 The fact that the two different sequences of questions in the BHTM method yielded the same estimates attests to the attentiveness and trustworthiness of our internet collaborators.

    Second, we excluded participants who replied that they would not take antidepressants even if antidepressant response rates were 100%, while the expected natural response rate was 30%. We followed established methods citing they would not be candidates for treatment in clinical practice.12 A sensitivity analysis including them with an improbable SWD, reflecting greater than 100% response, raised the estimate by 5 percentage points. Third, participants with moderate-to-severe symptoms but not in treatment were not necessarily diagnosed with major depression. We asked them to estimate the SWD for people with diagnosed depression as depicted in the clinical scenario. We assumed that while not diagnosed, they might provide personally invested SWD estimates because of their current depressive symptoms and potential treatment needs. Finally, systematic differences in depression treatment burdens must be considered. Policies differ between national healthcare systems, and between individuals within one country, based on insurance status and access to mental health services. The indirect costs (eg, loss in productivity) due to depression may also differ between individuals, and between societies. To evaluate these differences, we conducted exploratory analyses and found no appreciable association between the SWD and individual variables such as age, sex, race, nationality or insurance status.

    Clinical implications

    One-in-three of those who might consider antidepressant treatment and experience at least moderate depression symptoms would find the currently available antidepressants worthwhile, in exchange for the expected burdens of treatment including side effects, expenses and other inconveniences. Two-in-three require greater response rates or fewer burdens. There was great variability in individual SWD estimates but this variability could not be explained by demographic or clinical variables. Thus, while a minority may be satisfied with the best currently available antidepressants, more effective and/or less burdensome medications are needed. This is the first evaluation of the effectiveness that patients require for initiating antidepressant treatment. Greater value must be placed on the patient perspective for antidepressants in the treatment of depression. This information can help clinicians and researchers to understand patient expectations of therapies, evaluate the worthiness of antidepressants and establish evidence-based benchmarks for all medical research.

    Data availability statement

    Data are available upon reasonable request. Data collected for this study and the code used to analyse these data is available upon request from the corresponding author, for purposes of reproducing the results. The lead author affirms that the manuscript is an honest, accurate and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as originally planned have been explained.

    Ethics statements

    Patient consent for publication

    Ethics approval

    The present study was approved by the Kyoto University Graduate School of Medicine Ethics Committee (R3574), and all participants provided e-consent. Participants gave informed consent to participate in the study before taking part.

    Acknowledgments

    ACi is supported by the National Institute for Health Research (NIHR) Oxford Cognitive Health Clinical Research Facility, by an NIHR Research Professorship (grant RP-2017-08-ST2-006), by the NIHR Oxford and Thames Valley Applied Research Collaboration and by the NIHR Oxford Health Biomedical Research Centre (grant BRC-1215-20005). The views expressed are those of the authors and not necessarily those of the UK National Health Service, the NIHR or the UK Department of Health. MLF holds a National Health and Medical Research Council of Australia Investigator Fellowship.

    References

    1. ↵
      1. GBD 2019 Mental Disorders Collaborators
      . Mental disorders collaborators. global, regional, and national burden of 12 mental disorders in 204 countries and territories, 1990–2019: a systematic analysis for the global burden of disease study 2019. Lancet Psychiatry 2022;9:137–50. doi:10.1016/S2215-0366(21)00395-3
      OpenUrl
    2. ↵
      2. Herrman H ,
      3. Patel V ,
      4. Kieling C , et al
      . Time for United action on depression: A lancet-world psychiatric Association Commission. The Lancet 2022;399:957–1022. doi:10.1016/S0140-6736(21)02141-3
      OpenUrlPubMed
    3. ↵
      2. Cipriani A ,
      3. Furukawa TA ,
      4. Salanti G , et al
      . Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis. The Lancet 2018;391:1357–66. doi:10.1016/S0140-6736(17)32802-7
      OpenUrl
    4. ↵
      2. Cuijpers P ,
      3. Noma H ,
      4. Karyotaki E , et al
      . A network meta-analysis of the effects of Psychotherapies, Pharmacotherapies and their combination in the treatment of adult depression. World Psychiatry 2020;19:92–107. doi:10.1002/wps.20701
      OpenUrlPubMed
    5. ↵
      2. Furukawa TA ,
      3. Scott I ,
      4. Guyatt G , et al
      . Measuring patients’ experience. In: Guyatt G , Rennie D , Meade M , eds. Users’ guides to the medical literature: a manual for evidence-based clinical practice. New York: The McGraw-Hill Companies, Inc, 2014: 219–34.
    6. ↵
      2. Hengartner MP ,
      3. Plöderl M
      . Statistically significant antidepressant-placebo differences on subjective symptom-rating scales do not prove that the drugs work: effect size and method bias matter Front Psychiatry 2018;9:517. doi:10.3389/fpsyt.2018.00517
    7. ↵
      2. Moncrieff J
      . What does the latest meta-analysis really tell us about antidepressants Epidemiol Psychiatr Sci 2018;27:430–2. doi:10.1017/S2045796018000240
      OpenUrl
    8. ↵
      2. Jaeschke R ,
      3. Singer J ,
      4. Guyatt GH
      . Measurement of health status ascertaining the minimal clinically important difference. Control Clin Trials 1989;10:407–15. doi:10.1016/0197-2456(89)90005-6
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      2. Carrasco-Labra A ,
      3. Devji T ,
      4. Qasim A , et al
      . Minimal important difference estimates for patient-reported outcomes: A systematic survey. J Clin Epidemiol 2021;133:61–71. doi:10.1016/j.jclinepi.2020.11.024
      OpenUrlCrossRefPubMed
    10. ↵
      2. Ferreira ML ,
      3. Herbert RD ,
      4. Ferreira PH , et al
      . A critical review of methods used to determine the smallest worthwhile effect of interventions for low back pain. J Clin Epidemiol 2012;65:253–61. doi:10.1016/j.jclinepi.2011.06.018
      OpenUrlCrossRefPubMed
    11. ↵
      2. Devji T ,
      3. Carrasco-Labra A ,
      4. Qasim A , et al
      . Evaluating the credibility of anchor based estimates of minimal important differences for patient reported outcomes: instrument development and reliability study. BMJ 2020;369:m1714. doi:10.1136/bmj.m1714
    12. ↵
      2. McNamara RJ ,
      3. Elkins MR ,
      4. Ferreira ML , et al
      . Smallest worthwhile effect of land-based and water-based pulmonary rehabilitation for COPD. ERJ Open Res 2015;1:00007-2015. doi:10.1183/23120541.00007-2015
    13. ↵
      2. Hiroe T ,
      3. Kojima M ,
      4. Yamamoto I , et al
      . Gradations of clinical severity and sensitivity to change assessed with the Beck depression inventory-II in Japanese patients with depression. Psychiatry Res 2005;135:229–35. doi:10.1016/j.psychres.2004.03.014
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      2. Leucht S ,
      3. Fennema H ,
      4. Engel R , et al
      . What does the HAMD mean? J Affect Disord 2013;148:243–8. doi:10.1016/j.jad.2012.12.001
      OpenUrlCrossRefPubMed
    15. ↵
      2. Franco MR ,
      3. Howard K ,
      4. Sherrington C , et al
      . Smallest worthwhile effect of exercise programs to prevent falls among older people: estimates from benefit-harm trade-off and discrete choice methods. Age Ageing 2016;45:806–12. doi:10.1093/ageing/afw110
      OpenUrlCrossRefPubMed
    16. ↵
      2. Barrett B ,
      3. Brown D ,
      4. Mundt M , et al
      . Sufficiently important difference: expanding the framework of clinical significance. Med Decis Making 2005;25:250–61. doi:10.1177/0272989X05276863
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      2. Ferreira ML ,
      3. Herbert RD ,
      4. Ferreira PH , et al
      . The smallest worthwhile effect of nonsteroidal anti-inflammatory drugs and Physiotherapy for chronic low back pain: a benefit-harm trade-off study. J Clin Epidemiol 2013;66:1397–404. doi:10.1016/j.jclinepi.2013.02.018
      OpenUrlCrossRefPubMed
    18. ↵
      2. Kendrick T ,
      3. Collinson S
      . Antidepressants and the serotonin hypothesis of depression. BMJ 2022;378:1993. doi:10.1136/bmj.o1993 Available: https://doi.org/10.1136/bmj.o1993
    19. ↵
      2. Chandler J ,
      3. Shapiro D
      . Conducting clinical research using Crowdsourced convenience samples. Annu Rev Clin Psychol 2016;12:53–81. doi:10.1146/annurev-clinpsy-021815-093623
      OpenUrlPubMed
    20. ↵
      2. Mortensen K ,
      3. Hughes TL
      . Comparing Amazon’s mechanical Turk platform to conventional data collection methods in the health and medical research literature. J Gen Intern Med 2018;33:533–8. doi:10.1007/s11606-017-4246-0
      OpenUrl
    21. ↵
      2. Luo Y ,
      3. Kataoka Y ,
      4. Ostinelli EG , et al
      . National prescription patterns of antidepressants in the treatment of adults with major depression in the US between 1996 and 2015: A population representative survey based analysis. Front Psychiatry 2020;11:35. doi:10.3389/fpsyt.2020.00035
    22. ↵
      2. Furukawa TA ,
      3. Cipriani A ,
      4. Atkinson LZ , et al
      . Placebo response rates in antidepressant trials: a systematic review of published and unpublished double-blind randomised controlled studies. Lancet Psychiatry 2016;3:1059–66. doi:10.1016/S2215-0366(16)30307-8
      OpenUrl
    23. ↵
      2. Ten Have M ,
      3. Penninx BWJH ,
      4. Tuithof M , et al
      . Duration of major and minor depressive episodes and associated risk indicators in a psychiatric Epidemiological cohort study of the general population. Acta Psychiatr Scand 2017;136:300–12. doi:10.1111/acps.12753
      OpenUrlPubMed
    24. ↵
      2. Kendrick T ,
      3. Chatwin J ,
      4. Dowrick C , et al
      . n.d. Randomised controlled trial to determine the clinical effectiveness and cost-effectiveness of selective serotonin reuptake inhibitors plus supportive care, versus supportive care alone, for mild to moderate depression with somatic symptoms in primary care. Health Technol Assess;13. doi:10.3310/hta13220
    25. ↵
      2. Tibshirani R
      . Regression shrinkage and selection via the lasso. J Royal Statist Soc: Series B (Methodological) 1996;58:267–88. doi:10.1111/j.2517-6161.1996.tb02080.x Available: https://rss.onlinelibrary.wiley.com/toc/25176161/58/1
      OpenUrl
    26. ↵
      2. Michopoulos I ,
      3. Furukawa TA ,
      4. Noma H , et al
      . Different control conditions can produce different effect estimates in psychotherapy trials for depression. J Clin Epidemiol 2021;132:59–70. doi:10.1016/j.jclinepi.2020.12.012
      OpenUrlPubMed
    27. ↵
      2. Salanti G ,
      3. Chaimani A ,
      4. Furukawa TA , et al
      . Impact of placebo arms on outcomes in antidepressant trials: systematic review and meta-regression analysis. Int J Epidemiol 2018;47:1454–64. doi:10.1093/ije/dyy076
      OpenUrlPubMed
    28. ↵
      2. Stone MB ,
      3. Yaseen ZS ,
      4. Miller BJ , et al
      . Response to acute monotherapy for major depressive disorder in randomized, placebo controlled trials submitted to the US food and Drug Administration: individual participant data analysis. BMJ 2022:e067606. doi:10.1136/bmj-2021-067606
    29. ↵
      2. Furukawa TA
      . From effect size into number needed to treat. Lancet 1999;353:1680. doi:10.1016/S0140-6736(99)01163-0
      OpenUrlPubMed
    30. ↵
      2. Leucht S ,
      3. Hierl S ,
      4. Kissling W , et al
      . Putting the efficacy of psychiatric and general medicine medication into perspective: review of meta-analyses. Br J Psychiatry 2012;200:97–106. doi:10.1192/bjp.bp.111.096594
      OpenUrlAbstract/FREE Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • ES and TAF are joint first authors.

    • Twitter @Toshi_FRKW, @ChevanceAstrid, @DrSMarkham, @And_Cipriani

    • ES and TAF contributed equally.

    • Contributors ES, TAF and GS conceived of the study. ES, TAF, YL, MLF, SL and GS designed the study. ES, TAF, YL, MLF, KO, SL, ACh and GS contributed to protocol development. ES collected the data. ES and TAF analysed the data. ES, TAF and YL wrote the first draft of the manuscript. All authors interpreted the data and contributed to the writing of the final version of the manuscript. All authors agreed with the results and conclusions of this article. ES is the guarantor. The corresponding authors attest that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

    • Funding Intramural research support from Kyoto University Graduate School of Medicine.

    • Competing interests TAF reports personal fees from Boehringer-Ingelheim, DT Axis, Kyoto University Original, Shionogi, SONY and UpToDate, and a grant from Shionogi, outside the submitted work; In addition, TAF has patents 2020-548587 and 2022-082495 pending, and intellectual properties for Kokoro-app licensed to Mitsubishi-Tanabe. ACh has received research, educational and consultancy fees from INCiPiT (Italian Network for Paediatric Trials), CARIPLO Foundation, Lundbeck and Angelini Pharma. He is the CI/PI of two trials about seltorexant in depression, sponsored by Janssen. In the last 3 years SL has received honoraria as a consultant and/or advisor and/or for lectures and/or for educational material from Alkermes, Angelini, Eisai, Gedeon Richter, Janssen, Lundbeck, Medichem, Medscape, Merck Sharpp and Dome, Mitsubishi, Neurotorium, NovoNordisk, Otsuka, Recordati, Roche, Rovi, Sanofi Aventis, TEVA. All the other authors declared no conflict of interest.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.