Jump to comment:

• Suicide prediction following self-harm: are new tools any better?
  Leah Quinlivan, Sarah Steeg and Nav Kapur
  Published on: 11 October 2023

• Published on: 11 October 2023

  Suicide prediction following self-harm: are new tools any better?

  • Leah Quinlivan, Research Fellow University of Manchester
  • Other Contributors:
    • Sarah Steeg, Research Fellow
    • Nav Kapur, Professor of Psychiatry and Population Health

  Recent systematic reviews, clinical guidelines, and suicide prevention strategies suggest we should abandon the endeavour of using risk assessment to predict suicide and instead focus on clinical need. (1, 2) Does the recent carefully conducted paper by Fazel and colleagues mean that this advice should be overturned? We think not.

  At the turn of the century several of us were involved in developing risk tools for self-harm. Nearly two decades later, colleagues have used larger samples, novel methodology, and painstaking analysis to produce OxSATs. Of course, interpretation of how good tools are depends on multiple diagnostic accuracy statistics and clinical context. However, one accepted measure of overall performance is the ‘area under the curve’ (AUC).

  What is striking is that the AUC for predicting suicide in the 6 months after self-harm was nearly identical for OxSATs and the first generation scales (0.75 vs 0.71). (3, 4) What is perhaps even more striking is the very different interpretation of the findings. The authors of the recent study suggest OxSATs accurately predicts the risk of suicide whereas Steeg et al. concluded the opposite and suggested scales should not be used to determine treatment.

  What should we make of this discrepancy? In the end it perhaps comes down to what different researchers mean by ‘accurate’. A commonly used measure which may reflect real world utility is the positive predictive value (PPV) - of those rated as at...

  Show More

  Recent systematic reviews, clinical guidelines, and suicide prevention strategies suggest we should abandon the endeavour of using risk assessment to predict suicide and instead focus on clinical need. (1, 2) Does the recent carefully conducted paper by Fazel and colleagues mean that this advice should be overturned? We think not.

  At the turn of the century several of us were involved in developing risk tools for self-harm. Nearly two decades later, colleagues have used larger samples, novel methodology, and painstaking analysis to produce OxSATs. Of course, interpretation of how good tools are depends on multiple diagnostic accuracy statistics and clinical context. However, one accepted measure of overall performance is the ‘area under the curve’ (AUC).

  What is striking is that the AUC for predicting suicide in the 6 months after self-harm was nearly identical for OxSATs and the first generation scales (0.75 vs 0.71). (3, 4) What is perhaps even more striking is the very different interpretation of the findings. The authors of the recent study suggest OxSATs accurately predicts the risk of suicide whereas Steeg et al. concluded the opposite and suggested scales should not be used to determine treatment.

  What should we make of this discrepancy? In the end it perhaps comes down to what different researchers mean by ‘accurate’. A commonly used measure which may reflect real world utility is the positive predictive value (PPV) - of those rated as at high risk, how many go on to have the outcome of interest. In supplemental table 8 of the OxSATS paper the PPV for suicide within six months is 2%. In other words of 100 people rated as at high risk only 2 are truly at high risk. We acknowledge that the PPV is not the only the only way to judge a tool’s performance but we would argue that it is one that matters to clinicians.

  Where should we go from here? Listening to those with lived experience is essential. Service users report that risk scales may detract from therapeutic engagement and lead to exclusion and iatrogenic harms. (5) For many, stigma and poor experiences of care are the more pressing concerns.(6) In the end, there are probably no short-cuts to careful assessment and no easy way to identify those who go on to have adverse outcomes. However, we can agree with Fazel and colleagues on the bigger issue of making high quality, evidence-based interventions available and accessible.

  1. National Institute for Health and Care Excellence. Self-harm: assessment, management and preventing recurrence . NICE Guideline., 2022.
  2. Department of Health and Social Care. Suicide prevention in England: 5-year cross-sector strategy. https://www.gov.uk/government/publications/suicide-prevention-strategy-f..., 2023.
  3. Fazel S, Vazquez-Montes MDLA, Molero Y, et al. Risk of death by suicide following self-harm presentations to healthcare: development and validation of a multivariable clinical prediction rule (OxSATS). BMJ Mental Health 2023;26(1) doi: 10.1136/bmjment-2023-300673
  4. Steeg S, Quinlivan L, Nowland R, et al. Accuracy of risk scales for predicting repeat self-harm and suicide: a multicentre, population-level cohort study using routine clinical data. BMC Psychiatry 2018;18
  5. Graney J, Hunt IM, Quinlivan L, et al. Suicide risk assessment in UK mental health services: a national mixed-methods study. Lancet Psychiatry 2020;7(12):1046-53. doi: 10.1016/s2215-0366(20)30381-3
  6. Quinlivan LM, Gorman L, Littlewood DL, et al. 'Relieved to be seen'-patient and carer experiences of psychosocial assessment in the emergency department following self-harm: qualitative analysis of 102 free-text survey responses. BMJ Open 2021;11(5):e044434. doi: 10.1136/bmjopen-2020-044434 [published Online First: 2021/05/25]

  Show Less

  Conflict of Interest:
  NK leads the suicide work programme of the National Confidential Inquiry into Suicide and Safety. NK and LQ have undertaken quality improvement work for NHS England (NHSE). NK was expert topic advisor to the National Institute for Health and Care Excellence (NICE) self-harm guidelines and is a member of the National Suicide Prevention Strategy advisory group for England. The views of this letter are the personal views of the authors and do not necessarily reflect the views of NHSE, NICE or the Department for Health and Social Care.

  • Back to top