The Delirium Interview: a new reference standard for large studies evaluating delirium assessment tools

Posted on by

Contributed by Fienke L. Ditzel, MD; Arjen J. C. Slooter MD, PhD; and Suzanne C.A. Hut, PhD, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands

Rationale

The gold standard for diagnosing delirium is the classification by a delirium expert based on the DSM-5(TR) criteria.1,2 When validating a new delirium assessment tool, ideally, a panel of experts should be engaged to minimize subjectivity and correct for different experience levels.3,4 However, due to the fluctuating nature of delirium, this approach is logistically challenging as experts must examine patients simultaneously, due to the need to mitigate learning effects associated with repeated assessments. Moreover, even when they base their decisions on videos of standardized assessments, delirium experts disagree 21% of the time.3 This NIDUS blog summarizes the results of a new reference method for validating delirium assessment tools that accounts for possible disagreement between experts.5

The Delirium Interview

The Delirium Interview was developed through extensive dialogue with multiple delirium experts from various specialties including geriatrics, psychiatry, neurology, neuropsychology, critical care medicine, and nursing. We sought to ensure this new reference standard adhered to DSM-51,2 criteria for delirium and was able to be used in both verbal and non-verbal patients (e.g. invasively mechanically ventilated) [see Box 1 below].

To achieve this goal, each interview item was presented on an standard-sized paper in a multiple-choice format. Standardized administration of the test was ensured by having the standardized question visible to the researcher only, on the reverse side of each card. When a patient was non-verbal, they indicated their response(s) to each question through a series of nods or blinks as the researcher pointed to each option.

After the interview, the trained researcher wrote a brief narrative capturing their general impression of the patient including patient reaction, eye contact, response speed, motor movement(s), and communicative behaviors. To maintain consistency, specific prompts were provided to guide the standardized description of these patient impressions. Subsequently, the researcher added relevant information from the electronic health record (EHR).

Study design and methods

In this six-center study, we assessed how well experts could diagnose delirium based on information provided by the Delirium interview [the Written Interview Assessor Panel (WIAP)] compared to the current gold standard: live assessment of a patient [i.e., the Live Interview Assessor Panel (LIAP)]. The 10-minute Delirium Interview was conducted by a trained researcher in the presence of two delirium experts (i.e., the LIAP).

After each Delirium Interview, the LIAP was allowed to ask additional questions to the patient and inspect their EHR. The results of each Delirium Interview were then sent by the researcher to three other independent delirium experts (WIAP) (WIAP). The LIAP and WIAP consisted of a pool of 13 delirium experts with an average of 13±8 years of clinical/research experience.

Results

The study cohort included 98 patients of whom 56 (57%) were admitted to the Intensive Care Unit (ICU). Of all ICU patients, 22 (39%) patients had a reduced level of consciousness [Richmond Agitation Sedation Scale6 (RASS) < 0]. Of the total cohort, 26 (27%) patients were non-verbal. Compared to the reference standard (LIAP), the Delirium Interview (WIAP) demonstrated high sensitivity (89%, 95% confidence interval [CI]: 71%–98%) and specificity (82%, 95% CI: 71%–90%).

Interrater reliability was substantial within the LIAP (Kappa = 0.75, 95% CI: 0.63–0.87) and moderate in the WIAP (Kappa = 0.46, 95% CI: 0.35–0.58).7 When all experts were asked to provide a delirium probability rating from 1-10, correlation (as measured by the Intraclass Correlation Score) was excellent within the LIAP (0.90 95% CI: 0.86–0.93) and moderate within the WIAP (0.67, 95% CI: 0.57–0.75).8

Conclusion

The Delirium Interview is a standardized, comprehensive reference method for large-scale studies of delirium assessment tools. By employing a consensus approach through the majority vote of an expert panel, this method effectively resolves the challenges of expert disagreement regarding the presence of a delirium diagnosis. Our method offers a standardized approach across heterogeneous patient groups and demonstrates high sensitivity in delirium diagnosis without the necessity of bedside evaluation.

Box 1. The DSM-5 criteria and the items of the Delirium Interview
A: Disturbance in attention (i.e., reduced ability to direct, focus, sustain, and shift attention) and awarenessRASS,6 OSLA,9 ASE (item 2 of the CAM-ICU)10, MOTYB (item 3 of the 4AT)11
B. The disturbance develops over a short period of time (usually hours to a few days), represents a change from baseline attention and awareness, and tends to fluctuate in severity during the course of a day.Review of EHR 24 hours before- and 12 hours after the delirium assessment.
C. An additional disturbance in cognition (e.g., memory deficit, disorientation, language, visuospatial ability, or perception).Orientation (person, place and time), memory (item 4&5 of the CTD),12 conceptional reasoning (item 7 of the CTD),12 visuospatial ability (inspired by item 13 of the DRS-R-98),13 comprehension (item 8 of the MMSE)14
D. The disturbances in Criteria A and C are not explained by another pre-existing, established, or evolving neurocognitive disorder and do not occur in the context of a severely reduced level of arousal, such as comaTo fulfil this criterion, we adhered to the following exclusion criteria: Acute macro-brain injury within six weeks prior to the measurement, (including post-anoxic encephalopathy and traumatic brain injury), admission because of a primary neurological or neurosurgical disease or pre-existing dementia as documented in the EHR.
E. There is evidence from the history, physical examination, or laboratory findings that the disturbance is a direct physiological consequence of another medical condition, substance intoxication or withdrawal (i.e., due to a drug of abuse or to a medication), or exposure to a toxin, or is due to multiple etiologies.To fulfil this criterion, we adhered to the following inclusion criteria: admission to the ICU or non-ICU department
Additional informationSelf-report experiences of sleep assessment, hallucinations and paranoia

RASS: Richmond Agitation Sedation Scale. OSLA: Observational Scale of Level of Arousal. ASE: Attention Screening Examination. CAM-ICU: Confusion assessment method for the Intensive Care Unit. MOTYB: Months Of The Year Backwards. 4AT: 4 ‘A’s Test. EHR: Electronic Health Record. CTD: Cognitive Test for Delirium. DRS-R-98: Delirium Rating Scale Revised-98 MMSE: The Mini Mental State Examination.

References

  1. American Psychiatric Association. DSM-5 Diagnostic Classification. In: Diagnostic and Statistical Manual of Mental Disorders. Vol 5. American psychiatric association; 2013. doi:10.1176/appi.books.9780890425596.x00diagnosticclassification
  2. First MB, Yousif LH, Clarke DE, Wang PS, Gogtay N, Appelbaum PS. DSM-5-TR: overview of what’s new and what’s changed. World Psychiatry. 2022;21(2):217-218. doi:10.1002/wps.20990
  3. Numan T, van den Boogaard M, Kamper AM, Rood PJT, Peelen LM, Slooter AJC. Recognition of Delirium in Postoperative Elderly Patients: A Multicenter Study. J Am Geriatr Soc. 2017;65(9):1932-1938. doi:10.1111/jgs.14933
  4. van Houten CB, Naaktgeboren CA, Ashkenazi-Hoffnung L, et al. Expert panel diagnosis demonstrated high reproducibility as reference standard in infectious diseases. J Clin Epidemiol. 2019;112:20-27. doi:10.1016/j.jclinepi.2019.03.010
  5. Ditzel FL, Slooter AJC, van den Boogaard M, et al. The Delirium Interview as a new reference standard in studies on delirium assessment tools. JAGS. 2023;71:1923-1930. doi:10.1111/jgs.18263
  6. Sessler CN, Gosnell MS, Grap MJ, et al. The Richmond Agitation-Sedation Scale: validity and reliability in adult intensive care unit patients. Am J Respir Crit Care Med. 2002;166(10):1338-1344. doi:10.1164/rccm.2107138
  7. McHugh ML. Interrater reliability: The kappa statistic. Biochem Med (Zagreb). Published online 2012. doi:10.11613/bm.2012.031
  8. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016;15(2):155-163. doi:10.1016/j.jcm.2016.02.012
  9. Tieges Z, McGrath A, Hall RJ, Maclullich AMJ. Abnormal level of arousal as a predictor of delirium and inattention: An exploratory study. American Journal of Geriatric Psychiatry. 2013;21(12):1244-1253. doi:10.1016/j.jagp.2013.05.003
  10. Ely EW, Inouye SK, Bernard GR, et al. Delirium in Mechanically Ventilated Patients. JAMA: The Journal of the American Medical Association. 2001;286(21):2745-2746. doi:10.1001/jama.286.21.2703
  11. Bellelli G, Morandi A, Davis DHJ, et al. Validation of the 4AT, a new instrument for rapid delirium screening: A study in 234 hospitalised older people. Age Ageing. 2014;43(4):496-502. doi:10.1093/ageing/afu021
  12. Hart RP, Levenson JL, Sessler CN, Best AM, Schwartz SM, Rutherford LE. Validation of a cognitive test for delirium in medical ICU patients. Psychosomatics. 1996;37(6):533-546. doi:10.1016/S0033-3182(96)71517-7
  13. Trzepacz PT, Mittal D, Torres R, Kanary K, Norton J, Jimerson N. Validation of the Delirium Rating Scale-revised-98: Comparison with the Delirium Rating Scale and the Cognitive Test for Delirium. Journal of Neuropsychiatry and Clinical Neurosciences. 2001;13(2):229-242. doi:10.1176/jnp.13.2.229
  14. Folstein M, Anthony JC, Parhad I, Duffy B, Gruenberg EM. The meaning of cognitive impairment in the elderly. J Am Geriatr Soc. 1985;33(4):228-235. doi:10.1111/j.1532-5415.1985.tb07109.x

Suggested Citation

Ditzel, Fienke; Slooter, Arjen; Hut, Suzanne. The Delirium Interview: a new reference standard for large studies evaluating delirium assessment tools; February, 2024, Available at: https://deliriumnetwork.org/the-delirium-interview-a-new-reference-standard-for-large-studies/ (accessed today’s date)

Posted in Delirium Research.

Leave a Reply