Adaptation of In-Person Neuropsychological Assessments for Remote Administration

Reference: Joffe Y, Liu J, Arias F, et al. Adaptation, calibration, and validation of a cognitive assessment battery for telephone and video administration. J Am Geriatr Soc. 2024; 1-10. doi:10.1111/jgs.19275

During the COVID-19 pandemic, the SAGES study needed to rapidly develop a method for remote administration of in-person neuropsychological testing. The development of the process and its validation is now published in the citation above.  This process may hold broader applicability across many settings where in-person assessments may not be feasible or possible. These might include other circumstances of contact isolation; patients who live in remote rural settings which might be inaccessible; specialized units where access may be restricted (e.g., locked units, rehabilitation programs, incarceration settings); or simply feasibility constraints (e.g., large-scale epidemiologic studies with limited staffing, patients who move out of state during their follow-up). The practical approaches for adaptation detailed below may help with the generalizability of these well-validated approaches.

The Adaptation Approach

Neuropsychological testing is intended to follow standardized administration in a face-to-face setting to ensure validity of results. Previously, remote adaptation of neuropsychological test batteries were limited, usually on a small scale, and designed for specialized uses.  While remote testing is generally not recommended in clinical settings1, some studies have shown that remote or distance neuropsychological testing in research settings can be feasible and valid compared to standard assessments2-5. In our own longitudinal cohort study, the Successful Aging after Elective Surgery (SAGES) study, we developed a telephone assessment6 that was a partial remote adaptation, intended for only occasional use when a participant was out of state (either temporarily or permanently).

However, there are a number of circumstances when human contact with patients is not possible, such as patients under contact isolation (e.g., tuberculosis), or patients who live in rural areas that may be difficult to reach (e.g., inaccessible or underserved areas).  Other specialized settings where access may be restricted (e.g., locked dementia units; nursing homes; drug rehabilitation programs; incarceration) or environmental situations where access may not be possible (e.g., natural disasters, climate crises) can prompt a need for remote neuropsychological testing.

With the COVID-19 pandemic, it became necessary to abruptly transition to a completely remote approach for all study participants in order for our large-scale epidemiologic study to remain viable.  We quickly adapted our full assessment battery to be administered via video conference and telephone.  We also developed and tested a contact-free delivery, set up, administration, and pickup of equipment by trained staff.  All of these steps were carefully tested with trial and error, first with our staff, and then with patients in the field. Detailed protocols were developed for each step of the remote process. Patient and staff safety and infection control were the priorities for this adaptation.  Please note that while our specific approach described below required hands-free delivery and pickup of a tablet computer, videoconference camera, and equipment, this can also be done via mail with advanced planning.

Below we have outlined the procedures we developed and used for adaptation to videoconference testing, which we hope may be helpful for future studies.

  • Equipment and guidelines for remote videoconferencing
    • Equipment: After testing a number of videoconference options, SAGES decided to use iPads (Apple Inc, Cupertino, CA) mounted on a 360-degree rotating tablet stand (Suptek YF208D, Hong Kong). Attaching the iPad to the tablet stand allowed the interviewer to view what the participant was doing for the writing activities. SAGES also provided chargers, headphone, and hotspots. The hotspots were provided to ensure WiFi access, as some participants did not have reliable internet or did not know their internet password.
    • Guidelines for preparation and set-up of a video-based assessment used by SAGES: In contrast to other protocols that required administration outside of patient homes, used protocol-specific websites or omitted tasks that involve pen and paper activities7-9 , the SAGES video protocol was completely touch-free, with all equipment being delivered to the home of the participants. Research assistants used personal protective equipment for all steps and followed a detailed checklist for the contact-less procedures, ensuring safe preparation and delivery of all equipment.
      • Preparation of delivery: Before each scheduled videoconference, a study bag (with a SAGES logo) with all equipment, consent forms, assessment documents, pens, clip board, safety pouch (with wipes, masks, and gloves for the participants to keep) and detailed set-up guides (with step-by step picture-guided instructions) for the videoconference was assembled. Research assistant were instructed to disinfect all equipment and materials before placing them into the delivery bags. They ensured that the iPad was charged, no lingering information was on the iPad from a prior participant, the hotspot was charged, and the iPad automatically recognized the hotspot when the hotspot was turned on. They also created a Zoom meeting password, filled out an info sheet with meeting ID and passcode for the participant, and prepared consent forms on REDCap that could be signed electronically by the participants. To avoid participants reviewing the assessment sheets before the start of the test, the assessment documents were put inside a red closed envelop labeled “Do not open until asked.”
      • Contact free delivery: Before drop off, the research assistant asked participants for the preferred and safest drop-off location (e.g. garage, front or back door). In some cases (e.g., large apartment buildings when study participants felt leaving the bag outside of their door was not safe) the research assistant called the study participant about the impending drop-off and actual delivery so the participants could get the bag right away. To ensure the equipment was received safely by the participant, the RA often stayed at the drop-off location maintaining social distancing until the study participants picked up the equipment.
      • Remote set-up: Written, detailed instructions for each set-up step for all the needed equipment were provided with delivery bag. This included directions for how to unlock the iPad, connect to the internet (via the home internet or the SAGES provided hotspot), find the secure code to access the consent form on REDCap, find the IRB-approved Zoom (Zoom Video Communications, Inc., San Jose, CA) video conferencing software, how mount the iPad on the stand (if not pre-mounted already) and how to change the camera view so the research assistant could view remotely how the participant completed the writing tests (e.g., Trails A and B). The research assistants were trained to make sure that administration was clear and consistent, including the adapted administration of the previous in-person SAGES battery. If technical assistance was needed, the participant would call the research assistant who would guide them by phone through each step of the set-up. Rarely, the research assistant would arrange to enter the participant’s home to help set up the iPad while taking all possible safety precautions. This happened if the participant expressed anxiety about technology or when it was anticipated that self-guided setup would be difficult.
      • Remote administration of the neuropsychological assessment: Before administration, the research assistant made sure that the participant had all materials needed and that they could see and hear each other on the video. To account for lag time during the zoom call, research assistants were trained to allow for more time for participants to respond. The research assistants guided the participants on how to change the camera view (from the front camera view of the participants face to the back camera view of the writing surface/desk) and how to adjust and readjust the iPad stand and angle of the iPad. For verbal tasks the iPad camera was positioned so that the research assistant could see the participant’s face. For example, when administering the digit span test the research assistant asked the participant to look straight into the camera before reciting the numbers, to reduce the risk for distractions and ensuring that the participant would not write down the numbers.  For writing tasks, the iPad camera was positioned so that the research assistant could see the participant hands working on the tasks in real time. For tasks such as Trails A/B, real time corrections could be made if the participant made an error and needed to be redirected. For some tests the research assistant would share images via zoom and would verify the participant could see them well (e.g., Boston naming test or the training part of the Trails A/B).
      • Pick-up after video assessment and post pick-up maintenance: Upon completion of the study visit, the research assistant would pick up all materials, including the completed writing tasks. They verified at the pick-up location that all equipment including chargers were returned. Equipment was thoroughly sanitized and tested before the next scheduled participant. Research assistants used personal protective equipment for all steps.

We hope this procedure might be helpful to future studies where remote adaptation may be needed.  If sites have the luxury of time, then all protocols should be validated against in-person assessment prior to implementation.  In our study, due to COVID-19 restrictions, this was done retrospectively, but we were able to demonstrate high agreement and validity of our approaches10.

Literature Cited

  1. Binng D, Splonskowski M and Jacova C. Distance Assessment for Detecting Cognitive Impairment in Older Adults: A Systematic Review of Psychometric Evidence. Dement Geriatr Cogn Disord 2020; 49: 456-470. 2020/12/09. DOI: 10.1159/000511945.
  2. Lathan C, Wallace AS, Shewbridge R, et al. Cognitive Health Assessment and Establishment of a Virtual Cohort of Dementia Caregivers. Dement Geriatr Cogn Dis Extra 2016; 6: 98-107. 2016/04/22. DOI: 10.1159/000444390.
  3. Rentz DM, Dekhtyar M, Sherman J, et al. The Feasibility of At-Home iPad Cognitive Testing For Use in Clinical Trials. J Prev Alzheimers Dis 2016; 3: 8-12. 2016/03/22. DOI: 10.14283/jpad.2015.78.
  4. Sano M, Egelko S, Ferris S, et al. Pilot study to show the feasibility of a multicenter trial of home-based assessment of people over 75 years old. Alzheimer Dis Assoc Disord 2010; 24: 256-263. 2010/07/02. DOI: 10.1097/WAD.0b013e3181d7109f.
  5. Mielke MM, Machulda MM, Hagen CE, et al. Performance of the CogState computerized battery in the Mayo Clinic Study on Aging. Alzheimers Dement 2015; 11: 1367-1376. 2015/04/11. DOI: 10.1016/j.jalz.2015.01.008.
  6. Bunker L, Hshieh TT, Wong B, et al. The SAGES telephone neuropsychological battery: correlation with in-person measures. Int J Geriatr Psychiatry 2017; 32: 991-999. 2016/08/11. DOI: 10.1002/gps.4558.
  7. Requena-Komuro MC, Jiang J, Dobson L, et al. Remote versus face-to-face neuropsychological testing for dementia research: a comparative study in people with Alzheimer’s disease, frontotemporal dementia and healthy older individuals. BMJ open 2022; 12: e064576. 2022/11/26. DOI: 10.1136/bmjopen-2022-064576.
  8. Yoshida K, Yamaoka Y, Eguchi Y, et al. Remote neuropsychological assessment of elderly Japanese population using the Alzheimer’s Disease Assessment Scale: A validation study. J Telemed Telecare 2020; 26: 482-487. 2019/05/10. DOI: 10.1177/1357633×19845278.
  9. Zeghari R, Guerchouche R, Tran-Duc M, et al. Feasibility Study of an Internet-Based Platform for Tele-Neuropsychological Assessment of Elderly in Remote Areas. Diagnostics (Basel) 2022; 12 2022/04/24. DOI: 10.3390/diagnostics12040925.
  10. Joffe Y, Liu J, Arias F, et al. Adaptation, calibration, and validation of a cognitive assessment battery for telephone and video administration. J Am Geriatr Soc. 2024; 1-10. doi:10.1111/jgs.19275