Aspect-based Sentiment Analysis of Radiology Patient Experience Surveys: A Cohort Study

Institute of Healthcare Improvement considers experience of care as one of the Triple Aims. Radiology is among the highest volume departments in a health system. Despite being a cornerstone of quality, experience of care is poorly understood in Radiology. In radiology, Magnetic Resonance Imaging (MRI) is one of the most powerful and important medical imaging technologies for evaluate certain types of diseases. However, MRIs are among the most challenging imaging studies for patients due to long exams, loud noises during the exam, and the need to stay in a fixed position confined a narrow space for an extended time. Giving patients an opportunity to give feedback of their experiences after radiology visit can provide key patient-centered insights to elevate the experience of care for patients. The combination of simple, on-time electronic open-ended patient questionnaires and aspect-based sentiment analysis natural language processing (NLP) methods have enabled more accurate and prompt depictions of the patient experience, empowering operational and financial improvements in healthcare delivery. In this work, we create a labelled corpus of 146 post-MRI patient experience reviews by performing double-rated annotation, while iteratively developing annotation guidelines and resolving annotation inconsistencies. Sentiment-aspect pair segments were tagged by two expert annotators. Annotation guidelines, including the types of aspects and topics collected, were iteratively constructed to include the range and granularity of aspects for which patient had sentiment. Corpus analysis suggests waiting times, staff interactions and MRI discomfort were the primary sources of patient negative comment, but overall and staff-directed sentiment were positive. Subsequent work will extend the annotated corpus to a comprehensive gold standard dataset capable of distant supervision to supplement the corpus. Once sufficiently large, this corpus could be used to pretrain and train an aspect-based sentiment classification transformer model and deploy it as a real-time patient feedback engine to improve decision-making by health care providers and patient-centered care.