OBSERVER PERFORMANCE METHODS FOR DIAGNOSTIC IMAGING: Foundations, Modeling, and Applications with R-Based Examples
Methods, termed Receiver Operating Characteristic (ROC) analyses, exist for objectively evaluating the performance of radiologists based on subjective assessments of likelihood of disease on a set of cases. They are needed to evaluate new imaging advances. Extensive software tools have been developed, but almost all of them only analyze ROC data where the radiologist rates each image for confidence in presence of disease "somewhere in the image". Unlike ROC, free-response ROC (FROC) methodology accounts for correct and incorrect disease localizations. For over three decades the author has conducted pioneering FROC research. He distributes free, open-source, cross-platform R software, which, including an earlier Windows version, have been used in over 107 research publications and in courses and PhD projects worldwide. This book is aimed at users of the author's software who seek better understanding of the methods used to assess imaging systems, radiologists or computer aided detection (CAD) algorithms. Most existing books on the subject are oriented towards statisticians and deal minimally with CAD and the FROC paradigm. Assuming little statistical background, the book covers ROC-FROC analysis from a basic level to recent advances. Fundamental concepts are explained using R-coded examples. The book has an online component (https://www.expertcadanalytics.com), which is continually updated. Key-chapters (#16 - #18) demonstrate a radiological search model-based fitting of 236 ROC curves, yielding important insights into what is limiting radiologist performance. Besides educating the user community, the methods detailed in this book will spur improved methods of designing CAD algorithms and training, evaluating and certifying radiologists.