Deep learning in Nuclear Medicine—focus on CNN-based approaches for PET/CT and PET/MR: where do we stand?

Introduction In recent years, machine learning algorithms have led to innovative tools for medical imaging analysis. The purpose of the present review was to summarize the literature on the developing field of deep learning (DL), particularly the application of convolutional neural networks (CNNs) in PET/CT and PET/MR. Methods We performed the literature search, referring to "convolutional neural networks" and "positron emission tomography" on PubMed/MEDLINE, for potentially relevant articles published up until July 24th, 2020. Results After the screening process, 63 articles were finally included; these embraced both the technical (n = 23) and the clinical field (n = 40). Technical studies aimed at investigating the role of CNN-based methods for image quality improvement (n = 11) and on technical issues (n = 12), mainly attenuation correction. Clinical studies explored CNN applications in oncology lung cancer (n = 7), head and neck cancer (n = 4), esophageal cancer (n = 2), lymphoma (n = 3), prostate cancer (N = 4), cervical cancer (n = 1), sarcomas (n = 1), multiple cancer types (n = 4), in neurology (n = 10) and cardiology (n = 1); three additional studies belonged to "other" category. In oncology, the studies aimed at detection, diagnosis, and prognostication of cancer. In neurology, the majority of the studies aimed at diagnosing Alzheimer Disease and stratification of the risk. CNN-based algorithms demonstrated promising results with performances equal or even higher compared to conventional approaches. Discussion Overall, CNN applications for PET/CT and PET/MR are exponentially growing, demonstrating encouraging results for both technical and clinical purposes. Novel research strategies emerged to face the challenges of DL algorithms development. Education and confidence with DL-based tools are needed for proper technology implementation.