Multimodal Intelligent Model Facilitates Early Warning of Severe Acute Pancreatitis - Breakthrough Fusion of Imaging Genomics and Deep Learning

This article aims to construct and verify a multimodal model integrating imaging genomics, 3D deep learning, and clinical structured data for early prediction of severe acute pancreatitis (SAP). The study retrospectively included 609 cases of acute pancreatitis patients from three hospitals as the training set and independent test set, enhancing the model's generalization ability. The innovation lies in: for the first time, combining the imaging genomic features of 3D CT images with the deep learning features extracted by 3D ResNet50 after U-Net segmentation, and fusing the two types of imaging features and clinical indicators (such as SIRS, laboratory tests, etc.) through the XGBoost algorithm to achieve collaborative modeling of multimodal data. The model achieved an AUC of 0.914 in the test set, significantly better than traditional scoring systems (such as MCTSI, BISAP) and single-modal models. At the same time, the study uses variable importance ranking and local interpretable graphs to enhance the model's interpretability, finding pleural effusion, deep learning prediction values, triglycerides, etc., as key predictive factors. Compared with previous studies that mostly used 2D CNN, this study utilizes 3D deep learning to more comprehensively capture lesion spatial information, improving prediction accuracy. Overall, this multimodal model has high accuracy and clinical application potential, providing a new tool for individualized risk assessment of SAP.