Preoperative survival prediction method based on enhanced medical images and computing device using thereof

11790,528

17/165369

February 02, 2021

A preoperative survival prediction method and a computing device applying the method include constructing a data seta according to a plurality of enhanced medical images and a resection margin of each enhanced medical image and obtaining a plurality of training data sets from the constructed data set. For each training data set, multi-task prediction models are trained. A target multi-task prediction model is selected from the plurality, and a resection margin prediction value and a survival risk prediction value are obtained by predicting an enhanced medical image to be measured through the target multi-task prediction model. The multi-task prediction model more effectively captures the changes over time of the tumor in multiple stages, so as to enable a joint prediction of a resection margin prediction value and a survival risk prediction value.

Ping An Technology (Shenzhen) Co., Ltd. (Shenzhen, CN)

Ping An Technology (Shenzhen) Co., Ltd. (Shenzhen, CN)

1. A preoperative survival prediction method based on enhanced medical images applicable in a computing device, the method comprising: constructing a data seta according to a plurality of enhanced medical images and a resection margin of each enhanced medical image, and obtaining a plurality of training data sets from the constructed data set; for each training data set, inputting the training data set into a first network structure and a second network structure for training, extracting first feature maps of the training data sets through the first network structure, and extracting second feature maps of the training data sets through the second network structure; obtaining joint feature maps by connecting the first feature maps and the second feature maps, obtaining a resection margin risk loss value by calculating a resection margin risk loss function based on the joint feature maps, and obtaining a survival risk loss value by calculating a survival risk loss function based on the joint feature maps; determining whether the resection margin risk loss value and the survival risk loss value meet their respective loss thresholds; when the resection margin risk loss value and the survival risk loss value both meet their respective loss thresholds, stopping the training of the first network structure and the second network structure, to obtain a plurality of multi-task prediction models; selecting a target multi-task prediction model from the plurality of multi-task prediction models; obtaining a resection margin prediction value and a survival risk prediction value by predicting an enhanced medical image to be measured through the target multi-task prediction model.

2. The preoperative survival prediction method of claim 1 , wherein the method of constructing a data seta according to a plurality of enhanced medical images and a resection margin of each enhanced medical image comprises: obtaining a plurality of first target images by delineating a first target region in each enhanced medical image corresponding to a first phase; obtaining a plurality of second target images by segmenting a second target region in each enhanced medical image corresponding to a second phase; constructing an array by combining one enhanced medical image and the corresponding first target image, the corresponding second target image, and the corresponding resection margin, the data set including a plurality of the arrays.

3. The preoperative survival prediction method of claim 2 , further comprising: defining a first threshold value and a second threshold value greater than the first threshold value; comparing each pixel value in the enhanced medical image with the first threshold value and comparing each pixel value in the enhanced medical image with the second threshold value; updating a pixel value according to the first threshold value, when the pixel value in the enhanced medical image is smaller than the first threshold value; updating a pixel value according to the second threshold value, when the pixel value in the enhanced medical image is greater than the second threshold value; keeping a pixel value unchanged, when the pixel value in the enhanced medical image is greater than the first threshold but less than the second threshold; updating the enhanced medical image according to the updated pixel value.

4. The preoperative survival prediction method of claim 3 , wherein the method of selecting a target multi-task prediction model from the plurality of multi-task prediction models comprises: obtaining a plurality of testing data sets from the constructed data set, each testing data set corresponding to each training data set; obtaining a plurality of testing values by using each testing data set to test the corresponding multi-task prediction model; determining a largest testing value among the plurality of testing values; and determining a multi-task prediction model corresponding to the largest testing value as the target multi-task prediction model.

5. The preoperative survival prediction method of claim 4 , wherein the method of obtaining a plurality of testing values by using each testing data set to test the corresponding multi-task prediction model comprises: calculating a mean value and a variance value of each training data set; standardizing each testing data set according to the mean value and the variance value of the corresponding testing data set; and obtaining the plurality of testing values by using each standardized testing data set to test the corresponding multi-task prediction model.

6. The preoperative survival prediction method of claim 5 , further comprising: obtaining a plurality of resampled enhanced medical images by resampling each enhanced medical image into an isotropic enhanced medical image; and enhancing the plurality of resampled enhanced medical images, comprising rotating the plurality of resampled enhanced medical images according to a pre-rotation angle; or randomly zooming the plurality of resampled enhanced medical images.

7. The preoperative survival prediction method of claim 5 , wherein the first network structure being a 3D-CNN model with six convolutional layers equipped with batch normalization and relu, the second network structure being a CE-ConvLSTM model with a Res Tet model cascaded before, the resection margin risk loss function being a binary cross-entropy loss function, and the survival risk loss function being L(y i)=Σ i δ i (−y i +log Σ j:t j ≥t i exp(y j)), where j is from the set whose survival time is equal or larger than t i (t j ≥t i).

8. A computing device, comprising: at least one processor; and a storage device storing one or more programs which when executed by the at least one processor, causes the at least one processor to: construct a data seta according to a plurality of enhanced medical images and a resection margin of each enhanced medical image, and obtain a plurality of training data sets from the constructed data set; for each training data set, input the training data set into a first network structure and a second network structure for training, extract first feature maps of the training data sets through the first network structure, and extract second feature maps of the training data sets through the second network structure; obtain joint feature maps by connecting the first feature maps and the second feature maps, obtain a resection margin risk loss value by calculating a resection margin risk loss function based on the joint feature maps, and obtain a survival risk loss value by calculating a survival risk loss function based on the joint feature maps; determine whether the resection margin risk loss value and the survival risk loss value meet their respective loss thresholds; when the resection margin risk loss value and the survival risk loss value both meet their respective loss thresholds, stop the training of the first network structure and the second network structure, to obtain a plurality of multi-task prediction models; select a target multi-task prediction model from the plurality of multi-task prediction models; obtain a resection margin prediction value and a survival risk prediction value by predicting an enhanced medical image to be measured through the target multi-task prediction model.

9. The computing device of claim 8 , wherein the method of constructing a data seta according to a plurality of enhanced medical images and a resection margin of each enhanced medical image comprises: obtaining a plurality of first target images by delineating a first target region in each enhanced medical image corresponding to a first phase; obtaining a plurality of second target images by segmenting a second target region in each enhanced medical image corresponding to a second phase; constructing an array by combining one enhanced medical image and the corresponding first target image, the corresponding second target image, and the corresponding resection margin, the data set including a plurality of the arrays.

10. The computing device of claim 9 , the at least one processor further to: define a first threshold value and a second threshold value greater than the first threshold value; compare each pixel value in the enhanced medical image with the first threshold value and comparing each pixel value in the enhanced medical image with the second threshold value; update a pixel value according to the first threshold value, when the pixel value in the enhanced medical image is smaller than the first threshold value; update a pixel value according to the second threshold value, when the pixel value in the enhanced medical image is greater than the second threshold value; keep a pixel value unchanged, when the pixel value in the enhanced medical image is greater than the first threshold but less than the second threshold; update the enhanced medical image according to the updated pixel value.

11. The computing device of claim 10 , wherein the method of selecting a target multi-task prediction model from the plurality of multi-task prediction models comprises: obtaining a plurality of testing data sets from the constructed data set, each testing data set corresponding to each training data set; obtaining a plurality of testing values by using each testing data set to test the corresponding multi-task prediction model; determining a largest testing value among the plurality of testing values; and determining a multi-task prediction model corresponding to the largest testing value as the target multi-task prediction model.

12. The computing device of claim 11 , wherein the method of obtaining a plurality of testing values by using each testing data set to test the corresponding multi-task prediction model comprises: calculating a mean value and a variance value of each training data set; standardizing each testing data set according to the mean value and the variance value of the corresponding testing data set; and obtaining the plurality of testing values by using each standardized testing data set to test the corresponding multi-task prediction model.

13. The computing device of claim 12 , the at least one processor further to: obtain a plurality of resampled enhanced medical images by resampling each enhanced medical image into an isotropic enhanced medical image; and enhance the plurality of resampled enhanced medical images, comprising rotating the plurality of resampled enhanced medical images according to a pre-rotation angle; or randomly zooming the plurality of resampled enhanced medical images.

14. The computing device of claim 12 , wherein the first network structure being a 3D-CNN model with six convolutional layers equipped with batch normalization and relu, the second network structure being a CE-ConvLSTM model with a ResNet model cascaded before, the resection margin risk loss function being a binary cross-entropy loss function, and the survival risk loss function being L(y i)=Σ i δ i (−y i +log Σ j:t j ≥t i exp(y j)), where j is from the set whose survival time is equal or larger than t i (t j ≥t i).

15. A non-transitory storage medium having stored thereon instructions that, when executed by a processor of a computing device, causes the computing device to perform a preoperative survival prediction method based on enhanced medical images, the method comprising: constructing a data seta according to a plurality of enhanced medical images and a resection margin of each enhanced medical image, and obtaining a plurality of training data sets from the constructed data set; for each training data set, inputting the training data set into a first network structure and a second network structure for training, extracting first feature maps of the training data sets through the first network structure, and extracting second feature maps of the training data sets through the second network structure; obtaining joint feature maps by connecting the first feature maps and the second feature maps, obtaining a resection margin risk loss value by calculating a resection margin risk loss function based on the joint feature maps, and obtaining a survival risk loss value by calculating a survival risk loss function based on the joint feature maps; determining whether the resection margin risk loss value and the survival risk loss value meet their respective loss thresholds; when the resection margin risk loss value and the survival risk loss value both meet their respective loss thresholds, stopping the training of the first network structure and the second network structure to obtain a plurality of multi-task prediction models; selecting a target multi-task prediction model from the plurality of multi-task prediction models; obtaining a resection margin prediction value and a survival risk prediction value by predicting an enhanced medical image to be measured through the target multi-task prediction model.

16. The non-transitory storage medium of claim 15 , wherein the method of constructing a data seta according to a plurality of enhanced medical images and a resection margin of each enhanced medical image comprises: obtaining a plurality of first target images by delineating a first target region in each enhanced medical image corresponding to a first phase; obtaining a plurality of second target images by segmenting a second target region in each enhanced medical image corresponding to a second phase; constructing an array by combining one enhanced medical image and the corresponding first target image, the corresponding second target image, and the corresponding resection margin, the data set including a plurality of the arrays.

17. The non-transitory storage medium of claim 16 , further comprising: defining a first threshold value and a second threshold value greater than the first threshold value; comparing each pixel value in the enhanced medical image with the first threshold value and comparing each pixel value in the enhanced medical image with the second threshold value; updating a pixel value according to the first threshold value, when the pixel value in the enhanced medical image is smaller than the first threshold value; updating a pixel value according to the second threshold value, when the pixel value in the enhanced medical image is greater than the second threshold value; keeping a pixel value unchanged, when the pixel value in the enhanced medical image is greater than the first threshold but less than the second threshold; updating the enhanced medical image according to the updated pixel value.

18. The non-transitory storage medium of claim 17 , wherein the method of selecting a target multi-task prediction model from the plurality of multi-task prediction models comprises: obtaining a plurality of testing data sets from the constructed data set, each testing data set corresponding to each training data set; obtaining a plurality of testing values by using each testing data set to test the corresponding multi-task prediction model; determining a largest testing value among the plurality of testing values; and determining a multi-task prediction model corresponding to the largest testing value as the target multi-task prediction model.

19. The non-transitory storage medium of claim 18 , wherein the method of obtaining a plurality of testing values by using each testing data set to test the corresponding multi-task prediction model comprises: calculating a mean value and a variance value of each training data set; standardizing each testing data set according to the mean value and the variance value of the corresponding testing data set; and obtaining the plurality of testing values by using each standardized testing data set to test the corresponding multi-task prediction model.

20. The non-transitory storage medium of claim 18 , further comprising: obtaining a plurality of resampled enhanced medical images by resampling each enhanced medical image into an isotropic enhanced medical image; and enhancing the plurality of resampled enhanced medical images, comprising rotating the plurality of resampled enhanced medical images according to a pre-rotation angle; or randomly zooming the plurality of resampled enhanced medical images.

20190019300 January 2019 Simpson

A. Chaddad, P. Sargos and C. Desrosiers, “Modeling Texture in Deep 3D CNN for Survival Analysis,” in IEEE Journal of Biomedical and Health Informatics, vol. 25, No. 7, pp. 2454-2462, Jul. 2021, doi: 10.1109/JBHI.2020.3025901 (Year: 2021). cited by examiner
J. Jaworek-Korjakowska, et al. “Melanoma Thickness Prediction Based on Convolutional Neural Network With VGG-19 Model Transfer Learning,” 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, CA, USA, 2019, pp. 2748-2756, doi: 10.1109/CVPR (Year: 2019). cited by examiner
Hui, Bei, et al. “Identification of pancreaticoduodenectomy resection for pancreatic head adenocarcinoma: a preliminary study of radiomics.” Computational and mathematical methods in medicine 2020 (2020) (Year: 2020). cited by examiner
Wellner, Ulrich F et al. “Mesopancreatic Stromal Clearance Defines Curative Resection of Pancreatic Head Cancer and Can Be Predicted Preoperatively by Radiologic Parameters: A Retrospective Study.” Medicine vol. 95,3 (2016): e2529. doi:10.1097/MD.0000000000002529 (Year: 2016). cited by examiner
Wang, Xiao-Hang, et al. “MRI-based radiomics model for preoperative prediction of 5-year survival in patients with hepatocellular carcinoma.” British journal of cancer 122.7 (2020): 978-985. (Year: 2020). cited by examiner
Eilaghi, Armin et al. “CT texture features are associated with overall survival in pancreatic ductal adenocarcinoma—a quantitative analysis.” BMC medical imaging vol. 17,1 38. Jun. 19, 2017, doi:10.1186/s12880-017-0209-5 (Year: 2017). cited by examiner
Tadayyon, Hadi, et al. “A priori prediction of neoadjuvant chemotherapy response and survival in breast cancer patients using quantitative ultrasound.” Scientific reports 7.1 (2017): 45733 (Year: 2017). cited by examiner
Attiyeh, Marc A., et al. “Survival prediction in pancreatic ductal adenocarcinoma by quantitative computed tomography image analysis.” Annals of surgical oncology 25 (2018): 1034-1042. (Year: 2018). cited by examiner

edspgr.11790528