Sivaramakrishnan Rajaraman, Ghada Zamzmi, Feng Yang, Zhaohui Liang, Zhiyun Xue, Sameer Antani
Abstract
Model initialization techniques are vital for improving the performance and reliability of deep learning models in medical computer vision applications. While much literature exists on non-medical images, the impacts on medical images, particularly chest X-rays (CXRs) are less understood. Addressing this gap, our study explores three deep model initialization techniques: Cold-start, Warm-start, and Shrink and Perturb start, focusing on adult and pediatric populations. We specifically focus on scenarios with periodically arriving data for training, thereby embracing the real-world scenarios of ongoing data influx and the need for model updates. We evaluate these models for generalizability against external adult and pediatric CXR datasets. We also propose novel ensemble methods: F-score-weighted Sequential Least-Squares Quadratic Programming (F-SLSQP) and Attention-Guided Ensembles with Learnable Fuzzy Softmax to aggregate weight parameters from multiple models to capitalize on their collective knowledge and complementary representations.
Introduction
The prowess of Deep learning (DL) has been well established for medical imaging artificial intelligence (AI) applications with automation making way for improved and efficient image acquisition, quality assessment, object detection and tracking, disease screening, diagnostics, and prediction [1]. As a subset of machine learning (ML), DL comprises multilayered neural networks for automated feature extraction and predictions, outperforming traditional techniques in accuracy and robustness.
Materials and method
RSNA-CXR dataset: This publicly available CXR collection results from a collaboration between the RSNA, the Society of Thoracic Radiology (STR), and the National Institutes of Health (NIH) for the Kaggle pneumonia detection challenge [26]. The objective was to help support the design and development of image analysis and ML algorithms through a challenge targeting automatic classification of CXRs as normal, containing non-pneumonia-related, or pneumonia-related opacities.
Results
We first present a comparative analysis between the performances of the Cold-RP and Cold-IP models. Recall that the Cold-RP model initializes the VGG-16 backbone of the VGG-16-M model with random weights and trains it on the RSNA-P dataset. Conversely, the Cold-IP model initializes the VGG-16 backbone of the VGG-16-M model with ImageNet-pretrained weights and also trains it on the RSNA-P dataset.
Discussion
The terms B. Acc., P, R, and F denote balanced accuracy, precision, recall, and F-score, respectively. Bold numerical values denote superior performance in respective columns. Values in parentheses represent the 95% CIs for the MCC metric. The * denotes statistically significant MCC (p<0.00001).
Conclusions
Diverse model initialization techniques are instrumental for deep model optimization thereby affecting convergence speed, reducing the risk of overfitting, and improving generalizability. Our qualitative and quantitative analyses validate the claim that cold-start approaches can decelerate convergence while warm-start methods, such as ImageNet-pretrained weight initialization, enhance convergence and performance. Furthermore, improper weight initialization can introduce biases that inadvertently favor certain classes or feature sets which, in turn, increases the risk of model overfitting to the data and reducing generalizability. To mitigate this risk, we perform ensemble learning and propose novel weight-level ensemble methods to improve performance over individual constituent models. These ensembles can harness a broader range of feature representations, making them more adaptable and effective when handling unseen data. This adaptability is particularly relevant in medical computer vision, where models must demonstrate exceptional generalizability across diverse patient populations and imaging modalities.
Citation: Rajaraman S, Zamzmi G, Yang F, Liang Z, Xue Z, Antani S (2024) Uncovering the effects of model initialization on deep model generalization: A study with adult and pediatric chest X-ray images. PLOS Digit Health 3(1): e0000286. https://doi.org/10.1371/journal.pdig.0000286
Editor: Ulas Bagci, Northwestern University, UNITED STATES
Received: May 30, 2023; Accepted: December 4, 2023; Published: January 17, 2024
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: All data used for this study are in the manuscript and it's supporting information files.
Funding: This study is supported by the Intramural Research Program (IRP) of the National Library of Medicine (NLM) and the National Institutes of Health (NIH).
Competing interests: The authors have declared that no competing interests exist.
Source: https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0000286#sec009
