Comparison of Different Classification Models to Predict Mortality Among Patients Diagnosed with Acute Kidney Injury

Introduction

The prevalence of acute kidney injury (AKI) is increasing worldwide but is still underestimated due to the non-referral of patients to hospitals.¹ According to reports, 13.3 million people worldwide suffer AKI every year, with 85% of those living in developing countries. Furthermore, AKI is estimated to be responsible for up to 1.7 million deaths each year.² AKI is often multifactorial, especially in the setting of hospitalization.³ Some of the precipitating factors known to cause AKI are sepsis, urinary tract obstruction, heart failure, liver disease, ischemia, major surgery, myonecrosis, and several nephrotoxins, primarily dominated by sepsis in nearly 50% of the cases.⁴ Due to the limited data available on AKI from India, there is widespread disparity in the reported prognostic factors that lead to adverse outcomes in AKI. Most studies use a binary logistic regression (LR) as the predictive tool to identify adverse outcomes.

Machine learning is actively enhancing healthcare by creating new medical techniques, managing patient data, and improving disease treatments. Physicians have been using predictive models for a long time, and now these predictive models are used in medical decision-making processes.⁵

Regression analysis typically aims to identify the linear relationship between the outcome variable and one or more independent variables. However, in real-world data, the relationships can often be non-linear. To capture these non-linear relationships, machine learning models such as decision trees (DTs), random forests (RFs), and neural networks are employed. Traditional predictive models, like LR, have been widely used in medical literature. However, due to limited data on adverse outcomes of AKI, this study seeks to determine if newer machine learning algorithms, which are now extensively tested in medical specialties, offer any advantages over LR. While existing studies on AKI prediction using machine learning models have significantly contributed to early intervention strategies, such as AKI prediction.^6-9 The current study focuses on predicting the mortality associated with AKI and compares the robustness of DT, RF, artificial neural network (ANN), and LR analysis for predicting mortality in patients with AKI.

Materials and Methods

The study was a retrospective analysis of record-based data. It considers 994 de-identified patient records with AKI undergoing treatment in the Department of Nephrology at a tertiary healthcare center in South India from 2013 to 2021.

Patients with any condition undergo an initial examination. If AKI is suspected during this process, they are referred to the nephrology department. Patients with AKI who also had CKD were not included. AKI staging followed the KDIGO-2012 Guideline,^10, utilizing criteria that include assessments of urine output and serum creatinine levels. Mortality outcomes were recorded during the hospital stay, and the final database included only patients under nephrology care until discharge.

In this study, four classification models, multiple binary logistic regression (MBLR), DT, RF, and ANN, were developed to predict mortality among patients with AKI. During data preprocessing, missing values in the outcome variable (mortality status) for five individuals were excluded, while missing values in the independent variables were retained. Quantitative variables were converted to categorical variables where feasible for analysis in R. Variables for model development were selected through univariate analysis utilizing chi-square and t-tests. Additionally, in DT and RF models, the Gini impurity index and Entropy were calculated to evaluate the importance of variables in predicting mortality among patients with AKI.

Results

The distribution of demographic and clinical parameters is presented in Table 1. The results of the univariate analysis in Table 2 show that sex, AKI type, comorbidities, hemoglobin, major surgery, total platelet count, albumin, and alkaline phosphatase were not associated with mortality (p>0.05). The clinical characteristics such as ICU stay, diabetic status, malignancy status, AKI stage, hypertensive status, vasopressors requirement, usage of a ventilator, dialysis, contrast intake, Alkaline Phosphatase, AKI type, and etiology were found to be significantly associated (p<0.05) with mortality, and no quantitative variables considered in the study were found to be significantly different (p>0.05) with the outcome. Following that, the strength of the association of different variables with mortality was quantified using Cramer’s V index. Among the variables considered in the study, the requirement of vasopressors was found to be strongly associated with mortality, followed by ICU stay, usage of a ventilator, dialysis, contrast intake, hypertensive status, malignancy status, AKI stage, Alkaline Phosphatase, and diabetic status.

The results of the MBLR model are given in Table 3. Among the variables incorporated into the Multiple logistic regression model, only ICU stay, vasopressor requirement, and contrast intake were found to be statistically significant (p<0.05).

The DT was made based on CARTs. The nodes in DT were divided based on the Gini impurity index/entropy/information gain. The details of the impurity measures are shown in Table 4.

Table 4 shows the Gini impurity index, entropy, and information gain values for various variables. Based on the table, the vasopressors requirement has the lowest Gini impurity index (0.346) and entropy (0.773), suggesting it is the most important variable for predicting the outcome. AKI type has the highest Gini impurity index (0.499) and entropy (0.998), indicating it has the least impact on the outcome. The DT structure for training data has been shown in Figure 1.

Close

Figure 1:

Decision tree structure for the training dataset.

Export to PPT

The neural network architecture selection for the current study selected four nodes for the hidden layer and two nodes for the output layer to code the dependent variable mortality.

The network diagram used to predict mortality from the predictors has been shown in Figure 2. The diagram shows the 11 input nodes, four hidden nodes, and two output nodes representing mortality.

Close

Figure 2:

Artificial Neural Network structure.

Export to PPT

For building the RF Model, the Gini coefficient and the mean reduction in accuracy were used to rank the input variables in the outcome prediction model. The number of DTs was set at 1000.

The feature selection results by the RF algorithm are based on the mean decrease Gini. Vasopressor requirements have a higher mean decrease Gini, and the second most important variable is ICU stay, and the variable alkaline phosphate, malignancy status is found to have the least mean decrease Gini.

An error plot in Figure 3 for the RF model shows the Out-of-bag error rate for different classes and out-of-bag samples over varying numbers of trees. It indicates that the lowest error occurs around 100 trees for the given data, with red indicating “Survived” and green indicating “Dead.” Comparison of predictive accuracy, sensitivity, specificity, and AUC of LR, DT, RF, and ANN for both the training dataset and testing dataset have been shown in Table 5.

Close

Figure 3:

Error plot for random forest model.

Export to PPT

Discussion

This study was conducted to compare the predictive accuracy of supervised machine learning models (DT, RF, ANN) with LR for mortality in patients diagnosed with AKI. The findings of the univariate analysis corroborate previous studies indicating that higher serum creatinine levels and the need for vasopressors are key contributors to mortality risk in patients with AKI.^20,21 Also, study shows that using a straightforward yes/no assessment for ICU stay, vasopressor use, and contrast administration predicts mortality in patients with AKI as effectively as complex methods.

Table 5 shows the training dataset performance, with DT and RF models exhibiting notable accuracy (86%) and strong AUC values of 0.87 with CI (0.85-0.89) and 0.86 with CI (0.82,0.89). However, their performance declined in the testing dataset, with accuracy dropping to 77% and 78% respectively, suggesting overfitting. LR and ANN demonstrated stable accuracy and AUC (0.80) across both datasets, and these models showed competitive results in terms of accuracy, AUC, sensitivity, and specificity during training and testing, which highlights their generalizability and reliability in clinical applications.

In clinical practice, predicting the risk of mortality and its associated factors among patients diagnosed with AKI is critical for guiding treatment decisions and improving outcomes. MBLR models estimate this risk based on patient data. DTs make it easy to identify the factors increasing risk, such as needing vasopressors or prolonged ICU stay. RFs improve accuracy by combining multiple DTs, with results showing that using about 200 trees gives the best balance of performance and efficiency. ANNs analyze complex patterns in factors like diabetes, cancer, and AKI stage. Using ReLU activation and a sigmoid output, ANNs provide a probability score for death risk. These models help doctors identify high-risk patients and improve treatment plans.

This study has limitations such as the overfitting observed in DT and RF models, which poses a challenge in applying machine learning to clinical data. Additionally, key biomarkers such as TIMP-2 and IGFBP-7, now considered valuable for AKI prognosis, were not included, as they were not available during the study period (2013–2021). Future studies should integrate these biomarkers to enhance predictive accuracy and clinical decision-making.

This study mainly evaluated accuracy measures (AUC, sensitivity, specificity). Future work should include cross-validation, calibration, and larger multi-center datasets to make the models more reliable, generalizable, and useful for AKI risk stratification. Given the increasing role of machine learning in healthcare, future research should focus on optimizing feature selection, hyperparameter tuning, and ensemble techniques to further refine predictive models.

This study underscores the importance of selecting predictive models that balance accuracy, interpretability, and generalizability. While DT and RF excel in capturing complex patterns in training, their overfitting limits real-world applicability. LR and ANN, with their stable and consistent performance, remain the most reliable models for predicting mortality in patients with AKI.

Future research should focus on expanding datasets, integrating novel biomarkers, and validating models with external datasets. Until then, LR and ANNs remain the most dependable models for predicting AKI mortality, offering both interpretability and predictive accuracy.

In conclusion, this study demonstrates that while DT and RF models showed strong predictive performance in training data, their decline in testing accuracy suggests overfitting, which limits their clinical utility. LR and ANN, on the other hand, maintained consistent and reliable performance, making them more suitable for real-world clinical applications.