Outcomes and algorithms | Accuracy | Sensitivity | Specificity | PPV | NPV | F1 | AUC (95%CI) | p-value* | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Train | Test | Train | Test | Train | Test | Train | Test | Train | Test | Train | Test | Train | Test | ||
Sepsis or septic shock | |||||||||||||||
 MLP | 0.789 | 0.790 | 0.828 | 0.791 | 0.750 | 0.789 | 0.768 | 0.636 | 0.814 | 0.890 | 0.797 | 0.705 | 0.854 (0.839–0.869) | 0.852 (0.825–0.880) | 0.910 |
 Random forest | 0.920 | 0.779 | 0.948 | 0.780 | 0.892 | 0.778 | 0.898 | 0.621 | 0.945 | 0.884 | 0.922 | 0.691 | 0.984 (0.980–0.987) | 0.848 (0.821–0.875) |  < 0.001 |
 LightGBM | 0.868 | 0.764 | 0.896 | 0.803 | 0.840 | 0.745 | 0.848 | 0.595 | 0.889 | 0.891 | 0.871 | 0.683 | 0.948 (0.940–0.956) | 0.842 (0.815–0.870) |  < 0.001 |
 SVM | 0.855 | 0.765 | 0.907 | 0.764 | 0.803 | 0.766 | 0.821 | 0.602 | 0.896 | 0.874 | 0.862 | 0.674 | 0.945 (0.937–0.953) | 0.818 (0.787–0.849) |  < 0.001 |
 KNN | 0.800 | 0.738 | 0.889 | 0.744 | 0.711 | 0.734 | 0.755 | 0.566 | 0.865 | 0.861 | 0.817 | 0.643 | 0.890 (0.877–0.902) | 0.816 (0.786–0.846) |  < 0.001 |
 Logistic regression | 0.718 | 0.720 | 0.690 | 0.720 | 0.746 | 0.720 | 0.731 | 0.545 | 0.706 | 0.847 | 0.710 | 0.620 | 0.789 (0.771–0.806) | 0.802 (0.770–0.833) | 0.487 |
ICU admission | |||||||||||||||
 MLP | 0.692 | 0.680 | 0.714 | 0.688 | 0.670 | 0.680 | 0.684 | 0.120 | 0.700 | 0.971 | 0.698 | 0.205 | 0.744 (0.728–0.760) | 0.743 (0.663–0.822) | 0.973 |
 LightGBM | 0.960 | 0.676 | 0.924 | 0.667 | 0.997 | 0.677 | 0.997 | 0.116 | 0.929 | 0.970 | 0.959 | 0.198 | 0.985 (0.981–0.989) | 0.737 (0.671–0.803) |  < 0.001 |
 Random forest | 0.969 | 0.668 | 0.958 | 0.667 | 0.980 | 0.668 | 0.980 | 0.113 | 0.959 | 0.969 | 0.969 | 0.194 | 0.996 (0.995–0.997) | 0.730 (0.661–0.799) |  < 0.001 |
 Logistic regression | 0.728 | 0.654 | 0.727 | 0.646 | 0.730 | 0.654 | 0.729 | 0.107 | 0.728 | 0.967 | 0.728 | 0.183 | 0.801 (0.786–0.815) | 0.706 (0.626–0.786) | 0.024 |
 SVM | 0.689 | 0.611 | 0.770 | 0.604 | 0.607 | 0.612 | 0.662 | 0.090 | 0.725 | 0.960 | 0.712 | 0.157 | 0.766 (0.751–0.782) | 0.682 (0.598–0.765) | 0.052 |
 KNN | 0.791 | 0.601 | 0.973 | 0.604 | 0.610 | 0.601 | 0.714 | 0.088 | 0.957 | 0.960 | 0.823 | 0.154 | 0.949 (0.942–0.955) | 0.667 (0.585–0.749) |  < 0.001 |
All-cause mortality | |||||||||||||||
 MLP | 0.770 | 0.741 | 0.816 | 0.716 | 0.724 | 0.745 | 0.747 | 0.291 | 0.797 | 0.947 | 0.780 | 0.414 | 0.836 (0.823–0.850) | 0.796 (0.755–0.837) | 0.065 |
 Random forest | 0.952 | 0.740 | 0.940 | 0.716 | 0.965 | 0.744 | 0.964 | 0.290 | 0.941 | 0.947 | 0.952 | 0.412 | 0.990 (0.988–0.993) | 0.790 (0.750–0.831) |  < 0.001 |
 LightGBM | 0.924 | 0.690 | 0.884 | 0.716 | 0.964 | 0.686 | 0.961 | 0.250 | 0.892 | 0.943 | 0.921 | 0.371 | 0.977 (0.972–0.981) | 0.771 (0.725–0.816) |  < 0.001 |
 SVM | 0.925 | 0.709 | 0.896 | 0.706 | 0.953 | 0.709 | 0.950 | 0.262 | 0.902 | 0.943 | 0.923 | 0.382 | 0.982 (0.978–0.985) | 0.761 (0.714–0.808) |  < 0.001 |
 KNN | 0.780 | 0.715 | 0.932 | 0.716 | 0.629 | 0.716 | 0.715 | 0.268 | 0.902 | 0.945 | 0.809 | 0.390 | 0.907 (0.897–0.917) | 0.761 (0.713–0.808) |  < 0.001 |
 Logistic regression | 0.751 | 0.666 | 0.770 | 0.667 | 0.731 | 0.666 | 0.741 | 0.226 | 0.761 | 0.932 | 0.755 | 0.337 | 0.812 (0.797–0.827) | 0.760 (0.716–0.805) | 0.031 |