Work · Apr 2024 – Aug 2024
Sepsis mortality prediction
Will transcriptomic classifiers beat 0.7 AUC on sepsis mortality?
- Patient samples
- 772
- Public datasets
- 7
- Best AUC
- 0.74 (LightGBM)
Sepsis mortality prediction from gene expression is a dataset problem before it is a modeling problem: small cohorts, wildly different platforms, and a class imbalance that punishes anything that looks accurate by predicting survival for everyone.
At the USC Mangul Lab I benchmarked four classifier families — LightGBM, random forest, SVM, and logistic regression — across 7 public transcriptomic datasets totaling 772 patient samples, using SMOTE to handle the imbalance. LightGBM came out best at 0.74 AUC.
The number I care about less than the process is the AUC. What made the benchmark trustworthy was holding the dataset boundary as the evaluation boundary: results were reported per cohort, not pooled into one flattering average, because a model that wins by memorizing platform-specific expression artifacts is useless on the next cohort. SHISA4 surfaced as a severity-associated biomarker, which was the kind of finding worth following up rather than the score itself.
Alongside that, I modeled antibody sequence and structure features from immunogenetic datasets using AbStar and IgSimulator, and built reproducible Python notebooks for featurization and QC. That notebook discipline is the part I carried into later work: a result you cannot rerun from a clean checkout is not a result.
What I'd do differently: fix the feature set before looking at outcomes. With a few hundred patients, the freedom to select features after seeing results is worth more AUROC than most modeling choices, which is exactly why it should be constrained in advance.
Timeline
- Started benchmarking classifiers across public transcriptomic sepsis datasets
- LightGBM at 0.74 AUC; SHISA4 surfaced as a severity-associated biomarker
Python · LightGBM · scikit-learn · pandas