Robert Zhou

Work · Apr 2024 – Aug 2024

Sepsis mortality prediction

Will transcriptomic classifiers beat 0.7 AUC on sepsis mortality?

Patient samples
772
Public datasets
7
Best AUC
0.74 (LightGBM)

Sepsis mortality prediction from gene expression is a dataset problem before it is a modeling problem: small cohorts, wildly different platforms, and a class imbalance that punishes anything that looks accurate by predicting survival for everyone.

At the USC Mangul Lab I benchmarked four classifier families — LightGBM, random forest, SVM, and logistic regression — across 7 public transcriptomic datasets totaling 772 patient samples, using SMOTE to handle the imbalance. LightGBM came out best at 0.74 AUC.

The number I care about less than the process is the AUC. What made the benchmark trustworthy was holding the dataset boundary as the evaluation boundary: results were reported per cohort, not pooled into one flattering average, because a model that wins by memorizing platform-specific expression artifacts is useless on the next cohort. SHISA4 surfaced as a severity-associated biomarker, which was the kind of finding worth following up rather than the score itself.

Alongside that, I modeled antibody sequence and structure features from immunogenetic datasets using AbStar and IgSimulator, and built reproducible Python notebooks for featurization and QC. That notebook discipline is the part I carried into later work: a result you cannot rerun from a clean checkout is not a result.

What I'd do differently: fix the feature set before looking at outcomes. With a few hundred patients, the freedom to select features after seeing results is worth more AUROC than most modeling choices, which is exactly why it should be constrained in advance.

Timeline

  1. Started benchmarking classifiers across public transcriptomic sepsis datasets
  2. LightGBM at 0.74 AUC; SHISA4 surfaced as a severity-associated biomarker

Python · LightGBM · scikit-learn · pandas