Case study 5 of 6
From data to justice
Offender profiles learned from 573k solved FBI homicide cases, built to apply to the unsolved ones. Data from the Murder Accountability Project.
- stack
- Python
- XGBoost
- imbalanced-learn
- R
- FastAPI
- Docker
- AWS App Runner
0.635
macro-F1, held-out test
tick = 0.469 always-male baseline
0.75
ROC-AUC; the ceiling across 50+ combos
tick = coin flip
39.8%
female offenders caught
the always-male baseline catches 0%
Leaky and leak-free protocols on the same split
SMOTE builds synthetic minority rows by interpolating between real neighbours. Oversample before the split, and the test fold grades the model on near-copies of its training data. The sweep includes one run with SMOTE before the split so the leak has a number; that run uses the same data and model family as the leak-free XGBoost run.
macro-F1
recall, female offenders
accuracy
The leak-free runs top out near F1 0.63; the leaky run sits alone at 0.93.
Input fields available in an unsolved case
573,172 solved cases remain after cleaning; offender sex splits 88.3% male to 11.7% female, so the sweep compares nine oversamplers. Inputs are limited to fields knowable in an unsolved case: victim age, sex, and race, weapon, state, agency type, month, year. Offender attributes never enter; they are what the model predicts. Relationship never enters either, because in an unsolved case its value is simply unknown.
The split is 64/16/20, stratified on the target, and it happens first. Samplers and preprocessing fit on the training fold only. Nine oversampling strategies by five model families are ranked on validation, and the test fold is spent once, on the final chosen models.
Nine oversamplers ranked on validation macro-F1
Top strategies by validation macro-F1, best model per sampler shown; the full grid, failures included, lives in the repo.
SMOTEENN + XGBoost winner
0.634
SMOTENC + HistGradientBoosting
0.611
class_weight + RandomForest
0.607
SMOTETomek + ExtraTrees
0.596
no resampling + DecisionTree
0.562
always-male dummy
0.469
The winner gives up 4.6 points of raw accuracy against the always-male dummy and catches 40% of female offenders instead of none. Case attributes separate the sexes only so far; ROC-AUC 0.75 was the ceiling across every combo tried.
Offender age gets a regressor without oversampling. Applied to a continuous target with the split done first, oversampling lowers R² to 0.165 from 0.257. The final age model scores R² 0.257 and MAE 8.4 years against a 13-year target standard deviation.
The dataset, aggregated from the current SHR file
Every chart below is aggregated fresh from the Murder Accountability Project's current file: 929,433 cases, 1976-2025, a later vintage than the repo's training cut. The shapes match the repo's R analysis; this site draws the charts from the aggregates.
■ victims · ■ offenders (where known) · both concentrate at 20-40
FastAPI service on AWS App Runner
The package trains both pipelines in about two minutes and serves them from FastAPI: /predict takes case attributes and returns offender sex with its probability and an age estimate; /model-info returns the training provenance and held-out metrics of the loaded artifacts. The Docker image carries under 1 MB of model artifacts and never sees the 318 MB dataset; one script deploys it to AWS App Runner with TLS and liveness checks.