Customer Segmentation as a Monitored Model
Segmentation treated as a model with a lifecycle rather than a one-off clustering notebook: rolling monthly windows over ~1.1M transactions, a six-algorithm bakeoff, drift detection calibrated on real data (21/22 false alarms cut to 4 genuine regime changes), and champion/challenger retraining behind a Feast store, a Dagster asset graph, and a FastAPI service — all running locally.
01 Overview
Most segmentation projects fit K-Means once and stop. But segments decay — seasonality, cohort aging, changing spend — so a segmentation that is never re-checked is quietly wrong within months.
This project treats segmentation as a model with a lifecycle: customers are segmented from transaction history over rolling monthly windows, drift is detected and calibrated against the data's own variation, the model is versioned and retrained on a champion/challenger basis, and the whole thing is served, orchestrated, and deployable — all locally, no cloud account required.
It is built with a clear data-engineering / data-science separation behind a frozen interface contract, and two public datasets sit behind one canonical schema, swappable by config with zero downstream changes.
02 Results
K-Means with k=3 was selected, at silhouette ≈ 0.27 — which is typical for real, overlapping RFM data and is reported as such rather than dressed up. The bakeoff scored K-Means, GMM, Ward, Bisecting K-Means, Birch, and Spectral on silhouette, Davies–Bouldin, and cross-window stability; DBSCAN and HDBSCAN were evaluated and shown unfit for this data.
The result that matters most is the calibration one. A naive fixed drift reference fired on 21 of 22 months — an alarm that goes off every month is not a monitor. Thresholds tuned from the data's own month-to-month variation cut that to 4: the December-holiday and January-unwind regime changes that are genuinely there.
03 Architecture
Orchestrated as a Dagster asset graph with monthly partitions, backfill, a schedule, and a file sensor; deployed by Docker Compose.
Windowed feature engineering
RFM plus behavioral features per customer per month over 22 rolling windows — 47,170 feature rows from ~1.1M transactions. Two public datasets are normalized to one canonical schema, so switching source is a config change and nothing downstream notices.
Data contracts with quarantine
Pandera validates every ingested frame and quarantines bad rows rather than dropping them silently — a rejected row is inspectable, which is the difference between a data-quality signal and a data-quality loss.
Model selection, then a decision
Six algorithms scored on silhouette, Davies–Bouldin, and cross-window stability. K-Means won on interpretability and scale at k=3; DBSCAN and HDBSCAN were tried and documented as unfit. The benchmark is committed, so the choice is auditable.
Calibrated drift detection
Five signals — PSI, ARI, centroid movement, segment-size shift, and customer migration — with thresholds tuned from the data's own variation instead of textbook constants. That is what took the alert rate from 21/22 windows down to the 4 real regime changes.
Champion/challenger retraining
An MLflow registry with Staging/Production stages, a sustained-breach retrain gate (one bad month is noise), a cooldown so retrains cannot thrash, and rollback. Drift does not trigger a retrain; sustained drift does.
Serving, orchestration, deployment
A Feast feature store gives point-in-time-correct training retrieval and online serving; a FastAPI service exposes /assign with health and readiness; a Streamlit dashboard shows segments and drift; a Dagster asset graph runs it on monthly partitions with backfill and a file sensor. Verified end to end: containers healthy, /assign served the live champion, then torn down.
04 Decisions worth defending
Calibrate the monitor, not just the model
A drift detector that fires every month has no information in it. Tuning thresholds against observed month-to-month variation is what turned the signal from noise into four explainable events.
Sustained breach, then retrain
A single month over threshold is seasonality. The gate requires a sustained breach and enforces a cooldown, so the champion is not replaced on a holiday spike.
Point-in-time correctness
Feast retrieval is point-in-time, so training features cannot include information from after the label window — the leakage that makes an offline segmentation look better than it is.
Two datasets, one schema
Online Retail II and dunnhumby sit behind the same canonical schema. Swapping source is config, which is also the cheapest possible test that the pipeline is not overfit to one file format.
Honest cluster quality
Silhouette ≈ 0.27 is reported plainly. Real RFM data overlaps; a project claiming 0.7 on customer segmentation is usually measuring something other than customers.
Local by design
PostgreSQL or DuckDB, MLflow, Feast, Dagster, Docker Compose — the whole lifecycle runs on a laptop, so the architecture can be reviewed without a cloud bill.
05 Stack
The full source is on GitHub
Every number on this page is reproducible from the repository.