Telco Customer Churn Predictor

An end-to-end machine learning application: a tuned scikit-learn pipeline trained on the IBM Telco Customer Churn dataset, served through a FastAPI backend, with a single-page web UI where a user enters a customer's profile and receives a churn probability, a risk tier, and the top factors driving that prediction.

This project started as a learning exercise and was refactored into a portfolio-quality ML application.

The business problem

Acquiring a new telecom subscriber costs significantly more than retaining an existing one, so the highest-leverage lever a retention team has is knowing who is about to leave before they leave. Left to guesswork, retention outreach (discounts, calls, plan changes) either gets spread thinly across the whole customer base — wasting spend on customers who were never going to churn — or arrives too late, after a customer has already decided to switch providers. A churn model that scores every active customer, flags the highest-risk ones, and explains why a given customer is flagged turns retention from a blunt, reactive effort into a targeted, proactive one: a monthly export of "these 200 customers are high-risk, and here's the specific factor driving each one" is directly actionable in a way "27% of our customers churn eventually" is not.

Dataset

IBM Telco Customer Churn (Kaggle: blastchar/telco-customer-churn) — 7,043 customers, 21 columns:

• Demographics: gender, SeniorCitizen, Partner, Dependents
• Account: tenure, Contract, PaperlessBilling, PaymentMethod, MonthlyCharges, TotalCharges
• Services: PhoneService, MultipleLines, InternetService, OnlineSecurity, OnlineBackup, DeviceProtection, TechSupport, StreamingTV, StreamingMovies
• Target: Churn (Yes/No)

scripts/download_data.py fetches it reproducibly: the Kaggle API first (if kaggle.json credentials are configured), falling back automatically to a direct CSV mirror (IBM's own GitHub copy of the same sample dataset) if not — so a fresh clone works with zero account setup.

Dataset provenance and permitted use

• This repository does not contain or redistribute the dataset. data/raw/ is gitignored; the CSV is fetched from an upstream public source at setup time.
• The two upstream sources used are the Kaggle dataset blastchar/telco-customer-churn and IBM's public GitHub mirror linked above. Both are widely used as public sample data for learning and demonstration.
• The dataset's license has not been verified here. No explicit license file accompanies either upstream copy, and this repository makes no claim about one. The MIT license in LICENSE covers this repository's own code only — it does not extend to the data.
• If you intend to use the dataset for anything beyond learning or portfolio work, check the upstream terms yourself first.

Key EDA findings

Full analysis with commentary: notebooks/01_eda.ipynb.

• Data quality: TotalCharges ships as text with 11 blank entries. Every single blank corresponds exactly to tenure == 0 (verified, not assumed) — brand-new customers who haven't completed a billing cycle. Imputed as 0, not the median, since the mechanism is known exactly.
• Class imbalance: 73.5% retained / 26.5% churned (~2.7:1) — enough that accuracy is a misleading metric (see below).
• Contract type is the dominant churn driver: month-to-month customers churn at 42.7%, vs. 11.3% for one-year and 2.8% for two-year contracts — a ~15x spread.
• Tenure: churn risk is highest in the first 6 months (52.9%) and falls steadily to 9.5% by 49–72 months — an onboarding-risk curve, not a flat rate.
• Payment method: electronic check payers churn at 45.3%, vs. 15–19% for the three automatic/mailed alternatives.
• Internet service: fiber optic customers churn at 41.9% vs. 19.0% for DSL and 7.4% for no internet — likely a price-sensitivity signal (fiber is priced higher) rather than the technology itself.

How the data is split

Three disjoint, stratified slices (notebooks/02_modeling.ipynb §3):

Slice	Rows	Used for
model_train	4,507	cross-validation, model comparison, hyperparameter tuning, fitting the final pipeline
threshold_validation	1,127	selecting the decision threshold — nothing else
test	1,409	one single final evaluation, at the already-frozen threshold

The middle slice exists because choosing an operating point is itself a fitting decision. A threshold picked by minimizing cost on the test set and then reported on that same test set yields an optimistically biased recall. Selecting it on threshold_validation and freezing it before the test set is scored keeps the reported precision / recall / F1 an honest held-out result.

Model comparison

Four candidates, evaluated with 5-fold stratified cross-validation on model_train (all four shown — see notebooks/02_modeling.ipynb §5 for the full methodology):

Model	Precision	Recall	F1	ROC-AUC
Logistic Regression (winner)	0.523	0.804	0.634	0.845
Random Forest	0.639	0.476	0.545	0.826
SVM	0.522	0.778	0.624	0.825
XGBoost	0.559	0.570	0.564	0.805

A well-regularized linear model beating three more flexible ones is a real result on this dataset, not an error — the churn signal here is dominated by a handful of strong, roughly-linear-in-effect features (contract type, tenure), which favors a model that won't overfit noise in the rest.

Imbalance handling: class_weight="balanced" vs. SMOTE were compared directly (Logistic Regression baseline, proper CV — SMOTE applied only inside training folds to avoid leakage): 0.845 vs. 0.845 ROC-AUC, an insignificant gap. class_weight="balanced" was used going forward — same result, no synthetic data, less pipeline complexity.

Final model on the held-out test set, using a validation-selected threshold

RandomizedSearchCV, 5-fold CV on model_train, best params C=0.070, penalty=l2. Decision threshold 0.34, selected on threshold_validation and frozen before the test set was scored:

Metric	Value	Depends on the threshold?
ROC-AUC	0.846	No — threshold-free
Average precision	0.654	No — threshold-free
Brier score	0.165	No — threshold-free
Precision	0.445	Yes — at the validation-selected 0.34
Recall	0.912	Yes — at the validation-selected 0.34
F1	0.598	Yes — at the validation-selected 0.34

How to read this table. ROC-AUC, average precision and the Brier score are threshold-free metrics computed on the held-out test set. Precision, recall and F1 are also computed on the held-out test set, but they are operating-point metrics: they are reported at a threshold of 0.34, which was chosen on the separate threshold_validation slice by minimizing expected cost under an assumed 5:1 false-negative:false-positive cost ratio. Change that cost assumption and these three numbers change; the three above them do not.

For reference, the same model on the same test set at the default 0.50 cutoff scores precision 0.510 / recall 0.789 / F1 0.620 — so the threshold choice buys roughly +12 points of recall for about -7 points of precision.

Every number in this section is reproduced from models/metadata.json, which notebook 02 writes on each run (see Setup and run); the artifact itself is gitignored.

Why ROC-AUC and recall, not accuracy

With a ~73/27 class split, a model that predicts "no churn" for every customer scores ~73% accuracy while catching zero churners — accuracy rewards ignoring the class that actually matters for the business. ROC-AUC was used for model selection because it's threshold-independent and summarizes discrimination across the full range of possible cutoffs, unlike a single accuracy or F1 number tied to one arbitrary threshold.

Recall was prioritized when picking the deployment threshold because the two error types have different business costs here: a false negative (a customer who churns without being flagged) means losing that customer's revenue outright, while a false positive (a loyal customer flagged as at-risk) costs one unnecessary retention offer. Modeling this explicitly as a 5:1 cost ratio (missed churner costs 5x a false alarm) and minimizing expected cost over the threshold-validation slice picks a threshold of 0.34, well below the default 0.5 — pushing test-set recall from 0.79 (at 0.5) to 0.91, at the cost of precision dropping from 0.51 to 0.45. That trade is the right one when the retention outreach itself is cheap relative to the value of a retained customer; the API always returns the raw probability too, so this specific cost assumption isn't baked in irreversibly.

Project structure

├── data/
│   ├── raw/                  # downloaded CSV (gitignored — reproduced via scripts/download_data.py)
│   └── processed/
├── notebooks/
│   ├── 01_eda.ipynb          # exploratory analysis
│   └── 02_modeling.ipynb     # feature engineering, model comparison, tuning, threshold, persistence
├── src/
│   ├── config.py             # paths, categorical levels, thresholds — single source of truth
│   ├── data.py                # reproducible data loading (Kaggle API + fallback)
│   └── features.py            # feature engineering shared by the notebook and the API
├── models/                    # trained pipeline + metadata (gitignored — generated by notebook 02)
├── app/
│   ├── main.py                 # FastAPI app, lifespan, routes
│   ├── schemas.py               # Pydantic v2 request/response models
│   ├── predict.py               # inference + "top drivers" explanation logic
│   ├── static/                  # CSS/JS for the UI
│   └── templates/index.html     # single-page UI (Jinja2)
├── tests/
│   ├── test_features.py         # feature-engineering edge cases
│   └── test_api.py              # API tests against the real trained pipeline
├── scripts/download_data.py     # standalone reproducible data fetch
├── LICENSE                      # MIT — covers this repository's code, not the dataset
└── requirements.txt

Setup and run

Requires Python 3.11+.

# 1. Clone and create a virtual environment
python -m venv .venv
.venv\Scripts\activate          # Windows
# source .venv/bin/activate      # macOS/Linux

# 2. Install dependencies
pip install -r requirements.txt

# 3. Fetch the dataset (Kaggle API if configured, else a documented public fallback)
python scripts/download_data.py

# 4. Run both notebooks top to bottom (no manual steps required) to produce
#    models/churn_pipeline.joblib and models/metadata.json
#    — open notebooks/01_eda.ipynb then notebooks/02_modeling.ipynb and "Run All"

# 5. Start the API + web UI
uvicorn app.main:app --reload

Step 4 is required before the API or the tests will work: the trained pipeline and models/metadata.json (which holds the frozen threshold, the metrics quoted above, and the SHAP ranking the API's "top drivers" use) are generated locally and deliberately not committed. Everything is seeded from RANDOM_STATE in src/config.py, so a clean run reproduces the numbers in this README. There is no separate training script yet — see Next steps.

Then open http://127.0.0.1:8000 for the UI, or http://127.0.0.1:8000/docs for interactive API docs. GET /model-info returns the metrics, the full model comparison, and how the threshold was selected.

curl example

curl -X POST http://127.0.0.1:8000/predict \
  -H "Content-Type: application/json" \
  -d '{
    "gender": "Female", "SeniorCitizen": 0, "Partner": "No", "Dependents": "No",
    "tenure": 2, "Contract": "Month-to-month", "PaperlessBilling": "Yes",
    "PaymentMethod": "Electronic check", "MonthlyCharges": 85.00, "TotalCharges": 170.00,
    "PhoneService": "Yes", "MultipleLines": "No", "InternetService": "Fiber optic",
    "OnlineSecurity": "No", "OnlineBackup": "No", "DeviceProtection": "No",
    "TechSupport": "No", "StreamingTV": "Yes", "StreamingMovies": "Yes"
  }'

{
  "churn_probability": 0.9289,
  "churn_label": "Yes",
  "risk_tier": "High",
  "top_drivers": [
    { "feature": "Tenure of 2 months", "direction": "increases_risk" },
    { "feature": "Contract: Month-to-month", "direction": "increases_risk" },
    { "feature": "Average monthly spend of $85.00", "direction": "increases_risk" },
    { "feature": "Internet service: Fiber optic", "direction": "increases_risk" },
    { "feature": "Tenure range: 0-6mo", "direction": "increases_risk" }
  ],
  "threshold_used": 0.34
}

Tests

pytest tests/ -v

Screenshot

A high-risk prediction (month-to-month contract, 2-month tenure, fiber optic, $85/mo — the same profile as the curl example above):

Limitations

• One dataset. Everything here rests on a single public 7,043-row sample from one telecom operator. The specific coefficients, the threshold and the driver ranking are properties of that sample, not general facts about churn.
• The operating threshold encodes an assumption. 0.34 follows from the assumed 5:1 false-negative:false-positive cost ratio. That ratio is a plausible ballpark, not a measured business figure; a different ratio yields a different threshold and different precision/recall.
• The score is not a calibrated probability. A Brier score is reported (0.165) but no calibration analysis was performed, and class_weight="balanced" is expected to inflate predicted probabilities relative to the true base rate. Treat the output as a risk score for ranking and thresholding, not as a literal probability of churn.
• "Top drivers" are not per-row Shapley values. They come from a global SHAP ranking computed once at training time, filtered and oriented by the individual request's feature values (see app/predict.py). That is cheap and specific enough to be useful, but it is not a live per-prediction attribution.
• Local demo, not a deployment. The FastAPI app is intended to be run locally. There is no hosted instance, no container image, no authentication, no rate limiting, and no monitoring — and no claim that it is production-ready.
• No CI. The test suite runs locally against artifacts you generate; nothing runs automatically on push.

Next steps

• Calibration check: reliability curve plus Brier decomposition, and CalibratedClassifierCV if the scores turn out to be materially miscalibrated.
• Tune the challengers harder: the tree/boosting candidates got one shared RandomizedSearchCV budget; XGBoost in particular deserves a wider search before concluding a linear model wins.
• Add scripts/train.py so the artifacts can be regenerated without opening a notebook, then add CI that runs it plus the test suite on a fresh clone.
• Export the analytical charts from the notebooks to docs/ so the EDA and evaluation figures are visible without running Jupyter.
• Add a Dockerfile if deployment packaging ever becomes the point.

License

The source code in this repository is released under the MIT License.

That license covers the code only. The IBM Telco Customer Churn dataset is not included in this repository and is not redistributed here — it is fetched from an upstream public source at setup time by scripts/download_data.py. Reuse of the dataset is governed by the terms of those upstream sources (Kaggle / IBM), not by the MIT License above, and the dataset's license has not been verified here. If you plan to use the dataset for anything beyond learning or portfolio work, verify the upstream terms yourself first. See Dataset provenance and permitted use.