ML Pipeline Builder

A Claude Code skill for ML engineers who build production machine learning pipelines from raw data to deployed model APIs.

Most ML tutorials stop at "train model, save pickle". This skill teaches Claude (or any coding agent) how to build real ML systems: data validation, feature engineering, model registry, inference APIs, drift monitoring, and continuous training.

What this skill does

When you load this skill in Claude Code, the agent learns how to:

• Build ingestion pipelines with schema validation
• Validate data with Pandera or Great Expectations
• Engineer features with point-in-time correctness
• Train models with scikit-learn, XGBoost, LightGBM, PyTorch, or transformers
• Track experiments with MLflow
• Register models with stage promotion (Staging, Production, Archived)
• Serve models with FastAPI or batch scoring jobs
• Monitor for data drift and concept drift with Evidently
• Set up continuous training DAGs with Airflow or Prefect
• Productionize Jupyter notebooks into runnable pipeline code

Why this skill exists

ML is the second most common AI work after RAG. It is also the one most engineers ship broken.

The failure mode is always the same: data scientist trains a notebook, model gets 0.95 AUC, the team celebrates, the model is "deployed" as a pickle file loaded by an API, three months later the business metric drops 30% and nobody knows why.

The model is rarely the problem. The pipeline is. And the pipeline is determined by your data validation, feature consistency, registry, monitoring, and retraining strategy.

This skill teaches Claude all of those, with real code, real metrics, and real production war stories.

Install

Claude Code

git clone https://github.com/datawithusman/ml-pipeline-builder.git
cp ml-pipeline-builder/SKILL.md .claude/skills/ml-pipeline-builder.md

Or install globally:

mkdir -p ~/.claude/skills
curl -sL https://raw.githubusercontent.com/datawithusman/ml-pipeline-builder/main/SKILL.md \
  -o ~/.claude/skills/ml-pipeline-builder.md

Cursor

Save SKILL.md as .cursor/rules/ml-pipeline-builder.mdc in any ML project.

OpenCode / Windsurf / others

Copy the contents of SKILL.md into your tool's rules or system prompt file.

How to use

After installing, describe what you are building:

I want to build a fraud detection pipeline.
40M transactions per day, p99 latency 100ms.
Currently in a Jupyter notebook. Need to productionize it.

Claude will use this skill to:

1. Refactor the notebook into pipeline modules
2. Add data validation with Pandera
3. Set up feature engineering with point-in-time correctness
4. Train the model with MLflow tracking
5. Build a FastAPI inference API
6. Set up drift monitoring with Evidently
7. Create an Airflow retraining DAG
8. Write a productionization checklist

What is inside

Section	What you learn
Data ingestion	Incremental load with schema validation
Data validation	Pandera schemas, what to validate, alerting
Feature engineering	Feast, point-in-time correctness, offline vs online
Training	scikit-learn + MLflow, TimeSeriesSplit, hyperparameter tuning
Evaluation	Held-out test, per-slice metrics, regression detection
Model registry	MLflow registry, stage promotion, audit trail
Serving	FastAPI real-time API, batch scoring jobs
Monitoring	Operational, data drift, concept drift, alerting
Retraining	Continuous training DAGs in Airflow and Prefect
MLOps stack	Recommended stacks for startup, mid, enterprise
Project structure	Production ML repo layout
CI/CD for ML	GitHub Actions training pipeline
Anti-patterns	10 mistakes that kill ML in production
Productionization checklist	13-point pre-launch checklist
Pricing	2026 freelance rates for ML pipeline work
Real examples	Fraud detection (40M txns/day), Churn prediction (180k users)

Who this is for

• ML engineers building production pipelines for clients or products
• Data scientists productionizing their first model
• Backend developers adding ML inference to existing services
• Founders building ML-powered products
• Consultants shipping ML systems to enterprise clients
• Anyone who has trained a model in a notebook and wondered "now what"

Who this is NOT for

• Pure data analysis with no model
• LLM or chatbot work (use rag-production-setup)
• Research-only work
• Deep learning training on GPU clusters (partial coverage only)
• Computer vision model training (different skill)

Tech stack

• Orchestration: Airflow or Prefect 2.x
• Tracking: MLflow
• Registry: MLflow Model Registry
• Feature store: Feast (optional)
• Models: scikit-learn, XGBoost, LightGBM, CatBoost, PyTorch, transformers
• Validation: Pandera or Great Expectations
• Serving: FastAPI (low QPS), Triton or Ray Serve (high QPS)
• Monitoring: Evidently + Prometheus + Grafana
• CI/CD: GitHub Actions
• Language: Python 3.11+

About the author

Muhammad Usman. I build production ML pipelines for clients through Data With Usman. I also teach Python as a Stanford Code in Place Section Leader and build AI systems at Nobel AI.

• GitHub: @datawithusman
• Site: datawithusman.com

Contributing

If you have a pipeline recipe, feature store pattern, or monitoring strategy that is not covered here, open a pull request. Include measured metrics.

See CONTRIBUTING.md.

License

MIT. Use it for client work, paid projects, your own product.

Other skills in this series

• business-data-automator - Convert messy spreadsheets into dashboards and automations
• rag-production-setup - Production RAG systems with FastAPI + LangChain
• ml-pipeline-builder - This repo