Skip to content
ROOT

mlship Plan

Year 5 Capstone. One command to deploy any ML model to production. sklearn → Llama 70B. Zero config. Zero code. The OSS project that proves Staff/Principal-level synthesis. The launch that puts the abukix name on the map.

v0/v0.5 land in Year 4 (usable for the train→deploy composition recipe). v2 is the Year 5 P30 capstone launch.

mlship is the Group B service that doubles as the Year 5 capstone OSS launch. It’s a CLI a user invokes from anywhere — mlship deploy ./model.pkl produces a deployed, served, monitored ML endpoint with auto-detection picking the right runtime per framework.

The project lands in three stages. v0 ships in Year 4 Phase 21 — a private CLI doing sklearn auto-detection + container build + deploy to local Docker or KServe-on-basecamp, just enough to make the train → register → deploy composition recipe work end-to-end. v0.5 follows in Phase 25 with PyTorch, Fargate, and Cloud Run targets. v2 is the Year 5 Phase 30 capstone — public from day 1, launched with Show HN + LinkedIn + blog + demo video, and the artifact that makes the “Staff/Principal AI Platform Engineer” claim concrete.

The differentiator is scope discipline: 2 frameworks done excellently (sklearn + HuggingFace text-generation), not 6. Other frameworks ship as v2.1+ post-launch. The space is crowded (BentoML, Cog, Truss, KServe directly), but every existing tool requires config files or code; mlship’s auto-detection + zero-config is what makes it cinematic in a 3-minute demo video.

What it is

A CLI that takes a model file (or HuggingFace ID) and produces a deployed, served, monitored ML endpoint — in one command.

Terminal window
mlship deploy ./model.pkl
# → auto-detect sklearn; containerize; deploy local Docker; print URL


mlship deploy ./pytorch_model.pt --to k8s --cluster basecamp-k3s
# → containerize with PyTorch runtime; deploy KServe via basecamp; print URL


mlship deploy hf://meta-llama/Llama-3.2-1B --quantize awq --to k8s
# → fetch from HF; AWQ-quantize; deploy via vLLM on KServe; OpenAI-compat API


mlship list
mlship logs my-model
mlship rollback my-model
mlship destroy my-model

Why it exists

Three reasons:

  1. Career artifact. The Year 5 capstone is what makes “Staff/Principal AI Platform Engineer” claim concrete. mlship is the OSS proof.
  2. Brand artifact. Launch with Show HN, LinkedIn, blog post, demo video. Targets GitHub stars + real users + interest from companies hiring AI platform engineers.
  3. Real tool. ML engineers + DevOps engineers genuinely need this. The space is crowded (BentoML, Cog, Truss, KServe directly) but every existing tool requires config files or code; mlship’s auto-detection + zero-config is the differentiator.

Pattern it teaches

Scope

Y4 P21: v0 (private)

The minimum for the train→deploy composition recipe to work.

[ ] Go + cobra CLI
[ ] sklearn auto-detect (.pkl, .joblib)
[ ] Containerize: multi-stage Dockerfile, distroless runtime
[ ] Deploy targets: local Docker, K8s+KServe (basecamp)
[ ] REST API: POST /predict
[ ] Prometheus metrics
[ ] Tests: 70%+ coverage
[ ] CI: GitHub Actions

Y4 P25: v0.5 (private)

Add to the v0 base.

[ ] PyTorch auto-detect (.pt, .pth, TorchScript)
[ ] AWS Fargate deploy target
[ ] GCP Cloud Run deploy target
[ ] Better error messages
[ ] Tests: 80%+ coverage

Y5 P30: v2 capstone (PUBLIC, the launch)

The review’s scope discipline: 2 frameworks done excellently (sklearn + HuggingFace text-generation), not 6. Other frameworks ship as v2.1+ post-launch.

[ ] github.com/abukix/mlship PUBLIC + active
[ ] CLI: deploy, list, logs, rollback, destroy, update
[ ] Auto-detection (V2 launch):
    - sklearn (.pkl, .joblib)
    - HuggingFace text-generation (auto-route to vLLM)
    - HuggingFace transformers (general)
[ ] Auto-detection (V2.1+ post-launch — quietly added):
    - PyTorch (.pt, TorchScript)
    - TensorFlow (SavedModel, .h5)
    - ONNX
    - Plugin model for custom frameworks
[ ] Containerization:
    - Per-framework optimized Dockerfile
    - Multi-stage, distroless
    - Layer cache for fast rebuilds
[ ] Deployment targets:
    - Local (Docker)
    - K8s + KServe (basecamp or any cluster)
    - AWS Fargate (serverless)
    - GCP Cloud Run
[ ] LLM-aware:
    - vLLM runtime for HF text-generation
    - AWQ quantization option
    - OpenAI-compatible API (/v1/chat/completions)
[ ] Standard model API: REST POST /predict for traditional models
[ ] Observability out-of-the-box: Prometheus metrics, structured logs
[ ] Tests: 80%+ coverage, one integration test per shipped framework
[ ] CI: GitHub Actions, releases via GoReleaser
[ ] Docs site (mkdocs or VitePress)
[ ] 3-minute demo video
[ ] Show HN post
[ ] LinkedIn announcement
[ ] Blog post: "Why mlship exists"

When built

  • v0: Year 4 Phase 21 (Month 39-40)
  • v0.5: Year 4 Phase 25 (Month 47-48)
  • v2 capstone: Year 5 Phase 30, Months 57-60. The 4-month launch push.

Dependencies

mlship’s deploy targets depend on infrastructure provisioned by terralabs (clusters, Fargate config, Cloud Run setup) and on basecamp Tier 5-7 being operational (KServe, vLLM, observability). platform-ctl wraps mlship as platform-ctl model deploy from Y4 P21 onward, so mlship’s stability indirectly bounds platform-ctl’s usefulness.

Stack

  • Go 1.23+ (CLI core)
  • Python wrapper code for ML framework auto-detection (subprocess + introspection)
  • Cobra (CLI)
  • Docker SDK
  • K8s client-go + KServe types
  • AWS SDK + GCP SDK
  • pytest + go test

Success criteria (Y5 capstone)

[ ] 5+ real users (not just stars) deployed real models with mlship
[ ] >100 GitHub stars in launch week (good); >1000 in first 3 months (great)
[ ] One inbound conversation about "we want to use this at $COMPANY"
[ ] Documented in pattern paper as the capstone artifact
[ ] Show HN front-page slot OR clearly positive reception

Public vs private

  • Y4: private (v0/v0.5 used internally for composition recipes)
  • Y5 P30: PUBLIC FROM DAY 1. Repo created public; first commits + README + demo video go up together. Launch is coordinated across Show HN + LinkedIn + blog + abukix.dev/blog.

Cross-references