Project README Template

The shape of an OSS project README under the Abukix umbrella. Used 9-10 times across the OSS portfolio (rxp, pulse, triage, terralabs, platform-ctl, data-tier, llm-gateway, mcp-servers, studio, aiops). Consistency makes the portfolio read as one body of work.

Every OSS repo gets one README. Nine projects = nine READMEs. A consistent shape is the difference between “a coherent body of work” and “a pile of repos.” This template enforces that consistency without templating away each project’s character.

What a /root README is

A README is the front door of an OSS project. A senior engineer landing on the GitHub repo decides in 30 seconds whether to read further. The README has to earn the next 30 seconds, and the next 30 after that, and the 5 minutes after that. Sections are ordered by how much they earn per second of reading.

A /root README also serves a second audience: the portfolio reviewer. Every Abukix repo is part of the platform-engineering portfolio. The README links into the curriculum (which /root phase shipped it) and into basecamp (which tier it occupies). That cross-reference is part of the portfolio signal.

When to copy this template

When a /root phase ships a new OSS project (Y1 Phase 2: rxp; Y1 Phase 4: pulse; Y3 Phase 20: triage; etc.)
When a project hits v0.1.0 and the placeholder README needs to become a real README
When forking the template into a new project that’s NOT under /root (the shape generalizes)

The template — copy this verbatim into the new project’s `README.md`

# <project-name>

<!-- One-line tagline. What it does, in nine words or fewer. -->
**L7 reverse proxy specialized for LLM traffic — routing, caching, observability, fallback.**

<!-- Status badges. Real or placeholder. -->
[![CI](https://github.com/<user>/<project>/actions/workflows/ci.yml/badge.svg)](.../ci.yml)
[![Go Report](https://goreportcard.com/badge/github.com/<user>/<project>)](https://goreportcard.com/report/github.com/<user>/<project>)
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](LICENSE)
[![Version](https://img.shields.io/github/v/release/<user>/<project>)](https://github.com/<user>/<project>/releases)

---

## What this is

<!-- 1-2 sentence summary that earns the next 30 seconds of reading. -->

`<project>` is a Go service that sits between application code and LLM providers — providing model routing, response caching (exact + semantic), token-bucket rate limiting, fallback chains, and OpenTelemetry observability.

It mirrors the production-grade LLM-gateway pattern that frontier AI labs build internally, scoped to homelab + small-team deployments.

## Why it exists

<!-- 2-3 sentences. The pattern it instantiates + the gap it fills. -->

Every organization that uses LLMs at scale eventually builds some variant of this gateway. The pattern is well-understood: routing by query characteristics, caching for cost reduction, observability for cost/latency control, fallback chains for resilience. **What's missing is a small, K8s-native, opensource implementation** that fits a homelab and demonstrates the pattern operationally — instead of being a tutorial wrapper around the OpenAI SDK.

That's the gap `<project>` fills.

## Quickstart — under 5 minutes

<!-- The 5-minute path. Working in under 5 commands. -->

```bash
# 1. Install via Helm on a K8s cluster (basecamp or any other).
helm repo add abukix https://abukix.github.io/charts
helm install llm-gateway abukix/llm-gateway \
  --set anthropic.apiKey=<your-key> \
  --set openai.apiKey=<your-key>

# 2. Verify the service is up.
kubectl port-forward svc/llm-gateway 8080:80 &
curl http://localhost:8080/health

# 3. Send a request.
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-3-5-sonnet",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Architecture

                Client (your service)
                       │
                       ▼ HTTPS
              ┌────────────────────┐
              │   llm-gateway      │
              │                    │
              │ • Rate limit       │
              │ • Cache check      │
              │ • Route decision   │
              │ • Fallback chain   │
              │ • OTel trace       │
              └─────────┬──────────┘
                        │
        ┌───────────────┼───────────────┐
        ▼               ▼               ▼
  Local vLLM     Anthropic API   OpenAI API

The gateway is a single Go binary. State (cache, rate-limit counters) lives in Redis + pgvector. Configuration is declarative YAML reconciled by your GitOps tool. See docs/ARCHITECTURE.md for the full picture.

Features

Routing: rule-based (YAML); routes by query characteristics, cost budget, or model availability.
Exact caching: Redis-backed; per-tenant; configurable TTL.
Semantic caching: pgvector-backed; embedding-based similarity; configurable threshold.
Rate limiting: token-bucket per tenant + per model; counts both requests AND tokens.
Fallback chains: configurable per route (e.g., vllm → claude → gpt-4o-mini).
OpenTelemetry: traces every request including the decision history.
Content policy hooks: pre-request (PII redaction) and post-response (output filter).
Prompt-as-resource: versioned prompts served by name via /v1/prompts/<name>@<version>.

Installation

Helm (recommended for K8s deployments)

helm repo add abukix https://abukix.github.io/charts
helm install llm-gateway abukix/llm-gateway -f values.yaml

See charts/llm-gateway/values.yaml for full configuration.

Docker

docker run -p 8080:8080 \
  -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  ghcr.io/<user>/llm-gateway:latest

Binary

go install github.com/<user>/<project>/cmd/llm-gateway@latest
llm-gateway --config ./config.yaml

Configuration

Minimal config.yaml:

listen: ":8080"
routes:
  - match:
      path: "/v1/chat/completions"
    primary: anthropic
    fallback: [openai, vllm-local]
cache:
  exact:
    backend: redis
    ttl: 1h
  semantic:
    backend: pgvector
    threshold: 0.97
rate_limit:
  per_tenant:
    requests_per_minute: 60
    tokens_per_minute: 100000

Full reference: docs/CONFIG.md.

Development

git clone https://github.com/<user>/<project>.git
cd <project>
go mod download
make test       # runs tests
make build      # builds the binary
make docker     # builds the container image
make e2e        # spins up Redis + vLLM + runs e2e suite

Tooling required: Go 1.22+, Docker (or colima/podman), make. See CONTRIBUTING.md for full setup.

/root context

This project is part of the Abukix — Building an AI Platform in Public portfolio. It ships from Year 5 Phase 46 of the /root program and occupies Tier 7 (LLM/Agent) of basecamp, the multi-tier K8s-native AI platform.

Sibling Abukix projects:

Tier	Project	What it does
2	`pulse`	Probe scanner — Prometheus metrics
2	`triage`	On-call dashboard
3	`terralabs`	Terraform + Crossplane modules
4	`platform-ctl`	Paved-road CLI + custom operator
5	`data-tier`	Iceberg + Spark + Strimzi templates
8	`aiops`	Custom operator for AIOps

Contributing

PRs welcome. See CONTRIBUTING.md for: code style, test expectations, the PR review process, and the project’s stance on AI-generated code (see AI Learning Protocol).

Issues: bug reports + feature requests via GitHub Issues. For larger design conversations, open a Discussion.

License

Apache License 2.0. See LICENSE.

Acknowledgments

The LLM-gateway pattern is industry-shared. LiteLLM, Portkey, and Helicone are parallel OSS / managed implementations.
vLLM (UC Berkeley) for the local-LLM serving runtime this project routes to.
The CNCF / kubebuilder ecosystem for the operator patterns this project builds on.

Built as part of Abukix — Building an AI Platform in Public. Five years, one platform, opensource end-to-end.


---

## What good looks like

- **The first 100 words pass the 30-second test.** Tagline + What this is + Why it exists. A senior engineer who reads only these and bounces should still know what the project is for.
- **Quickstart actually works in under 5 minutes.** Test it. Test it again after every release. Stale quickstarts kill credibility.
- **The architecture diagram is text-art** unless the project is GUI-heavy. ASCII / Unicode renders in every viewer; PNG diagrams break when GitHub re-images them.
- **The /root context section is non-negotiable.** Each project's place in the portfolio is part of the project's identity. Without the cross-reference, the project looks like a one-off.
- **License clear and Apache 2.0 default.** Apache 2.0 protects you from patent claims AND permits commercial use AND is OSS-friendly. Don't deviate without a reason.
- **Acknowledgments are honest.** The patterns this project instantiates have origins. Naming them is good intellectual hygiene + good citizenship.

## Variations by project type

### CLI tool (rxp, pulse, platform-ctl)
- Quickstart leads with `brew install` / `go install` / `cargo install`.
- Architecture is short or skipped.
- Usage examples are command-line transcripts.

### Service (triage, llm-gateway, mcp-servers, aiops)
- Quickstart leads with Helm install.
- Architecture diagram is essential.
- Configuration section is detailed (services have surfaces to tune).

### Template / module library (terralabs, data-tier)
- Quickstart leads with `module "..." { source = "..." }`.
- Architecture is per-module.
- Skip the binary install section.

### Custom operator (platform-ctl operator, aiops operator)
- Quickstart leads with the CRD example.
- Architecture diagram shows the reconcile loop.
- Include a "What this operator does NOT do" section to set scope.

## Anti-patterns

| Anti-pattern | Why |
|---|---|
| 2,000-word README | Nobody reads it. Split into README (the front door) + docs/ (the deep dive). |
| No quickstart | Reader can't try it. Bouncing within 30 seconds. |
| Quickstart that doesn't work | Worse than no quickstart. Test it on every release. |
| Status badges that 404 | Either real or remove them. Broken badges signal abandonment. |
| Marketing language ("revolutionary," "game-changing") | Voice anchor fail. The brand is direct, pattern-first, no fluff. |
| Hiding the license / making it ambiguous | Apache 2.0 file in the repo root. Visible in the README. Don't make license a discovery problem. |
| No `/root context` section | The portfolio's coherence depends on these cross-references. |
| Acknowledgments that omit the pattern's origin | Bad intellectual hygiene. Name where the pattern came from. |
| README that's last-edited 18 months ago | Add a "Status" line at the top: `Active`, `Maintained`, `Archived`. If stale, mark archived honestly. |

## File companions

A complete repo has these alongside the README:

/ ├── README.md ← this template ├── LICENSE ← Apache-2.0 ├── CONTRIBUTING.md ← how to contribute ├── CHANGELOG.md ← keep-a-changelog format ├── .pre-commit-config.yaml ← gitleaks + standard hooks ├── docs/ │ ├── ARCHITECTURE.md │ ├── CONFIG.md │ └── adrs/ ← project-specific ADRs (separate from /root ADRs) └── …source…


Templates for `CONTRIBUTING.md`, `CHANGELOG.md`, etc. live in this repo too — or copy from a sibling Abukix project.

## Cross-references

- [brand/identity](/brand/identity/) — naming conventions + the /root context cross-reference convention
- [blog-post template](/meta/blog-post-template/) — when a project ships, its launch post follows the blog-post template
- [adr template](/meta/adr-template/) — project-specific ADRs follow the same template
- [projects/index](/projects/) — the index of all 9 Abukix OSS projects this template instantiates
- [Writing Templates index](/meta/)