Twenty-four weeks on the modern AI engineering stack. The agent loop as a discipline. Retrieval as engineering. Memory as a context-budget cache. Multi-agent orchestration graphs. Local inference on Ollama, vLLM, and NeMo. Tool surfaces over the Model Context Protocol. Open-Claude-compatible (OpenClaw) runtimes. Eval-in-prod. Free, forever.
§ I · The Program
Crunch Agents is the AI agent systems engineering specialization of the Code Crunch academy — a 2026 redesign for the Crunch Labs tier. It is built for engineers who want to ship LLM-backed products: not notebook demos, but agentic systems with versioned prompts, hybrid retrieval over a private corpus, memory tiers that survive a forty-turn conversation, multi-agent supervisor graphs, and a tool surface served over the Model Context Protocol.
This track is deliberately distinct from C5 (Crunch AI · Data Science). C5 owns classical machine learning through deep learning — feature engineering, gradient boosting, CNNs, transformers as architectures, PyTorch fundamentals. C23 picks up where C5 leaves the room: LLMs as components in a system, not as research artifacts. The discipline here is systems engineering on top of models you mostly do not train. By Week 24 you have shipped a multi-agent research assistant served from your own vLLM cluster, instrumented with OpenTelemetry Gen-AI conventions, evaluated with Ragas and a calibrated LLM-as-judge, and survived a chaos drill in which a GPU node dies, a tool is prompt-injected, and a vector index corrupts — all in the same week.
"If C5 makes you a model builder, C23 makes you the engineer who keeps an agentic product alive in production." — Crunch Agents, course README
§ II · Who It's For
Agents is opinionated about its audience. C1 (Code Crunch Convos) is the floor — you should be able to read Python comfortably, run Docker without flinching, and survive a Linux shell.
Has shipped Flask, FastAPI, or Django. Has called the OpenAI API. The gap between demo and production is fog — wants the durable patterns: retrieval as engineering, eval as engineering, the agent loop with rigor.
Came up on scikit-learn, XGBoost, PyTorch. Can train, calibrate, fight a leakage bug. Lacks muscle memory for RAG, MCP, vLLM, agent graphs, tool-use safety, and the operating model of an LLM-backed product.
Runs platforms — K8s, Prometheus, on-call, autoscaling. The new pager is GPU pools and token budgets. Wants vLLM topology, NeMo Inference, continuous batching, KV-cache economics, and a real runbook for an agentic system.
Has a Ph.D. or near-Ph.D. and reads a paper a week. The applied side — testable prompts, observability, retrieval pipelines, cost engineering, the messy reality of shipping — is the gap. C23 closes it without insulting the background.
§ III · Four Phases
The arc of the program is composed in four phases — six weeks each — each building on the last like floors of a building.
LLMs as system components. Tokens, context, sampling, structured output. Prompt-as-code with promptfoo and Langfuse. Tool calling across vendors. The agent loop (ReAct, plan-and-execute, reflection). Local inference bring-up on Ollama, llama.cpp, and vLLM.
Embeddings, chunking, hybrid search, reranking. Vector stores in production — pgvector, Qdrant, Weaviate. Memory tiers: episodic, semantic, procedural. Multimodal RAG. Evaluation as engineering with Ragas and a calibrated LLM-as-judge.
LangGraph state graphs. Mastra and durable execution. The Model Context Protocol in depth, plus the open-Claude-compatible OpenClaw runtime family. Fine-tuning with LoRA / QLoRA. Safety, prompt injection, threat modeling. Observability via OTel Gen-AI conventions.
vLLM clusters with continuous batching. NeMo Inference and the NVIDIA stack. Cost engineering, model routing, semantic cache. Two capstone sprints. Week 24 chaos drill: GPU node loss, prompt-injection on a tool, vector-index corruption. Postmortem.
§ IV · The Curriculum
Each entry corresponds to a folder in the GitHub repository with lecture notes, exercises, challenges, a quiz, homework, and a mini-project. Detailed acceptance criteria live in the syllabus.
Tokenizer → context → forward pass → sampling · decoder-only transformer at a systems level · open-weights (Llama 4, Qwen 3, Mistral, Gemma 3, DeepSeek) vs closed-weights · reading a model card without falling for the benchmark.
Build llmpick — multi-model recommender
BPE · SentencePiece · tiktoken · the price of long context · KV cache · temperature, top-p, top-k, min-p · structured output with outlines, guidance, xgrammar · grammar-constrained decoding.
Tokenization explorer + grammar-constrained JSON
The prompt is code · system/user/assistant roles · few-shot, CoT (honest), self-consistency · jailbreak surface · versioning with promptfoo and Langfuse · spec-then-implement with Claude Code and Cursor.
Promptfoo regression suite — 6 versions, 30 golden examples
Function calling across vendors · open-source equivalents on Llama and Qwen · MCP as the cross-vendor protocol (stdio, SSE, streamable HTTP) · JSON-mode vs grammar-constrained · tool-use as RCE.
4-tool agent on Claude + local Qwen 2.5 7B
ReAct, plan-and-execute, reflection, self-critique · the "infinite tool-call" failure · step / token / time / cost budgets · claude-agent-sdk, OpenAI Agents SDK, AWS Strands, Google ADK at survey depth.
~150-line ReAct loop, no framework, vs claude-agent-sdk
Ollama for iteration · llama.cpp for portability · vLLM for throughput · SGLang, TensorRT-LLM, TGI · quantization (GGUF, AWQ, GPTQ, bitsandbytes) · speculative decoding · Apple MLX.
Same 7B on Ollama, llama.cpp, vLLM — measure tokens/sec, p95, VRAM
Open families — BGE, GTE, jina-embeddings-v3, nomic-embed, E5-Mistral · vendor embeddings · the MTEB leaderboard, read skeptically · ANN indexes (HNSW, IVF, ScaNN).
Three open + one vendor embedding over a legal corpus in pgvector
Token-window, semantic-paragraph, recursive, sliding-window, late chunking · Unstructured, MinerU, LlamaParse, PyMuPDF · OCR (Tesseract, Surya) · table extraction · chunk size as a hyperparameter.
Chunking A/B harness — MRR, Recall@5, faithfulness delta
BM25 + dense · reciprocal rank fusion · bge-reranker-v2, Cohere, ColBERT late-interaction · text-to-SQL · query rewriting · HyDE.
Cumulative lift: dense → +BM25 → +RRF → +reranker → +HyDE
pgvector, Qdrant, Weaviate, Milvus, Chroma · filtered ANN · backup, replication, rebuild · GraphRAG and knowledge-graph hybrids · agentic-RAG patterns.
Same pipeline across pgvector, Qdrant, Weaviate — ingest, query, recovery
Three tiers — episodic, semantic, procedural · summarization (rolling, hierarchical, map-reduce) · eviction (LRU, salience-weighted) · MemGPT / Letta · the "lost in the middle" effect.
40-turn chat with three-tier memory vs no-memory baseline
VLMs — LLaVA, Qwen2-VL, Phi-Vision, InternVL · CLIP / SigLIP · Whisper, Piper, XTTS · SDXL / Flux · Ragas as standard · DeepEval, promptfoo, TruLens · calibrated LLM-as-judge.
Full Ragas suite — faithfulness, context recall, precision, relevancy
State graphs, conditional edges, checkpoints, persistence · supervisor, swarm, hierarchical patterns · honest critique of AutoGen and CrewAI · when a state machine beats an agent.
Re-implement Week 5 as a LangGraph with SQLite checkpointer
Mastra (TypeScript-first) · Inngest for event-driven durable execution · Trigger.dev · Temporal at scale · Python-first vs TS-first agent stacks, honestly.
Supervisor in Mastra + LangGraph · Inngest-triggered research run
MCP server / client, transport modes, tool / resource / prompt primitives · writing MCP servers in Python and TS · the open OpenClaw family — open MCP gateways, self-hosted MCP servers, community Claude-compatible loops · MCP security review.
Two MCP servers (FS + private-corpus search) over stdio and HTTP
When not to fine-tune (almost always) · LoRA, QLoRA, DoRA · SFT, DPO, ORPO, KTO · NeMo Framework, Axolotl, Unsloth · RLHF/RLAIF at survey depth.
Unsloth SFT of Qwen2.5-7B on a 500-example custom DSL
Direct vs indirect prompt injection · jailbreak surface · output filtering (regex, classifier, judge) · Llama Guard, OpenAI Moderation, Perspective · the OWASP LLM Top 10 · tool-use threat modeling.
Red-team your Week 15 MCP agent — 25 attacks, 3 defenses
OpenTelemetry Gen-AI semantic conventions · Langfuse (self-hosted), Arize Phoenix, LangSmith, Helicone · token accounting per route/user/model · latency SLOs · eval-on-traces.
Three dashboards: token-by-route, p95-by-step, retrieval over time
Continuous batching · paged attention · prefix caching · tensor / pipeline parallel · multi-replica vLLM behind LiteLLM · speculative decoding · OpenAI-compatible API.
vLLM + LiteLLM on H100 — concurrency 1/8/32/128 + break-even memo
NVIDIA NeMo Inference for production serving · TensorRT-LLM kernels · Triton Inference Server · NeMo Guardrails as policy · honest NeMo vs vLLM comparison.
Qwen2.5-14B on NeMo + Triton — benchmark vs vLLM, add a guardrail
Routing — small for easy, big for hard · semantic cache (GPTCache or self-built pgvector) · prompt compression (LLMLingua) · batched inference · per-feature cost accounting · speculative decoding as a cost lever.
Router: Qwen2.5-7B local vs Claude vendor + 0.92-cosine semantic cache
Architecture review · hybrid retrieval over a 10 GB private corpus · three-tier memory wiring · Mermaid architecture diagram · 6-page architecture document.
Retrieval + memory layers of the agentic research assistant
Supervisor + retrieval + code + writing agents · MCP tool surface · vLLM cluster deploy · Ragas + calibrated LLM-as-judge on 100-question gold set · OTel traces to Langfuse + Phoenix · cost-tracked routing.
End-to-end system shipped as deploy URL or docker compose image
Eval-in-prod with shadow traffic · blue/green model deploys · canary by user cohort · the on-call runbook for an agentic system · the chaos drill · the postmortem · interview prep.
GPU node loss · prompt-injection on a tool · vector-index corruption
§ V · The Toolchain
Every primary tool below is open-source or has an open-weights, self-hosted path. Vendor APIs (OpenAI, Anthropic, Gemini, Bedrock) are taught as the production scale path and the frontier-capability path — never as the only path.
§ VI · Skills You Will Carry
By the end of Week 24, you are able to do each of the following — credibly, on a real system, in front of a real reviewer.
§ VII · The Capstone
Weeks 22–24 are reserved for a single substantial system — the kind a real product team would scope across a quarter. Architecture document, live deploy, video walkthrough, chaos-drill postmortem.
Capstone Brief
A multi-agent system — supervisor plus retrieval, code, and writing agents — operating over a 10 GB private corpus with hybrid retrieval, three-tier memory, an MCP tool surface, and self-hosted serving on a vLLM cluster with continuous batching. Cost-tracked routing between a local 7B/13B and a frontier vendor model. Full eval suite. OTel tracing. Red-team report. Chaos-drill postmortem.
§ VIII · Getting Started
The setup is intentionally lightweight. If you have a Linux, macOS, or WSL2 laptop and can run a terminal command, you can begin Week 1 today. No GPU required on day one.
# 1. Clone the curriculum repository git clone https://github.com/CODECRUNCHWORLDWIDE/C23-CRUNCH-AGENTS.git cd C23-CRUNCH-AGENTS # 2. Pull a local model (Apple Silicon / Linux / WSL2) ollama pull llama3.1 # or qwen2.5:7b, mistral, gemma3 # 3. Set up a Python venv with uv (fast, reproducible) uv venv && source .venv/bin/activate uv pip install -r requirements.txt # 4. Open Week 1 README and begin $EDITOR curriculum/week-01-llm-as-system-component/README.md
Need the hardware budget, rented-GPU recipes, or CPU-only fallbacks? See the README.
§ IX · Frequently Asked
Not on day one. A 16 GB laptop carries you through Week 6, and Apple Silicon via Metal carries you further. From Week 7 onward a 24 GB GPU (RTX 3090 / 4090, used or new, or a rented A10 / L4 at ~$0.50–$1.00 per hour) unlocks the local-embedding and reranker labs. Two specific labs (vLLM cluster, NeMo Inference) benefit from H100 access on rented spot instances. Every lab has a CPU-only fallback. Total compute budget across the course on rented GPUs is roughly $30.
It helps but is not required. C5 buys you intuition for tokenization, attention, training loops, and PyTorch idioms — useful in Weeks 16–17 when we fine-tune. C23 re-derives the LLM internals from a systems perspective in Weeks 1–3 so you do not need C5 to follow. What you actually need on day one is Python fluency (C1), Linux comfort, and Docker basics. If you have those, you are ready.
Both, by design. Every lab has an open-weights, self-hosted path (Ollama, llama.cpp, vLLM, NeMo). Vendor APIs (OpenAI, Anthropic, Gemini, Bedrock) are taught as the production scale path and the frontier-capability path. By Week 21 you are running cost-tracked routing between a local 7B/13B and a vendor frontier model, and you can defend the split in an architecture review.
C5 owns classical ML through deep learning — feature engineering, gradient boosting, CNNs, transformers as architectures, PyTorch fundamentals. C5 makes you a model builder. C23 picks up where C5 leaves the room: LLMs as components in a system, not as research artifacts. C23 makes you the engineer who keeps an agentic product alive in production — retrieval, memory, orchestration, MCP, serving, eval, observability, cost, on-call. Many engineers take both.
If you finish C23, yes — credibly. Week 18 builds the OTel-instrumented observability stack. Week 21 is cost engineering. Week 24 is the chaos drill (GPU node loss, prompt-injection on a tool, vector-index corruption) and the postmortem. The career engineering pack ships a real production-runbook.md with alerts, dashboards, incident classes, escalation, and a postmortem template. The course is built to graduate engineers who can carry the pager on a Friday night.
OpenClaw is the umbrella we use for the emerging family of open-source, Claude-compatible MCP-native agent runtimes — open MCP gateways, self-hosted MCP servers, community-maintained Claude-compatible agent loops. Week 15 covers MCP in depth (transports, primitives, security review) and teaches you to write MCP servers in Python and TypeScript, consume them from Claude Desktop and Cursor, and operate them through an OpenClaw-style gateway. MCP is the USB-C of agent tooling — it is the present, and it is open.
§ X · Begin
Open the repository. Read Week 1. The agent loop is yours.