What the Next Five Modules Unlock

You have a working baseline. It passes the ship gate. It probably failed 50-70% of the boss challenge. Each of the remaining five modules takes aim at a specific failure category and teaches the technique that fixes it.

The failure it solves: definitional drift and cross-chunk synthesis.

When your baseline fails because the corpus says "context caching" but the question asked about "prompt caching," that is an embeddings problem. When an answer spans two chunks that fixed-size chunking put far apart, that is a chunking problem. Module 2 makes you compare:

• Voyage vs. OpenAI embeddings on your corpus
• Fixed-size vs. semantic vs. hierarchical chunking
• Retrieval-metric deltas across all combinations

Ship gate: a documented comparison across three configurations, with a clear winner.

The failure it solves: needle-in-haystack.

Pure vector retrieval ranks ten lexically similar chunks roughly equal. When the right one is at rank 6, your stuff prompt never sees it. Module 3 adds:

• BM25 keyword retrieval alongside vector retrieval
• Reciprocal Rank Fusion (RRF) to combine the two
• A cross-encoder reranker (Cohere Rerank 3) to reshuffle the top 20 into the top 5 you actually send

Ship gate: measurable Recall@5 improvement over Module 1 on your own corpus.

The failure it solves: multi-hop questions.

"Which chapter governs public works solicitations on HIePRO?" requires retrieving what HIePRO is, then retrieving the chapter number. Baseline RAG cannot do two hops. Module 4 teaches:

• HyDE (Hypothetical Document Embeddings)
• Multi-query rewriting
• Question decomposition

Ship gate: a query transformer with A/B comparison against the raw-question baseline.

The failure it solves: you have no idea if changes help.

Up through Module 4, you have been eyeballing whether your RAG got better. That does not scale. Module 5 builds:

• Retrieval metrics (Recall@k, MRR, nDCG)
• LLM-as-judge for answer quality, using Claude Haiku 4.5 as the cheap judge
• A regression harness that compares runs across module configurations

Ship gate: a 30-question eval set scored across every prior module's configuration. You can finally answer "is Module 3 actually better than Module 1 on my corpus?"

The failure it solves: "it works on my laptop."

Module 6 takes your best RAG and makes it production-shaped:

• Prompt caching (Anthropic's cache_control) to cut costs
• Structured logging for observability
• Cost dashboards — reading the JSONL log you have been writing since Module 1
• Prompt versioning so you can A/B test prompt changes
• Failure-mode handling for empty retrievals, rate limits, and timeouts
• Deploy behind a reverse proxy

Ship gate: a production-deployed RAG with cost dashboards and prompt versioning, measurably better than Module 1 by at least 25% on your boss challenge.

By the end of Module 6, you will have:

• Six configurations of the same pipeline, all measured against the same boss
• Documented cost and latency for each
• A deployed production instance
• A completion artifact (a signed PDF plus a /verify/<hash> URL) describing what you built

The completion artifact is portfolio-grade. It says "I built a RAG system, here is how it performs on my corpus, here is what I learned at each module." That is the output.

Go ship.