Part of the series: Production-Grade Concurrent AI Systems in Go
→ Full code for this post: github.com/madmmas/go-concurrent-ai-systems/tree/part-19 → Diff from Part 18: compare/part-18...part-19 → Run it: go run ./cmd/news-processor -articles=4 -chunks=3 inside arc-2-production/part-19-rag-pipeline
Every part of Arc 2 introduced one pattern in isolation: fan-out, pipeline stages, structured concurrency, retries, rate limiting, circuit breaking, backpressure, token streaming, goroutine leak prevention. Each was demonstrated against a simplified pipeline where the focus was on understanding the mechanism rather than building a complete system.
Part 19 builds the complete system.
Retrieval-Augmented Generation is the dominant architecture for production AI applications that need grounded, accurate responses. Rather than asking an LLM to rely on its training data — which may be stale, incomplete, or hallucinated — RAG retrieves relevant context from a live knowledge base and gives it to the LLM as part of the prompt. The result is an answer that is anchored to real, current information.
For a news intelligence platform, RAG is the natural fit. When a user asks "what happened in the semiconductor market this week?", the answer should come from articles the platform actually processed — not from the LLM's knowledge of events that may predate its training cutoff.
The Pipeline Architecture
Articles
↓
[Chunker] workers=5 — split each article into overlapping text chunks
↓
[Embedder] workers=3 — generate vector embedding per chunk (LLM call)
↓
[Collector] — group chunks back by article
↓
[Generator] workers=3 — produce grounded answer from chunks (LLM call)
↓
RAGResults
Each stage is a concurrent pool connected to the next by a channel — the same structure as Part 11's multi-stage pipeline. Worker counts are asymmetric for the same reasons: chunking is cheap CPU work and can use many workers, embedding and generation are LLM calls and should be rate-limited.
The collector between embedder and generator is not a worker pool — it is a single goroutine that groups chunks by article ID as they arrive from the embedder, then feeds each complete article's chunk set to the generator. This fan-in step is necessary because embedding processes chunks individually (one chunk, one LLM call) but generation needs all chunks for an article together to produce a grounded answer.
Stage by Stage
Chunker splits each article into ChunksPerDoc text segments. In production these are overlapping chunks — each chunk contains some text from the previous one to preserve context at boundaries. The simulator uses fixed-size chunks for simplicity.
func (p *RAGPipeline) runChunker(ctx context.Context, in <-chan model.Article) <-chan Chunk {
out := make(chan Chunk, p.ChunksPerDoc*cap(in)+1)
var wg sync.WaitGroup
for w := 0; w < p.ChunkWorkers; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for article := range in {
if ctx.Err() != nil { return }
fmt.Printf("[chunk] article %d → %d chunks\n", article.ID, p.ChunksPerDoc)
for i := 0; i < p.ChunksPerDoc; i++ {
out <- Chunk{
ArticleID: article.ID,
ChunkID: i,
Text: fmt.Sprintf("Chunk %d of article %d", i, article.ID),
}
}
}
}()
}
go func() { wg.Wait(); close(out) }()
return out
}
Embedder takes each chunk and generates a vector embedding via an LLM call. The embedding captures the semantic content of the chunk — it is a high-dimensional float vector where similar texts have similar vectors. The embedder is the rate-limited stage: each chunk requires one LLM call, and with ChunksPerDoc=3 and 4 articles, that is 12 embedding calls.
func (p *RAGPipeline) runEmbedder(ctx context.Context, in <-chan Chunk) <-chan Chunk {
out := make(chan Chunk, cap(in)+1)
var wg sync.WaitGroup
for w := 0; w < p.EmbedWorkers; w++ {
wg.Add(1)
go func() {
defer wg.Done()
for chunk := range in {
articleCtx, cancel := context.WithTimeout(ctx, p.Timeout)
if err := p.llm.Call(articleCtx, "Embed", chunk.ArticleID); err != nil {
cancel()
out <- chunk // pass through without embedding on failure
continue
}
cancel()
chunk.Embedding = []float32{0.1, 0.2, 0.3} // simulated embedding vector
fmt.Printf("[embed] article %d chunk %d\n", chunk.ArticleID, chunk.ChunkID)
out <- chunk
}
}()
}
go func() { wg.Wait(); close(out) }()
return out
}
Collector groups embedded chunks by article. Because chunks from different articles interleave in the embedder output — article 2's chunk 0 may finish embedding before article 1's chunk 2 — the collector maps ArticleID → []Chunk until all chunks for an article have arrived, then hands the complete set to the generator.
Generator takes each article's chunk set and makes one LLM call to produce a grounded answer. The prompt in production would contain the actual chunk text; here it is simulated.
What the Output Shows
RAG Pipeline: 4 articles × 3 chunks each
Stages: chunk(5) → embed(3) → generate(3)
[chunk] article 1 → 3 chunks
[chunk] article 2 → 3 chunks
[chunk] article 3 → 3 chunks
[chunk] article 4 → 3 chunks
[1] Embed started (799ms)
[1] Embed started (540ms)
[1] Embed started (395ms)
[1] Embed completed
[embed] article 1 chunk 2
[2] Embed started (257ms)
...
[3] Generate started (292ms)
[4] Generate started (260ms)
[1] Generate started (550ms)
[4] Generate completed
[generate] article 4 answer from 3 chunks
...
Article 4: 3 chunks → "RAG answer for article 4 using 3 chunks"
Article 3: 3 chunks → "RAG answer for article 3 using 3 chunks"
Article 1: 3 chunks → "RAG answer for article 1 using 3 chunks"
Article 2: 3 chunks → "RAG answer for article 2 using 3 chunks"
Total: 3.194s
All four articles chunk simultaneously. All twelve chunks embed concurrently across three embedder workers. As soon as all three chunks for an article are embedded, that article's chunk set moves to the generator. Articles 3 and 4 — whose chunks happened to embed faster — get their answers before articles 1 and 2, even though all four started at the same time. The pipeline produces results in completion order, not input order.
Four articles, twelve embedding calls, four generation calls, 3.194 seconds total. A sequential version would take every embedding and generation call serially — probably 10–15 seconds for the same work, depending on latency.
The Patterns This Pipeline Uses
The RAG pipeline is where every pattern from Arc 2 has a concrete role, not just a pedagogical one.
Fan-out (Part 10): Each article fans out into ChunksPerDoc chunks simultaneously in the chunker. The embedder workers further fan out embedding calls concurrently.
Stage isolation (Part 11): chunk → embed → collect → generate are separate stages connected by channels. The embed workers do not know how many articles are coming; they range over the input channel until it closes.
Context propagation (Part 8 / Part 9): Every LLM call receives a per-call timeout derived from the pipeline context. Cancelling the pipeline context stops all stages within one timeout cycle.
Rate-limiting (Part 14): In a production version, the embedder and generator stages would share a TokenBucket limiting total LLM calls per second. The stage isolation makes adding the bucket trivial — it sits at the LLM call boundary inside each worker, unchanged from Part 14.
Backpressure (Part 16): The chunk channel's capacity is bounded — p.ChunksPerDoc * cap(artCh) + 1. If the embedder falls behind, the chunker backs up. No unbounded queue growth.
Goroutine safety (Part 18): Every stage uses the pattern from Part 18 — buffered output channels, deferred WaitGroup-based close, context cancellation path. The goroutine count before and after ProcessAll should be identical.
Scaling the Knobs
The RAG pipeline exposes five tuning parameters:
rag := pipeline.New(
llm,
5, // chunk workers — cheap, use many
3, // embed workers — LLM calls, rate limited
3, // generation workers — LLM calls, rate limited
3, // chunks per document
5*time.Second, // per-call timeout
)
For a real deployment, embed and generation worker counts are driven by your provider's rate limit. If the provider allows 10 embedding calls per second and each call takes about 1 second on average, 8 embed workers keeps you just under the limit. The same math applies to generation. Adding a TokenBucket from Part 14 in front of each LLM call makes these limits hard enforcements rather than approximations.
Chunks per document controls the granularity of retrieval. More chunks per document means finer-grained retrieval — a query can match a specific paragraph rather than a full article — but more embedding calls per document. Fewer chunks per document is cheaper but less precise. Typical production values range from 3 to 20 depending on average document length and retrieval precision requirements.
What's Next
Part 19 is the culminating application of everything built in Arc 2. Part 20 closes the arc with observability: the metrics, latency percentiles, and error tracking that tell you whether the pipeline is actually working as designed in production — and where to look when it is not.
See you in Part 20.
This is Part 19 of the series "Production-Grade Concurrent AI Systems in Go." Read Part 18 — Goroutine Leaks or continue to Part 20 — Observability: Seeing What the Pipeline Is Actually Doing.