Phase
Production Concurrent AI
Fan-out/fan-in, multi-stage pipelines, errgroup, retries, rate limiting, circuit breakers, backpressure, streaming, and observability for production AI systems.
1 published · 10 upcoming
- Part 10Published
From Sequential Scripts to Concurrent AI Pipelines in Go — Part 10
Arc 1 ran three AI tasks per article one after another. Part 10 runs them all at once — fan-out launches them concurrently, fan-in collects all three results, and per-article time drops from the sum to the slowest.
- Part 11In progress
From Sequential Scripts to Concurrent AI Pipelines in Go — Part 11
Part 10 ran tasks concurrently inside one worker. Part 11 runs stages concurrently across the whole pipeline — scrape, clean, embed, and summarise each get their own worker pool, tuned to their specific bottleneck.
- Part 12In progress
From Sequential Scripts to Concurrent AI Pipelines in Go — Part 12
Part 10 fanned out tasks and collected results. Part 12 adds the missing piece: when one task fails, cancel the rest immediately. errgroup does this in a handful of lines — and understanding how it works is more useful than treating it as a black box.
- Part 13In progress
From Sequential Scripts to Concurrent AI Pipelines in Go — Part 13
Part 12 showed how to cancel sibling tasks on first failure. Part 13 shows what to do next: retry with exponential backoff and jitter, distinguish retryable from permanent errors, and route exhausted articles to a dead letter queue.
- Part 14In progress
From Sequential Scripts to Concurrent AI Pipelines in Go — Part 14
Part 13 added retries for when the provider returns 429. Part 14 prevents the 429 from happening in the first place — a token bucket rate limiter that caps outgoing LLM calls per second, regardless of how many workers are running.
- Part 15In progress
From Sequential Scripts to Concurrent AI Pipelines in Go — Part 15
Rate limiting smooths steady-state traffic. Retries recover from transient failures. Neither helps when a provider is down for minutes. The circuit breaker detects sustained failure and stops calling entirely — failing fast, protecting the pipeline, and probing for recovery.
- Part 16In progress
From Sequential Scripts to Concurrent AI Pipelines in Go — Part 16
When a producer generates work faster than consumers can process it, the queue grows without bound. A bounded channel is the fix — when it fills, the producer blocks automatically. No explicit signal, no coordination: the channel capacity is the backpressure contract.
- Part 17In progress
From Sequential Scripts to Concurrent AI Pipelines in Go — Part 17
Every previous part treated an LLM response as a single value that arrives when the call completes. Real LLM APIs stream tokens incrementally — you receive the first word in hundreds of milliseconds and the rest follow one by one. A channel between producer and consumer is the natural Go model for this.
- Part 18In progress
From Sequential Scripts to Concurrent AI Pipelines in Go — Part 18
A goroutine that starts and never exits is a leak. Unlike memory leaks, goroutine leaks are invisible — no crash, no error, just a process that slowly accumulates blocked goroutines until memory and scheduler overhead degrade everything. This part shows how to detect them, reproduce them intentionally, and fix them.
- Part 19In progress
From Sequential Scripts to Concurrent AI Pipelines in Go — Part 19
The flagship of Arc 2: a full concurrent RAG pipeline — chunk, embed, generate — where every stage from Part 11, every resilience pattern from Parts 13-15, and every flow control mechanism from Parts 16-18 comes together in one coherent system.
- Part 20In progress
From Sequential Scripts to Concurrent AI Pipelines in Go — Part 20
A pipeline that cannot be observed cannot be operated. Part 20 adds throughput, latency percentiles, and error breakdown by type — the three metrics that tell you whether the pipeline is healthy, where it is slow, and what is failing. This is the foundation for Prometheus and OpenTelemetry integration.