Part of the series: Production-Grade Concurrent AI Systems in Go
→ Full code for this post: github.com/madmmas/go-concurrent-ai-systems/tree/part-08 → Diff from Part 7: compare/part-07...part-08 → Run it: go run ./cmd/news-processor inside arc-1-foundations/part-08-context-timeouts
The worker pool from Part 7 is fast, race-free, and bounded. But it has a quiet assumption baked into every LLM call: that the provider will eventually respond.
Real providers don't always do that. A single call can hang for thirty seconds, two minutes, or indefinitely — a model instance that's overloaded, a network partition, a provider incident. With our current design, one hung call means one worker is stuck. In a five-worker pool, one hung call ties up twenty percent of your capacity. Two hung calls, forty percent. A slow cascade like this is invisible until the pipeline stops making progress entirely, with no error, no log line — just workers that stopped returning results.
The fix is a deadline on every call. Not a hope that it'll finish — a hard boundary after which Go cancels the call, the worker moves on, and the article is marked as failed rather than blocking indefinitely.
Go's mechanism for this is context.Context.
What Context Is
context.Context is an interface that carries a deadline, a cancellation signal, and arbitrary key-value pairs through a call chain. You create a context at the top of an operation and pass it down through every function that does work on behalf of that operation.
ctx := context.Background() // the root — never expires, never cancelled
ctx, cancel := context.WithTimeout(ctx, 3*time.Second)
defer cancel()
// ctx now carries a deadline: 3 seconds from now.
// Every function that receives ctx can check whether that deadline has passed.
The critical rule: once you have a context-aware API — any HTTP client, database driver, or (now) our LLM simulator — pass the context to every call. Not some calls. Every call. The entire value of context propagation is that a deadline set at the top flows automatically to every piece of work beneath it.
Adding Context to the Simulator
The simulator's Call method in Parts 1–5 had no concept of cancellation:
// Before Part 8 — no deadline awareness
func (c *LLMClient) Call(task string, articleID int) {
time.Sleep(latency)
}
Part 8 changes the signature and the sleep:
// Part 8 — respects context deadline
func (c *LLMClient) Call(ctx context.Context, task string, articleID int) error {
// ...
select {
case <-time.After(latency):
return nil // completed normally
case <-ctx.Done():
return ctx.Err() // deadline exceeded or cancelled
}
}
The select is the key. Instead of unconditionally sleeping for latency, it races two channels: the timer and the context's done channel. Whichever fires first wins. If the context deadline passes before the timer, ctx.Done() closes and Call returns ctx.Err() — context.DeadlineExceeded — immediately. The goroutine is not sleeping, not blocked, not leaking. It's done.
This is exactly how real HTTP clients work under the hood. net/http uses the same select pattern against the request's context to abort in-flight requests.
Two Levels of Deadline
Our pipeline now has two distinct timeout boundaries, and understanding the difference between them matters.
Per-article timeout — set by the caller for a single article's full processing:
articleCtx, cancel := context.WithTimeout(ctx, p.ArticleTimeout)
defer cancel()
result := p.processArticle(articleCtx, article)
If any one of the three AI tasks for article 7 runs long, the article context fires, the in-progress task returns an error, and the remaining tasks are skipped. The worker moves on to article 8.
Pipeline context — the outer context passed to ProcessAll:
results, report := pool.ProcessAll(ctx, articles)
This is the whole-pipeline deadline. Cancelling it stops all workers, all articles, everything — we'll explore this fully in Part 9. For now, most callers pass context.Background() here, meaning "no pipeline-level deadline, just honour the per-article ones."
The relationship between them is hierarchical. A child context inherits its parent's deadline. If you create a per-article context with a 5-second timeout, but the pipeline context has already reached its deadline, the per-article context fires immediately — the tighter deadline wins automatically.
The Pipeline After the Change
Three changes in processArticle — every llm.Call now passes the context and checks the error:
func (p *WorkerPool) processArticle(ctx context.Context, article model.Article) model.AIResult {
result := model.AIResult{ArticleID: article.ID}
if err := p.llm.Call(ctx, "Summarization", article.ID); err != nil {
result.Err = err
return result // deadline fired — skip remaining tasks, return partial result
}
result.Summary = "AI-generated summary"
if err := p.llm.Call(ctx, "Sentiment Analysis", article.ID); err != nil {
result.Err = err
return result
}
result.Sentiment = "Positive"
if err := p.llm.Call(ctx, "Keyword Extraction", article.ID); err != nil {
result.Err = err
return result
}
result.Keywords = []string{"AI", "Go", "Concurrency"}
return result
}
When a task times out, the function returns immediately with Err set and the remaining tasks untouched. The worker doesn't hang. The article is accounted for in the results — just with a failure flag rather than data.
This last point matters. Every article still produces exactly one result. Timed-out articles produce a result with Err != nil. The total result count always equals the input article count. Nothing disappears silently.
What You Actually See
Run with a generous timeout — healthy pipeline, no failures:
Pipeline: 5 articles, 3 workers, 4s per-article timeout
[worker 1] starting article 1
[worker 2] starting article 2
[worker 3] starting article 3
[1] Summarization started (1.1s)
[2] Summarization started (890ms)
[3] Summarization started (1.3s)
...
═════════════════════════════════════════════════════════
Total : 5 articles
Succeeded : 5
Failed : 0
Duration : 5.559s
═════════════════════════════════════════════════════════
Now run with an unreliable provider — 20% timeout rate, 1500ms per-article deadline:
⚠️ Unreliable mode: 20% of calls will time out
[worker 1] starting article 1
[worker 2] starting article 2
[1] Summarization started (1.1s)
[2] Summarization started (890ms)
[2] Sentiment Analysis started (1.2s)
[2] Sentiment Analysis cancelled: context deadline exceeded
[worker 2] article 2: context deadline exceeded
[worker 2] starting article 3
[1] Keyword Extraction started (760ms)
[worker 1] article 1: done
...
Total: 10 | Succeeded: 4 | Failed: 6 | Duration: 1.52s
Four things worth noting in that output:
Article 2's summarization completes but sentiment analysis times out — so the worker returns a partial result and immediately picks up article 3 without waiting. The worker is not blocked. The pipeline keeps moving.
The duration is 1.52 seconds for 10 articles. Without timeouts, the six timed-out articles would each block their worker for the full simulated hang duration — potentially minutes. With timeouts, they fail fast and the slots are reused.
Four articles succeeded. Six failed. All ten are accounted for. The caller can decide what to do with the failed ones — log them, queue them for retry, alert on the failure rate.
The defer cancel() Rule
One pattern you'll see throughout Part 8's code and throughout the Go standard library:
articleCtx, cancel := context.WithTimeout(ctx, p.ArticleTimeout)
defer cancel()
context.WithTimeout allocates internal resources — a timer goroutine and associated memory. If you never call cancel(), those resources leak until the parent context is cancelled or the process exits. With a worker pool processing thousands of articles, that leak compounds.
defer cancel() ensures the resources are freed the moment processArticle returns, regardless of whether it returned early due to an error, completed normally, or panicked. It's not optional hygiene — it's mandatory cleanup.
If cancel() is called before the timeout fires, that's fine — calling cancel on an already-expired context is a no-op. The rule is simply: every context.WithTimeout or context.WithCancel must have a matching cancel() call, and defer is the idiomatic way to guarantee it.
What's Next
Part 8 gives every article a deadline. Part 9 gives the entire pipeline one.
Right now, if you want to stop the pipeline mid-run — a SIGTERM from Kubernetes, a user cancellation, an upstream system telling you to stop — there's no mechanism for it. ProcessAll runs until every article either completes or times out. You can't interrupt it from the outside.
In Part 9, we wire up OS signal handling, propagate cancellation through the pipeline context, and build a ShutdownReport that tells you exactly what completed, what was mid-flight, and what never started when the shutdown arrived. That last detail — accounting for every article even under cancellation — is the difference between a pipeline you can safely operate and one that loses work silently under pressure.
See you in Part 9.
This is Part 8 of the series "Production-Grade Concurrent AI Systems in Go." Read Part 7 — Worker Pools or continue to Part 9 — Cancellation and Graceful Shutdown.