← Building Concurrent Ai Pipelines In Go

Phase

Advanced Concurrency Engineering

Scheduler internals, sync.Once/Map/Pool and atomics, semaphores, singleflight, slog with pprof, and OpenTelemetry tracing — the advanced toolkit, with zero infrastructure.

0 published · 6 upcoming

  1. Part 21In progress

    Part 21 - Go Scheduler Internals: How Many Workers Should the Pipeline Run?

    Every worker pool in this series has a -workers flag, and so far we have picked its value by feel. Part 21 opens Arc 3 by looking under the hood at the M:P:G scheduler and measuring why LLM calls scale far past your CPU count while CPU-bound work stops dead at GOMAXPROCS.

  2. Part 22In progress

    Part 22 - sync.Once, sync.Map, sync.Pool and Atomics: The Rest of the sync Package

    Arc 1 used Mutex, RWMutex and WaitGroup. Part 22 covers the rest of the sync package through the news platform: sync.Once for a shared LLM client, sync.Map for the URL dedup cache, sync.Pool for prompt buffers, and atomic compare-and-swap for lock-free limits. Plus the benchmark that lied until we fixed it.

  3. Part 23In progress

    Part 23 - Semaphores: Limiting One Section, Not the Whole Worker

    A worker pool limits how many goroutines exist. A semaphore limits how many of them can be inside one piece of code at once. Part 23 caps concurrent embedding calls with a channel semaphore, then builds a weighted semaphore for token-sized requests and fixes the cancellation bug hiding in the obvious sync.Cond version.

  4. Part 24In progress

    Part 24 - Singleflight: One LLM Call for Many Identical Requests

    When twenty workers ask for the embedding of the same breaking story in the same second, one LLM call should serve them all. Part 24 builds singleflight from the standard library, then fixes the three traps in the textbook version: the first caller's deadline cancelling everyone, a panic hanging every waiter, and waiters that cannot give up.

  5. Part 25In progress

    Part 25 - slog, Ticker and pprof: A Pipeline That Explains Itself

    Twenty-four parts of fmt.Printf end here. Part 25 moves the pipeline to structured log/slog, flushes metrics on a time.Ticker without losing the final interval, and serves pprof safely so you can see every goroutine's stack while the pipeline is running.

  6. Part 26In progress

    Part 26 - Tracing: Following One Article Across Every Goroutine

    Logs record events, metrics record totals, profiles record a moment. Part 26 closes Arc 3 with tracing: an OpenTelemetry-style tracer where context.Context carries each span across goroutines, so one batch becomes one waterfall showing which worker ran which article, how long each stage took, and exactly where it failed.