I Stopped Restarting HTTP Connections Between AI Models. Here Is What I Use Instead.

A 5-stage AI pipeline where each model takes 200ms of compute time should take about 1 second. In practice it often takes 1.75 seconds or more. The extra 750ms is not your models. It is your transport. This post is about what happens when you replace per-request HTTP connections between model services with persistent tunnels, why the difference matters more than most people think, and how to implement it with a Go orchestrator that looks like normal HTTP but routes over encrypted P2P connections