From tokens to terabytes: Building reactive generative media pipelines

For the first three years of the generative AI wave, the output of a model was a string. You called an API, you got tokens back, you rendered them in a chat window or wrote them to a row in Postgres. The economics of that pipeline were dominated by inference cost.