Building a second data pipeline is harder than building the first. Not because the engineering is harder, but because nothing from the first build transfers. The state machine is bespoke, the SLA tracking is bespoke, the feedback schema is bespoke. By the third pipeline, we’re maintaining three dashboards, three training data formats, and three definitions of “done,” all doing essentially the same thing.
This is a structural problem. The incentives push toward it: a new pipeline has a deadline, the fastest path is to adapt the last one, and the compounding cost shows up later. When the ML team can’t consolidate training signals across workflows, or when on-call rotation covers five systems that share no tooling.
The pattern described here doesn’t eliminate the first build. It eliminates the second, third, and tenth. To see why that’s hard, it helps to be specific about where bespoke systems break down.
Why Bespoke Systems Fail at Scale
Every workflow needs to track task lifecycle: created, assigned, in-progress, completed, failed, escalated. Sounds standard. But each team implements this slightly differently, and the differences compound. Operators who work across workflows carry multiple mental models in their head. Dashboards can’t be shared. On-call runbooks can’t be generalized. When something misses SLA, there’s no single place to look because every workflow has its own SLA monitoring logic, and some of them got it wrong.
The ML training signal problem is subtler but more expensive. Human reviewers produce corrections. Those corrections are training data, or they should be. If every workflow structures that feedback with a different schema, different confidence signals, different outcome labels, the data science team ends up building a separate ingestion pipeline per workflow. Models train in isolation. We can’t ask “is the same content type underperforming across all our models?” because the data doesn’t join.
And then there’s the operational surface. Every new bespoke workflow brings a new deployment, new alerting setup, new runbook, new on-call scope. None of it borrows from the last one. Engineering time that should go toward building new capabilities goes toward keeping the lights on across a fleet that keeps expanding but never gets cheaper to run.
The pattern that breaks this cycle has four interlocking properties: a shared state substrate that every workflow runs on top of; declarative routing so new workflows are configuration, not code; worker-type agnosticism so ML models, humans, and AI agents compose without framework changes; and a unified feedback contract so training signals accumulate in one pool rather than N isolated silos. Each one addresses a different failure mode above.
A Two-Tier Foundation
Put any two bespoke pipelines side by side and we’ll find the same thing buried inside both: something tracking where every task is in its lifecycle, and something driving the sequence of steps that moves work forward. They’re tangled together in each implementation, which is why nothing transfers when we build the third one.
The reason they stay tangled is that they seem like the same problem. They’re not. State is what tasks exist, where they are, what decisions have been made. It belongs to nobody and everybody simultaneously. Operators need it to debug a stall at 2am. Dashboards need it to report SLA compliance. ML pipelines need it to consume corrections. It changes constantly but has nothing to do with how any particular workflow is structured. Execution is the ordered steps that move a task from created to complete. It’s the only part that actually differs between workflows.
Once we see them as separate problems, the right structure follows.
That’s the core leverage. The tenth workflow lands on the same monitoring, the same observability, the same on-call runbooks as the first. The next question is how new workflows get registered on that substrate, which is where the platform’s real operational leverage concentrates.
Configuration Over Code
The pattern plays out the same way on every team. A new requirement comes in: route this content type differently, lower the confidence threshold, add a new escalation tier. In a bespoke system, that conversation ends with someone opening a ticket to modify the state machine. It’s a code change. It needs a PR, a review, a deploy, a rollout. If the threshold turns out to be wrong, we do it again.
On the unified platform, the same conversation ends with someone opening a config file.
The triggering system is event-driven and declarative. When an asset reaches a certain condition (a new item arrives, a threshold is crossed, a dependency completes), the platform evaluates a predicate and routes it to the right workflow. The predicate is what the team writes. Everything behind it, the SLA clocks, the retry logic, the state transitions, the downstream notifications, is handled by the framework.
Predicates can be simple (route all items of type X to workflow Y) or compound (route items of type X with confidence below threshold T to human review, otherwise auto-approve). Either way, the routing logic lives in configuration, not application code. If a predicate is wrong, someone notices the routing behavior is off, traces it to the config, and fixes it that afternoon. If it were a code change, the same fix would mean another ticket, another PR, another deploy.
This changes the team’s job in a subtle but important way. Engineers stop spending time on plumbing and start spending time on the quality of the decision logic: the threshold that separates auto-approve from human review, the conditions that trigger escalation, the routing rules that determine which reviewer pool sees which content. That’s where judgment actually matters. And there’s a compounding benefit: because every workflow is expressed in the same configuration language, a new engineer can understand the full platform by reading one set of files, not N different codebases. The system becomes legible to people who didn’t build it.
But declarative routing is only useful if the workers behind those rules are interchangeable, which brings us to the third property.
Worker-Agnostic Execution
Early in the design, the temptation is to treat worker types as fundamentally different problems. An ML model responds in 200 milliseconds. A human reviewer takes two days. An AI agent might take minutes or hours depending on what it’s doing. The natural instinct is to build different orchestration paths for each: a synchronous call for the model, a queue-based async pattern for humans, some hybrid for agents.
That instinct is where three separate execution patterns come from. All doing essentially the same thing.
The insight that breaks the pattern is that the orchestrator doesn’t need to know how long it’s waiting. Only that it’s waiting. A synchronous ML call and a human review sitting in a queue for 48 hours look identical from the orchestrator’s perspective: work was dispatched, and eventually a result came back. The mechanism that makes this work is WAIT: the orchestrator suspends, persists its position, and resumes when it receives a completion signal. What generates that signal is irrelevant to the framework.
Once we build around that abstraction, worker type stops being a framework concern. An ML model, a human reviewer, and an AI agent all satisfy the same interface: accept a task, process it, return a result. Mixed workflows (algorithm first, escalate to human on low confidence) compose from the same primitives without any special-casing. Swapping an ML model for an AI agent is a config change, not a refactor.
This matters more now than it did two years ago. Teams integrating AI agents often treat orchestration as an open question: how do we sequence agentic work alongside human review? The answer, if we’ve already made workers interchangeable, is that we don’t do anything differently. The framework was already waiting. Worker interchangeability also creates a dividend that’s easy to overlook until we’re missing it: every worker, regardless of type, produces corrections through the same interface, which is what makes a unified feedback contract possible.
One Feedback Language
At some point, the data science team asks a question that seems like it should have a simple answer: “Which content categories are reviewers most consistently overriding the model on?”
In a bespoke system, that question takes weeks. The corrections exist, every reviewer’s decision was logged somewhere. But somewhere is different in each workflow. One pipeline captures corrections as a binary override flag. Another stores a free-text reason code. A third has the full before-and-after diff, but uses a confidence schema that doesn’t match the other two. To answer the question across all three, someone has to build a one-off join across incompatible schemas, usually with hardcoded mapping logic and several assumptions that will turn out to be wrong. Either the data science team does it themselves and marks it as tech debt, or they scope the analysis to one workflow and note that findings might not generalize.
Meanwhile, the thing they were actually trying to surface (the model’s consistent blind spots, reviewer calibration drift, a content type that underperforms across multiple systems) stays invisible.
A unified feedback contract makes this a non-problem.
Every correction, regardless of which workflow generated it, produces the same structured output: what the worker saw, what decision they made, what they changed, and why. ML teams have one ingestion pipeline. Cross-workflow questions are just queries. “Is the same content type underperforming across multiple models?” becomes a five-minute analysis instead of a multi-sprint project.
The shared pool compounds in a way isolated pools can’t. Models that serve multiple workflows train on signal from all of them. A quality improvement in one workflow creates training data that benefits others. The more workflows run on the platform, the faster the models improve, which is the opposite of what happens in a bespoke system, where each new workflow adds another isolated pool that never talks to anything else.
There’s one more thing the contract enables that’s easy to miss: safe retraining. Because corrections are structured and attributed, the platform can snapshot model outputs at task-creation time and preserve them even as new model versions arrive mid-review. Reviewers always see stable, consistent inputs. In a bespoke system, this is the kind of edge case we discover in production.
The Operational Shift
Picture an engineer joining a team that runs five bespoke pipelines. She spends her first month learning the first one: its state machine quirks, its SLA monitoring setup, where the alerts fire, what the dashboard means. Then she moves to the second pipeline. It’s different. Different state model, different alert structure, different runbook. She starts over. By the time she’s on-call for the fleet, she’s carrying five separate mental models. When something pages at 2am, she has to remember which system it is before she can even start debugging.
On a unified platform, that doesn’t happen. An engineer who learns how to debug a stalled task in one workflow knows how to debug it in all of them. The state model is the same, the observability surface is the same, the retry behavior is the same. On-call runbooks apply across the fleet. When something goes wrong at 2am, there’s one mental model to apply.
This compounds across the team’s output, not just its on-call burden. In a bespoke system, a meaningful fraction of each new workflow’s build time goes to infrastructure that already exists somewhere else on the team: setting up SLA tracking, wiring up alerting, implementing the state machine, again. On a unified platform, that work was done once, before the first workflow shipped. Every subsequent workflow spends its engineering time entirely on what actually varies: the routing predicates, the quality thresholds, the worker configuration. That’s where the domain expertise matters. Everything else is already built.
None of this is unique to content understanding pipelines. The same dynamics appear wherever heterogeneous workers process tasks through a defined lifecycle.
Where Else This Applies
Here are some other examples where our architecture can fit naturally.
Content moderation on a social platform: ML flags posts, human moderators review borderline cases, an AI agent drafts policy citations for appeals.
Medical imaging is structurally the same problem. An AI model analyzes radiology scans and highlights anomalies, a radiologist reviews flagged regions, and urgency-based routing escalates critical findings. The WAIT mechanism, pause until the human completes, however long that takes, is exactly what the framework already handles. Radiologist corrections are among the most valuable training data available. A unified feedback contract would capture them consistently across scan types and specialties without extra instrumentation per modality.
Financial compliance: transaction monitoring teams add new risk categories frequently, new fraud patterns, new regulatory requirements, new jurisdictions. With this architecture, adding one becomes an operational change instead of a code change: update a threshold, add a routing rule, deploy a config. The audit trail lives in the shared state store and stays queryable across all workflows without per-workflow reporting infrastructure.
Document processing: legal, insurance, and financial services teams deal with dozens of document types, each with its own classification logic and review requirements. The orchestration underneath, route to classifier, surface to reviewer for ambiguous cases, pass to extraction agent for structured data, looks almost identical across document types. Built once on this architecture, new document types become configuration instead of a new build.
The first workflow on the platform still takes real effort: designing the state store, the orchestrator, the feedback contract, the triggering system. That investment is front-loaded on purpose. The third workflow takes a week. By the twentieth, onboarding is a configuration exercise, and the only judgment calls are in the workflow logic itself, the routing predicates, the quality thresholds, the worker configuration. That’s where the engineering actually matters. The plumbing is already built.