This is the best question an operations buyer asks, and most AI vendors answer it badly. Not because they are dishonest, but because the answer has to be designed into the system before launch, and if you are only thinking about it when asked, you have not designed it.
There are four failures, and they need different answers
1. The workflow stops
An API is down, a credential expired, a service is unreachable. The easiest failure, because it is loud. Work queues rather than disappearing, the queue is monitored for depth and age, and the alert goes to us. When it clears, the backlog drains in order.
2. The workflow makes a wrong decision
The dangerous one, because it is quiet. The system decided something and was wrong, and nothing looks broken. Three defences, all built before launch: confidence thresholds, so uncertain items go to a person instead of being guessed at; validation against your own systems, so a decision that contradicts a known fact is caught; and a logged record of every decision with its inputs, so when you find one wrong you can find the rest.
3. The workflow degrades
Accuracy drifts. A supplier changes their document layout, a model provider updates a model, your own data changes shape. Nothing errors. The exception rate moves, and if nobody is watching that number, the first person to notice is a customer. We track it and treat a move in it as an incident, not a curiosity.
4. Something upstream changes
Your ERP is upgraded, a field is renamed, a vendor deprecates an endpoint. This is not really a failure, it is maintenance, and it is the reason "build it and hand it over" so often ends with a system that stopped working in month five and nobody could say when.
What we decide before anything goes live
- Blast radius. What is the worst thing a wrong decision here can do? That answer sets how much human approval the workflow carries.
- The fallback. When the system is unsure or unavailable, what happens? Usually: queue it, route it to a person, and make the state visible.
- The threshold. What confidence level sends an item to a human? Set from measured accuracy on a real sample, not from a default.
- Who gets the alert, and how fast. If it is a workflow we operate, that is us.
- The rollback. How do we turn it off and go back to the previous process without losing the work in flight?
A critical workflow with no human approval step and no rollback is not an automation. It is an unsupervised process with a good story.
Why we operate what we build
Every failure above needs somebody who knows the system, is watching the numbers, and can change it. A client who was handed a repository and a document has none of those things at two in the morning.
It is also the honest test of whether the thing was built properly. A vendor who never has to operate their own work does not feel the cost of building it badly. We do, which is a much more reliable incentive than a promise about quality.
Ask us this on the first call
Bring the 2am question to any vendor you are evaluating, including us. Ask what the fallback is, what the confidence threshold will be and how it will be set, who is alerted, and how you turn it off. If those answers arrive as generalities, the design does not exist yet.