Design the review queue before you train the model

Ask a team what their automation project needs and they talk about the model. Watch the project go live and the hard work turns out to be somewhere else.
Every working example we have seen ends in a fork. Clean cases flow through on their own. Doubtful ones go to a person. That person needs a screen, and that screen decides whether anyone uses the system.
Take refund handling. A model reads the request and the evidence, and a rule checks the original transaction and the refund policy. Requests inside policy get paid. Anything outside it goes to a person with the reason attached.
If the reason is vague, or the reviewer has to open three other tools, the queue backs up and the team goes back to doing it by hand.
So the questions to answer first are about people:
- Who sees the doubtful case?
- What do they see on one screen?
- What does one click do?
- What gets recorded, and under whose name?
Then measure. Count a week of real cases before the build, with minutes per case and the share of exceptions. Count the same week again after launch.
We cover seven examples, from invoice capture to inspection reporting, in Intelligent Automation Use Cases.
Dev Station builds the integration and the review layer for operators in the US and UK. Send us one process and a week of its cases, and we will tell you which part is automatable: dev-station.tech/contact.
About the author
- Dev Station Technology builds digital inspection platforms, field data apps and industrial IoT software for companies in the UK, the US and Australia. We build around your inspection process. You own the code. Typical projects are offline-first inspection apps, Bluetooth capture from test instruments, automated PDF certificates, and AI that shows an inspector what to look at while keeping a record of what the person decided. Our engineers work from Ho Chi Minh City with overlap into UK and US hours. This blog covers inspection software, applied AI and what it takes to run both in production. Website: https://dev-station.tech/ Contact: https://dev-station.tech/contact/
Comments
Join the discussionDesigning the human review queue and exception routing before chasing model metrics is an underrated lesson. Once real production edge cases hit, confidence thresholds and reviewer turnaround become the true bottleneck—not validation loss.


