Assessment6 min · 20 points · no sign-up
AI automation assessment
Roughly a third of the AI projects we are asked to quote should not be AI projects. Here is how to tell, before anyone spends money finding out.
01
Score the task (be strict)
- Volume: does this happen at least a few hundred times a month? Below that, the engineering rarely pays back.
- Consistency: do two experienced people doing it agree on the answer? If they disagree, define the rule before you automate it.
- Cost of being wrong: what happens downstream when the answer is wrong, and who notices?
- Input quality: is the input a clean digital file, a scan, a photograph taken badly, or a phone call?
- Current cost: hours per month × loaded cost. Write the number down; it is your budget ceiling.
02
Check whether it is an AI problem at all
- If the rule can be written in a page of if-statements, write the if-statements. They are cheaper, faster and auditable.
- If the data is already structured, you need a query and a form, not a model.
- If the real problem is that three systems do not talk, integration beats intelligence.
- If the process is badly designed, automating it makes a bad process faster.
03
What a credible pilot must include
- A graded answer set built from your own data by the people who do the job today — at least 50 examples, ideally 200.
- A measured baseline: how accurate are humans on the same set? You are competing with that, not with perfection.
- A refusal path: the system must be able to say it does not know, and that behaviour must be tested.
- Citations or provenance for anything consequential, so a wrong answer can be traced.
- A human approval step wherever being wrong costs money, a customer or a clinical decision.
- A per-transaction cost estimate, so the running bill is not a surprise in month three.
04
Questions to ask any AI vendor
- Will our data be used to train anyone's model? Get it in writing.
- What is the measured accuracy on our data, not on a benchmark?
- What happens when the model is wrong — who finds out, and how fast?
- Can we change model provider later, or is the system welded to one?
- Who owns the prompts, the evaluation set and the pipeline at the end?
If you want a second pair of eyes
We will walk through this with you, free, for thirty minutes.
No deck. Bring the specifics and we will tell you where the risk actually is — including when the answer is that you should not start this project yet.
- GSTIN 09CPBPD9064H1ZS
- Lucknow, India
- Reply in 1 working day