Accountable AI: knowing when to trust a prediction
A prediction is only useful in operations if it tells you how far to trust it. Accountable AI means every answer comes with four things: the decision, how far to trust it, a warning when the data is outside what the model knows, and which variables weighed in. Without them, a wrong model sounds exactly like a right one.
Why isn’t a model that is usually right good enough?
A model with a good hit rate still gets things wrong. The problem isn’t that it fails: it’s that it fails in the same voice it uses when it’s right. The false alert and the real one arrive in the same format, the same colour and with the same urgency.
In operations that can’t afford to fail, that has concrete consequences. You send a technician to check equipment that was fine. Or the opposite: someone mutes an alert because the last three were noise, and the fourth one wasn’t.
An alert nobody can check ends up ignored. Not out of carelessness, but because your team has no way to tell when the model knows what it’s talking about and when it’s guessing.
So the useful question isn’t “how accurate is the model?” but “does this answer, right now, deserve action?”. Only the model itself can answer that, one prediction at a time.
What should every prediction carry before you act on it?
In CogniX, our predictive AI engine, every answer arrives in four parts. Not in a separate report or a dashboard of global metrics: in the answer itself.
| Part | What it tells you | What you do with it |
|---|---|---|
| Decision | Normal or anomalous, the predicted value or the category. | It’s the answer itself. |
| How far to trust it | The confidence of this specific answer. | Decide whether to act, check or wait. |
| Unknown-data warning | Whether the input falls outside what the model saw in training. | Know the model is extrapolating. |
| Which variables weighed in | Which inputs pushed the answer, and by how much. | Let your technician understand why and verify it. |
How far to trust it, calibrated where the maths allows
Calibrated confidence means that when the model says 80%, it is right roughly eight times out of ten. That is what turns a number into something you can work with.
Not every type of model supports that calibration, though. In CogniX, uncertainty is calibrated where the maths allows, and it says where it doesn’t. We would rather show you where the limit is than sell you precision that isn’t there.
What weighed in: exact, indicative, never causal
The fourth part answers “why?”. Here too there are nuances worth knowing before you trust any explanation:
- In some models, the weight of each variable can be calculated exactly.
- In others it can only be approximated, and CogniX marks it as indicative.
- In no case is it a causal explanation. Temperature weighing heavily in an alert doesn’t prove temperature is the cause. It tells you where to look first.
That last point matters more than it seems. An explanation sold as a cause leads to fixing things that weren’t broken.
Why does telling the new from the anomalous help with alert fatigue?
A binary anomaly detector has two answers: normal or anomalous. The trouble is everything in between. A machine starting up for the first time, a change of season, a process being retuned: none of that is a fault, but none of it looks like what the model saw in training either.
CogniX’s anomaly detection, DesviantiX, answers with three verdicts:
- Normal: within what the model knows.
- New but plausible: something it hasn’t seen before, which doesn’t make it a fault.
- Anomalous: outside the normal.
The practical difference lies in what you do with each one, and that’s your call: anomalous goes to alerting; new but plausible goes to review, without waking anyone up. With only two verdicts, everything new ends up as an alert, and that’s where fatigue sets in: your team learns that most alerts mean nothing.
CogniX also follows each device separately. A value can be normal for one pump and odd for the one next to it. Comparing each device with its own behaviour stops the noisiest unit in the fleet from setting the bar for everyone else.
How do you know the model in production is the one you validated?
A model can pass every test and still answer differently in production. All it takes is for the code that prepares data at serving time to do something slightly different from the code that prepared it for training: a different unit, a rounding step, a column in another order. The model keeps answering, just as confidently, about data that isn’t what it learnt from.
CogniX treats this as a publishing requirement: every training run records its expected answers, and the production service has to reproduce them before it is published.
Retraining works along the same lines. When your data changes, retraining is triggered by a person, a schedule or NovaeOps, when CogniX runs inside the platform. The new model goes in without stopping the service, but it doesn’t blindly replace the one that already works: before promoting it, you compare it with the one in production. If the new one is worse, you don’t promote it.
What results has CogniX shown in the lab?
Two tests, with date, hardware and data. They are lab measurements, reproducible with the command included in each test, not results from a customer in production.
Intrusion detection (public NSL-KDD dataset, September 2026). Trained on normal traffic only, without a single label, and at the same false-positive rate of 4%:
| Method | Attacks detected at a 4% false-positive rate |
|---|---|
| DesviantiX (CogniX) | 84.0% |
| One-Class SVM | 79.9% |
| Isolation Forest | 72.6% |
And the part that doesn’t flatter us: at a 1% false-positive rate, One-Class SVM comes out ahead, with 63.8% against our 58.2%. We publish it because the threshold you choose changes the result, and you need to know that before you decide.
Performance (August 2026, Ryzen 7 7800X3D, 8 cores): up to 53,700 predictions per second for anomaly detection, with a p99 of 2.84 ms, on a light model. The heaviest model we measured reaches 19,400 per second. We measure it again on your hardware.
Those milliseconds are what let you run the model reading by reading, rather than in a batch job that tells you tomorrow. We explain how in putting AI in the data path without breaking ingestion.
What don’t we promise?
To always be right. Unsupervised detection learns from your “normal” data, and if that data has anomalies in it, it learns to treat them as normal too.
We have measured it: with 20% contamination in the training data, detection drops to just over a third. That’s why we help you clean the baseline before training. We measure the effect and tell you how much it matters.
Nor do we promise that the AI will decide on its own. The verdict informs; you decide with rules what triggers an automatic action. Where it matters, the AI proposes and a person approves.
Five questions to ask any predictive AI system
Before you put a model into your operations, ours or anyone else’s, ask it these:
- Does every answer say how far to trust it, or is there only a global metric?
- Does it warn you when the data is outside what it saw in training?
- Does it say which variables weighed in, and whether that explanation is exact or indicative?
- How do you check that the model in production answers the same as the one you validated?
- What happens to detection if the training data is dirty, and has anyone measured it?
If any of the answers is “we don’t know”, you know where the risk is.
Where do you start?
With a dataset and a concrete question: which device is drifting from normal, or what value you expect tomorrow. CogniX is at version 3, available for pilots, and runs on your network or at the edge without sending us telemetry. Inside NovaeOps it puts the verdict on the readings and is billed per delivered verdict.
If you want to see it with data like yours, take a look at how CogniX works and ask us for a demo.