Uncategorized

At 20 Engineers, Vision Accuracy Alone Is the Wrong KPI

A vision model can improve on a test set while making the product worse: it may automate easy cases and send the expensive mistakes straight to users. For a tech lead with a small engineering team, I would make the release decision on audited outcomes per eligible request, not model accuracy. If you cannot connect a prediction to an action, a later label, and its cost, you cannot tell whether the deployment works.

The release gate should measure harmful actions, not correct predictions

Start with the decision your application makes. Suppose a camera check either accepts an image automatically or sends it to a reviewer. “The model was correct” is too broad: a wrong automatic acceptance may require a costly correction, while a wrong referral to review mainly consumes staff time. I would define the primary metric as harmful automatic acceptances per 1,000 eligible requests, with review minutes and p95 response latency as guardrails. This makes the denominator include requests the system declines to automate, so the team cannot improve its apparent error rate simply by reviewing everything.

Write down what counts as harm before looking at model scores. For example, an automatic acceptance might be harmful if a trained reviewer would reject it under the current policy. Record the action taken, the policy version, and the reviewer’s reason. If a downstream complaint can overturn that judgment, retain both labels and the date of the correction. Otherwise, a delayed complaint will appear to be a new incident rather than evidence that the original decision was wrong.

A starting constraint could be fewer than 2 harmful acceptances per 1,000 eligible requests; that is a value to negotiate with the product owner, not a universal safety threshold. Pair it with a review-capacity limit—for example, an initial ceiling of 25% of eligible requests—because a model that avoids errors by transferring all work to people has not improved the service. Set a latency budget separately: a provisional 600 ms p95 may be appropriate only if the calling application can tolerate it. These figures should be changed when the cost of a wrong action, reviewer capacity, or request deadline changes.

I would not gate this release on a single validation-set precision score, because its class balance and image quality need not match the requests that reach production. Nor would I call a lower review rate a win without auditing accepted cases: review avoidance is exactly where an overconfident model can hide damage. Read the adjudication log first, then read How to prove your vision model works in production metrics. The log, rather than any general metric recipe, has to settle what this team means by success.

An auditable decision stream is more useful than another model dashboard

Give every eligible request a stable request ID and emit an event whether inference succeeds, times out, falls back, or never starts. An OpenTelemetry trace should connect image intake, model call, policy decision, and response; attributes such as service.name and deployment.environment.name distinguish deployments. Store the model identifier, policy version, confidence or score, decision, review reason, and eventual label against that ID. Record an image reference or restricted hash rather than copying raw images into general-purpose telemetry, because broad access to diagnostic data creates a second, unnecessary exposure path.

Prometheus histograms can track inference and end-to-end latency using the le bucket label; Grafana can show p95 latency beside timeout and fallback counts. Those tools cannot tell you whether accepted images were correct. Keep adjudications in a queryable ledger—PostgreSQL is sufficient at this scale—and export joined events to Parquet when an audit needs cohort analysis in DuckDB. Use CVAT to manage reviewer annotations and export COCO-format labels where the task fits that format. Put annotation-guideline versions alongside labels, because a changed guideline can look like model drift even when the images have not changed.

The following Python snippet runs as written. Its rows are illustrative, not measured production results; replace the embedded CSV with an export of adjudicated decisions. It deliberately reports each cohort separately rather than hiding a bad cohort in an average.

import csv
from io import StringIO
data = """decision,correct,cohort
accept,1,night
accept,0,night
review,1,day
accept,1,day
review,0,night"""
rows = list(csv.DictReader(StringIO(data)))
for cohort in sorted({r["cohort"] for r in rows}):
    s = [r for r in rows if r["cohort"] == cohort]
    a = [r for r in s if r["decision"] == "accept"]
    print(cohort, "false_accept", sum(r["correct"] == "0" for r in a) / len(a) if a else None,
          "review", sum(r["decision"] == "review" for r in s) / len(s))

The snippet exposes a trap rather than solving sampling. If reviewers label only referred cases, accepted cases have no trustworthy “correct” field. Randomly sample automatic acceptances for blinded adjudication, and keep the sampling probability so an analyst can reconstruct production rates. Reserve a planning budget of 400 independently sampled accepted cases per major cohort before treating a zero-error result as reassuring; even zero errors in 400 observations gives only an approximate two-sided 95% upper bound of 0.75% under independent sampling. That bound does not repair correlated images, inconsistent reviewers, or a cohort defined after inspecting failures. Use scikit-learn’s precision_recall_curve for threshold exploration, but calculate the release metric from the sampled decision ledger, because threshold curves alone do not include operational fallbacks or review cost.

A pretrained API wins only if it wins under the same decision policy

Compare a named pretrained vision API with a named custom deployment on the same incoming images, but do not compare their raw confidence numbers: the scores need not share a calibration scale. For a concrete choice, put Google Cloud Vision API against a PyTorch model served through ONNX Runtime. The API can win when labeled data is scarce and its audited decisions meet the harm and latency limits; its costs include per-request charges, network dependence, and the work of checking vendor behavior after changes. The custom deployment can win when the team has representative labels and can materially reduce harmful decisions at its required volume; its costs include training, serving, annotation, and ownership of every regression.

Run both candidates in shadow mode on identical requests while the existing policy still makes the user-visible decision. Give each candidate its own threshold, selected on an earlier labeled set, but enforce the same maximum review fraction and action definitions during evaluation. Otherwise, a candidate can appear safer merely because it sends more work to reviewers. Log API timeouts, malformed responses, and custom-serving failures as outcomes, not missing rows. Use MLflow Model Registry aliases such as @champion to identify the custom artifact under test, and pin the API request configuration and observed provider response metadata so a later comparison does not silently mix versions.

A proposed trial might shadow 10% of eligible traffic for seven days; treat that as a tunable coverage and duration choice, not evidence that seven days captures every rare condition. Check whether that interval includes the lighting, device, and input-quality cohorts that drive failures. Compare candidates on paired requests, then inspect cases where their proposed actions disagree. A difference in average accuracy matters less than which candidate automatically accepts the cases a reviewer would reject. Test both candidates on the same inputs before you read Custom Vision Model or Pretrained API Which Fits Your First App; a first-app preference does not establish a production winner without paired outcomes.

Price the comparison in the unit the organisation can act on: expected correction cost plus reviewer minutes and serving cost per 1,000 eligible requests. Include engineering time for maintaining the custom model, but report it separately from request-variable cost, because combining them into one opaque number prevents a useful volume forecast. If the API and custom model are statistically indistinguishable on harmful actions, I would keep the simpler deployment until a larger audit can justify switching; a claimed improvement smaller than the label uncertainty is a poor reason to add operational ownership.

A rollout works only if it improves on the policy it replaces

Shadow results test predictions, not the full intervention. Once the team has an audit stream and a safe fallback, randomly assign a small eligible slice to the new decision policy and leave a comparable slice on the existing one. Keep request assignment stable for repeated images or users where feasible, because switching policies between related requests can contaminate both groups. Measure harmful actions per eligible request, review minutes, latency, fallbacks, and corrections in both groups. Include requests that timed out or bypassed inference in the assigned group’s denominator; excluding them would reward an unreliable deployment.

Set the label-lag rule in advance. An example operating rule is to wait two business days for routine adjudications before closing a daily comparison, while continuing to update it for later reversals. That interval is an assumption to check against the team’s actual review queue, not a reason to discard slow-arriving complaints. Report counts and uncertainty by cohort as well as overall results. A dashboard that shows a favorable aggregate while night images deteriorate should block expansion if night images are part of the promised service.

Give the on-call engineer a reversible control: route decisions back to review or the old policy without redeploying the model. Alert on a breached harm estimate only when enough audited labels exist to support it; alert immediately on missing events, fallback spikes, or exhausted review capacity, because those failures can be observed without waiting for labels. Evidently can flag input-distribution changes for investigation, but drift is a diagnostic rather than a verdict: a changed input distribution matters to this release when it changes audited decisions or makes the audit sample unrepresentative.

Precommit the stop conditions with the people who own corrections. If harmful acceptances exceed the agreed limit, if review demand exceeds staffed capacity, or if adjudication coverage falls below the level needed to judge either, pause expansion. “We have not seen errors” is not evidence of success when accepted cases are not being checked. The rollout has worked when the new policy produces a defensible improvement against the old one at an acceptable operating cost—not when its model score looks better in isolation.

The first task is to price one wrong action

Before selecting another model, take one recent batch of eligible requests and have the product owner and a reviewer agree on what an incorrect automatic action costs. Write that definition into a decision-event schema, then sample accepted cases for independent adjudication. If those two steps cannot produce a credible baseline harm rate, keep the current policy: there is no sound release comparison yet.