QuantumReadyWhat we get right, measured rather than claimed
Every other scanner page says what it can find. This one says how often it is right, on a fixed corpus with hand written answers, and it is regenerated on every release so a regression is visible rather than discovered by a customer.
Enterprise simulation details
Eight fictional company profiles, from a Python analytics extension through a Java Spring portal, a PHP and Perl site, a .NET finance service on SQL Server, the container and VM platform, the data tier with three clouds, and a whole-company run. Each profile has an answer key kept outside the scanner's inputs, and every run is published with its failures and later verified fixes.
Those eight profiles are a regression suite: each was tuned, so their pass rate is a floor rather than an accuracy estimate. The independent measure is the weekly uncharted evaluation, where cases are drawn from a pool written before the engine saw them.
Loading the latest measurement...
Loading the live totals...
Two measurements, for two different questions
| Measurement | What it answers | How it is produced |
|---|---|---|
| Latest measurement | Is the engine right? | A fixed corpus of small repositories with hand written answers. Precision, recall and evidence placement, recomputed on every release. |
| Live measurement | What is it actually seeing, publicly? | A running total across every scan the engine performs, hosted or offline, public or private. Counts and shares only, with no identifying detail, so it is a cumulative market picture rather than a customer record. |
They are deliberately not blended into one figure. Precision and recall need ground truth, and no amount of real scanning can supply that, so a single combined number would be a guess dressed up as a measurement. What both share is that they update continuously: the latest measurement when the engine or the corpus changes, the live one with every scan.
How the accuracy figures are computed
The corpus is a set of small repositories, one per common situation: a Node service with
RSA-signed tokens, a Python service with SHA-1 fingerprints, a Java service using a
com.example package, a service with nothing to migrate at all, and a repository that
only mentions legacy algorithms in prose. Each has an answer key listing every algorithm the
engine should and should not report, file by file.
A pair is one algorithm in one file. Precision is the share of reported pairs that are correct. Recall is the share of correct pairs that were reported. Evidence accuracy is the share of findings placed on the right rung of the ladder, from Detected through to Runtime confirmed. Every rate carries a 95% interval, and the build fails if any of them drops below the baseline recorded with the corpus, so these numbers cannot quietly decay. The corpus is split into sets, because a fixture the rules were tuned against measures something different from a fixture written from real code afterwards, and both are different again from a case no pattern can reach at all.
What this measurement does not say
It is our own corpus, not the world. A hundred percent here means the engine matches a set of situations we wrote down, and it is a regression test rather than a promise about your codebase. Real repositories contain code we have never seen, in languages and frameworks we have not covered, which is why every report states its coverage and its limits rather than a single confidence figure.
There are also things no static scanner can detect, and they are documented rather than hidden:
an algorithm name assembled at runtime, reflection, an encrypted configuration file, or a choice
made inside a compiled binary. Those are listed in the limitations section of every report, and
the fixtures folder records them explicitly under known-blind-spots.