Methodology

What we measure, how we classify it, and exactly where the method stops. Written so a technical reviewer can reproduce or challenge it.

0. The engine, and the two scanners

The engine finds cryptography. A scanner is where the engine runs. There are two, they answer identically, and the choice between them is about where your code can go and what you need to reach, not about capability.

The engine One codebase, versioned, and named in every report it produces
84 rules, 366 patterns 11 languages read as code Owner and action on every finding Second pass before publication Readiness score CycloneDX CBOM export

Hosted scan

On our Cloudflare deployment. Nothing to install.

Reaches
Public repositories, repositories linked to your account, and public endpoints.
Reads code
Yes, eleven language families, on the plans that cover it.
What leaves
The repository is read by us while the scan runs, and the report is stored for your retention period.

Offline scanner

On your machine, from one download.

Reaches
What a hosted scan cannot: private forks, build artifacts, container images, server and cloud configuration, and hosts behind Cloudflare, whose edge a Worker cannot dial.
Reads code
Yes, the same eleven, and every file rather than the candidates a hosted scan can afford.
What leaves
Findings only, and only if you answer yes at the end. Source never leaves.

The same rules, the same syntax reading, the same score, and a test on every release that runs both over the whole accuracy corpus and fails the build if a finding or a site count differs. The two do use different WebAssembly runtimes, because a Worker cannot compile WebAssembly at runtime and a Node process can, which is exactly the kind of difference that quietly produces two answers. The deploy also asks the deployed scanner to read every language it claims, and fails the release if one is missing. The browser workflow checkpoints work in bounded batches. The hosted and local source scans both have finite limits, shown below and in every report. Identical rules do not imply identical coverage when a limit or a network restriction is reached.

1. Public endpoint scan

There is no way to read the negotiated key exchange group from an ordinary HTTPS request, so we perform the TLS handshake ourselves. We open a TCP connection to port 443 and send a TLS 1.3 ClientHello that offers the post-quantum hybrid groups first (X25519MLKEM768, SecP256r1MLKEM768 and the Kyber drafts) with an X25519 key share attached. Per RFC 8446 the server must then reply either with a HelloRetryRequest naming its preferred group, or with a ServerHello naming the negotiated group. Either way we learn which group the server chose. This raw probe is partial. A retry request identifies a requested group, not an established hybrid exchange. We then attempt complete TLS 1.3 and TLS 1.2 sessions, inventory the certificate and report chain trust, hostname and validity dates separately. If the runtime does not expose the completed-session group, hybrid establishment remains unmeasured.

Two limits are worth stating plainly. Workers cannot open connections to Cloudflare's own edge, so when a host resolves into a Cloudflare range we fall back to a clearly labelled DNS inference instead of a measurement. And some hardened endpoints refuse a handshake that did not come from a real browser, in which case we report that we could not confirm rather than guessing.

2. Repository scan

We download the repository as a tarball over HTTPS, decompress it in memory, and parse the tar. Directories such as node_modules, vendor, dist, build, target, virtual environments and test data are excluded. The two source workflows are bounded. The hosted scan reads up to 250 files or 2 MiB of source, whichever comes first, with a 512 KiB file limit, and reads code as well as text within that budget; the offline scanner reads up to 3,000 files and 12 MiB. The browser uses resumable batches of up to 120 files and 1.5 MiB per request, then combines the evidence and call graph. Hosted archives are streamed through a separate 32 MiB decompressed limit. Final hosted call graphs are bounded at 5,000 definitions and 20,000 calls. Findings are capped at 3,000. A reached source or finding limit makes the readiness score provisional, with no complete score displayed. Parser, graph, archive and artifact limits are disclosed independently.

Every remaining file is tested against a rule set covering Java and the JVM, Python, JavaScript and TypeScript, Go, C and C++, C# and .NET, Ruby, PHP, Rust, Swift and Objective-C, plus configuration files and dependency manifests. Rules look for algorithm names, provider and library calls, TLS configuration directives, key and certificate material, and cryptographic dependency declarations.

3. Risk classification

Classification follows the standard quantum model, not a vendor opinion.

  • Quantum vulnerable. Public key algorithms whose security rests on factoring or discrete logarithms. Shor's algorithm breaks these outright. This covers RSA, DSA, Diffie-Hellman, ECDH, ECDSA, Ed25519, X25519 and elliptic curve cryptography generally.
  • Deprecated or weak. Broken or badly outdated regardless of quantum: MD5, SHA-1, DES, 3DES, RC4, RC2, Blowfish, TLS 1.0 and 1.1, and non-cryptographic randomness used for keys.
  • Adequate today. Still sound against classical attack, but worth documenting and reviewing, such as AES-128.
  • Quantum resistant. Symmetric and hashing primitives with sufficient margin, such as AES-256, ChaCha20-Poly1305, SHA-256 and above, SHA-3, and modern key derivation functions.
  • Post-quantum ready. Algorithms standardised or proposed for the post-quantum era, including ML-KEM, ML-DSA, SLH-DSA and FALCON.

Grover's algorithm gives a quadratic reduction in idealised key-search queries. That is not a prediction of practical attack cost. NIST's FAQ continues to permit AES-128, AES-192 and AES-256. Our preference for AES-256 for long-lived data is a conservative planning policy, not a finding that AES-128 is broken.

4. Readiness score

Each distinct algorithm is weighted by category importance and by how often it appears in production code. Key exchange, signatures and key material carry the most weight, then protocol configuration, then symmetric encryption, then hashing and key derivation. Quantum-vulnerable algorithms count fully against the score, deprecated ones at 75 percent, adequate ones at 20 percent, and quantum-resistant or post-quantum ones at zero. Small bonuses apply for post-quantum signals and strong symmetric encryption.

Findings that appear only in tests, fixtures or examples are excluded from scoring, though they are still reported. If fewer than two distinct algorithms are found, we decline to score and say so, because a score built on one data point is misleading. This is our own technical index. It is not the CSA Quantum Readiness Index and it carries no regulatory weight.

Inventory records, such as noting that an identity provider is in the integration, are excluded from the score as well. They describe what a repository has rather than what is wrong with it. The score itself does not discount an algorithm because a provider owns the fix, since it measures the cryptography that was found; the owner column is where the work is attributed.

5. Evidence and traceability

Every finding records the file, the line number, the matched text, the detected language, the rule that matched, and whether the file is production or test context. Reports state how many files and bytes were scanned, what was skipped and why, and whether any limit was reached.

Every finding is checked a second time before it is published. A pattern match is a candidate, not a result, and the second pass is what turns one into the other. It re-runs the rule against the quoted code and confirms that the line number points at it. It confirms the match is in code rather than in a comment or a docstring. It checks that a credential pattern found a value rather than the name of a secret or a reference to a secret store, that a private key header has a key under it rather than being code that reads PEM text, and that base64 was applied to something secret before it counts as misuse. Where a finding says a provider or a platform owns the fix, it has to point at the token in the file that supports that claim, and the token is printed with the finding so the inference can be checked. Finally, the remediation text has to belong to the algorithm that was found, and the report says so when the guidance is generic rather than specific.

Whatever fails those checks is removed from the findings and listed in the report with its reason, so a reader can see what was taken out rather than having to trust the list. Exported reports carry the same record on a Verification sheet. Two limits are worth stating: the checks judge context and shape, not intent, and a rule that never matches cannot be corrected by a later pass, which is what the accuracy fixtures are for.

The published accuracy numbers, the corpus behind them, and the claims we make about them are on Detection accuracy. Readers of a report can also send back a verdict on any finding, which is kept as a field measurement of precision on real code rather than on fixtures.

Where a language is parsed, the call is read rather than the text around it. Eleven families are parsed by both scanners: Java and Kotlin, JavaScript, TypeScript, Python, Go, Ruby, PHP, Rust, C#, C and C++, and PowerShell. That resolves the algorithm, the mode and the padding from the call itself and reports a cipher whose algorithm cannot be known from source as a named review item rather than as a claim. The same pass identifies the function a call sits inside, and builds a call graph over those functions: definitions and calls are read from the tree, so a call is an edge between two definitions rather than a count of how often a name appears. That is what turns "this function checks a signature" into "this function checks a signature and eleven places reach it, three of them entry points". Section 7 says exactly what the graph resolves and what it leaves alone.

6. What this method cannot do

  • Source scanning does not execute or decompile binaries. The runner separately inspects container and archive contents within disclosed limits. The optional Node collector observes selected successful crypto operations during a run; it does not cover WebCrypto, native addons, Java or every process. Hardware security modules, firmware, business databases and runtime key stores are not inspected.
  • It does not resolve dynamic construction of algorithm names, reflection, generated code, or encrypted configuration.
  • It reports third-party dependencies from manifest files rather than scanning dependency source.
  • A finding proves that a pattern exists in the repository. It does not prove the code path is reachable or in production use. What it does do is place every finding on an evidence ladder, from Detected through to Runtime confirmed, so the report never claims more than the evidence supports.
  • It cannot establish regulatory compliance. That requires attestation, documentation and human review.

6a. Where to find each answer in a report

QuestionWhere it appears in the report
What does this project depend on?The Declared dependencies panel: direct and transitive packages, with the version declared in the manifest or lockfile and the file it came from.
Is any of it really in use?The Evidence column in the inventory, one rung per algorithm, explained in the Evidence and confidence panel just below the table.
Why is this algorithm first?The Prioritised actions table, where each row carries the reason for its position.
How was any of it decided?The panel titled How we decide what is actually in use, which lists every source of evidence row by row and states plainly what is not done.
Was the code read, or only matched?The Reading the code row in Coverage and limitations: how many files were parsed, in which languages, how many call sites that resolved, and where the parse did not run it says so with the reason.
Where should the fix start?Where the change would land: the functions that hold a crypto call, who calls them by name, and which entry points reach them through the call graph.

The call graph, and where it stops. It resolves a call to a definition when the receiver names the class, when the name is imported from a file in the scan, or when the definition is in the same file. Where none of that holds and the name is defined exactly once in the repository, the edge is recorded as resolved by uniqueness, and the report separates that from the resolved ones because it is a probability rather than evidence. A call to something the repository does not define is counted as outside it. What remains is an internal call that could not be attributed, and it is counted rather than guessed at. Every report states the resolution rate it achieved. Two things the graph still does not do: it does not follow a value from a request handler down to the line that calls RSA, and it does not resolve calls made through an interface, a reflection call, or a callback whose target is chosen at runtime. Tools that claim that produce confident answers that are frequently wrong in dynamic languages, and we would rather say Likely used and be right than say Confirmed and be wrong.

The dependency view is an inventory, not a resolution graph. It reports the versions declared in the repository, which is what a manifest can honestly tell you, and not the versions resolved on a running system, which it cannot.

7. Human review

Free scans are automated and unreviewed. Human review must be arranged separately on a plan that includes it. Automated results are not human-reviewed or compliance attestations.

8. Sources

  • CSA Quantum-Safe Handbook V1, 16 July 2026, and the Quantum Readiness Index V1. Available at csa.gov.sg.
  • MAS Circular MAS/TCRS/2024/01, Advisory on Addressing the Cybersecurity Risks Associated with Quantum, 20 February 2024.
  • NIST FIPS 203 (ML-KEM), FIPS 204 (ML-DSA) and FIPS 205 (SLH-DSA).
  • RFC 8446, The Transport Layer Security Protocol Version 1.3, for the handshake behaviour described above.

Engine version checking. This page describes the current implementation and will change as the engine changes.

How the engine is evaluated

Three lanes, kept apart on purpose, because a test you have tuned the engine against is not evidence of accuracy.

Regression. Eight documented company profiles, each with an answer key kept outside the scanner's inputs, are rescanned on every change to the engine. They were tuned, so their pass rate is a floor: it catches a capability that has been lost, and it is not an accuracy estimate.

Uncharted, weekly. A fresh set of weaknesses is drawn from a pool written before the engine saw it, and is not tuned against. The published number is the share of planted weaknesses the scanner reported. A miss is recorded as a measurement; fixing it moves the case into the regression lane and it is replaced in the pool.

Enterprise simulation, weekly. One company profile runs end to end in isolated containers, with real TLS endpoints, databases, cloud simulations and SSH inventory, and its evidence is published with any failures.

Detection accuracy · Public evidence history · Independent weekly evaluation

Continuous enterprise validation

We test documented fictional companies on isolated GitHub-hosted containers, with independent reference measurements and an answer key kept outside scanner inputs. Our public evidence history records the company profile, technology stack, measured precision and recall, failures, limitations and later verified fixes. Weekly runs rotate ready profiles; more varied stacks are added after their reference checks are ready.

Unknown evidence remains unknown. A synthetic test does not certify a production company or establish complete accuracy.