Your Scanner Isn’t the Control. Your Evidence Is.

Scope note: this article is about ordinary infrastructure and application work that does not hold, process or manage personal data, payment data, health data or anything under a regulated regime such as PCI-DSS, GDPR, DORA or sector-specific rules. Where those apply, the rules are the rules, and this argument does not override them.

There is a conversation I keep having, and I suspect other small contractors do too.

A large company hires me to build or run something. The contract is signed by their legal and procurement people. It describes the deliverables, the service levels, the security obligations. It does not say “you must use vendor X’s scanner” or “you must purchase product Y.” It says, in effect: keep it secure, keep it patched, show us that you did.

Then, some months in, an email arrives from a department that was not in the room when the contract was signed. Usually security, sometimes architecture, sometimes an internal audit function. The email asks for a report. Not a report; a specific report, from a specific commercial tool, in a specific format. The one their team is used to seeing. The one that costs more per year than the contract pays per quarter.

I send what I have: a scan run with well-maintained open-source tools, pinned versions, exact commands, findings mapped to CVE identifiers, a clear statement of what was in scope, what was fixed, what was accepted and why. Plus something no scanner gives them: a list of things I patched or mitigated before the upstream fix landed, because I had the time to look, and I had assistance that let me look further than one person normally can.

The reply, more often than not, is a polite version of “that’s nice, but we need the one from the tool.”

I used to find this infuriating. I still find it wrong. But I have stopped thinking of it as a story about people who refuse to move with the times, because that framing is lazy and, more importantly, it does not work. Nobody changes their process because a contractor called them old-fashioned.

First, be fair to the security department

Put yourself in the chair of the person sending that email.

They are responsible for the security posture of a company that works with dozens, maybe hundreds, of external contractors. Each one produces artifacts in their own style. When something goes wrong, the question that will be asked of them is not “did the contractor do good work?” It is “how did you verify it?”

From that chair, a report produced by the contractor about the contractor’s own work is a vendor grading their own homework. That is not an insult to me. It is a control, and a sensible one. Independence of the assessor is one of the oldest principles in audit, older than any of the tools involved.

Then there is comparability. If fifty contractors each send a thoughtfully written PDF in fifty different structures, one analyst has to read fifty documents. If fifty contractors send the same machine-readable output, one analyst can triage all of it before lunch. “My report is more detailed” is, from their side, a cost, not a gift.

And there is liability. “We required the industry-standard tool and reviewed its output” is a sentence that survives a board meeting and a conversation with an insurer. “Our contractor used open-source software and an AI assistant” does not, yet. That is not the department’s fault. That is where the industry is.

Finally, and this one stings a little: an AI-assisted workflow is exactly the kind of thing a careful security person should be sceptical of. Language models can misjudge exploitability, invent a CVE identifier, or suggest a patch that quietly breaks a control. Speed is not evidence. “I covered more ground than a human could” is a claim, and claims need to be checkable.

So none of the department’s instincts are about living in the past. They are about their own accountability. Any article, or any contractor, that ignores that will be ignored in return.

Now, be fair to the contract

Having said all that: a department’s habits are not a contract.

When a company signs a statement of work with a small contractor, the price reflects the obligations in that document. If the document does not require a particular commercial tool, the price does not include it. A commercial scanner licence for a mid-size estate can cost more per year than the whole engagement. Asking a contractor to absorb that, repeatedly, informally, because “it’s what we always ask for,” is not a security control. It is an unbudgeted change of scope, requested by someone who does not own the budget.

The cleanest response is not an argument about tooling at all. It is procedural:

“Happy to provide that. It is outside the current statement of work, so please route the request through your procurement or legal contact and we will quote it as an addendum.”

In my experience, that sentence ends most of these conversations, because the person asking almost never wants to become the owner of a budget line. It also does something important: it moves the disagreement to where it belongs, inside the client’s own organisation, between the department that has a preference and the department that signed the contract. That is their alignment problem to solve, not mine.

But I do not want to stop there, because “not my problem” is a poor place for a professional relationship to sit. The better question is: what would make my evidence acceptable, on its own merits, to a reasonable security lead?

What defensible evidence looks like without a commercial logo

Here is what I have converged on. It is not exotic, and none of it requires a licence fee.

Reproducibility. Every report states which tools were run, at which exact versions, with which exact commands, against which exact targets, at which time. Anyone with access can re-run it and get the same result. A finding that cannot be reproduced is an opinion.

Standard formats alongside the narrative. The written report is for humans. Next to it goes the raw output in formats their tooling already ingests: SARIF for code and configuration findings, a CycloneDX or SPDX software bill of materials, CVE identifiers with CVSS scores. The analyst who has to triage fifty contractors can drop mine into the same pipeline as everyone else’s.

Explicit boundaries, especially around AI. The report separates three things clearly: what a tool detected, what a human decided, and where an assistant helped. Assistance is legitimate for triage, prioritisation, cross-referencing advisories, drafting the summary, and spotting patterns across a large surface. It is not a detection engine, and it is never presented as one. If a model suggested a mitigation, the report says a person reviewed and applied it, and names the person.

Verifiable artifacts. Reports are signed. Raw scanner output is retained for the engagement period. The client can check that the document they received is the document that was produced, and that the summary matches the underlying data.

A path to independence. The report closes with an offer, not a demand: if independent validation is required, here is what it would cost to engage a third party, or here is how the client’s own team can re-run the pipeline in an hour, on their own infrastructure, with their own eyes.

That last point matters more than it looks. Independence is what the security department actually needs. A commercial tool gives them the appearance of independence, because a third party wrote the software. A reproducible pipeline they can execute themselves gives them the substance of it.

Where the “living in the past” argument finally applies

Once evidence looks like that, the legitimate objections are gone. What is left, if the answer is still “no, we need the report from the tool,” is a preference for a logo over verifiability. That is the moment, and only that moment, where the word “compliance theatre” is earned.

Because consider what the commercial-only position actually asks for. Most commercial scanners are reactive by design: they detect what is already in their signature database, which means what is already known, which means what an attacker may already have been using for days. The workflow I am describing, in which a contractor keeps systems current, reads advisories as they land, and uses assistance to check exposure across an entire estate in an afternoon, is not a cheaper substitute for scanning. It is a different thing: prevention with an audit trail. Rejecting that in favour of a quarterly PDF from a licensed product does not raise the security bar. It lowers it, and charges more for the privilege.

There is a balance to be found, and I do not think it is complicated. Compliance requires evidence. Evidence must be independent, comparable and verifiable. Open-source tooling and careful documentation can meet all three, and assistance from AI, used with clear boundaries, can extend how far a small team can see. What compliance has never required is a specific vendor’s invoice.

What I am asking

To the people in security, architecture and audit departments who send those emails: I understand what you are protecting, and I am not asking you to lower your standards. I am asking you to state them in terms of evidence rather than products, and to check that what you ask of contractors matches what your own legal team agreed with them. If it does not, that is an internal conversation, and it is a fair one to have.

To fellow contractors: stop arguing about tools. Make your evidence reproducible, machine-readable, honestly scoped and independently checkable. Then, when someone still insists on the logo, you will know exactly what the objection is, and so will they.

The scanner was never the control. The evidence is.