I started the year hearing the same concern across the industry. Without access to the newest, most powerful cyber-capable models, companies would fall behind in finding and remediating vulnerabilities.
The assumption was understandable. Finding vulnerabilities at scale is hard, and the strongest model would surely deliver the strongest security results. Access to it, many believed, would determine who stayed ahead.
What Changed My Mind
Our security research added an important nuance. Models matter, but their real value also depends on the harness around them, the skills, workflows and business context baked in from the start. With that foundation, a smaller, specialized model can outperform a much larger general-purpose model.
A larger AI model may have broader general knowledge, but it does not inherently know our architecture, our risk priorities or the controls required in a regulated environment. A smaller model with the right context and guardrails can be more precise, faster and more useful.
The harness filters out irrelevant code, prioritizes security-sensitive areas and builds an understanding of how data moves through an application. It then searches for vulnerabilities, tries to disprove each potential finding and uses a second model to spot-check the final results. The system also applies the security knowledge and patterns we have built over time, so it is not starting from scratch with every question.
The step our engineers call the secret sauce is context. The harness pulls patterns specific to the language and the kind of application. A payments application gets payments checks, and company-specific patterns like refund abuse get checked where they apply. Pointing a model at raw source code and asking it to find vulnerabilities misses all of that.
The results changed how I think about the problem. Before we built the harness, a team of 25 engineers reported three Common Vulnerabilities and Exposures (CVEs) in a year. With it, we have received approximately 230 CVEs across widely used open-source software. One popular framework alone produced about 20 findings, including six or seven critical remote-code-execution flaws. Those vulnerabilities are now patched for the whole community, helping make the software safer for the entire industry.
In internal testing, we found our harness more effective than a generic frontier-model approach. The advantage did not come from the model alone. It came from pairing the right model with a system designed to teach it how to do the work.
What I’m Doing Differently
In Q4, I’m investing not only in model access, but also in the process and context that make those models effective.
As more code is written with AI, the industry must apply greater scrutiny on the back end. Every finding is still validated, reproduced, assessed for impact and fixed with the developer who owns the code.
Models matter. But our experience has shown that the greatest advantage, and the real productivity leap, comes from the skills, context and controls you build around them.
