Stop Using Vulnerability Counts to Measure Software Security

What is it about?

A vulnerability fix is a paradox: it’s a record of a mistake that is now gone. It is both a defeat and a triumph. The security posture of a system is not the same as a record of what was fixed historically. Unfortunately, I see vulnerability fix counts used to make questionable comparisons across products, projects, and companies and to justify new research. Security is so much harder to measure than that, and there are better ways.

Featured Image

Photo by Rapha Wilde on Unsplash

Photo by Rapha Wilde on Unsplash

Why is it important?

Software vulnerabilities are part of our everyday life. They expose us to theft, fraud, abuse, and threaten national security. We want to measure software security so we can understand it, which is an important and noble pursuit. In a climate where mistakes are never on the agenda, the incentives to fix vulnerabilities get mixed up. If we get it right, our developers will be incentivized to improve their software. If we get it wrong, we'll end up exposing ourselves even more.

Perspectives