When AI Suggests a Security Bug, Can the Code Back It Up?

What is it about?

In an illustrative example in our paper, a request parameter enters one Java file, becomes part of a database query in another, and reaches execution in a third. Deciding whether that path is vulnerable requires evidence from across the project. GraphLedger compresses a map of code connections, aiming to retain the paths and evidence relevant to a suspected bug. An AI model proposes a possible vulnerability. The Assumption Ledger then checks the assumptions in that proposal against repository evidence. Based on the evidence found within a bounded search, it labels the hypothesis Verified, Rejected, or Inconclusive.

Featured Image

Photo by Xavier Cee on Unsplash

Photo by Xavier Cee on Unsplash

Why is it important?

Security warnings are more useful when developers can see the code evidence behind them. In the study’s controlled test on 100 Java repositories, 23% of the cases GraphLedger flagged as vulnerable were false positives. The figure was 41% for the study’s VulAgent baseline under the same evaluation protocol. GraphLedger’s recall was 0.76, meaning it identified 76% of the labeled vulnerable cases in that test. The recorded evidence may help reviewers inspect a finding. These results concern bounded analyses of how data reaches sensitive operations in Java. They do not establish performance for every language or kind of bug.

Perspectives