Research · first author · 2025
ZeroFalse
An LLM pipeline that filters false positives out of static-analysis alerts.
- LLMs
- Static analysis
- Evaluation
Overview
Static analyzers flag far more than they should. ZeroFalse takes raw CodeQL alerts and runs them through contextual reasoning and structured evidence validation before an LLM decides whether each one is real — then measures that across ten frontier models.
Contributions and results
- Evaluated 10 LLMs from 6 model families on the OWASP Java Benchmark (1,974 cases) and 755 real-world CodeQL alerts from 37 Java repositories.
- Best F1 of 0.912 on OWASP and 0.837 on real-world code.
- CWE-specialized prompting improved F1 by up to +0.26 on real-world code.
- Built an interactive dashboard for comparing models, CWEs and prompt versions.
