Agentic LLM workflows for vulnerability detection, exploitation, and patching — co-supervised by Marc Juarez (Lecturer, Security, Privacy and Trust) and Adriana Sejfia (Institute for Computing Systems Architecture), with Dr. Mohammad Tayebi at SFU. Designed and ran a controlled ablation on whether exploit evidence improves LLM-generated security patches, holding the agent, model, and prompt wording constant so the evidence was the only variable: the result was a null one — no significant difference between the exploit-informed and alert-only arms (+3.0 points on a 0–100 scale across 12 matched CVE pairs, win/loss/tie 6–5–1, sign test p ≈ 1.0) at roughly 3× the cost. The contribution is the mechanism diagnosis behind it: a single proof-of-vulnerability is a narrowing signal, so an evaluation gate defined on that one path rewards patches that close it rather than the bug class behind it. Also built the evaluation harness (reference-anchored LLM-as-judge scoring against the real upstream fix across 8 dimensions, K=3 ensemble with medoid selection, aggregates computed in code, hard gates on vulnerability-elimination and regressions, and a measured noise floor so any reported difference can be read against it), curated a linked-CVE dataset and directed graph of security fixes that were themselves later found incomplete, and shipped the AutoSec Lab dashboard (FastAPI + React) for run orchestration and side-by-side arm comparison.
University of Edinburgh →