Enhancing domain knowledge-based automatic vulnerability discovery

Abstract

Recently, there has been much research on improving automatic vulnerability discovery (e.g., fuzzing and static analysis) by leveraging domain knowledge. Although most of them focused on which domain knowledge to leverage, this dissertation presents that considering how to apply and represent domain knowledge also helps to enhance domain knowledge-based automatic vulnerability discovery. First, we utilize more domain knowledge. We extend the previous JavaScript engine fuzzing, which uses only JavaScript syntax, to leverage a JavaScript type system and show the effectiveness of our approach. Second, we enhance by more advanced adoption of domain knowledge. Particularly, we improve tools for finding a kind of use-after-free bugs caused by compacting garbage collection. For that, we precisely define these use-after-free bugs and implement a new tool with the tailored symbolic execution, which found bugs that the previous tools missed. Lastly, we enhance extensible static binary checking tools, which take domain knowledge as vulnerability patterns, by analysts-friendly representation of domain knowledge. Previous tools only support patterns based on their own low-level intermediate representations, while most analysts work with decompiled code, which have high-level information such as value types. We thus propose an extensible static checking tool based on decompiled code, which can support diverse patterns and find several bugs in COTS binaries such as Windows kernel.