- Intelligent Tutoring System
- Deployment Issues of Automated Program Repair
- Concolic Program Repair
- Security Vulnerability Detection
- Neural Network Analysis
- Differential Software Testing
- Worst-Case Complexity Analysis
Intelligent Tutoring System
The high-level idea of the Intelligent Tutoring System is to introduce an automated technique to provide feedback and grading suggestions for programming assignments. As shown in the above figure, for a given programming assignment, the tutor would provide a reference solution and some test cases, while the student would submit a solution and receive feedback. The feedback should go beyond the simple execution of test cases and tell the student where and how to fix the submission. We apply this project in teaching foundations of software engineering, while applying latest research in automated program repair.
- NUS CS3213 Website: https://cs3213-fse.github.io
In an related earlier project, we studied automated program repair via reference implementation.
Deployment Issues of Automated Program Repair
Automated program repair is an emerging technology that seeks to automatically rectify bugs and vulnerabilities using learning, search, and semantic analysis. Trust in automatically generated patches is necessary for achieving greater adoption of program repair. As a first step, we surveyed more than 100 software practitioners to understand the artifacts and setups needed to enhance trust in automatically generated patches.
Concolic Program Repair
In this project, we developed an integrated approach for detecting and discarding overfitting patches via systematic co-exploration of the patch space and input space. We leverage concolic path exploration to systematically traverse the input space (and generate inputs), while ruling out significant parts of the patch space. We implemented our technique in the form of a tool called ‘CPR’ and evaluated its efficacy in reducing the patch space by discarding overfitting patches from a pool of plausible patches.
Security Vulnerability Detection
Side-channel attacks allow an adversary to uncover secret program data by observing the behavior of a program with respect to a resource, such as execution time, consumed memory or response size. Side-channel vulnerabilities are difficult to reason about as they involve analyzing the correlations between resource usage over multiple program paths. We developed fuzzing-based techniques to efficiently detect and quantify side-channel vulnerabilities.
- Differential Fuzzing for Side-Channel Analysis: ICSE’2019 (pdf), GitHub
- Quantitative Fuzzing for Side Channels: ISSTA’2021 (pdf), GitHub
On a more general attempt for security vulnerabilities, we developed Vudenc (Vulnerability Detection with Deep Learning on a Natural Codebase), a deep learning-based vulnerability detection tool that automatically learns features of vulnerable code from a large and real-world Python codebase.
- GitHub: https://github.com/LauraWartschinski/VulnerabilityDetection
- Paper: IST’2022 (pdf)
Neural Network Analysis
Neural networks have many applications, being used for example in pattern analysis, image classification, or sentiment analysis for textual data, and also in medical diagnosis or perception and control in autonomous driving, which bring safety and security concerns. In several research projects, we investigated how to assess determine the robustness of neural networks and how to improve their accuracy by fixing the logic of the network at an intermediate layer or at the last layer. We also developed an extension of Symbolic PathFinder for the efficient symbolic analysis of neural networks.
- Neural Network Repair (CAV’2021, pdf)
- NeuroSPF: A tool for the Symbolic Analysis of Neural Networks (ICSE’2021 Demo-Track, pdf, GitHub)
- Robustness Analysis (ICSE’2020, pdf)
Differential Software Testing
Detecting regression bugs in software evolution, analyzing side-channels in programs and evaluating robustness in deep neural networks (DNNs) can all be seen as instances of differential software analysis, where the goal is to generate diverging executions of program paths. Two executions are said to be diverging if the observable program behavior differs, e.g., in terms of program output, execution time, or (DNN) classification. The key challenge of differential software analysis is to simultaneously reason about multiple program paths, often across program variants. With HyDiff we developed the first hybrid approach for differential software analysis. HyDiff integrates and extends two very successful testing techniques: Feedback-directed greybox fuzzing for efficient program testing and shadow symbolic execution for systematic program exploration. HyDiff is implemented on top of the fuzzer AFL and the symbolic execution framework Symbolic PathFinder.
- GitHub: https://github.com/yannicnoller/hydiff
- Paper: ICSE’2020 (pdf)
- Doctoral Symposium Paper: ASE’2018 (pdf)
Worst-Case Complexity Analysis
The goal of Worst-Case Complexity Analysis (WCA) is to discover vulnerabilities which occur when the worst-case time or space complexity of an application is significantly higher than the average case. In this project we developed Badger - a new hybrid approach for complexity analysis using fuzzing and symbolic execution in tandem. It uses fuzz testing to generate a diverse set of inputs that aim to increase not only coverage but also a resource-related cost associated with each path. Since fuzzing may fail to execute deep program paths due to its limited knowledge about the conditions that influence these paths, we complement the analysis with a symbolic execution, which is also customized to search for paths that increase the resource-related cost. Symbolic execution is particularly good at generating inputs that satisfy various program conditions but by itself suffers from path explosion. We implemented our approach for the analysis of Java programs, based on Kelinci and Symbolic PathFinder.