Security Testing: The Power of Fuzzing
Fuzzing is a powerful testing technique that can be used to check software and system robustness. A fuzzer is used to test how a target system reacts to randomly generated invalid or unexpected inputs. Thanks to this procedure, ESCRYPT has already discovered weak points and implementation errors in numerous customer projects – up to and including the sudden crash of safety-relevant ECUs.
Generating inputs and monitoring
Every fuzzer has two main components: the test data generator and the monitoring tool. Ultimately, the quality of these components determines the quality and significance of the test result. The test data generator creates input data that is non-typical for the application under test. The aim is to trigger unexpected errors and anomalies. A good generator is protocol-aware, i.e. it speaks to the target system’s protocol and modifies only single aspects of the protocol (e.g. individual data fields or specific aspects of the message flow). That way, it is treading the fine line between invalid and valid input – which is called “smart fuzzing” (figure 1).
At the same time, it is essential to identify the misbehavior of the target system by means of monitoring. Ideally, it should be monitored in several ways – for example, by camera imaging if the system has a screen, but also by monitoring its amperage draw, using the good old “ping” message to which the target should always respond, or hooking up debuggers to monitor the target’s internal behavior. The bandwidth of possible misbehaviors ranges from complete hard failures or crashes to subtle bugs such as timing issues or memory leaks.
Figure 1: Fuzzing examines the fine line between valid and invalid input.
Blackbox and whitebox fuzzing
Fuzzing can be used as a blackbox test and as a whitebox test. With blackbox fuzzing, the tester has no insight into the target device being tested. All it can do is send randomized input to the system and observe whether it reacts visibly to that data in any way.
In whitebox fuzzing, on the other hand, monitoring is much deeper, but also much more complex. It allows for the inspection of binary files running on the target device, the installation of monitoring services, and even the recompilation of the targeted software, to detect subtle flaws. In contrast to blackbox fuzzing, which is performed purely as a lab test, whitebox fuzzing requires insights into the customer’s development processes.
Efficient and versatile testing tool
Fuzz testing is an extremely effective tool that can detect a wide variety of vulnerabilities. In one instance, ESCRYPT testers even caused an entire ECU to crash by fuzzing the CAN protocol of an embedded ECU. The team informed the developer and the bug was patched.
ESCRYPT is currently focused on protocol-aware blackbox fuzzing of different embedded protocols. The ESCRYPT security testers use a combination fuzzer based on commercial and freely available elements as well as elements developed in-house. This enables them to cover almost all automotive-relevant protocols: CAN, ISO-TP, UDS, USB, Bluetooth, WiFi, and many Ethernet-based protocols (IP, TCP, UDP, FTP, TLS, etc.). In addition, ESCRYPT is currently running several whitebox fuzzing projects to evaluate the integration of fuzz testing into different development environments. It has also developed its own fuzzing tool that specifically targets USB and CAN protocols.