docs: update fuzz tests (split into separate page)
Build all targets / Scan for Board Targets (push) Has been cancelled
Build all targets / Build Group [${{ matrix.group }}][${{ matrix.arch == 'nuttx' && 'x86' || 'arm64' }}] (push) Has been cancelled
Build all targets / Upload Artifacts to S3 (push) Has been cancelled
Build all targets / Create Release and Upload Artifacts (push) Has been cancelled
Checks / build (NO_NINJA_BUILD=1 px4_fmu-v5_default) (push) Has been cancelled
Checks / build (NO_NINJA_BUILD=1 px4_sitl_default) (push) Has been cancelled
Checks / build (check_format) (push) Has been cancelled
Checks / build (check_newlines) (push) Has been cancelled
Checks / build (module_documentation) (push) Has been cancelled
Checks / build (px4_fmu-v2_default stack_check) (push) Has been cancelled
Checks / build (px4_sitl_allyes) (push) Has been cancelled
Checks / build (shellcheck_all) (push) Has been cancelled
Checks / build (tests) (push) Has been cancelled
Checks / build (tests_coverage) (push) Has been cancelled
Checks / build (validate_module_configs) (push) Has been cancelled
Clang Tidy / build (push) Has been cancelled
MacOS build / build (px4_fmu-v5_default) (push) Has been cancelled
MacOS build / build (px4_sitl) (push) Has been cancelled
Ubuntu environment build / Build and Test (ubuntu:22.04) (push) Has been cancelled
Ubuntu environment build / Build and Test (ubuntu:24.04) (push) Has been cancelled
Container build / Set Tags and Variables (push) Has been cancelled
Container build / Build Container (amd64) (push) Has been cancelled
Container build / Build Container (arm64) (push) Has been cancelled
Container build / Deploy To Registry (push) Has been cancelled
EKF Update Change Indicator / unit_tests (push) Has been cancelled
Failsafe Simulator Build / build (failsafe_web) (push) Has been cancelled
FLASH usage analysis / Analyzing px4_fmu-v5x (push) Has been cancelled
FLASH usage analysis / Analyzing px4_fmu-v6x (push) Has been cancelled
FLASH usage analysis / Publish Results (push) Has been cancelled
ITCM check / Checking nxp_tropic-community (push) Has been cancelled
ITCM check / Checking px4_fmu-v5x (push) Has been cancelled
ITCM check / Checking px4_fmu-v6xrt (push) Has been cancelled
MAVROS Mission Tests / build (map[mission:MC_mission_box vehicle:iris]) (push) Has been cancelled
MAVROS Offboard Tests / build (map[test_file:mavros_posix_tests_offboard_posctl.test vehicle:iris]) (push) Has been cancelled
Nuttx Target with extra env config / build (px4_fmu-v5_default) (push) Has been cancelled
Python CI Checks / build (push) Has been cancelled
ROS Integration Tests / build (push) Has been cancelled
ROS Translation Node Tests / Build and test (map[ros_version:humble ubuntu:jammy]) (push) Has been cancelled
ROS Translation Node Tests / Build and test (map[ros_version:jazzy ubuntu:noble]) (push) Has been cancelled
SITL Tests / Testing PX4 tailsitter (push) Has been cancelled
SITL Tests / Testing PX4 iris (push) Has been cancelled
SITL Tests / Testing PX4 standard_vtol (push) Has been cancelled
Docs - Crowdin - Upload Guide sources (en) / upload-to-crowdin (push) Has been cancelled
Docs - Deploy PX4 User Guide / build (push) Has been cancelled
Docs - Deploy PX4 User Guide / deploy (push) Has been cancelled
Handle stale issues and PRs / stale (push) Has been cancelled
Fuzzing / Fuzzing (push) Has been cancelled

This commit is contained in:
Hamish Willee
2025-06-19 09:24:37 +10:00
committed by Beat Küng
parent 3cfaaf73b4
commit 572a06b2f3
3 changed files with 75 additions and 47 deletions
+1
View File
@@ -813,6 +813,7 @@
- [Test MC_04 - Failsafe Testing](test_cards/mc_04_failsafe_testing.md)
- [Test MC_05 - Indoor Flight (Manual Modes)](test_cards/mc_05_indoor_flight_manual_modes.md)
- [Unit Tests](test_and_ci/unit_tests.md)
- [Fuzz Tests](test_and_ci/fuzz_tests.md)
- [Continuous Integration](test_and_ci/continous_integration.md)
- [Integration Testing](test_and_ci/integration_testing.md)
- [MAVSDK Integration Testing](test_and_ci/integration_testing_mavsdk.md)
+70
View File
@@ -0,0 +1,70 @@
# Fuzz Tests
Fuzz tests are a generalised form of [unit test](../test_and_ci/unit_tests.md) that run code against a large number of random inputs.
This helps to ensure that the code is robust against any input, not just those expected by the developer.
They can be written like unit tests with possible assertions (`EXPECT_EQ`, etc), and have a set of input parameters.
The fuzzer then tries to find inputs that cause the code to crash (with [Address Sanitizer](https://clang.llvm.org/docs/AddressSanitizer.html) enabled automatically) or trigger an assertion.
The tests are run as part of normal unit tests, and in a more comprehensive fuzzing mode test.
For more information see [Running Fuzz Tests](#running-fuzz-tests) below.
## Writing a Fuzz Test
To write a fuzz test:
1. Start by writing a "normal" [functional test](../test_and_ci/unit_tests.md#functional-test).
1. Make sure the file name contains `fuzz` (lower case).
For example `my_driver_fuzz_tests.cpp`.
1. Add one or more fuzz tests to the file.
For example:
```cpp
#include <gtest/gtest.h>
#include <fuzztest/fuzztest.h>
void myDriverNeverCrashes(const std::string& s) {
MyDriver driver;
driver.handleInput(s);
}
FUZZ_TEST(MyDriverFuzzTests, myDriverNeverCrashes);
```
The file can also contain normal tests.
For more information, see https://github.com/google/fuzztest.
A complete example in the PX4 repository can be found in the [septentrio driver](https://github.com/PX4/PX4-Autopilot/blob/main/src/drivers/gnss/septentrio/septentrio_fuzz_tests.cpp).
## Running Fuzz Tests
Fuzz tests can be run in two modes:
- As part of normal unit tests with `make tests`.
This will only create a small number of inputs and not use coverage information.
- In fuzzing mode: this runs a single fuzz test with coverage information over a longer period of time (either fixed or indefinitely).
The fuzzer will try to find inputs that cover all reachable code paths.
It requires compilation with Clang and can be run with the following commands:
```sh
rm -rf build/px4_sitl_tests
export CC=clang
export CXX=clang++
make tests TESTFILTER=__no_tests__
cd build/px4_sitl_tests
./functional-<my-test> --fuzz=<test-name>
```
## Seeds
Depending on the code complexity, it might be hard for the fuzzer to find inputs that pass certain conditions.
For this it is possible to provide one or more seeds, which the fuzzer will use as first inputs.
[The google documentation](https://github.com/google/fuzztest/blob/main/doc/fuzz-test-macro.md#initial-seeds-initial-seeds) contains more details.
You can also use the `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` macro for conditional code compilation, for example to exclude CRC checks.
More information about efficient fuzzing can be found on [this page](https://chromium.googlesource.com/chromium/src/+/main/testing/libfuzzer/efficient_fuzzing.md).
## CI
Fuzz tests are run as part of the normal unit tests in CI for each pull request.
In addition, the fuzz tests are run daily for 15 minutes on the main branch.
+4 -47
View File
@@ -23,7 +23,7 @@ The steps to create new unit tests are as follows:
Tests can be run via `make tests`, after which you will find the binary in `build/px4_sitl_test/unit-MyNewUnit`.
It can be run directly in a debugger.
## Writing a GTest Functional Test{#functional-test}
## Writing a GTest Functional Test {#functional-test}
GTest functional tests should be used when the test or the components being tested depend on parameters, uORB messages and/or advanced GTest functionality.
Additionally, functional tests can contain local usage of STL data structures (although be careful of platform differences between e.g. macOS and Linux).
@@ -172,51 +172,8 @@ For example:
- `make tests TESTFILTER=Attitude` only run the `AttitudeControl` test
## Fuzz Testing
Fuzz tests are a generalized form of unit tests that are run on a large number of inputs to ensure the code is robust against any input.
They can be written like unit tests with possible assertions (`EXPECT_EQ`, etc), and have a set of input parameters.
The fuzzer then tries to find inputs that cause the code to crash (with enabled Address Sanitizer) or trigger an assertion.
### Writing a Fuzz Test
1. Start by writing a [functional test](#functional-test).
1. Make sure the file name contains `fuzz` (lower case). For example `my_driver_fuzz_tests.cpp`.
1. Add one or more fuzz tests to the file. The file can also contain normal tests.
For example:
```cpp
#include <gtest/gtest.h>
#include <fuzztest/fuzztest.h>
Fuzz tests are a generalised form of unit test that ensures code is robust against any input.
They are run as part of the unit tests, and also more extensively in their own testing mode.
void myDriverNeverCrashes(const std::string& s) {
MyDriver driver;
driver.handleInput(s);
}
FUZZ_TEST(MyDriverFuzzTests, myDriverNeverCrashes);
```
For further details, see https://github.com/google/fuzztest.
### Running a Fuzz Test
Fuzz tests can be run in two modes:
- As part of normal unit tests with `make tests`. This will only create a small number of inputs and not use coverage information.
- In fuzzing mode: this runs a single fuzz test with coverage information over a longer period of time (either fixed or indefinitely).
The fuzzer will try to find inputs that cover all reachable code paths.
It requires compilation with Clang and can be run with the following commands:
```sh
rm -rf build/px4_sitl_tests
export CC=clang
export CXX=clang++
make tests TESTFILTER=__no_tests__
cd build/px4_sitl_tests
./functional-<my-test> --fuzz=<test-name>
```
### Seeds
Depending on the code complexity, it might be hard for the fuzzer to find inputs that pass certain conditions.
For this it is possible to provide one or more seeds, which the fuzzer will use as first inputs.
[The documentation](https://github.com/google/fuzztest/blob/main/doc/fuzz-test-macro.md#initial-seeds-initial-seeds) contains more details.
You can also use the `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` macro for conditional code compilation, for example to exclude CRC checks.
More information about efficient fuzzing can be found on [this page](https://chromium.googlesource.com/chromium/src/+/main/testing/libfuzzer/efficient_fuzzing.md).
### CI
Fuzz tests are run as part of the normal unit tests in CI for each pull request.
In addition, the fuzz tests are run daily for 15 minutes on the main branch.
For more information see [Fuzz Tests](../test_and_ci/fuzz_tests.md).