Commit Graph
1805 Commits
Author SHA1 Message Date
Peter Barker 251092473a Tools: shorten column names in global size summary table
Use Periph, tracker and iofw in the header to keep the table narrow.
2026-09-24 13:35:21 +10:00
Randy Mackay 8e67c21500 Tools: allowed_subsystems accepts Periph as prefix 2026-09-24 11:28:09 +10:00
George Zogopoulos 49b8a2c365 Tools: add convert_params_4_7.py to convert 4.6 parameter files to 4.7
Renames and rescales the Copter and QuadPlane parameters that changed
name or units in 4.7 (attitude, position and waypoint controllers,
loiter, circle, RTL, land, pilot, rangefinder, MAV_ and stream rate
parameters), converts ARMING_CHECK to ARMING_SKIPCHK and moves the
MAVLink bits of SERIALn_OPTIONS into MAVn_OPTIONS. Files are converted
in place or into an output directory, --patch selects the 4.7.0 or 4.7.1
names and the vehicle is detected from the file content.
2026-09-23 17:21:55 +10:00
George Zogopoulos 95af0b8efb Tools: refactor convert_param_scale.py into a reusable module
Move the parameter file parsing, rescaling and rewriting into importable
helpers so that other conversion scripts can reuse them. The standalone
interface is unchanged apart from a new --sig-digits option (default 3).
QGroundControl files are now handled, names are matched exactly instead
of by prefix, and separators, comments and untouched lines are preserved.
2026-09-23 17:21:55 +10:00
Peter Barker 25dc1c4640 Tools: run_coverage.py: cope with lcov 2.x fatal error classes
lcov 2.x, as shipped in the Ubuntu 24.04 CI containers, promotes two
conditions the weekly coverage run always hits from warnings to fatal
errors:

 - "mismatched end line": two functions starting on the same source
   line with different end lines.  Every gtest TEST() body trips this,
   as the macro also defines the fixture constructor and destructor on
   the TEST() line, so both lcov --capture invocations abort.
 - "unused": a --remove pattern which matched nothing; ".waf*" never
   matches, so the pattern-removal step aborts.

Pass --ignore-errors for those classes on the affected invocations.
lcov 1.x rejects unknown error classes, so the arguments are only
emitted when lcov reports version 2 or later.
2026-09-23 08:37:53 +10:00
Pierre KancirandClaude Opus 5 c07e8722bf Tools: build_tests: compare the binaries the board actually built
The size table marks a board "*" when the two builds are byte-identical
and "0" when they are the same size but differ, and two whole classes of
board could never reach "*".

The name reaches binaries_are_identical() lower-cased, because that is
the key the column lookup needs, but the files are AP_Periph.bin and
AP_Bootloader.bin, so on a case-sensitive filesystem neither candidate
was found and the answer was "differs" whatever the bytes said.  CI
proves those two are identical: base and pull request builds log the same
APP_DESCRIPTOR crc32 pair, which covers the whole image.

A Linux board writes neither .bin nor .elf -- disco's bin/ holds bare
ELFs -- so nothing was compared there either.  Match a file without
regard to case, and fall back to the extensionless ELF.

While here: enumerate the directory being measured rather than always the
master one, and import tabulate where it is used, so the module can be
imported by a test without it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-23 08:35:34 +10:00
Graham 640c74e76c Tools: add WebAssembly SITL support 2026-09-22 12:59:21 +10:00
Peter Barker d437708812 Tools: add @MAV_LOG build option
@MAV_LOG has no class of its own to find in the symbol table, so it is
detected by its prefix string, which only the backend table row puts in the
binary. That row is also only built where logs go to a filesystem, so the
option depends on Logging.
2026-09-22 11:10:46 +10:00
Peter Barker c892590775 Tools: update AHRS auto-trim feature symbol for its move into Copter
The auto-trim scheduler task moved from RC_Channels_Copter::auto_trim_run
to Copter::AHRSTrimming::auto_run, so extract_features.py was reporting
AP_COPTER_AHRS_AUTO_TRIM_ENABLED as absent from binaries which have it.
2026-09-22 09:30:40 +09:00
Pierre KancirandClaude Opus 5 08d11f1ca8 Tools: build_tests: name the commits and emit the table as json
The table says which commit it was built from and which it was compared
against, and --json-output writes the same as data for the workflow that
comments it on a pull request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 07:55:09 +10:00
Thomas Watson d25c1668e3 Tools: fix size_compare_branches.py jobs argument
waf does not persist `-j` in the configuration, it's only used for the
command it's specified on. So specifying `-j` to this script never did
anything to limit the number of build processes because the job limit
was only given to configure commands.

Now the script passes the appropriate number of jobs to build as well
as configure commands, so the limit is actually applied.

It's unclear if passing the number of jobs to the configure command
accomplishes a lot, but it doesn't hurt so it's kept.
2026-09-21 10:23:57 +10:00
Peter Barker b2b1b3d279 Tools: decode_devid.py: parse device types from the C++ headers
pre-commit / ci (push) Canceled after 0s
test Renode / cubeorangeplus-quadplane (push) Canceled after 0s
test scripts / build (astyle-cleanliness) (push) Canceled after 0s
test scripts / build (check_autotest_options) (push) Canceled after 0s
test scripts / build (logger_metadata) (push) Canceled after 0s
test scripts / build (param-file-validation) (push) Canceled after 0s
test scripts / build (param_parse) (push) Canceled after 0s
test scripts / build (python-cleanliness) (push) Canceled after 0s
test scripts / build (shellcheck) (push) Canceled after 0s
test scripts / build (validate_board_list) (push) Canceled after 0s
Rather than maintaining copies of the bus and device type tables,
decode_devid.py now parses the enums in AP_HAL/Device.h, AP_SerialManager.h
and the compass, IMU, baro and airspeed backend headers using
logger_metadata/enum_parse.py.  A small rename map keeps established
display names (DRONECAN, AK0991x), and the retired LIS2MDL ID, which is
in no enum, is listed explicitly.  Comments on enum entries are shown as
warnings when decoding.

--dump-json and --dump-json5 write the tables, with a format_version
and a content-hash data_version, for tools without a source tree;
--json reads such a file back.  A copy of the script outside an
ArduPilot tree (as synced by MethodicConfigurator) reads devid.json from
its own directory.  The files are published alongside LogMessages.* by
build_log_message_documentation.sh, and CI dumps them in the
logger_metadata step so that header changes which break parsing are
caught.

The tables had drifted from the headers: ACC_LSM9DS1,
INS_ZEROONE_FPGA_SCH16T and INS_ICM56686 were missing, MMC5883 is now
MMC5983, AK8963/BMM150 had trailing spaces, and several names now
follow the headers.
2026-09-16 17:18:56 +10:00
Dr.-Ing. Amilcar do Carmo Lucas 2dc256b862 Tools: Generate __METADATA__ directories with parameter and LogMessages documentation
Generates parameter XML and LogMessages documentation.
Stores compressed metadata in __METADATA__ directories.
Adds atomic directory replacement and locking.
Adds --metadata-only and vehicle-selection options.
Copies metadata into stable-release directories.
Adds metadata URLs to generated manifests.
Improves tag validation and skips unsupported Blimp stable/beta builds.
2026-09-15 16:36:38 +10:00
g945643frankie e87935e3bd Tools: add ICM-56686 to decode_devid.py
DEVTYPE_INS_ICM56686 is 0x44.
2026-09-15 11:11:00 +10:00
Peter BarkerandClaude Opus 5 3f73e5ad1f Tools: make run_coverage.py stop at errors, add a flag to keep going on errors
use as a diagnostic tool was limited when it would exit on some failures but not on others.

Make it exit on failures by default, add a flag to allow a user to get consistent behaviour the other way

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-14 18:58:28 +10:00
Peter Barker 7dbc76fb1c Tools: enable math index checks for every build.Rover CI job
The rover workflow's build job primes ccache with
--enable-math-check-indexes, matching sitltest-rover and
sitltest-sailboat, which both pass "Rover" as the run_autotest name.
sitltest-balancebot passes "BalanceBot", so it configured without the
flag, missed the cache on 872 of 1382 compiles and spent 6m33s
building ardurover against 14s in the other two jobs.

Key the flag on the build step rather than the name, so BalanceBot
builds the same binary as the other build.Rover jobs.
2026-09-14 18:58:17 +10:00
Peter Barker f7b5cc7ddc Tools: size_compare_branches.py: do not emit 'missing build product' when creating progress files
this output is only useful at the end of a run, not during it
2026-09-08 21:12:02 +10:00
yogesh 3e77c0ef54 Tools: update build server description to specify TFS20L-I2C 2026-09-05 19:09:31 +10:00
Peter Barker fde165277d Tools: size_compare_branches.py: correct behaviour on heater-less IOMCU firmwares
this wasn't returning correct size comparisons on CubeRedSecondary-IO
2026-09-05 14:00:50 +10:00
Eric Katzfey 9329fc345d Tools: support QURT binaries in branch size comparison 2026-09-04 12:46:29 +10:00
Andrew Tridgell eec4032d98 Tools: fix CAN playback bus filtering
DataFlash field names are case-sensitive, so the lowercase lookup always fell back to bus zero. Read the CANF and CAFD Bus field using its declared capitalization.
2026-09-04 11:11:33 +10:00
Andrew Tridgell e3681fc848 Tools: CAN_playback capture uses CAN_Pn_OPTIONS
The CAN_logger.lua example it referenced has been removed.
2026-09-04 11:11:33 +10:00
Peter BarkerandClaude Opus 5 836c75eb94 Tools: add a feature gate for copy-from-EKF calibration
AP_COMPASS_LEARN_COPY_FROM_EKF_ENABLED, defaulting to
COMPASS_LEARN_ENABLED && AP_AHRS_ENABLED, so the disarm-time copying of
EKF-learned offsets can be compiled out on its own.

save_ekf_learned_offsets() moves out of the COMPASS_LEARN_ENABLED block
in Compass_learn.cpp so the two are genuinely separable; the in-flight
learner drags in the EKF-GSF yaw estimator, which this does not need.

Verified with nm: Compass::save_ekf_learned_offsets is present in a
default build and absent when the define is 0, with COMPASS_LEARN_ENABLED
still enabled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 09:29:27 +10:00
Dr.-Ing. Amilcar do Carmo Lucas 0c9b31f832 Tools: Make generate_pdef.xml_metadata.py more robust and flexible
Turn a fragile working-tree-mutating script into a safer, repeatable historical-release metadata generator.

- Replaces hard-coded paths and implicit directory changes with CLI options for repository, output directory, vehicle selection, starting version, and upload.
- Enumerates and semantically sorts valid vehicle release tags, including the Sub/ArduSub naming difference.
- Uses disposable detached Git worktrees, preserving the caller’s checkout.
- Detects parser capability for SHA/tag provenance and inserts deterministic fallback provenance when needed.
- Atomically publishes generated XML.
- Makes uploading opt-in and requires explicit rsync credentials/password-file input.
- Retains release exclusions for unsupported historical parameter-parser versions.
2026-09-02 17:36:27 +10:00
Dr.-Ing. Amilcar do Carmo Lucas 032a861cc0 Tools: address robot issues in decode_devid.py
Some of the issues were pre-existing.
Behavior is preserved and warnings issued.
2026-09-02 17:28:56 +10:00
Dr.-Ing. Amilcar do Carmo Lucas 6679ef5658 Tools: Refactor decode_devid to be callable by python code
Also fix the lining issues

Signed-off-by: Dr.-Ing. Amilcar do Carmo Lucas <amilcar.lucas@iav.de>
2026-09-02 17:28:56 +10:00
Andy Piper ec69d60327 Tools: scripts: add build option for AP_GroundEffect 2026-09-01 20:09:07 +09:00
Peter Barker 6a413fb5ac Tools: use new Location object in place of mavutil.location 2026-09-01 19:15:49 +10:00
“Jim 09ab657f2b Tools: add AF9838 build option 2026-08-31 08:18:54 +10:00
“Jim ccc8977fd1 Tools: add AF9838 compass device ID decoding 2026-08-31 08:18:54 +10:00
Andrew Tridgell a13b9757f8 Tools: add mass storage build option 2026-08-28 17:18:39 +10:00
Peter BarkerandClaude Opus 4.8 45b5045521 Tools: add a regression test for output of clang-scan-build
Add process_scan_build_output.py, which parses the .plist reports a
scan-build run leaves behind and fails if any finding is not accounted
for, so that findings cannot creep back in over time.

Findings are matched against scan_build_suppressions.py.  Each entry
names a file and one issue hash per clang major version: the hash clang
derives for a finding is not stable across releases, and not every
finding exists in every release.  An entry is stale, and so a failure,
only when it carries a hash for the version actually run and that hash
matched nothing.  The list is therefore an accurate record of the
findings we have reviewed and accepted, and fixing a suppressed finding
fails until its entry is removed.

The check fails closed.  It refuses to pass unless the report directory
exists, holds at least one report, and names a clang version the
suppression list has hashes for; a run under an unrecorded version
would otherwise quietly check nothing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 14:01:54 +10:00
Iampete1 e624e95ae5 Tools: scripts: decode_devid: add MAVLink devide decodeing 2026-08-27 10:59:59 +10:00
Peter Barker 4cb9402076 Tools: correct swapped brand_name and manufacturer in brand_map
generate_manifest.py unpacks brand_map values as (brand_name,
manufacturer).  Four entries were added with the fields the other way
around, so the manifest published the vendor as the board name and the
board name as the vendor.

Only the tuple order changes; the strings are left as they were.
2026-08-26 17:21:01 +10:00
Peter Barker 958dafe2c6 Tools: param_check_all.py: skip checking iflight-chimera7-4.2.param params
these are notionally 4.2 parameters - it's there in the name
2026-08-26 09:47:38 +10:00
Peter Barker 3418106a20 Tools: run test.BattCAN in the sitl_periph CI job
test.BattCAN has been broken since it was added in June 2024 and
nothing noticed, because no CI job ran it.  Run it alongside test.CAN
in the existing sitltest-can target: no new CI job, roughly two
minutes added under peripheral lockstep.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
(cherry picked from commit 5a6d13039ae87bda202bd3aaa379a35128ad1b9b)
2026-08-25 11:32:08 +10:00
Andrew Tridgell 0ca141baaf Tools: add configurable crashdump backends 2026-08-25 10:46:36 +10:00
Tom Pittenger 2a2f7e9232 Tools: rename the ADSB_UAVIONIX build option to ADSB_MAVLINK
Follows the rename of AP_ADSB_uAvionix_MAVLink to AP_ADSB_MAVLink and of
HAL_ADSB_UAVIONIX_MAVLINK_ENABLED to HAL_ADSB_MAVLINK_ENABLED.

Spelled ADSB_MAVLINK rather than ADSB_MAVLink to match the options either
side of it - ADSB_SAGETECH, ADSB_SAGETECH_MXS and ADSB_UAVIONX_UCP.

Note this changes a user-visible option name: --disable-ADSB_UAVIONIX now
fails with waf's "no such option" rather than silently building a feature
the user asked to leave out.
2026-08-18 21:06:43 -07:00
Andrew Tridgell 5c37c3c4c0 Tools: select matching board for upload on linux
pre-commit / ci (push) Canceled after 0s
test scripts / build (astyle-cleanliness) (push) Canceled after 0s
test scripts / build (check_autotest_options) (push) Canceled after 0s
test scripts / build (logger_metadata) (push) Canceled after 0s
test scripts / build (param-file-validation) (push) Canceled after 0s
test scripts / build (param_parse) (push) Canceled after 0s
test scripts / build (python-cleanliness) (push) Canceled after 0s
test scripts / build (shellcheck) (push) Canceled after 0s
test scripts / build (validate_board_list) (push) Canceled after 0s
this makes "./waf XXX --upload" on linux smarter by selecting the
right USB device based on the information in the APJ file. When there
is only one candidate USB device it behaves as it does now, but when
there is more than one it selects based on board name and USB ID.

This stops the annoying reboot of all of your other flight controllers
of different types on USB when working on just one of them.
2026-08-19 09:19:51 +10:00
Randy Mackay 327bd39082 Tools: du32_change.py update for unused_new_radio_frame 2026-08-11 02:01:53 +01:00
Peter Barker 913408d39f Tools: suggest the correct subsystem for a prefix-less commit
check_branch_conventions.py used a static 'AP_Compass:' example when a
commit message subject had no subsystem prefix.  Add
BuildScriptBase.subsystem_for_commit(), which resolves a commit's changed
files through allowed_subsystems.py and returns the subsystem they share,
and name that subsystem in the suggestion instead.  Fall back to the
static example when the files do not resolve to one subsystem.

Also reword the message so it is clear the missing prefix is on the git
commit message, not on the pull request title, and say how to amend it.
2026-08-11 10:06:27 +10:00
Peter Barker 23c69071f5 Tools: move get_changed_paths_for_commit into BuildScriptBase
Verbatim move; it sits naturally alongside get_added_paths_for_commit
and is about to be needed by another base-class method.
2026-08-11 10:06:27 +10:00
Peter Barker 016078de71 Tools: build_options.py: remove bad dep for MAVLINK_SIGNING
copy/paste error from duplicated line
2026-08-11 10:04:49 +10:00
Peter BarkerandClaude Opus 5 3da0a5d5ff Tools: build_script_base: work out the bootloader blacklist on first use
Working out the blacklist parses every ChibiOS hwdef, which takes a
third of a second, and both build_boards.py and
size_compare_branches.py did it while starting up whether or not the
run would build a bootloader.

Make it a property filled in on first use.  All three places which
consult it already do so only for the bootloader vehicle, so a run
which does not build one now never parses a hwdef for this.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:24:04 +10:00
Peter BarkerandClaude Opus 5 b769156f58 Tools: build_script_base: take USE_BOOTLOADER_FROM_BOARD from the hwdef
A board which takes another board's bootloader says so in its own
hwdef, so there is no need to name it in the blacklist as well; 23 of
the 42 entries were doing that, and each new such board had to be
added here by hand.

Read the directive with board_list's hwdef_for_board() rather than by
matching the hwdef text, so that undefs, overrides and includes are
handled the way the build handles them.  This became affordable when
hwdef processing got faster: parsing all 443 ChibiOS hwdefs takes
0.33s, against the minutes or hours of building which follow.

Note that this deliberately does not treat a missing hwdef-bl.dat as
meaning a board has no bootloader.  A board which needs one but is
missing one must fail, not be silently skipped, so those boards stay
listed explicitly.

The returned set is unchanged: 89 boards, the same 89 as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:24:04 +10:00
Peter BarkerandClaude Opus 5 570d393f56 Tools: build_script_base: do not build bootloaders for QURT boards
The loop which blacklists boards by HAL predates the addition of the
QURT HAL, so ModalAI-VOXL2 and ModalAI-VOXL3 were neither excluded by
HAL nor named in the explicit blacklist.  A bootloader build task was
therefore generated for them, which fails; QURT has no bootloader and
no hwdef-bl.dat.

No QURT board was named in the explicit blacklist, so nothing is
removed from there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:24:04 +10:00
Peter BarkerandClaude Opus 5 f317336863 Tools: build_script_base: drop redundant Linux boards from bootloader blacklist
canzero and t3-gem-o1 are AP_HAL_Linux boards, not ChibiOS boards, so
they are already added to the bootloader blacklist by the loop which
excludes every board whose hal is Linux, ESP32 or SITL.

No functional change; the returned set is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 19:24:04 +10:00
VinayakPundhir d529404195 Tools: Add Aeron Systems PLX3 INS to build_options
pre-commit / ci (push) Canceled after 0s
test scripts / build (astyle-cleanliness) (push) Canceled after 0s
test scripts / build (check_autotest_options) (push) Canceled after 0s
test scripts / build (logger_metadata) (push) Canceled after 0s
test scripts / build (param-file-validation) (push) Canceled after 0s
test scripts / build (param_parse) (push) Canceled after 0s
test scripts / build (python-cleanliness) (push) Canceled after 0s
test scripts / build (shellcheck) (push) Canceled after 0s
test scripts / build (validate_board_list) (push) Canceled after 0s
2026-08-04 16:08:14 +10:00
muramura 501077eac5 Tools: make AltHold an optional build feature 2026-08-03 08:12:37 +10:00
Iampete1 941ba1b944 Tools: Scripts: decode_devid: add missing WSPI bus type 2026-07-30 23:50:38 +10:00