Commit Graph
1753 Commits
Author SHA1 Message Date
Henry WurzburgandClaude Sonnet 5 d6d783a9cf Tools: prefer hwdef over HAL name when splitting/relabelling commits
libraries/<HAL>/hwdef/... changes were being tagged with the HAL library
name (e.g. AP_HAL_ChibiOS) since it was listed before hwdef as a
candidate subsystem, and an already-declared HAL name was treated as
valid and left alone. Make hwdef -- the deepest directory in the path --
the canonical tag in both cases, since it applies uniformly across
every HAL rather than being tied to one library.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-29 16:45:27 +10:00
Randy Mackay 657d2bf006 Tools: allowed_subsystems.py fix to support AC_PosControl 2026-07-28 06:47:45 +09:00
Peter BarkerandClaude Fable 5 1f6e646d2a Tools: add --progress-file option to build_boards and size_compare_branches
pre-commit / ci (push) Canceled after 0s
test scripts / build (astyle-cleanliness) (push) Canceled after 0s
test scripts / build (check_autotest_options) (push) Canceled after 0s
test scripts / build (logger_metadata) (push) Canceled after 0s
test scripts / build (param-file-validation) (push) Canceled after 0s
test scripts / build (param_parse) (push) Canceled after 0s
test scripts / build (python-cleanliness) (push) Canceled after 0s
test scripts / build (shellcheck) (push) Canceled after 0s
test scripts / build (validate_board_list) (push) Canceled after 0s
size_compare_branches wrote its progress CSV to the hardcoded path
/tmp/some.csv; make the destination a command-line option, and write
no progress file at all when the option is not given.  The write
helper lives in the shared base class, and build_boards gains the
same option, writing task counts, remaining tasks and failures as
builds complete (in both parallel and sequential modes).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 11:41:31 +10:00
Peter Barker 483386fdfd Tools: add CI check that changed source files keep their trailing newline
Some editors strip the final newline by default, which shows up as
"No newline at end of file" noise in GitHub review.  Fail the branch
conventions check when a PR adds a source file without a trailing
newline or removes one from a file which had it.  Pre-existing
violations do not block unrelated edits.
2026-07-21 08:32:28 +10:00
Bob Long f762235f6c Tools: inline annotations for location ratchet
pre-commit / ci (push) Canceled after 0s
test scripts / build (astyle-cleanliness) (push) Canceled after 0s
test scripts / build (check_autotest_options) (push) Canceled after 0s
test scripts / build (logger_metadata) (push) Canceled after 0s
test scripts / build (param-file-validation) (push) Canceled after 0s
test scripts / build (param_parse) (push) Canceled after 0s
test scripts / build (python-cleanliness) (push) Canceled after 0s
test scripts / build (shellcheck) (push) Canceled after 0s
test scripts / build (validate_board_list) (push) Canceled after 0s
2026-07-18 20:17:19 +10:00
Bob Long 97ef31cc0f Tools: add mavutil.location ratchet script
This ensures no PR adds new references to the frameless mavutil.location
while we work to phase it out of autotest completely.
2026-07-17 17:45:04 +10:00
Brown 4176fc33ae AP_InertialSensor: add LSM6DSV32X and LSM6DSK320X support
Detect LSM6DSV32X separately from LSM6DSV16X using the CTRL8 variant bit,
as both parts share the same WHO_AM_I value.

Detect LSM6DSK320X using its WHO_AM_I value, register both new variants
with dedicated device types, and use the correct accel/gyro full-scale
register encodings for each variant.

Update decode_devid.py so INS device IDs decode to the new variant names.
2026-07-15 17:46:10 +10:00
Peter BarkerandClaude Opus 4.8 4be67e3e25 Tools: enforce one allowed subsystem per commit in CI
Add check_commit_subsystems() to check_branch_conventions.py, which uses
the allowed_subsystems library to verify, for every commit on the branch,
that:

  - the declared subsystem prefix is an allowed subsystem (a libraries/
    directory, a directory the commit creates, or a curated extra), and
  - every file changed in the commit belongs to that subsystem.

When a file does not belong to the declared subsystem the error lists the
subsystem(s) it could be part of; when the prefix itself is unknown the
error suggests the closest allowed name ("did you mean ...?").  Revert and
fixup! commits are skipped (handled by the existing checks).  This runs in
the existing test_branch_conventions.yml workflow; no CI configuration
change is required.

The generic git helpers get_added_paths_for_commit() and
created_library_dirs() live on BuildScriptBase so other build scripts can
reuse them.  Register allowed_subsystems as a known-first-party module in
pyproject.toml so the new import sorts correctly under ruff/isort.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 08:08:08 +10:00
Peter BarkerandClaude Opus 4.8 f490ad3400 Tools: add allowed_subsystems definitive subsystem list
Add Tools/scripts/allowed_subsystems.py, the single source of truth for
which subsystem prefixes are permitted in commit messages and which
subsystem(s) a changed file may belong to.

A subsystem is allowed if it is a directory under libraries/ (computed
dynamically), a libraries/ directory created by the commit itself, or one
of the curated non-library extras (vehicles, Tools, autotest, waf, hwdef,
modules, a few common class prefixes, ...).  subsystems_for_path() returns
an ordered list of candidate subsystems for a path, since some files are
conventionally committed under more than one name (an autotest under
autotest or Tools; a ChibiOS hwdef file under hwdef or AP_HAL_ChibiOS).
suggest_subsystem() offers the closest allowed name for a mistyped prefix.

The functionality is exposed as an AllowedSubsystems object; running the
file directly prints every valid subsystem name.  It is consumed by
check_branch_conventions.py, the git-subsystems-split tool and the sample
pre-commit hook.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-15 08:08:08 +10:00
George Zogopoulos 549752196e Tools: Register AC_COPTER_CUSTOMCONTROL_ENABLE as a build feature 2026-07-10 10:14:26 +10:00
Pierre Kancir 89846c8fe5 Tools: pretty_diff_size: do use .hex
comes from the fact we now have intelhex in the container
2026-07-07 11:28:44 +10:00
Peter BarkerandClaude Opus 4.8 5152cde404 Tools: check SerialProtocol default-protocol gate coverage
serial_protocol_compiled_in() in
AP_SerialManager_default_protocol_check.cpp gates each serial port default
protocol against the feature that provides it, but falls through to "true"
for any protocol it does not name -- so a protocol added to the enum without
a term there is silently treated as always-compiled-in, defeating the check
for that protocol.  Nothing in the C++ can notice that omission.

Add check_serial_protocol_gates.py, which parses the SerialProtocol
enumerators from AP_SerialManager.h and the protocols named by
serial_protocol_compiled_in(), and requires every enumerator to be either
gated there or listed in an explicit UNGATED register (with the reason).  A
new protocol in neither, or a stale name in the C++/register, is an error.

The register also makes visible the protocols that have a feature but are not
yet gated (GPS, Rangefinder, AirSpeed, AHRS, ...); gating those is a
behaviour-neutral follow-up.

Run it from pre-commit when the header, the check, or the script changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 14:03:37 +10:00
Peter BarkerandClaude Opus 4.8 6afff3799c Tools: enable serial default-protocol checks for new boards
The serial default-protocol checks are off by default.  test_new_boards.py
enables them (via --extra-hwdef) when building newly-added boards, so a board
cannot pin a serial port default to a protocol it does not compile in.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 14:03:37 +10:00
Peter Barker 13a59be5db Tools: make it clearer that commits are faulty rather than PR title 2026-07-02 19:11:11 +10:00
Peter Barker 827c11a600 autotest: create a new build_boards.py
shares most code with size_compare_branches.py - just configures/builds boards based on board_list
2026-07-01 21:47:13 +10:00
Willian Galvani e19be12027 Tools: TMP119 sensor support 2026-06-30 09:39:04 +10:00
Pierre Kancir 04979f4351 Tools: cygwin_build: don't -j8 on GH 2026-06-26 10:14:22 +10:00
Pierre Kancir 8c9f40e48a .github: fix ccache reuse between SITL build and autotest jobs 2026-06-24 09:51:35 +10:00
Peter BarkerandClaude Opus 4.8 78c56ef55d Tools: require committed bootloader binary for new bootloader hwdef
test_new_boards.py now checks that any branch adding a new hwdef-bl.dat
also commits the matching prebuilt bootloader binary at
Tools/bootloaders/<board>_bl.bin, mirroring the build-time requirement.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 09:51:03 +10:00
Peter Barker 25dd465429 Tools: consolidate new board checks into test_new_boards.py
pre-commit / ci (push) Has been cancelled
test scripts / build (astyle-cleanliness) (push) Has been cancelled
test scripts / build (check_autotest_options) (push) Has been cancelled
test scripts / build (logger_metadata) (push) Has been cancelled
test scripts / build (param-file-validation) (push) Has been cancelled
test scripts / build (param_parse) (push) Has been cancelled
test scripts / build (python-cleanliness) (push) Has been cancelled
test scripts / build (shellcheck) (push) Has been cancelled
test scripts / build (validate_board_list) (push) Has been cancelled
some checks went into check_branch_conventions.py instead.

Adds a comment so this doesn't happen again
2026-06-22 22:19:05 +10:00
Pierre Kancir f969b6f980 Tools: build_tests: add json output to pretty_diff_size and table from json gathering 2026-06-21 09:29:40 +10:00
Andrii Anoshyn 70b124bbf1 Tools: adjust hwdef metadata checks 2026-06-19 13:00:30 +10:00
Pierre Kancir c9875dc02e Tools: fix generate_lua_docs.sh output path
mkdir -p lualogs to ensure it exist
explicitly use doc_out_path for output, using only logpath seems broken
2026-06-17 14:36:22 +10:00
Peter Barker acd41e4b27 Tools: make sure board IDs are allocated in correct range 2026-06-16 18:06:29 +10:00
Peter Barker c56c10694a Tools: add board_list.BoardList.hwdef_for_board 2026-06-16 08:46:48 +10:00
Pierre Kancir c566a9cec5 Tools: remove all modules from coverage 2026-06-16 08:40:40 +10:00
Peter Barker 740cbb712b Tools: build_options.py: RPM_DRONECAN does not need GENERATOR!
pre-commit / ci (push) Has been cancelled
test scripts / build (astyle-cleanliness) (push) Has been cancelled
test scripts / build (check_autotest_options) (push) Has been cancelled
test scripts / build (logger_metadata) (push) Has been cancelled
test scripts / build (param-file-validation) (push) Has been cancelled
test scripts / build (param_parse) (push) Has been cancelled
test scripts / build (python-cleanliness) (push) Has been cancelled
test scripts / build (shellcheck) (push) Has been cancelled
test scripts / build (validate_board_list) (push) Has been cancelled
2026-06-13 17:30:19 +10:00
Peter BarkerandClaude Opus 4.8 97ec3af9d6 Tools: add build_binaries history HTML site generator
Generate a static, fully-offline HTML site from a build_binaries history
SQLite database (as produced by build_binaries_history.py). Intended to be
run periodically; each run regenerates the whole tree. Pages are
self-contained (chart data inlined, uPlot vendored locally), so the output
opens directly from local disk with no database or network access.

Pages:
- Coverage (the landing page): board x vehicle status at the most recent
  COMPLETED latest-tag run, split into boards with/without failures.
- Build trends: per-board grid, one box per run (newest left) linking to its
  commit, recently-failing boards first; a "completed" row marks finished vs
  partial runs; an optional checkbox interleaves the git commits we did not
  build (rendered on demand).
- Flash free: per vehicle/board program-flash headroom (region computed via
  the ChibiOS HWDef system), tightest first, low-headroom flagged.
- Runs: run duration, boards per run, builds per run, and commits since the
  previous run.
- Overview dashboard, plus per-board detail pages (flash-free and build-
  duration history and notable size changes). Chart points link to the commit.

Lives in its own subdirectory with a README onboarding guide (architecture,
data flow, the inlined-data contract, how to extend, and dev gotchas).

Written with AI assistance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 11:36:18 +10:00
Peter Barker 3616a1822f Tools: add failure injection to build_options.py 2026-06-05 20:32:19 +10:00
Peter Barker 0332990510 Tools: signing/README.md, add a note on accepting commands when no pukeys 2026-06-05 19:43:33 +10:00
Javid e4a718d635 Tools: add HAL_WITH_ESC_TELEM build option
so esc telem can be turned off from the build server, and add the
symbol to extract_features so it shows in feature lists.
2026-06-03 19:09:17 +10:00
Peter Barker a3848abe29 Tools: extract_features: extract WPNEXT features 2026-06-03 09:54:28 +10:00
Peter Barker d4eda2d71f autotest: add test for MAV_CMD_DO_SET_ROI_WPNEXT_OFFSET 2026-06-03 09:54:28 +10:00
Peter Barker a4147441a5 Tools: add LIS2MDL back in with a comment that it generally wasn't used 2026-06-01 17:05:32 +10:00
Jacob Dahl 6563ed1d2f Tools: remove LIS2MDL driver, use IIS2MDC
LIS2MDL and IIS2MDC are the same ST silicon with identical register
maps, WHO_AM_I, I2C address and sensitivity. Remove the separate
LIS2MDL driver entirely and switch boards that referenced it to IIS2MDC.

AIRBRAINH743 COMPASS line changed from LIS2MDL to IIS2MDC.
2026-06-01 17:05:32 +10:00
Pierre Kancir 8d225aa7e7 Tools: build_script_base: make BuildScriptBase base class and progress_prefix abstract
this prevent linter to trigger on none existing progress_prefix on the base class
Child class will now have TypeError if they don't implement  progress_prefix
2026-05-27 08:50:02 +10:00
Pierre Kancir da8bd63a35 Tools: build_script_base: fix call error return 2026-05-27 08:50:02 +10:00
Peter Barker dea45a24e5 autotest: disallow BRD_TYPE in defaults.parm for new/modified boards
should always be specified in hwdef as BOARD_TYPE_DEFAULT
2026-05-26 22:31:26 +10:00
Peter Barker 8f5f690771 Tools: ensure new boards have both a README.md and an image 2026-05-23 09:55:03 +10:00
Andrew TridgellandClaude Opus 4.7 fce9883d2f AP_Networking: add SITL TAP backend for host-side reachability
New AP_Networking_SITL_TUN backend that opens a Linux TAP device and
bridges the lwIP stack to it. Slots into the same "ethernet" position
in PPP_ETHERNET_GATEWAY mode that the ChibiOS backend takes on real
hardware, so a SITL build acting as a PPPGW can be reached by developer
tools on the host (browsers, curl, ...) at the periph's PPP-side IP.

A new Tools/scripts/Networking/sitl_network.sh handles the one-time
host setup: it creates the TAP device under the current user (so the
SITL binary needs no extra privileges), assigns the host's IP, and
installs iptables MASQUERADE + FORWARD rules so the SITL subnet can
reach the wider host network / internet via NAT. The matching `down`
action undoes exactly what `up` put in place, using a small state file
in /tmp to remember the outbound interface and the previous
net.ipv4.ip_forward setting. Tools/scripts/Networking/README.md
documents the usage and customisation knobs.

    Tools/scripts/Networking/sitl_network.sh up     # create TAP + NAT
    sim_vehicle.py -v Plane -f quadplane-PPP
    curl http://10.77.193.20/                       # PPPGW web UI
    Tools/scripts/Networking/sitl_network.sh down

The backend's init falls back to a "PPPGW-only, no host bridge" mode
if the TAP device is not pre-created (TUNSETIFF returns EPERM): it
seeds activeSettings from the user-configured IP params and prints a
one-line warning. That way the same hwdef works in autotest (which
runs without root and without sitl_network.sh) and in developer
sessions (which run it).

Gated by AP_NETWORKING_BACKEND_SITL_TUN (default 0); turned on in a
per-board hwdef when wanted. AP_NETWORKING_NEED_LWIP is widened to
include this backend so the lwIP sources get pulled into the build.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 17:08:34 +10:00
Andrew TridgellandClaude Opus 4.7 3ad3bb46ff AP_HAL_SITL: add sitl_periph_PPP PPPGW build target
New SITL AP_Periph board modelled on the ChibiOS PPPGW boards (eg
Pixhawk6X-PPPGW), with the PPP networking backend, scripting, and a
ROMFS-embedded copy of the pppgw_webui.lua web interface from
Tools/AP_Periph/Web. Intended for end-to-end PPP testing between two
ArduPilot SITL processes and for exercising the PPPGW web server in
SITL.

The hwdef:
 - enables AP_PERIPH_NETWORKING_ENABLED, AP_NETWORKING_BACKEND_PPP and
   AP_NETWORKING_CAN_MCAST_ENABLED;
 - points NET_PPP_PORT at serial 1 so the periph can be wired to the
   plane SITL via --serial1=tcpclient:127.0.0.1:<port>;
 - enables AP_SCRIPTING + AP_FILESYSTEM_ROMFS + AP_PARAM_DYNAMIC, and
   embeds Tools/AP_Periph/Web/ via ROMFS_DIRECTORY;
 - disables Lua bindings whose backing libraries AP_Periph does not link
   (AP_Vehicle/AC_Fence/AP_Button/AP_Camera/AP_Mount/AP_OpticalFlow/
   AP_VisualOdom/AP_Frsky_Telem/AP_Arming/RC_Channel). ChibiOS periph
   builds handle this via defaults_periph.h; SITL periph has no
   equivalent so we list the gates inline.

ROMFS support for SITL hwdefs (the ROMFS / ROMFS_WILDCARD /
ROMFS_DIRECTORY keywords) is added to sitl_hwdef.py, modelled on the
ChibiOS hwdef parser. Files are published via env_vars['ROMFS_FILES']
and picked up by the existing waf embed_ROMFS_files step in boards.py.

A new sitl_periph_PPP-specific defaults file
(Tools/autotest/default_params/periph-ppp.parm) holds runtime overrides
that should always apply to this build: NET_PPP_BAUD=921600 (saner
than the 12.5Mbaud real-hardware default for the loopback-TCP
transport), and SCR_ENABLE=1 so the web interface script comes up at
boot.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 17:08:34 +10:00
Andy Piper d9b30d1010 scripts: add build option for EKF bootstrap reset
Add AP_AHRS_EKF_RESET_ENABLED to the custom build server so the
EKF bootstrap reset aux function can be compiled out on
flash-constrained boards. Enabled by default, depends on EKF3.
2026-05-13 18:04:37 +01:00
Peter Barker e69c03557c Tools: do not work with AP_PERIPH_ defines on non-periph boards 2026-05-06 17:48:58 +10:00
Peter Barker bbed8c1333 Tools: allow support for MAV_CMD_GET_HOME_POSITION to be compiled out
superseded by MAV_CMD_REQUEST_MESSAGE
2026-05-06 17:47:25 +10:00
Peter Barker 9c142a4c18 Tools: remove rallypoint/fencepoint protocols 2026-04-30 19:52:50 +10:00
Peter Barker c9c9ff2984 Tools: test_new_boards.py: prevent bad lines in defaults.parm
where we have the ability to set a default using a define in the hwdef we should.
2026-04-28 07:47:26 +10:00
cuav-chen2 84f74fc0d7 Tools: Scripts: Decode devid: add ADIS16607 2026-04-27 10:24:51 +10:00
Peter Barker fbc82ee347 Tools: convert to hwdef system 2026-04-21 10:06:15 +10:00
Peter Barker df9dd14608 Tools: understand markdown code guards when looking for underlines
this was marking code included within blocks (```) as having heading underlines.  Which we shouldn't do inside fenced code blocks.
2026-04-20 19:25:51 +10:00
Peter BarkerandClaude Sonnet 4.6 fe2e31fe30 Tools: extend RST markup check to catch :ref: cross-references
Widen check_markdown_rst_hyperlinks to also flag RST cross-references
(:[a-z]+:`...`) alongside the existing RST external hyperlink check.
Rename the pass message to "RST-style markup" to reflect the broader
scope.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 21:29:44 +10:00