543 Commits

Include renames

Author	SHA1	Message	Date
Ramon Roche	ec2687abc3	ci(usb-ids): check board USB IDs on PRs that change them ... The workflow triggers on pull requests that touch a board defconfig or the USB ID tooling, and checks only the defconfigs the PR adds or modifies. The file list comes from diffing the checked-out PR merge commit against its base parent, so no API call or token is needed. A PR that changes the tooling itself is checked against every board, so a checker change is proven on the real tree and not only by its unit tests. A test keeps the tooling list and the workflow paths filter in sync. A manual run checks every board. There is no push or scheduled run: the registry repo checks its own changes against PX4 main, and board changes reach main only through PRs. The mypy and flake8 step for the checker and runner lives in python_checks.yml with the other Python tooling checks. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-28 14:44:48 -07:00
Ramon Roche	14cf29bb31	fix(build): check submodules once per configure, never reset changes ... Every px4_add_git_submodule call ran check_submodules.sh twice, at configure and build time, and with CI=true force-updated each submodule every time, a 2017 workaround for an interactive prompt that is gone. VS Code and CLion force-updated too, resetting submodule changes under development. The check now runs once per configure. A missing submodule is fetched; one at another commit is kept as it is, with a warning locally and an error in CI. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-28 11:50:38 -07:00
Ramon Roche	9a17deda06	ci(clang-tidy): prepare the clang build dirs in sequence ... `make -j clang-ci` configured px4_sitl_default-clang and px4_sitl_default-clang-test at the same time, and both configures fetch the same submodules. About 1 in 200 Static Analysis runs failed with `index.lock: File exists` or a half-cloned libevents. The workflow now runs the configure-only test dir first, which fetches the submodules, then builds the clang dir, which finds them present. clang-ci had no other user, so it is removed. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-28 10:55:25 -07:00
Ramon Roche	89c68cd37f	ci(build-all): build two targets per job concurrently ... Each build job compiled its group one target at a time, and the serial parts of every build (configure, the NuttX make, linking) left the 4-vCPU runners about half idle. Two concurrent builds keep them around 80% busy and cut total build-step time about 24% and runner cost about 17%. Larger runners and more slots were measured and cost more than they save. build_all_runner.py replaces build_all_runner.sh. NuttX compiles inside its source tree, so each extra slot is a git worktree under .build_slots/. All submodules are fetched once before building, each slot clones them locally from the checkout, and builds run with GIT_SUBMODULES_ARE_EVIL=1 so CMake's per-configure submodule check does not sync and update them concurrently on the shared .git/config. Targets that write the same build/ directory (<board>_deb with <board>_default, the metadata targets with px4_sitl_default) always share a slot, and the runner refuses a plan that would split them. Every target is built even after a failure. Build output streams with a [seconds|slot target] prefix, each target prints its memory usage or, on failure, the excerpt from the first error, and a Build Logs step prints each target's full log as one collapsed group. Slot build directories are moved into build/, so packaging is unchanged. The matrix now carries targets as a list plus a slot count from build_all_config.yml; the comma-joined string existed only because workflow expressions could not pass an array to a shell command, which join() now does. The scan job runs new tests for the generated matrix and the slot assignment. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-28 09:45:09 -07:00
Ramon Roche	939b420488	ci(codecov): upload unit test coverage (#28869) ... * ci(codecov): upload unit test coverage and limit SITL coverage to main Coverage on main came from a single standard_vtol SITL flight, unit tests were never uploaded, and the Coverage build never hit ccache because ccache 4.9.1 (Ubuntu 24.04) rejects -fprofile-update=atomic. The flag only keeps hit counts exact under thread races; line coverage is unaffected and lcov already ignores the resulting negative counts. Unit tests now build with coverage and upload as the unittests flag on every PR and push to main. standard_vtol builds as Coverage on main only and uploads as sitl-vtol, carried forward on PRs. codecov.yml sets the default branch, informational project/patch statuses and ignores. Per-build-type ccache keys stop the Coverage and RelWithDebInfo jobs from overwriting each other's cache. Fixes #28862 Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com> * ci(codecov): run unit test coverage as its own job The Coverage build shifts EKF replay float output, so running it in the Unit Tests job made the change indication step auto-commit new baselines to the PR branch. Unit Tests goes back to the plain build and a separate job produces the unittests upload. disable_search keeps the Codecov uploader to the lcov report instead of running gcov over the build tree. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com> --------- Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-26 19:11:53 -07:00
Ramon Roche	bf9b1610ca	fix(ros2): pin BuildKit for larger dev container SBOMs ... Publishing px4-dev-ros2-gazebo fails because its SPDX attestation exceeds BuildKit 0.32's 40 MiB limit. Pin the builder to v0.33.0 (80 MiB), matching the fix already applied to build_deb_package.yml in 61907962c9. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-26 18:51:08 -07:00
Ramon Roche	786893436c	ci(macos): save ccache only on main ... The macOS build runs on GitHub-hosted runners, so its ccache lives in the repository's 10 GB Actions cache. Every run saved a new entry, about 300 a week and 80% from pull requests, which kept the cache full and evicting. Pull requests already restore from main's newest entry, so only main needs to save. Pull requests still print the final ccache stats. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-26 18:13:56 -07:00
Ramon Roche	5ecc673e23	ci(macos): drop the Homebrew download cache and double ccache ... The Homebrew download cache restored from any older key and saved the result under the new one, so it only ever grew: each entry is now about 2.9 GB, three of them fill most of the 10 GB repository cache, and GitHub evicts the ccache entries to make room. Restoring and saving it cost about 72 s per run, more than the Homebrew install it saves. The 200M ccache is full after px4_sitl, so px4_fmu-v5_default only hits about 81%. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-26 15:34:46 -07:00
Ramon Roche	a8b0e91ebe	feat(setup): install macOS Gazebo from a locked conda-forge environment ... Homebrew rebuilds and drops the gz bottles whenever homebrew-core bumps protobuf or abseil, which breaks the macOS Gazebo install until the pins catch up. conda-forge keeps every package it has published, so Tools/setup/macos/pixi.toml and its lock install the same gz, protobuf, abseil, OpenCV and GStreamer binaries until the lock is updated on purpose. OpenCV is the headless build, and GStreamer brings the good, bad and ugly plugin sets the gz camera plugin's pipeline uses. macos.sh installs the environment with pixi and loads it from the venv activation through a small script, since pixi shell-hook would export the installing shell's PATH. The build and the gz launch do not depend on that activation: CMake searches the environment like it pins the .venv Python, and px4-rc.gzsim sources gz_env.sh before it looks for gz, which loads the environment when it is not already active and puts its lib dir ahead of Homebrew's in the dyld fallback path. conda-forge's gz-transport13 does not pull in cppzmq, so the manifest lists it. The osrf/simulation pin, the protobuf pin and the weekly bot that refreshed them are removed. The homebrew-core commit moves into homebrew-pins.txt unchanged. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-26 15:12:00 -07:00
Ramon Roche	a4b722d400	ci(packaging): seed the container build caches from main (#28859) ... * ci(packaging): seed the container build caches from main The workflow only ran on pull requests, tags and manual dispatch, so nothing ever saved a cache under main. Pull requests can only read their own ref and main, so every first run on a PR built all images cold (30 min vs 13 min warm). Running on main and weekly gives PRs a cache to fall back to; the weekly run also keeps it inside the runner cache's 10-day expiry. The ROS ccache fallback also skipped the image name, so sih and gazebo could restore each other's cache. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com> * ci(packaging): clarify why main is rebuilt weekly Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com> --------- Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-26 14:27:05 -07:00
Ramon Roche	55a577ffd0	ci(docs): skip site artifact upload on pull requests ... Only T4: Deploy reads px4_docs_build, and it runs on push and workflow_dispatch. On pull requests the upload spent ~93 s and ~1 GB of artifact storage per run for nothing. The site is still built, so the check keeps failing on broken docs. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-26 13:43:41 -07:00
Jacob Dahl	7c8d6daca9	docs(agents): move the shared driver code rule to drivers.instructions.md (#28853) ... It only matters when working under src/drivers/, which the scoped file already covers, and AGENTS.md is loaded for every task. The section was also not wrapped like the rest of AGENTS.md. Assisted-by: Claude:claude-opus-5-5	2026-09-25 23:41:55 -06:00
Andrea Bernasconi	b38dcb1f6c	fix(navigator): use "Return" instead of "RTL" in user-facing text (#28543)	2026-09-26 12:40:56 +10:00
Jacob Dahl	4e91ae1c42	ci(flash_analysis): rename size table columns to Δ and Total (#28844) ... Shorter headers keep the five-column table narrow, and "used" read as the amount used by the change rather than the target's total. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>	2026-09-25 15:59:06 -06:00
Jacob Dahl	253479bbeb	feat(ci): report flash and RAM use against capacity, analyze fmu-v2 (#28838) ... * feat(ci): report flash and RAM use against capacity, analyze fmu-v2 The size comment showed only the delta, which does not say how close a target is to its limit. Capacity comes from the board's own linker script, because the flash-analysis build links against an oversized flash region so that a change which overflows still gets a report. fmu-v2 adds an F4 next to the F7 and H7 targets, and with 1 MB of flash it is the FMU that overflows first. Assisted-by: Claude:claude-opus-5-5 * fix(ci): shorten the size comment's usage columns Five columns with capacities spelled out made the table too wide to read at a glance. Assisted-by: Claude:claude-opus-5-5	2026-09-24 17:54:56 -06:00
Jacob Dahl	2ec03ff1c3	fix(ci): skip the flash comment for sub-30 B deltas and agent-doc changes (#28806) ... The commit hash compiled into px4_firmware_version_binary() changes its code size, so a PR with no code change reported -8 B on v6x and got a comment. Changes that only touch agent instructions also no longer build the targets. Assisted-by: Claude:claude-opus-5-5 Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>	2026-09-22 17:52:55 -06:00
Jacob Dahl	1d761cdf2e	fix(ci): report flash and static RAM usage separately (#28789) ... * fix(ci): report flash and static RAM usage separately Bloaty's VM total includes zero-initialized RAM and understates the combined cost of data and code copied from flash into RAM. Report the flash load image and static RAM allocations independently so the PR headline reflects each resource's actual change. Assisted-by: Codex * docs(ci): trim memory analysis comment Assisted-by: Codex * fix(ci): diff PR flash usage against the merge commit's base The base branch tip is fetched when the job runs, so a PR merged while its job was queued was compared against itself and reported a near-zero change, and commits landed in the meantime showed up as reverted. * fix(ci): measure flash image from section headers ld maps the ELF header into the first LOAD segment when its max page size exceeds the flash origin's alignment (binutils 2.34 defaults to 64 KiB), so segment-based accounting rejected valid firmware as outside the flash region. * feat(ci): flag notable flash and RAM changes Growth over 100 B and 1000 B gets yellow and red markers, savings over 100 B green, so reviewers can spot size regressions without reading the numbers. * refactor(ci): derive flash and RAM totals without per-target constants The matrix duplicated each board's flash-analysis linker region, which would drift silently. Section headers alone distinguish code executing in place (VMA == LMA) from data copied to or reserved in RAM, and the image span matches objcopy -O binary. * fix(ci): limit the bloaty breakdown to VM sizes Debug, symbol and string table rows only change the file size and made up most of the comment. * fix(ci): post flash analysis only when size changes A push that puts flash and static RAM back to zero was still rewriting the sticky comment, so a PR with no size change carried a zero report. Remove that comment instead. Fork tokens cannot delete it, so those PRs hand a delete artifact to the poster. Assisted-by: Grok:grok-4.7 Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com> * fix(ci): retain zero-delta flash analysis comments Existing reports can be updated when size deltas return to zero, avoiding a separate privileged deletion path. Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com> Assisted-by: Codex:gpt-6 --------- Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>	2026-09-22 16:56:58 -06:00
Jacob Dahl	fbfc1386ee	ci(macos): install from pinned Homebrew commits ... The job ran brew update and then installed whatever Homebrew was publishing, except Gazebo, which stayed on one commit. Those were built against different protobufs, so a release upstream failed the build with no commit here. Check the package repos out at commits recorded in the tree and do not update them. Run on macos-15 only, so the OS label cannot move either. Assisted-by: Grok:grok-4.7 Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>	2026-09-21 16:55:31 -07:00
Jacob Dahl	49339b843f	ci(macos): remove the MacOS workflows ... The build runs brew update and then pours Gazebo from one pinned tap commit while the rest of the install, including protobuf, comes from current Homebrew. A release in either repo fails the check with no commit here, and the GitHub-hosted image moves on its own, so the job cannot be a closed set. Drop the build and the weekly pin refresh. The setup script, the pins, and the refresh script stay for local use. Assisted-by: Grok:grok-4.7 Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>	2026-09-21 16:55:31 -07:00
Jacob Dahl	db99f32cf2	ci(macos): disable the MacOS build check ... make px4_sitl fails when Homebrew's protobuf is not the one the pinned Gazebo bottles were generated against. Those generated headers are a fatal error against any other version, and that has been failing these legs since 2026-09-02. Take the check off push and pull requests. workflow_dispatch still runs it. Assisted-by: Grok:grok-4.7 Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>	2026-09-21 16:55:31 -07:00
Julian Oes	1d7b23e198	build(macos): pin protobuf to what the gz bottles need (#28774) ... Every gz .pb.h refuses to compile against anything but the exact protobuf its gencode came from, so the bottles gz-tap-pin.txt holds osrf/simulation at only build against the protobuf homebrew-core shipped the day OSRF built them. homebrew-core moves protobuf on its own schedule and OSRF rebuilds days later, so `make px4_sitl` has been failing on macOS on every branch since homebrew-core shipped protobuf 36.2. Pin protobuf as well, to the homebrew-core revision that installs the version the pinned gz bottles carry, and derive that pin from the bottles themselves so the two can never disagree. A protobuf release then becomes a no-op here instead of days of red CI while OSRF catches up. The pinned install has to run after the other simulation packages and with the dependents check off: opencv@4 depends on protobuf too, brew resolves a dependency against the versions recorded in the dependent's bottle, and it reinstalls dependents whose linkage it has just broken. Pinning the formula with `brew pin` is not an option either, brew then refuses to install anything that depends on it. Assisted-by: Claude:claude-opus-5 Signed-off-by: Julian Oes <julian@oes.ch>	2026-09-22 10:15:13 +12:00
Beniamino Pozzan	403b957e77	ci(ros_dev): add ros2-gazebo-dev target to ros_dev_container.yml ... Signed-off-by: Beniamino Pozzan <beniamino.pozzan@gmail.com>	2026-09-21 11:18:58 -07:00
Julian Oes	2fc441493e	ci(commit_checks): pass the PR title through the environment (#28772) ... The PR title was interpolated directly into the shell command, so bash parsed it as syntax before argparse ever saw it. A title containing double quotes split into several arguments and failed the check for the wrong reason, and a hostile title could run arbitrary commands on the runner. Pass it via the environment like the labeler workflow already does, so it is data rather than syntax. Also make sure a report exists when the check fails without writing one, otherwise the posting step dies on a missing comment.md and hides the actual error. Assisted-by: Claude:claude-opus-5 Signed-off-by: Julian Oes <julian@oes.ch>	2026-09-21 11:15:57 +12:00
Claudio ChiesandClaudio Chies	651b8687b8	feat(failure_injection): selectable detected/un-detected motor failure via SYS_FAIL_MOT_OFF (#28210) ... * feat(failure-injection): implement motor failure injection with soft and hard modes * feat(failure-injection): update motor failure injection descriptions and behaviors for annunciated and un-annunciated modes * feat(failure-injection): update motor failure injection terminology to detected and undetected modes * feat(failure-injection): refine motor failure injection handling for detected and undetected modes * fix(boards): remove FailureInjector::update() from itcm_functions FailureInjector was replaced by the shared failure_injection library, so the symbol no longer exists and the ITCM check fails on the boards that still list it. Its successor failure_injection::Config::update() is an inline no-op unless MODULES_FAILURE_INJECTION_MANAGER is enabled, which none of these boards do, so there is no replacement symbol to map. Assisted-by: Claude:claude-opus-5 Signed-off-by: Claudio-Chies <61051109+Claudio-Chies@users.noreply.github.com> --------- Signed-off-by: Claudio-Chies <61051109+Claudio-Chies@users.noreply.github.com> Co-authored-by: Claudio Chies <chiesc@chies.com>	2026-09-16 10:10:18 +02:00
Ramon Roche	d57d7f3b11	ci(ros): use PX4 development images for translation tests (#28683) ... * ci(ros): reuse the Jazzy toolchain for translation tests Use the same pinned PX4 development image as the ROS integration tests while preserving the Humble environment and both distro-specific caches. Assisted-by: Copilot:gpt-6-astra * ci(ros): pin published Humble and Jazzy development images Use the multi-architecture toolchains published from d8c01514ac for both translation-test distributions. Reuse bundled ccache while preserving separate caches, message-versioning checks and checkout-matched translation builds. Assisted-by: Copilot:gpt-6-astra	2026-09-11 18:49:48 -07:00
Ramon Roche	d8c01514ac	feat(ros2): add Humble development images and distro releases (#28684) ... * feat(ros2): add Humble development images and distro releases Build Humble on Jammy without replacing its ROS DDS libraries: Agent 2.4.3 links private, pinned Fast DDS/CDR dependencies. Publish Humble and Jazzy tags independently while leaving packaged runtime images Jazzy-only. Assisted-by: Copilot:gpt-6-astra * refactor(ros2): share pinned Agent DDS libraries across distros Build the Agent against the same private Fast DDS and Fast CDR pins on Humble and Jazzy, without replacing either ROS underlay. Remove distro-specific build branches and name the dependency manifest for its shared role. Assisted-by: Copilot:gpt-6-astra	2026-09-11 14:10:22 -07:00
Ramon Roche	269fba3e53	fix(ros2): publish the development image to Docker Hub ... Make the standalone ROS toolchain available from the same registries as the ROS runtime images. Preserve opt-in publication, existing GHCR tags and both architecture SBOM indexes. Assisted-by: Copilot	2026-09-10 18:28:12 -07:00
Ramon Roche	61907962c9	fix(packaging): support larger container SBOM attestations ... Gazebo ROS image publication exceeds BuildKit 0.32's 40 MiB attestation limit. Pin the builder daemon to 0.33.0 for its 80 MiB limit while preserving the existing scanner and complete package/file coverage. Assisted-by: Copilot:gpt-6-astra	2026-09-10 17:49:41 -07:00
Ramon Roche	b28ba0eedb	ci(ros2): test PR firmware in the ROS development container ... Build each PR firmware and matching ROS workspace from source against the digest-pinned Jazzy toolchain. Run the interface library unit and SIH integration suites without reusing stale firmware or generated messages. Assisted-by: Copilot:gpt-6-astra	2026-09-10 16:07:58 -07:00
Ramon Roche	f7052b281c	feat(ros2): add PX4 ROS development containers ... Provide a supported Jazzy environment for SIH/Gazebo development and source builds, with pinned tooling and reproducible multi-architecture image publishing. Separate package and container assets from ROS source preparation, and keep checkout workspaces fresh through a Python CLI. Assisted-by: Copilot:gpt-6-astra	2026-09-10 16:07:58 -07:00
Julian Oes	cc976e150a	feat(posix): separate the filesystem root from the storage directory (#28582) ... On NuttX these are two things: / is the filesystem root, and /fs/microsd is the SD card mounted under it. MAVLink FTP serves the root and confines writes to the SD card, so the read-only ROMFS at /etc is visible but cannot be written. On POSIX they were the same directory. PX4_ROOTFSDIR and PX4_STORAGEDIR both resolved to CONFIG_BOARD_ROOT_PATH, which on SITL is ".", so the working directory was simultaneously the FTP root and the only writable area. The consequences were that the ROMFS symlink sat inside the FTP root, and that _validatePathIsWritable() had nothing meaningful to check against and so was compiled out on POSIX entirely, leaving no write restriction at all. Give POSIX the same split. CONFIG_BOARD_ROOT_PATH keeps its meaning as the storage directory, and a new CONFIG_BOARD_FS_ROOT_PATH names the root FTP serves, defaulting to the storage path so every existing board is unchanged. SITL sets the root to "." and storage to "./fs", which mirrors NuttX: logs, parameters, dataman and eeprom move under ./fs, and etc/ stays in the root as read-only data. With storage distinct from the root, the write restriction now applies on every platform rather than NuttX only, and no longer compares against a hardcoded prefix length that was wrong for any board not using /fs/microsd. Assisted-by: Claude:claude-opus-5[1m] Signed-off-by: Julian Oes <julian@oes.ch>	2026-09-08 17:01:55 -06:00
Ramon Roche	6f93287fdd	ci(macos): rewrite gz tap pin refresh in Ruby under brew ruby ... The bash version spent most of its lines on plumbing: parsing a report back out of a per-commit `brew ruby` child, awk and herestrings for set logic, bash 3.2 workarounds, and a trap to restore the tap. The job itself is small and lives inside Homebrew anyway, so run the whole thing under Homebrew's Ruby and call its tap, formula and bottle APIs directly. One process instead of one Homebrew startup per candidate commit, the tap checkout restored by ensure, tarballs checked with Net::HTTP, and no second language embedded in a string. Formulary.clear_cache and Tap#clear_cache between checkouts keep each candidate's formulae fresh. GITHUB_OUTPUT still arrives because brew forwards GITHUB_* whenever CI is set. Same walk, same exemption for formulae unbottled at the pin, same outputs; verified to pick the same commit as the bash version. Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-04 13:26:30 -07:00
Ramon Roche	2d6329fab2	ci(macos): brew update before macos.sh so both legs see current core ... The macOS runner images set HOMEBREW_NO_AUTO_UPDATE in their bashrc and carry a homebrew-core snapshot from image build time. Today the macos-15 image (20260828) installs protobuf 36.0 while macos-26 (20260707) still installs 35.1, so the two legs of this workflow cannot both be compatible with any single gz tap pin. The pin is refreshed against tap HEAD, which OSRF keeps consistent with current homebrew-core, so update core before installing. Dev machines already do this through brew's auto-update. Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-04 13:26:30 -07:00
Ramon Roche	045f249929	ci(macos): move gz tap pin refresh into a script that checks bottles ... The weekly refresh workflow carried its logic inline, compared only the tap SHA against the pin, and duplicated the --sim-tools package list. `brew install --dry-run` never prints "from source", so its bottle check could not fire. Tools/ci/refresh_gz_tap_pin.sh now owns the job and the workflow calls it. It reads the osrf/simulation formulae from macos.sh, walks the tap from HEAD back to the pin, asks Homebrew at each commit whether every formula in the runtime closure has a bottle for the host, HEAD-requests each tarball, and rewrites the pin to the first commit that passes. Formulae with no bottle at the current pin (the gz-harmonic meta-formula, which OSRF never bottles) are not required. peter-evans/create-pull-request opens or updates the bot PR from the script's outputs, so an unmerged bump is refreshed instead of duplicated. The job runs on macos-15, the oldest macOS in compile_macos.yml, because Homebrew pours an older macOS bottle onto a newer one, not the reverse. Assisted-by: Claude:claude-fable-5-1 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-09-04 13:26:30 -07:00
FARHANG	d840da70b6	build(macos): pin gz tap unconditionally, add weekly refresh ... macos.sh only pinned osrf/simulation when --ci was passed, so dev machines still hit OSRF's bottle-gap window and built Gazebo from source. Apply it whenever --sim-tools runs. The pin is a point-in-time SHA with nothing to advance it, so add a weekly workflow that checks whether osrf/simulation HEAD is fully bottled and opens a PR bumping gz-tap-pin.txt when it is. Assisted-by: Claude:claude-sonnet-5 Signed-off-by: FARHANG <farhang.nba@gmail.com>	2026-09-04 13:26:30 -07:00
Jacob Dahl	9338031b11	fix(ci): attach FMU bootloader .bin to GitHub releases (#28495) ... * fix(ci): attach FMU bootloader .bin to GitHub releases Releases only shipped `_bootloader.px4`, the USB uploader envelope. SWD recovery needs the raw `.bin` at 0x08000000. Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com> * fix(ci): do not attach unused bootloader .px4 USB flashing would write it into the application slot. SWD uses the raw .bin. Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com> * fix(ci): omit bootloader SBOMs from GitHub releases GitHub Releases glob artifacts/**/*.sbom.spdx.json. Bootloader metadata dirs are not a recovery artifact. *_bootloader_* variants (e.g. bootloader_secureboot) stay omitted: those images are baked with in-tree test keys. Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com> * docs(bootloader): point SWD recovery at the release .bin Match the cannode pre-built blurb. The .px4 envelope is not the SWD image. Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com> * docs(bootloader): scope release .bin to SWD programmers gdb load needs ELF program headers. The release image is a raw .bin for ST-Link / CubeProgrammer / OpenOCD, and only in-tree *_bootloader targets are attached. Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com> --------- Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>	2026-09-01 14:41:26 -06:00
Jacob Dahl	3969a1b8ff	fix(ci): publish flashable CAN node firmware (#28445) ... * fix(ci): publish flashable CAN node firmware Release packaging only collected .px4 files, which the DroneCAN bootloader cannot flash. Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com> * fix(ci): attach canbootloader .bin as the SWD image The cannode bootloader is a raw .bin for st-flash, not a .px4 envelope. Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com> * fix(ci): drop cannode.zip from release assets The target-named .uavcan.bin and _canbootloader.bin files are the flashable images; the zip only duplicated them. Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com> --------- Signed-off-by: Jacob Dahl <dahl.jakejacob@gmail.com>	2026-08-28 22:17:34 -06:00
Ramon Roche	6db88017b8	ci(macos): key ccache per matrix OS ... Both legs of the macOS matrix built the same ccache key, since the prefix carried no OS component and setup-ccache appends only ref and sha. Two consequences, reproducible on every run: One leg always lost its save, reporting "Unable to reserve cache with key ccache-macos-<ref>-<sha>, another job may be creating this cache". The survivor then poisoned the other. compiler_check=content means ccache hashes the compiler binary, and macos-latest is now macOS 26 while macos-15 is macOS 15, so neither can reuse the other's objects. Measured on run 32291495373, same commit, after ccache -z: macos-latest hit 860/944 (91.10%) on px4_sitl while macos-15 managed 332/944 (35.17%), recompiling two thirds of what it should have had cached. The 200M budget was shared between both toolchains as well, and reported full, so they were also evicting each other. Assisted-by: Claude:claude-opus-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-08-20 12:11:36 -07:00
Ramon Roche	97c35057ce	ci(build-all): install only what the matrix generator needs ... The scan job is the one job in this workflow that runs outside the build container, so it installs its own Python dependencies. It was installing all 31 entries of requirements.txt to run a script that reaches four of them, and every one of the other 27 was a chance to fail: the job has died on packaging and on lxml, neither of which it imports. Drop the pip cache with it. Restoring it costs about as long as fetching four small wheels, and its key used hashFiles('**./Tools/setup/requirements.txt'), a glob that matches nothing. The hash was empty, so the key collapsed to "Linux-pip-", matched its own restore-key, and never saved again after the first write in April. Verified in a clean python:3.12-slim container: generate_board_targets_json.py exits 0 for both --group and --group --seeders with exactly these four installed. pyyaml and kconfiglib alone are not enough, since loadconfig.ensure_env() shells out to the zenoh generator, which needs empy and pyros-genmsg. Assisted-by: Claude:claude-opus-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-08-20 12:11:15 -07:00
Ramon Roche	6ed16e2748	build(macos): pin osrf/simulation tap under --ci ... When homebrew-core bumps a shared dependency, OSRF's bot bumps each gz formula's revision and deletes its bottle block within minutes of the upstream merge. The matching rebuild is a separate, unautomated step: across 2026 the gap ran 0.5 to 11 days, median 2.5, leaving gz unbottled roughly 18% of the time. In that window macos.sh compiles gz-sim8, gz-msgs10 and five others from source and the setup step goes from ~8min to 20-30min. The removal only edits the formula. The bottle tarballs are never deleted from OSRF's S3 bucket, so pinning to the last fully bottled revision keeps installs binary while upstream catches up. Gated behind a new --ci flag rather than applied unconditionally. Automation wants a reproducible, fast install; development machines want the current formulae and can absorb a source build, and should not have their tap left on a detached HEAD as a side effect of running the setup script. Only the gz tap is pinned. opencv@4, protobuf, gstreamer and the rest still track homebrew-core, so CI keeps catching upstream formula breakage on every PR, which is the reason --sim-tools runs there in the first place. Assisted-by: Claude:claude-opus-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-08-19 13:10:10 -07:00
Ramon Roche	3f81372147	ci(itcm): bump px4-dev container and drop pip install ... The image now ships pyelftools, so the job no longer has to fetch it from PyPI on every run. That install was a recurring failure: 502s from files.pythonhosted.org killed 6 ITCM jobs in the last 30 days. Removing the step matters as much as the bump. Left in place it would still reach PyPI for the symforce line, which the check has never needed: Tools/itcm_check.py imports only elftools. Assisted-by: Claude:claude-opus-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-08-18 17:04:21 -07:00
Ramon Roche	a7aecfb9fb	build(container): bake optional requirements into px4-dev image ... The Dockerfile only ever copied requirements.txt, so optional-requirements.txt was never installed into the image even though it sits in the build context. Every CI job needing pyelftools therefore fetched it from PyPI at job time and failed whenever files.pythonhosted.org returned 502s. symforce publishes no Linux aarch64 wheel and no sdist, so installing the file unguarded would break the linux/arm64 image build. Gate it with an environment marker so aarch64 installs pyelftools only. Assisted-by: Claude:claude-opus-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-08-18 15:13:33 -07:00
Ramon Roche	d0487fc844	ci(build-all): seed ccache only for chip families with cold caches (#28278) ... * ci(build-all): seed ccache only for chip families with cold caches Measured across recent runs, the seeder stage contributes nothing in steady state: all 264 build-job cache restores in the 8 most recent successful runs came from group-level caches, zero from seeder caches, yet the stage gates the build matrix for ~3.4 min of every run's critical path and burns ~22 8cpu-runner-minutes warming caches nobody reads. Its fail-fast value is similarly thin: 2 of the last 40 failed runs failed at seed stage; the other 34 passed seeding and failed in the matrix anyway. Probe each family's ccache namespace in the RunsOn cache bucket from the group_targets job and emit only cold families into the seed matrix. Warm runs skip the stage entirely (the matrix starts right after the scan); a cold family still seeds first and gates the matrix, which is the one case the warmup pays for. The probe fails open: if the bucket env, the listing, or the assumed object layout is wrong, every family reads as cold and the workflow behaves exactly as before. Assisted-by: Claude:claude-fable-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com> * ci(build-all): fail open when seeder probe output is unparseable The has_cold computation ran unguarded under bash -e: malformed probe output would fail the whole workflow instead of falling back to the full seeder matrix. Guard it so every failure path degrades to seeding everything, on RunsOn and on downstream forks without it alike. Assisted-by: Claude:claude-fable-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com> * ci(build-all): move seeder probe logic into the helper script The probe step carried fail-open shell logic inline in the workflow. Move output writing, has_cold computation, and all fail-open handling into filter_cold_seeders.py; the workflow step is now a single script invocation. The script always exits zero and degrades every failure (missing bucket env, aws error, unparseable input) to the full seeder matrix. Assisted-by: Claude:claude-fable-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com> * ci(build-all): probe the magic cache's real object layout The first probe guessed cache/{key} for the magic-cache object layout and read every family COLD, degrading to unconditional seeding. The real layout is cache/v1/{org}/{repo}/{ref}/{version-hash}/{cache-key}; the version hash is the actions/cache digest and not knowable a priori, so list the default-branch scope once (main-scope caches are visible to every ref) and match cache-key basenames. Verified locally against the live bucket: all 10 families read warm and the seed stage skips. Assisted-by: Claude:claude-fable-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com> * ci(build-all): single-source ccache key roots in the matrix generator The ccache key convention was encoded three times: hand-assembled in the workflow's restore/save steps, again in the seeder's cache-key-prefix argument, and a third time in the probe script. Drift between them is silent thanks to the probe's fail-open design: a renamed key scheme would read every family COLD forever and quietly revert to unconditional seeding. Mint the namespace root once in generate_board_targets_json.py as cache_prefix on every group and seeder matrix entry, with seeders occupying the reserved 'seeder' group inside their family namespace. The workflow composes {cache_prefix}-{group}-{ref}-{sha} from matrix fields only, and the probe greps the root it is handed, knowing nothing of the format. Composed keys are byte-identical to the old scheme (asserted for all 33 groups and 10 seeders), so no cache is invalidated. Assisted-by: Claude:claude-fable-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com> --------- Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-08-18 13:36:37 -07:00
Ramon Roche	7740117b17	ci(macos): enable --sim-tools on macos-latest and macos-15 to measure Gazebo bottle install time	2026-08-19 01:05:54 +09:00
Ramon Roche	f8e1d1a899	ci(build-all): drop per-job pynacl install, bump container to v1.18.0-beta2 ... The pynacl install step from #27237 was a stopgap until the build container shipped pynacl preinstalled. That happened in #27237 itself: it added pynacl to Tools/setup/requirements.txt, which dev_container.yml bakes into the image, so v1.18.0-beta1 and later already ship it. Only the v1.17.0-rc2 pin here kept the step alive: 30+ unpinned PyPI pulls per workflow run that already cost us a matrix job to PyPI 502s. Verified by building px4_fmu-v6x_secureboot locally in px4io/px4-dev:v1.18.0-beta2: the signing step runs against the baked-in pynacl with no pip install. The voxl2 container keeps its own pin and needs no fallback since only CONFIG_BOARD_SECUREBOOT targets (NuttX platform only) invoke the signing script. Expect one cold ccache run after merge: the new container's compilers invalidate all caches at once via compiler_check = content. Assisted-by: Claude:claude-fable-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-08-18 09:44:18 +09:00
James Botwina	47ce421a88	fix(simulator_sih): restore xvert front transition (#28090) ... * fix(simulator_sih): correct tailsitter pitot axis Signed-off-by: jbotwina <jimbotwina16@gmail.com> * fix(airframes/10042): tune xvert transition Signed-off-by: jbotwina <jimbotwina16@gmail.com> * test(mavsdk_tests): cover xvert transition Signed-off-by: jbotwina <jimbotwina16@gmail.com> * fix(airframes/10042): disable airspeed scaling Signed-off-by: jbotwina <jimbotwina16@gmail.com> --------- Signed-off-by: jbotwina <jimbotwina16@gmail.com>	2026-07-29 13:01:51 -06:00
Ramon Roche	eba76cbb69	fix(ci): restore board target matrix generation after zenoh Kconfig change ... Since b11e615810 src/modules/zenoh/Kconfig sources the generated topic catalog through ZENOH_KCONFIG_TOPICS, which only cmake exports, so every standalone Kconfig parse crashed and build_all_targets produced an empty build matrix on every run while the scan job still reported success. Add Tools/kconfig/loadconfig.py as the single entry point for parsing the Kconfig tree outside cmake: it generates the topic catalog exactly like cmake/kconfig.cmake and provides shared board/target enumeration and config loading. Port generate_board_targets_json.py (CI matrix) and updateconfig.py (make updateconfig) to it; both were independently broken and each reimplemented board scanning on its own. Harden the workflow so the next failure of this class is loud: run the generator as a standalone assignment, so bash -e fails the step (the previous echo "matrix=$(...)" form masked the exit code), and write the JSON outputs through the heredoc GITHUB_OUTPUT form. Matrix output verified byte-identical to the pre-b11e6158101 baseline in all three modes; updateconfig output verified identical against the same baseline. Assisted-by: Claude:claude-fable-5 Signed-off-by: Ramon Roche <mrpollo@gmail.com>	2026-07-20 20:24:10 -04:00
Julian Oes	734d511114	ci(sitl_tests): tolerate gcov negative counts in coverage capture ... Switching the SITL tests to the SIH simulator exercises sih.cpp, whose gcov data contains a spurious negative counter (line 510) that newer lcov treats as a fatal error, failing the coverage step. Pass --ignore-errors negative so lcov zeroes the bad count and continues, as the tool itself suggests. Signed-off-by: Julian Oes <julian@oes.ch>	2026-07-17 10:51:42 -05:00
Julian Oes	93c42252ff	feature(CI): use SIH sim	2026-07-17 10:51:42 -05:00
Julian Oes	eaaf6f676a	test(mavsdk): leave tests backwards compatible but enable SIH ... This reverts some of the changes to keep runners of classic Gazebo in there but leave it optional, so we can still use it if required.	2026-07-17 10:51:42 -05:00