Kept the coverage report from the runs that most need one (#659)

A failing test threw away coverage that had already been collected, and the
run whose behaviour changed is exactly the run whose coverage is worth
reading. Measured on the failing run of 2026-08-18: it uploaded test_reports
for all three suites and no coverage_report artifact at all.

Two causes, and the workflow one is the smaller of them.

cmake_bootstrap.sh runs under set -e, so a failing ctest aborted test()
before ./coverage.sh was reached. The gcda files exist by that point, so
nothing was missing except the step that reads them. ctest's status is now
captured and returned at the end, and the summary grep is allowed to fail
rather than being the thing that stops the coverage behind it.

The serial branch of the test dispatch collected no status either, so under
set -e the first failing configuration stopped the remaining four from being
tested at all -- and their coverage from being collected. That was cheap
while the suites ran in parallel, because the parallel branch already
collects exit codes from its background jobs. Moving to serial execution in
#643 quietly made one failure cost the other four configurations. The serial
branch now collects status the same way the parallel branch does.

With those fixed the report exists, so the workflow steps that publish it no
longer skip on failure. They are guarded with !cancelled() rather than
always(), so a cancelled run still stops promptly, which is the idiom
deploy_code_coverage already uses. The ${{ }} wrapping is required and not
decoration: a bare ! opens a YAML tag, and the file will not parse without
it.

Verified locally by replacing one test binary with a stub that exits 1:

  before  the failing run of 2026-08-18 produced no coverage_report artifact
  after   run.sh test default_build_coverage exits 8, and produces
          coverage_report/default_build_coverage.xml with 177 files and
          3804 of 3827 lines
  after   run.sh test all exits 8, and all five configurations run rather
          than stopping at the first

The failure still fails. Only the reporting around it changed.

Assisted-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Frédéric Desbiens
2026-08-24 16:36:18 -04:00
committed by GitHub
parent adc6469b91
commit 7959aef3bc
2 changed files with 37 additions and 12 deletions
+15 -8
View File
@@ -104,8 +104,15 @@ jobs:
- name: Configure GitHub Pages
uses: actions/configure-pages@v5.0.0
# The coverage steps below run even when a test failed. cmake_bootstrap.sh
# now produces the report in that case, and the run whose behaviour changed
# is the one whose coverage is worth reading; previously a single flaky test
# suppressed the report for the whole run. !cancelled() rather than always(),
# so a cancelled run still stops promptly -- the same idiom the
# deploy_code_coverage job below already uses. The ${{ }} is required: a bare
# ! opens a YAML tag, and the expression will not parse without it.
- name: Generate Code Coverage Results Summary
if: (!inputs.skip_coverage)
if: ${{ !cancelled() && (!inputs.skip_coverage) }}
uses: irongut/CodeCoverageSummary@v1.3.0
with:
filename: ${{ inputs.cmake_path }}/coverage_report/${{ inputs.coverage_name }}.xml
@@ -115,13 +122,13 @@ jobs:
output: file
- name: Write Code Coverage Summary
if: (!inputs.skip_coverage)
if: ${{ !cancelled() && (!inputs.skip_coverage) }}
run: |
echo "## Coverage Report ${{ inputs.result_affix }}" >> $GITHUB_STEP_SUMMARY
cat code-coverage-results.md >> $GITHUB_STEP_SUMMARY
- name: Create CheckRun for Code Coverage
if: ((github.event_name == 'push') || (github.event_name == 'workflow_dispatch') || (github.event.pull_request.head.repo.full_name == github.repository)) && (!inputs.skip_coverage)
if: ${{ !cancelled() && ((github.event_name == 'push') || (github.event_name == 'workflow_dispatch') || (github.event.pull_request.head.repo.full_name == github.repository)) && (!inputs.skip_coverage) }}
uses: LouisBrunner/checks-action@v2.0.0
with:
token: ${{ secrets.GITHUB_TOKEN }}
@@ -132,7 +139,7 @@ jobs:
output_text_description_file: code-coverage-results.md
- name: Add Code Coverage PR Comment
if: ((github.event_name == 'push') || (github.event.pull_request.head.repo.full_name == github.repository)) && (!inputs.skip_coverage)
if: ${{ !cancelled() && ((github.event_name == 'push') || (github.event.pull_request.head.repo.full_name == github.repository)) && (!inputs.skip_coverage) }}
uses: marocchino/sticky-pull-request-comment@v2.9.4
with:
header: Code Coverage ${{ inputs.result_affix }}
@@ -140,7 +147,7 @@ jobs:
# Add sudo to move coverage folder created by root user
- name: Prepare Coverage GitHub Pages
if: (!inputs.skip_coverage)
if: ${{ !cancelled() && (!inputs.skip_coverage) }}
run: >-
if [ "${{ inputs.result_affix }}" != "" ] && ${{ inputs.skip_deploy }}; then
sudo mv ${{ inputs.cmake_path }}/coverage_report/${{ inputs.coverage_name }} \
@@ -149,12 +156,12 @@ jobs:
- name: Coverage Report name
id: artifact
if: (!inputs.skip_coverage)
if: ${{ !cancelled() && (!inputs.skip_coverage) }}
run: echo "coverage_report=coverage_report-$(date +%s)" >> $GITHUB_OUTPUT
- name: Upload Code Coverage Artifacts
uses: actions/upload-artifact@v4.6.2
if: (inputs.skip_deploy && !inputs.skip_coverage)
if: ${{ !cancelled() && (inputs.skip_deploy && !inputs.skip_coverage) }}
with:
name: ${{ steps.artifact.outputs.coverage_report }}
path: ${{ inputs.cmake_path }}/coverage_report
@@ -162,7 +169,7 @@ jobs:
- name: Upload Code Coverage Pages
uses: actions/upload-pages-artifact@v3.0.1
if: (!inputs.skip_deploy && !inputs.skip_coverage)
if: ${{ !cancelled() && (!inputs.skip_deploy && !inputs.skip_coverage) }}
with:
path: ${{ inputs.cmake_path }}/coverage_report/${{ inputs.coverage_name }}