mirror of
https://github.com/eclipse-threadx/threadx.git
synced 2026-10-07 14:52:34 +08:00
A failing test threw away coverage that had already been collected, and the run whose behaviour changed is exactly the run whose coverage is worth reading. Measured on the failing run of 2026-08-18: it uploaded test_reports for all three suites and no coverage_report artifact at all. Two causes, and the workflow one is the smaller of them. cmake_bootstrap.sh runs under set -e, so a failing ctest aborted test() before ./coverage.sh was reached. The gcda files exist by that point, so nothing was missing except the step that reads them. ctest's status is now captured and returned at the end, and the summary grep is allowed to fail rather than being the thing that stops the coverage behind it. The serial branch of the test dispatch collected no status either, so under set -e the first failing configuration stopped the remaining four from being tested at all -- and their coverage from being collected. That was cheap while the suites ran in parallel, because the parallel branch already collects exit codes from its background jobs. Moving to serial execution in #643 quietly made one failure cost the other four configurations. The serial branch now collects status the same way the parallel branch does. With those fixed the report exists, so the workflow steps that publish it no longer skip on failure. They are guarded with !cancelled() rather than always(), so a cancelled run still stops promptly, which is the idiom deploy_code_coverage already uses. The ${{ }} wrapping is required and not decoration: a bare ! opens a YAML tag, and the file will not parse without it. Verified locally by replacing one test binary with a stub that exits 1: before the failing run of 2026-08-18 produced no coverage_report artifact after run.sh test default_build_coverage exits 8, and produces coverage_report/default_build_coverage.xml with 177 files and 3804 of 3827 lines after run.sh test all exits 8, and all five configurations run rather than stopping at the first The failure still fails. Only the reporting around it changed. Assisted-by: Claude Opus 5 <noreply@anthropic.com>
251 lines
9.4 KiB
YAML
251 lines
9.4 KiB
YAML
# This is a basic workflow that is manually triggered
|
|
|
|
name: regression_template
|
|
|
|
on:
|
|
workflow_call:
|
|
inputs:
|
|
install_script:
|
|
default: './scripts/install.sh'
|
|
required: false
|
|
type: string
|
|
build_script:
|
|
default: './scripts/build.sh'
|
|
required: false
|
|
type: string
|
|
test_script:
|
|
default: './scripts/test.sh'
|
|
required: false
|
|
type: string
|
|
cmake_path:
|
|
default: './test/cmake'
|
|
required: false
|
|
type: string
|
|
skip_test:
|
|
default: false
|
|
required: false
|
|
type: boolean
|
|
skip_coverage:
|
|
default: false
|
|
required: false
|
|
type: boolean
|
|
coverage_name:
|
|
default: 'default_build_coverage'
|
|
required: false
|
|
type: string
|
|
skip_deploy:
|
|
default: false
|
|
required: false
|
|
type: boolean
|
|
deploy_list:
|
|
default: ''
|
|
required: false
|
|
type: string
|
|
result_affix:
|
|
default: ''
|
|
required: false
|
|
type: string
|
|
|
|
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
|
|
jobs:
|
|
# This workflow contains a single job called "linux_job"
|
|
run_tests:
|
|
if: ${{ !inputs.skip_test}}
|
|
permissions:
|
|
contents: read
|
|
issues: read
|
|
checks: write
|
|
pull-requests: write
|
|
|
|
# The type of runner that the job will run on
|
|
runs-on: ubuntu-24.04
|
|
|
|
# Steps represent a sequence of tasks that will be executed as part of the job
|
|
steps:
|
|
- name: Check out the repository
|
|
uses: actions/checkout@v4.2.2
|
|
with:
|
|
submodules: true
|
|
|
|
# apt and PyPI have stalled this step twice, for 55 minutes and for over two
|
|
# hours, against a normal 27 to 152 seconds. Nothing here had a timeout, so a
|
|
# stall ran until the six hour job limit and cost a whole run rather than
|
|
# failing and being retried.
|
|
- name: Install softwares
|
|
timeout-minutes: 10
|
|
run: ${{ inputs.install_script }}
|
|
|
|
- name: Build
|
|
timeout-minutes: 15
|
|
run: ${{ inputs.build_script }}
|
|
|
|
- name: Test
|
|
timeout-minutes: 60
|
|
run: ${{ inputs.test_script }}
|
|
|
|
- name: Publish Test Results
|
|
uses: EnricoMi/publish-unit-test-result-action@v2.11.0
|
|
if: always()
|
|
with:
|
|
check_name: Test Results ${{ inputs.result_affix }}
|
|
files: |
|
|
${{ inputs.cmake_path }}/build/*/*.xml
|
|
|
|
- name: Upload Test Results
|
|
if: success() || failure()
|
|
uses: actions/upload-artifact@v4.6.2
|
|
with:
|
|
name: test_reports ${{ inputs.result_affix }}
|
|
path: |
|
|
${{ inputs.cmake_path }}/build/*.txt
|
|
${{ inputs.cmake_path }}/build/*/Testing/**/*.xml
|
|
${{ inputs.cmake_path }}/build/**/regression/output_files/*.bin
|
|
|
|
- name: Configure GitHub Pages
|
|
uses: actions/configure-pages@v5.0.0
|
|
|
|
# The coverage steps below run even when a test failed. cmake_bootstrap.sh
|
|
# now produces the report in that case, and the run whose behaviour changed
|
|
# is the one whose coverage is worth reading; previously a single flaky test
|
|
# suppressed the report for the whole run. !cancelled() rather than always(),
|
|
# so a cancelled run still stops promptly -- the same idiom the
|
|
# deploy_code_coverage job below already uses. The ${{ }} is required: a bare
|
|
# ! opens a YAML tag, and the expression will not parse without it.
|
|
- name: Generate Code Coverage Results Summary
|
|
if: ${{ !cancelled() && (!inputs.skip_coverage) }}
|
|
uses: irongut/CodeCoverageSummary@v1.3.0
|
|
with:
|
|
filename: ${{ inputs.cmake_path }}/coverage_report/${{ inputs.coverage_name }}.xml
|
|
format: markdown
|
|
badge: true
|
|
hide_complexity: true
|
|
output: file
|
|
|
|
- name: Write Code Coverage Summary
|
|
if: ${{ !cancelled() && (!inputs.skip_coverage) }}
|
|
run: |
|
|
echo "## Coverage Report ${{ inputs.result_affix }}" >> $GITHUB_STEP_SUMMARY
|
|
cat code-coverage-results.md >> $GITHUB_STEP_SUMMARY
|
|
|
|
- name: Create CheckRun for Code Coverage
|
|
if: ${{ !cancelled() && ((github.event_name == 'push') || (github.event_name == 'workflow_dispatch') || (github.event.pull_request.head.repo.full_name == github.repository)) && (!inputs.skip_coverage) }}
|
|
uses: LouisBrunner/checks-action@v2.0.0
|
|
with:
|
|
token: ${{ secrets.GITHUB_TOKEN }}
|
|
name: Code Coverage ${{ inputs.result_affix }}
|
|
conclusion: ${{ job.status }}
|
|
output: |
|
|
{"summary":"Coverage Report"}
|
|
output_text_description_file: code-coverage-results.md
|
|
|
|
- name: Add Code Coverage PR Comment
|
|
if: ${{ !cancelled() && ((github.event_name == 'push') || (github.event.pull_request.head.repo.full_name == github.repository)) && (!inputs.skip_coverage) }}
|
|
uses: marocchino/sticky-pull-request-comment@v2.9.4
|
|
with:
|
|
header: Code Coverage ${{ inputs.result_affix }}
|
|
path: code-coverage-results.md
|
|
|
|
# Add sudo to move coverage folder created by root user
|
|
- name: Prepare Coverage GitHub Pages
|
|
if: ${{ !cancelled() && (!inputs.skip_coverage) }}
|
|
run: >-
|
|
if [ "${{ inputs.result_affix }}" != "" ] && ${{ inputs.skip_deploy }}; then
|
|
sudo mv ${{ inputs.cmake_path }}/coverage_report/${{ inputs.coverage_name }} \
|
|
${{ inputs.cmake_path }}/coverage_report/${{ inputs.result_affix }}
|
|
fi
|
|
|
|
- name: Coverage Report name
|
|
id: artifact
|
|
if: ${{ !cancelled() && (!inputs.skip_coverage) }}
|
|
run: echo "coverage_report=coverage_report-$(date +%s)" >> $GITHUB_OUTPUT
|
|
|
|
- name: Upload Code Coverage Artifacts
|
|
uses: actions/upload-artifact@v4.6.2
|
|
if: ${{ !cancelled() && (inputs.skip_deploy && !inputs.skip_coverage) }}
|
|
with:
|
|
name: ${{ steps.artifact.outputs.coverage_report }}
|
|
path: ${{ inputs.cmake_path }}/coverage_report
|
|
retention-days: 1
|
|
|
|
- name: Upload Code Coverage Pages
|
|
uses: actions/upload-pages-artifact@v3.0.1
|
|
if: ${{ !cancelled() && (!inputs.skip_deploy && !inputs.skip_coverage) }}
|
|
with:
|
|
path: ${{ inputs.cmake_path }}/coverage_report/${{ inputs.coverage_name }}
|
|
|
|
deploy_code_coverage:
|
|
runs-on: ubuntu-24.04
|
|
if: ((github.event_name == 'push') || (github.event_name == 'workflow_dispatch')) && !inputs.skip_coverage && !inputs.skip_deploy && !failure() && !cancelled()
|
|
needs: run_tests
|
|
environment:
|
|
name: github-pages
|
|
url: ${{ steps.deployment.outputs.page_url }}
|
|
permissions:
|
|
pages: write
|
|
id-token: write
|
|
|
|
steps:
|
|
# Selects the coverage artifacts by pattern rather than by name.
|
|
#
|
|
# This used to ask for ${{ steps.artifact.outputs.coverage_report }},
|
|
# which is set by the "Coverage Report name" step of run_tests -- a
|
|
# different job. The steps context does not cross jobs, so that
|
|
# expression evaluated to the empty string and the action fell back to
|
|
# its documented behaviour for an unspecified name: "No input name,
|
|
# artifact-ids or pattern filtered specified, downloading all
|
|
# artifacts". It pulled down four, the two coverage reports and the two
|
|
# test_reports bundles of JUnit XML, each into a directory named after
|
|
# the artifact -- so the published site carried the test reports as well
|
|
# as the coverage, under paths containing a run timestamp that changed
|
|
# on every publish.
|
|
#
|
|
# merge-multiple puts the contents of both coverage artifacts directly
|
|
# into coverage_report rather than under a directory named for each
|
|
# artifact. Each one holds a single directory named for its suite,
|
|
# ThreadX or SMP, renamed from default_build_coverage by the "Prepare
|
|
# Coverage GitHub Pages" step, so the merge yields exactly the two
|
|
# suite directories the deploy expects, and the artifact name -- with
|
|
# its timestamp -- stops appearing in the published path at all.
|
|
#
|
|
# The two artifacts also each carry a default_build_coverage.xml, and
|
|
# the merge means one overwrites the other. That file is consumed by
|
|
# CodeCoverageSummary back in run_tests and is not read here, so this is
|
|
# untidy rather than wrong.
|
|
- uses: actions/download-artifact@v4.3.0
|
|
if: ${{ inputs.skip_test }}
|
|
with:
|
|
pattern: coverage_report-*
|
|
merge-multiple: true
|
|
path: ${{ inputs.cmake_path }}/coverage_report
|
|
|
|
- name: Upload Code Coverage Pages
|
|
uses: actions/upload-pages-artifact@v3.0.1
|
|
if: ${{ inputs.skip_test }}
|
|
with:
|
|
path: .
|
|
|
|
# The artifacts are named coverage_report-<epoch> by run_tests, so the
|
|
# bare name matched nothing: useGlob defaults to true in this action, and
|
|
# as a glob "coverage_report" matches only the literal string. The step
|
|
# has therefore been deleting nothing. It does not fail on a miss, which
|
|
# is why that went unnoticed; retention-days: 1 on the upload is what has
|
|
# actually been clearing these up.
|
|
- name: Delete Duplicate Code Coverage Artifact
|
|
uses: geekyeggo/delete-artifact@v5.1.0
|
|
with:
|
|
name: coverage_report-*
|
|
|
|
- name: Deploy GitHub Pages site
|
|
id: deployment
|
|
uses: actions/deploy-pages@v4.0.5
|
|
|
|
- name: Write Code Coverage Report URL
|
|
run: >-
|
|
if [ "${{ inputs.deploy_list }}" != "" ]; then
|
|
for i in ${{ inputs.deploy_list }}; do
|
|
echo 'Coverage report for ' $i ':${{ steps.deployment.outputs.page_url }}'$i >> $GITHUB_STEP_SUMMARY
|
|
done
|
|
else
|
|
echo 'Coverage report: ${{ steps.deployment.outputs.page_url }}' >> $GITHUB_STEP_SUMMARY
|
|
fi
|