mirror of
https://github.com/eclipse-threadx/threadx.git
synced 2026-10-06 06:59:08 +08:00
Excluded the SMP regression-test hook code from the coverage figure
regression_test / tx (push) Canceled after 0s
regression_test / smp (push) Canceled after 0s
regression_test / deploy (push) Canceled after 0s
regression_template / run_tests (push) Canceled after 0s
regression_template / deploy_code_coverage (push) Canceled after 0s
regression_test / tx (push) Canceled after 0s
regression_test / smp (push) Canceled after 0s
regression_test / deploy (push) Canceled after 0s
regression_template / run_tests (push) Canceled after 0s
regression_template / deploy_code_coverage (push) Canceled after 0s
Every SMP build configuration is compiled with -DTX_REGRESSION_TEST, which opens a block in the Linux SMP port header defining ten macros that expand to live code -- flag tests, state changes and calls -- at eleven sites in common_smp/src. An application defines none of them and tx_api.h expands them all to nothing, but gcov attributes a macro expansion to the file that invokes it, so the harness's own scaffolding was being counted as certified source. coverage_union.py now drops those sites and prints what it dropped, covered and total on both axes, so the exclusion can be audited rather than inferred. One site keeps its line: tx_api.h defines TX_TIMER_INITIALIZE_EXTENSION as a real statement under TX_MISRA_ENABLE, so that line ships and only its branches come out. A listed site that no longer matches the report is a hard failure naming the macro, not a silent skip. The branch gate moves 99.05 to 99.04 because the margin is ten outcomes rather than a percentage, and a smaller denominator buys nine at the old value; the line gate is unchanged. The excluded code measured 10 lines and 36 branch outcomes, every one of them covered, so the uncovered set is untouched: the same 24 outcomes at the same 22 sites in the same 13 files, over the same 194 files. Eight configurations from a wiped tree, 136/136 tests in each, 0 failed. The union goes from 5449/5450 lines and 3588/3612 branch outcomes to 5439/5440 and 3552/3576, re-derived key by key against the previous measurement. gcovr's merged report is published unchanged. Assisted-by: Claude Code (Opus 5) <noreply@anthropic.com>
This commit is contained in:
@@ -26,13 +26,13 @@ filter=$repo_root/common_smp/src
|
||||
# renumbers it. Both figures are produced and the merged report is published
|
||||
# unchanged; the gate uses the union because it is the one that counts branches
|
||||
# in the source. Over the eight configurations the merged denominator reads more
|
||||
# than 6500 where the source carries 3612, and adding misra_trace_build alone
|
||||
# moved the merged figure by 994 branches against the dozen the source really
|
||||
# gained.
|
||||
# than 6500 where the certified denominator is 3576, and adding misra_trace_build
|
||||
# alone moved the merged figure by 994 branches against the dozen the source
|
||||
# really gained.
|
||||
#
|
||||
# The branch gate is far below the 83.00 that preceded it and no coverage was
|
||||
# lost doing it. 83.00 was a percentage of 4773 counted branch instances; this is
|
||||
# a percentage of the 3612 branches that exist in common_smp/src.
|
||||
# a percentage of the 3576 certified branches in common_smp/src.
|
||||
#
|
||||
# The branch denominator is 3612 rather than 3610 from the fault-injection sweep
|
||||
# onwards. TX_BYTE_ALLOCATE_EXTENSION in the Linux SMP port gained a second
|
||||
@@ -41,6 +41,16 @@ filter=$repo_root/common_smp/src
|
||||
# and nothing else in ports_smp enters the denominator -- the gcovr -f filter
|
||||
# excludes it.
|
||||
#
|
||||
# It is 3576 rather than 3612 from the hook exclusion onwards, and the line
|
||||
# denominator 5440 rather than 5450. Every configuration is built with
|
||||
# -DTX_REGRESSION_TEST, which opens a block in the port header defining ten
|
||||
# macros that expand to live code at eleven sites in common_smp/src. That code is
|
||||
# in no build anyone ships, so coverage_union.py drops it: 10 lines and 36 branch
|
||||
# outcomes, all of them covered, leaving the uncovered set untouched. One of the
|
||||
# eleven keeps its line, because tx_api.h gives that macro a real statement under
|
||||
# TX_MISRA_ENABLE and the line therefore ships. The sites are listed in
|
||||
# coverage_union.py and printed with the figure.
|
||||
#
|
||||
# The margin below the measured figure is not slack for regressions. It is there
|
||||
# because the same unchanged tree has not always measured identically twice.
|
||||
#
|
||||
@@ -102,14 +112,21 @@ filter=$repo_root/common_smp/src
|
||||
# it by winning a race rather than by executing in order.
|
||||
#
|
||||
# The gate is therefore still set ten lines and ten outcomes below the set
|
||||
# covered in *every* run -- 5449 lines and 3588 outcomes -- rather than below the
|
||||
# covered in *every* run -- 5439 lines and 3552 outcomes -- rather than below the
|
||||
# lowest total any run reported. The two coincide in the last two samples and
|
||||
# have not in any earlier one. Ten is a measured size now rather than an
|
||||
# inherited one: it covers the largest per-configuration movement observed,
|
||||
# eight outcomes, with headroom, against a set that has no second configuration
|
||||
# to fall back on.
|
||||
#
|
||||
# The margin is ten outcomes, not a percentage, so the branch threshold moved
|
||||
# with the denominator. Against 3576 a 99.05 gate admits 3543 and buys nine; it
|
||||
# takes 99.04 to admit 3542 and keep the ten the margin is sized at. The line
|
||||
# axis needs no change: against 5440 the 99.79 gate still admits 5429, which is
|
||||
# ten below the floor. Both are held on the achieved figure of 5439/5440 and
|
||||
# 3552/3576.
|
||||
min_line=${TX_COVERAGE_MIN_LINE:-99.79}
|
||||
min_branch=${TX_COVERAGE_MIN_BRANCH:-99.05}
|
||||
min_branch=${TX_COVERAGE_MIN_BRANCH:-99.04}
|
||||
|
||||
# --merge unions the per-configuration reports into the one number that means
|
||||
# something. Each configuration writes an intermediate JSON beside its XML, and
|
||||
|
||||
@@ -26,6 +26,11 @@ Both figures are kept. gcovr's merged report is still produced and published
|
||||
unchanged -- it is the tool's own output and nothing here rewrites it. This is
|
||||
the figure the coverage ratchet gates on, so that adding a build configuration
|
||||
moves the number only by the code it actually brings in.
|
||||
|
||||
The union also drops the code the regression-test hook macros inject into the
|
||||
certified source. See HOOK_SITES below for what that is and why it is not part
|
||||
of the denominator; the excluded sites are printed with the figure, because an
|
||||
exclusion nobody can see in the output is an exclusion nobody can audit.
|
||||
"""
|
||||
|
||||
import glob
|
||||
@@ -34,6 +39,93 @@ import os
|
||||
import sys
|
||||
|
||||
|
||||
# The regression-test hook code, which is not part of the certified denominator.
|
||||
#
|
||||
# CMakeLists.txt puts -DTX_REGRESSION_TEST on every build configuration. That
|
||||
# opens a block in the Linux SMP port header, ports_smp/linux/gnu/inc/tx_port.h
|
||||
# lines 206-319, which defines ten macros that expand to live code -- flag
|
||||
# tests, state changes and calls -- at eleven sites in common_smp/src, carried
|
||||
# by this list and the one after it. An application does not define the macro,
|
||||
# tx_api.h's #ifndef fallbacks then expand every one of them to nothing, and
|
||||
# none of this code exists in a build anyone ships. gcov attributes a macro
|
||||
# expansion to the file that invokes it, so without this list the harness's own
|
||||
# scaffolding is counted as certified source.
|
||||
#
|
||||
# Derive the list by enumerating the block's #define lines and grepping each
|
||||
# name. Do not grep for a naming pattern: TX_PORT_SPECIFIC_MEMORY_SYNCHRONIZATION
|
||||
# is not named *_EXTENSION, and a grep for that suffix misses it and the two
|
||||
# sites it occupies.
|
||||
#
|
||||
# Only a macro the *test build* defines belongs here. A macro the port header
|
||||
# defines is in the application's build too, so removing its code would take
|
||||
# shipped code out of the denominator -- the opposite error. The case to keep
|
||||
# in mind is TX_TRACE_PORT_EXTENSION: the same header defines it,
|
||||
# unconditionally, and it expands to a clock_gettime() inside tx_misra.c. It
|
||||
# ships, so it stays.
|
||||
#
|
||||
# The line numbers are stable because the certified baseline is a frozen tag,
|
||||
# and a site that moves is caught rather than silently skipped: every entry must
|
||||
# match a line the report carries, or this fails.
|
||||
HOOK_SITES = [
|
||||
("common_smp/src/tx_byte_allocate.c", 175, "TX_BYTE_ALLOCATE_EXTENSION"),
|
||||
("common_smp/src/tx_byte_release.c", 236, "TX_BYTE_RELEASE_EXTENSION"),
|
||||
("common_smp/src/tx_initialize_kernel_enter.c", 133, "TX_INITIALIZE_KERNEL_ENTER_EXTENSION"),
|
||||
("common_smp/src/tx_initialize_kernel_enter.c", 170, "TX_PORT_SPECIFIC_MEMORY_SYNCHRONIZATION"),
|
||||
("common_smp/src/tx_initialize_kernel_enter.c", 183, "TX_PORT_SPECIFIC_MEMORY_SYNCHRONIZATION"),
|
||||
("common_smp/src/tx_mutex_priority_change.c", 343, "TX_MUTEX_PRIORITY_CHANGE_EXTENSION"),
|
||||
("common_smp/src/tx_mutex_put.c", 349, "TX_MUTEX_PUT_EXTENSION_1"),
|
||||
("common_smp/src/tx_mutex_put.c", 600, "TX_MUTEX_PUT_EXTENSION_2"),
|
||||
("common_smp/src/tx_thread_priority_change.c", 363, "TX_THREAD_PRIORITY_CHANGE_EXTENSION"),
|
||||
("common_smp/src/tx_thread_stack_analyze.c", 145, "TX_THREAD_STACK_ANALYZE_EXTENSION"),
|
||||
]
|
||||
|
||||
# One site where the hook displaces shipped code rather than occupying an empty
|
||||
# line, so only its branches come out.
|
||||
#
|
||||
# Nine of the ten macros have an empty #ifndef fallback in tx_api.h, so without
|
||||
# the hook their invocation line holds nothing at all and the whole line goes.
|
||||
# TX_TIMER_INITIALIZE_EXTENSION is the exception: under TX_MISRA_ENABLE
|
||||
# tx_api.h defines it as "status = _tx_misra_status_get((a));", a statement with
|
||||
# no branches. A shipped misra build therefore still executes this line, and
|
||||
# dropping it would remove shipped code from the denominator. The line stays and
|
||||
# the hook's branch outcomes go.
|
||||
HOOK_BRANCH_SITES = [
|
||||
("common_smp/src/tx_timer_initialize.c", 272, "TX_TIMER_INITIALIZE_EXTENSION"),
|
||||
]
|
||||
|
||||
|
||||
def exclude_hook_sites(lines, branches):
|
||||
"""Drop the hook expansions from the union, and report what was dropped.
|
||||
|
||||
Returns one row per site: the macro, and the covered/total it took out of
|
||||
each axis. Covered/total rather than a count, so the output shows on its
|
||||
face that the exclusion removed nothing that was uncovered -- which would
|
||||
raise the figure for the wrong reason.
|
||||
"""
|
||||
|
||||
report = []
|
||||
missing = []
|
||||
|
||||
for path, number, macro, drop_line in (
|
||||
[(p, n, m, True) for p, n, m in HOOK_SITES] +
|
||||
[(p, n, m, False) for p, n, m in HOOK_BRANCH_SITES]):
|
||||
|
||||
outcomes = sorted(k for k in branches if k[0] == path and k[1] == number)
|
||||
if (path, number) not in lines:
|
||||
missing.append((path, number, macro))
|
||||
continue
|
||||
|
||||
line_covered = line_total = 0
|
||||
if drop_line:
|
||||
line_covered, line_total = lines.pop((path, number)), 1
|
||||
|
||||
outcome_covered = sum(branches.pop(k) for k in outcomes)
|
||||
report.append((path, number, macro, drop_line,
|
||||
line_covered, line_total, outcome_covered, len(outcomes)))
|
||||
|
||||
return report, missing
|
||||
|
||||
|
||||
def union(tracefiles):
|
||||
"""Union line and branch coverage across per-configuration tracefiles."""
|
||||
|
||||
@@ -76,6 +168,14 @@ def main():
|
||||
print("coverage_union.py: the tracefiles contain no files.", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
excluded, missing = exclude_hook_sites(lines, branches)
|
||||
if missing:
|
||||
for path, number, macro in missing:
|
||||
print("coverage_union.py: %s:%d is not in the report -- %s has moved."
|
||||
% (path, number, macro), file=sys.stderr)
|
||||
print("coverage_union.py: re-derive HOOK_SITES from the port header.", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
line_covered, line_total = sum(lines.values()), len(lines)
|
||||
branch_covered, branch_total = sum(branches.values()), len(branches)
|
||||
|
||||
@@ -85,6 +185,22 @@ def main():
|
||||
print("coverage_union.py: unioned over %d configuration(s):" % len(tracefiles))
|
||||
for path in tracefiles:
|
||||
print(" %s" % os.path.basename(path)[:-len(".json")])
|
||||
|
||||
print(" excluded, regression-test hook expansions (-DTX_REGRESSION_TEST):")
|
||||
hook_lines_covered = hook_lines_total = 0
|
||||
hook_outcomes_covered = hook_outcomes_total = 0
|
||||
for path, number, macro, drop_line, lc, lt, oc, ot in excluded:
|
||||
print(" %-36s lines %d/%d, outcomes %2d/%-2d %s%s"
|
||||
% ("%s:%d" % (path[len("common_smp/src/"):], number),
|
||||
lc, lt, oc, ot, macro, "" if drop_line else ", branches only"))
|
||||
hook_lines_covered += lc
|
||||
hook_lines_total += lt
|
||||
hook_outcomes_covered += oc
|
||||
hook_outcomes_total += ot
|
||||
print(" %-36s lines %d/%d, outcomes %2d/%-2d"
|
||||
% ("total", hook_lines_covered, hook_lines_total,
|
||||
hook_outcomes_covered, hook_outcomes_total))
|
||||
|
||||
print(" lines %d/%d - %.2f%%" % (line_covered, line_total, line_rate))
|
||||
print(" branches %d/%d - %.2f%%" % (branch_covered, branch_total, branch_rate))
|
||||
|
||||
|
||||
Reference in New Issue
Block a user