feat: split module report pies into ASIL and QM by antonkri · Pull Request #946 · eclipse-score/docs-as-code

antonkri · 2026-10-13T09:47:58Z

Summary

The statistics in the module and platform verification reports mixed QM and
ASIL B Needs in the same pie chart, so neither safety class could be assessed
on its own. Every pie row is now rendered once per safety class:

one row restricted to safety == "ASIL_B"
one row restricted to safety == "QM"

Each row carries a colour-coded heading (ASIL B / QM). If a component or
feature has no Need of a safety class, the pies are replaced by placeholder
tiles of the same size, so the absence is visible instead of a blank chart.

Module verification report

Applies to Component Requirements Statistics and Component Architecture
Statistics, i.e. 8 pies per component (status + test coverage, per safety
class).

Platform verification report

Applies to the stakeholder requirement, feature requirement and feature
architecture rows. The reference-coverage pie is partitioned per safety class
as well, so its slices match the row it sits in.

Preview

Rendered with real data in eclipse-score/reference_integration#396
(pins score_docs_as_code to the head of this PR):

Platform Verification Report
Module Verification Reports:
Baselibs,
Logging,
Persistency,
all modules

Colours

The ASIL B colours are unchanged: valid yellow, valid_inspected green,
invalid red.

QM Needs are not formally inspected, so for QM the pie only has valid and
invalid. There valid already is the completed state, so it is coloured
green rather than the yellow that means "not inspected yet" for ASIL B. A QM
Need that is nevertheless valid_inspected is counted as valid (as in the
metamodel graph checks), so it is not silently dropped from the status pie.

Both reports have a collapsible legend explaining the status values and chart
colours.

Layout

Both reports use the same three-column raster for every pie row, so rows of
two and of three pies render their charts at the same size; two-pie rows are
centred. These changes are scoped to the reports — needpie charts elsewhere
are unaffected.

Table sorting

The component overview, requirements traceability and architectural elements
tables (module report) and the requirement tables (platform report) are sorted
by safety now, so all ASIL B entries precede the QM ones. The previous
secondary ordering by id is gone; sphinx-needs only supports a single sort
key.

Implementation note

The pie rows would have multiplied near-identical needpie directives, so
they are generated by a single safety_pie_row helper and the two safety
classes cannot drift apart.

MiniJinja (used by sphinx-needs) has no template loader, so macros cannot be
shared between .need templates. safety_pie_row, id_filter_list and
workproduct_rows are therefore implemented once as render helpers in
score_sphinx_needs_templates (next to linked_needs etc.) instead of being
duplicated per template. This also removes the id_filter_list copy from the
tool qualification report; the helpers render byte-identical output to the
former macros and are covered by unit tests.

The per-component status and test-coverage pie charts mixed QM and ASIL B Needs in a single figure, so neither safety class could be assessed on its own. Each component now gets two rows of charts: one restricted to ASIL B and one restricted to QM. QM Needs never reach ``valid_inspected``, so the QM status pie only knows ``valid`` and ``invalid``. There ``valid`` is the completed state and is therefore coloured green instead of the yellow "not inspected yet" that it denotes for ASIL B. The ASIL B colours stay as they were. The four near-identical needpie directives are replaced by a single ``safety_pie_row`` macro to keep the two safety classes in sync. The component overview, requirements traceability and architectural elements tables are now sorted by safety, so that all ASIL B entries precede the QM ones.

github-actions · 2026-10-13T12:29:52Z

Documentation preview for this pull request is available at:
pr-946: https://eclipse-score.github.io/docs-as-code/pr-946/

Each QM/ASIL B pie row now carries its own colour-coded rubric instead of only a suffix in the chart title. When a component has no Need in a safety class, the two pies are replaced by placeholder tiles of the same size so the absence is visible rather than rendering blank circles.

Applies the module report's treatment to the platform report: the stakeholder, feature requirement and feature architecture rows are each rendered once per safety class, with a colour-coded rubric and placeholder tiles where a safety class has no Need. The reference-coverage pie is partitioned per safety class as well, so its slices match the row it sits in.

…irst

Sphinx-Needs hardcodes figsize=(8, 4) and anchors the legend to the right of the pie, leaving the circle at ~40% of the image width. Three charts in one row shrank it past readability. Re-laying out the finished figure grows the circle to ~80% and makes the image nearly square, so rows of two and of three now match.

Moving the legend below the pie reshaped every needpie chart in S-CORE, not just the ones in the verification reports. Keep only the report-local grid changes: a smaller gutter and charts scaled to their column, so rows of two and of three come out the same size.

…l-qm # Conflicts: # src/needs_templates/module_verification_report.need

MiniJinja has no template loader, so the macro could not be imported and was duplicated in both templates. Implement it once as a render helper.

Also apply ruff format to the helper.

The macros were duplicated in the module, platform and tool qualification templates. The helpers render byte-identical output.

Copilot

🟡 Changes recommended

QM Needs with the supported valid_inspected status are omitted from status charts.

1 open finding

Include valid_inspected Needs in QM completed status slice · New

What changed in this PR

Separates module and platform verification statistics by safety class so ASIL B and QM can be assessed independently.

Changes:

Adds safety-specific charts, empty-state placeholders, legends, and consistent layouts.
Sorts report tables by safety class.
Extracts shared rendering helpers with documentation and unit tests.

File	Description
src/needs_templates/tool_qualification_report.need	Uses the shared ID-filter helper.
src/needs_templates/platform_verification_report.need	Splits charts and reference coverage by safety class.
src/needs_templates/module_verification_report.need	Splits component charts and updates layout and sorting.
src/extensions/score_sphinx_needs_templates/tests/test_needs_templates.py	Tests shared rendering helpers.
src/extensions/score_sphinx_needs_templates/README.md	Documents the new helpers.
src/extensions/score_sphinx_needs_templates/__init__.py	Implements and registers shared rendering helpers.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

antonkri · 2026-10-15T11:52:49Z

+                f":colors: {status_colors}",
+                ":legend:",
+                "",
+                *(f'{scope} and status == "{state}"' for state in statuses),


Fixed it, we handle valid_inspected as valid for QM requirements.

The QM status pie only had valid and invalid slices, so a QM Need marked valid_inspected was silently dropped from it while the coverage pie still counted it. Treat it as valid, like the metamodel graph checks do.

Puts the green "done" entry first in every status legend and pie, as it already is for QM, test coverage and reference coverage.

The merge of main (a539af8) kept this branch's module_verification_report.need and thereby dropped the ASIL B/QM split merged in #946. Restore main's template; the central metrics stay available but are no longer wired into it. sphinx-needs passes filter-func arguments verbatim after splitting on ",", so the quoted arguments from the template arrived as '"comp__x"' and every pie counted 0. Strip whitespace and quotes. Arguments still must not contain a dot: sphinx-needs resolves the module with rsplit(".", 1) over the whole call, so a version such as v1.0 breaks it. Documented and covered by a test. Also drop the unused version_filter path of calculate_status_and_coverage_metrics, which duplicated req_in_report_version.

The module verification report now takes its status and test-coverage numbers from the shared score_metrics filter functions instead of inline needpie filters, while keeping the ASIL B / QM split from #946. - count_status_filtered / count_coverage_filtered take an optional safety class; get_component_*_metrics accept it as arg4. QM counts valid_inspected as valid, ASIL lists valid_inspected first, matching the existing labels and colours. - The filter functions always count the needs handed in by needpie and no longer read metrics.json, so the report does not depend on it. - sphinx-needs splits the call on its last ".", so the version is passed as v1_0 and decoded back to v1.0. - safety_pie_row gets optional component/category/version arguments that switch the status and coverage pies to these filter functions. The platform report keeps its inline filters. A test renders both variants of safety_pie_row and checks that the filter functions count exactly what the inline filters count, per safety class and with a version scope.

* feat(metrics): add verification metrics for centralized report generation - Extend traceability_metrics_schema.json with verification_metrics section defining per-component status and coverage metrics - Add calculate_status_and_coverage_metrics() and calculate_verification_metrics() to compute status distribution and test coverage per component - Add get_component_status_metrics() and get_component_coverage_metrics() filter functions for needpie directives - Integrate verification_metrics into CALCULATED_METRICS global state This provides a single source of truth for verification report metrics, enabling both CI/CD checks and template consumption via sphinx_filters. * feat(metrics): add version scoping to centralized verification metrics - Add version_filter parameter to calculate_status_and_coverage_metrics() implementing the same valid_from logic as the template's req_in_report_version - Update calculate_verification_metrics() to accept report_version parameter - Update filter functions to lookup version-specific metrics via verification_metrics_v{version} key with fallback to unversioned - Update template to pass module_version to filter functions This ensures centralized metrics match the version-scoped behavior of the original inline needpie filters. * fix(metrics): handle string-or-list for satisfied_by and belongs_to Sphinx-Needs may store single-value links as strings instead of lists. Ensure consistent list handling when checking component membership. * docs: document metrics.json content including verification_metrics Add a section describing the structure of metrics.json output, including the new verification_metrics section for per-component status and coverage data used by verification reports. * style: fix lint and type issues for centralized metrics - Add noqa: C901 for complex functions (inherent complexity from version filtering) - Add proper type hints for verification_metrics return type - Apply ruff-format fixes * feat: make version-scoping helpers public for reuse - Make parse_version, RequirementInReportVersion, and any_req_in_report_version public APIs in score_sphinx_needs_templates - Update sphinx_filters.py to use public req_in_report_version - Fix version scoping in verification report pie charts to properly filter requirements by valid_from when applies_to_module_version is set - Add helper functions to reduce complexity in filter functions - Update tests to use public API instead of vars()[...] access - Add documentation note about duplicate version logic in traceability_metrics Fixes issue where Component Requirements Status/Test Coverage pie charts showed ALL requirements instead of only those in scope for a report version. * fix: add missing Bazel dependency on score_sphinx_needs_templates score_metrics imports req_in_report_version from score_sphinx_needs_templates, but this dependency wasn't declared in the BUILD file, causing ModuleNotFoundError in score_metamodel tests. * move to the helpers * precommit fix * fix lint * copillot findings * feat(metrics): add verification metrics for centralized report generation - Extend traceability_metrics_schema.json with verification_metrics section defining per-component status and coverage metrics - Add calculate_status_and_coverage_metrics() and calculate_verification_metrics() to compute status distribution and test coverage per component - Add get_component_status_metrics() and get_component_coverage_metrics() filter functions for needpie directives - Integrate verification_metrics into CALCULATED_METRICS global state This provides a single source of truth for verification report metrics, enabling both CI/CD checks and template consumption via sphinx_filters. * feat(metrics): add version scoping to centralized verification metrics - Add version_filter parameter to calculate_status_and_coverage_metrics() implementing the same valid_from logic as the template's req_in_report_version - Update calculate_verification_metrics() to accept report_version parameter - Update filter functions to lookup version-specific metrics via verification_metrics_v{version} key with fallback to unversioned - Update template to pass module_version to filter functions This ensures centralized metrics match the version-scoped behavior of the original inline needpie filters. * fix(metrics): handle string-or-list for satisfied_by and belongs_to Sphinx-Needs may store single-value links as strings instead of lists. Ensure consistent list handling when checking component membership. * docs: document metrics.json content including verification_metrics Add a section describing the structure of metrics.json output, including the new verification_metrics section for per-component status and coverage data used by verification reports. * style: fix lint and type issues for centralized metrics - Add noqa: C901 for complex functions (inherent complexity from version filtering) - Add proper type hints for verification_metrics return type - Apply ruff-format fixes * feat: make version-scoping helpers public for reuse - Make parse_version, RequirementInReportVersion, and any_req_in_report_version public APIs in score_sphinx_needs_templates - Update sphinx_filters.py to use public req_in_report_version - Fix version scoping in verification report pie charts to properly filter requirements by valid_from when applies_to_module_version is set - Add helper functions to reduce complexity in filter functions - Update tests to use public API instead of vars()[...] access - Add documentation note about duplicate version logic in traceability_metrics Fixes issue where Component Requirements Status/Test Coverage pie charts showed ALL requirements instead of only those in scope for a report version. * fix: add missing Bazel dependency on score_sphinx_needs_templates score_metrics imports req_in_report_version from score_sphinx_needs_templates, but this dependency wasn't declared in the BUILD file, causing ModuleNotFoundError in score_metamodel tests. * move to the helpers * precommit fix * fix lint * copillot findings * fix centralization * fix: restore the ASIL/QM module report and decode filter-func arguments The merge of main (a539af8) kept this branch's module_verification_report.need and thereby dropped the ASIL B/QM split merged in #946. Restore main's template; the central metrics stay available but are no longer wired into it. sphinx-needs passes filter-func arguments verbatim after splitting on ",", so the quoted arguments from the template arrived as '"comp__x"' and every pie counted 0. Strip whitespace and quotes. Arguments still must not contain a dot: sphinx-needs resolves the module with rsplit(".", 1) over the whole call, so a version such as v1.0 breaks it. Documented and covered by a test. Also drop the unused version_filter path of calculate_status_and_coverage_metrics, which duplicated req_in_report_version. * feat: count the module report pies with the central filter functions The module verification report now takes its status and test-coverage numbers from the shared score_metrics filter functions instead of inline needpie filters, while keeping the ASIL B / QM split from #946. - count_status_filtered / count_coverage_filtered take an optional safety class; get_component_*_metrics accept it as arg4. QM counts valid_inspected as valid, ASIL lists valid_inspected first, matching the existing labels and colours. - The filter functions always count the needs handed in by needpie and no longer read metrics.json, so the report does not depend on it. - sphinx-needs splits the call on its last ".", so the version is passed as v1_0 and decoded back to v1.0. - safety_pie_row gets optional component/category/version arguments that switch the status and coverage pies to these filter functions. The platform report keeps its inline filters. A test renders both variants of safety_pie_row and checks that the filter functions count exactly what the inline filters count, per safety class and with a version scope. * test: satisfy BasedPyright in the component filter tests --------- Co-authored-by: FScholPer <FScholPer@users.noreply.github.com> Co-authored-by: Anton Krivoborodov <anton.krivoborodov@bmw.de>

antonkri requested review from AlexanderLanin, MaximilianSoerenPollak, a-zw, dcalavrezo-qorix and nradakovic as code owners October 13, 2026 09:48

github-project-automation Bot added this to INF - Infrastructure Community Oct 13, 2026

antonkri marked this pull request as draft October 13, 2026 10:43

antonkri mentioned this pull request Oct 13, 2026

DO NOT MERGE: preview larger report pies (docs-as-code#979) eclipse-score/reference_integration#396

Draft

Merge branch 'main' into feat/report-split-asil-qm

b6eb13a

antonkri marked this pull request as ready for review October 13, 2026 12:16

antonkri and others added 12 commits October 13, 2026 14:23

style(platform report): show all three pies in one row

46dbf1c

style(platform report): sort tables by safety class so ASIL B comes f…

bb255e5

…irst

feat: uniform pie size and a chart legend in both verification reports

2380ca3

fix: centre the two-pie rows in the three-column raster

c3a72cc

Merge remote-tracking branch 'origin/main' into feat/report-split-asi…

22ec27e

…l-qm # Conflicts: # src/needs_templates/module_verification_report.need

refactor: share safety_pie_row between the verification reports

d480bc1

MiniJinja has no template loader, so the macro could not be imported and was duplicated in both templates. Implement it once as a render helper.

test: cover the shared safety_pie_row helper

2828fcc

Also apply ruff format to the helper.

refactor: share id_filter_list and workproduct_rows as render helpers

4443cb9

The macros were duplicated in the module, platform and tool qualification templates. The helpers render byte-identical output.

antonkri requested a balanced review from Copilot October 15, 2026 09:58

Copilot started reviewing on behalf of antonkri October 15, 2026 09:58

View session

Copilot AI reviewed Oct 15, 2026

View reviewed changes

antonkri and others added 3 commits October 15, 2026 11:46

fix: count valid_inspected QM Needs as valid in the status pie

fdd8096

The QM status pie only had valid and invalid slices, so a QM Need marked valid_inspected was silently dropped from it while the coverage pie still counted it. Treat it as valid, like the metamodel graph checks do.

Merge branch 'main' into feat/report-split-asil-qm

480123a

style: list valid_inspected before valid for ASIL B

5b99c50

Puts the green "done" entry first in every status legend and pie, as it already is for QM, test coverage and reference coverage.

Merge branch 'main' into feat/report-split-asil-qm

5322f22

nradakovic approved these changes Oct 15, 2026

View reviewed changes

antonkri merged commit d257e2d into main Oct 15, 2026
35 checks passed

antonkri deleted the feat/report-split-asil-qm branch October 15, 2026 13:17

github-project-automation Bot moved this to Done in INF - Infrastructure Community Oct 15, 2026

antonkri mentioned this pull request Oct 16, 2026

docs: use mod_ver_report need type and pin docs-as-code main eclipse-score/baselibs#672

Merged

This was referenced Oct 16, 2026

Feat/centralize verification metrics #949

Merged

feat: larger verification report pies #979

Merged

Navigation Menu

Repository navigation

feat: split module report pies into ASIL and QM - #946

feat: split module report pies into ASIL and QM#946
antonkri merged 18 commits into
mainfrom
feat/report-split-asil-qm

antonkri commented Oct 13, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

antonkri Oct 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

antonkri commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Module verification report

Platform verification report

Preview

Colours

Layout

Table sorting

Implementation note

Uh oh!

github-actions Bot commented Oct 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

antonkri Oct 15, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

antonkri commented Oct 13, 2026 •

edited

Loading