[uss_qualifier] Add ParallelFlightPlannerCombinations, parallel execution of actions generator and use everyting in f3548_21

Author: the-glu

.basedpyright/baseline.json

@@ -4666,14 +4666,6 @@
                     "endColumn": 9,
                     "lineCount": 3
                 }
-            },
-            {
-                "code": "reportArgumentType",
-                "range": {
-                    "startColumn": 20,
-                    "endColumn": 41,
-                    "lineCount": 1
-                }
             }
         ],
         "./monitoring/uss_qualifier/action_generators/interuss/mock_uss/with_locality.py": [

monitoring/uss_qualifier/action_generators/README.md

@@ -3,3 +3,26 @@
 The bulk of uss_qualifier's automated testing logic is contained in [test scenarios](../scenarios/README.md).  A [test suite](../suites/README.md) is essentially a static "playlist" of test actions to perform (test scenarios, action generators, and other test suites), all of which ultimately resolve to test scenarios.  An action generator is essentially a dynamic "playlist" of test actions -- it can generate test actions that vary according to provided resource values, situations, or other conditions only necessarily known at runtime.
 
 For documentation purposes, all action generators must statically declare the test actions they may take.  However, whether each (or any) of these actions will actually be taken at runtime cannot be statically determined in general.
+
+## Parallel execution in action generators
+
+An action generator's `actions()` method yields one of:
+
+- a `TestSuiteAction` — executed sequentially, as before.
+- a `list[list[TestSuiteAction]]` — a *parallel group*. The outer list holds the branches to execute concurrently; each inner list is a sequence of actions executed in order within its own branch.
+
+Example:
+
+```python
+def actions(self) -> Iterator[TestSuiteAction | list[list[TestSuiteAction]]]:
+    yield base_action                               # sequential
+    yield [[A1, A2, A3], [B1, B2, B3]]              # A and B in parallel
+```
+
+When a parallel group is yielded, each branch runs on its own thread. Reports are appended to the parent report in branch order.
+
+### Constraints
+
+Each branch shares the same `Resource` instances unless the action generator hands out distinct ones. If a resource has mutable state that two branches would race on, the generator must produce isolated copies - typically by declaring `ResourceModifier`-based variants and calling `.adjust(index)` for each branch.
+
+If a branch fails with `on_failure: Abort` (or hits a critical problem), the other branches are signalled to stop at the next action boundary. In-progress actions still finish.

monitoring/uss_qualifier/action_generators/flight_planning/README.md

@@ -19,3 +19,13 @@ This action generator accepts a [FlightPlannersResource](../../resources/flight_
 | `ussC`          | `ussC`          | `ExampleTestScenario` |
 
 The usage intent for this action generator is to enable design of simple test scenarios with a small number of participants, but to automatically repeat that simple scenario with all applicable role assignment combinations given a list of flight planner USSs to test.
+
+## `ParallelFlightPlannerCombinations`
+
+Variant of `FlightPlannerCombinations` that runs combinations in parallel where possible.
+
+Same configuration as `FlightPlannerCombinations`. The only difference is scheduling: combinations sharing no flight planner participant are grouped together and executed concurrently. Combinations sharing at least one participant remain in different groups (so no participant is hit by two tests at the same time).
+
+Groups are built greedily (first-fit): each combination is placed in the first existing group with no participant overlap, otherwise a new group is started. This is not minimal in the worst case but the problem is graph coloring, NP-hard.
+
+Each combination receives its own `adjust(index)` variant of every `ResourceModifier` resource in the pool (inherited from `FlightPlannerCombinations`), so parallel branches don't share mutable resource state.

monitoring/uss_qualifier/action_generators/flight_planning/__init__.py

@@ -1 +1,4 @@
 from .planner_combinations import FlightPlannerCombinations as FlightPlannerCombinations
+from .planner_combinations import (
+    ParallelFlightPlannerCombinations as ParallelFlightPlannerCombinations,
+)

monitoring/uss_qualifier/action_generators/flight_planning/planner_combinations.py

@@ -17,6 +17,7 @@
 )
 from monitoring.uss_qualifier.resources.resource import (
     MissingResourceError,
+    ResourceModifier,
     ResourceType,
 )
 from monitoring.uss_qualifier.suites.definitions import TestSuiteActionDeclaration
@@ -40,7 +41,7 @@ class FlightPlannerCombinationsSpecification(ImplicitDict):
 class FlightPlannerCombinations(
     ActionGenerator[FlightPlannerCombinationsSpecification]
 ):
-    _actions: list[TestSuiteAction]
+    _actions_with_participants: list[tuple[TestSuiteAction, frozenset[str]]]
     _current_action: int
 
     @classmethod
@@ -91,22 +92,34 @@ def __init__(
                 "default flight planner combination selector",
             )
 
-        self._actions = []
+        self._actions_with_participants = []
         role_assignments = [0] * len(specification.roles)
+        combination_index = 0
         while True:
             participants = flight_planners_resource.make_subset(role_assignments)
             flight_planners_combination = {
                 k: v for k, v in zip(specification.roles, participants)
             }
 
             if combination_selector.is_valid_combination(flight_planners_combination):
-                modified_resources = {k: v for k, v in resources.items()}
+                modified_resources = {
+                    k: v.adjust(combination_index)
+                    if isinstance(v, ResourceModifier)
+                    else v
+                    for k, v in resources.items()
+                }
                 for k, v in flight_planners_combination.items():
                     modified_resources[k] = v
 
-                self._actions.append(
-                    TestSuiteAction(specification.action_to_repeat, modified_resources)
+                self._actions_with_participants.append(
+                    (
+                        TestSuiteAction(
+                            specification.action_to_repeat, modified_resources
+                        ),
+                        frozenset(p.participant_id for p in participants),
+                    )
                 )
+                combination_index += 1
 
             index_to_increment = len(role_assignments) - 1
             while index_to_increment >= 0:
@@ -121,5 +134,32 @@ def __init__(
 
         self._current_action = 0
 
-    def actions(self) -> Iterator[TestSuiteAction]:
-        yield from self._actions
+    def actions(
+        self,
+    ) -> Iterator[TestSuiteAction] | Iterator[list[list[TestSuiteAction]]]:
+        for action, _ in self._actions_with_participants:
+            yield action
+
+
+class ParallelFlightPlannerCombinations(FlightPlannerCombinations):
+    """Like FlightPlannerCombinations, but yields actions grouped so actions
+    sharing no participant run in parallel."""
+
+    @classmethod
+    def get_name(cls) -> str:
+        return "For each appropriate combination of flight planner(s), in parallel where possible"
+
+    def actions(self) -> Iterator[list[list[TestSuiteAction]]]:
+        # Greedy first-fit grouping
+        groups: list[list[tuple[TestSuiteAction, frozenset[str]]]] = []
+        for action, participants in self._actions_with_participants:
+            for group in groups:
+                used = frozenset().union(*(p for _, p in group))
+                if used.isdisjoint(participants):
+                    group.append((action, participants))
+                    break
+            else:
+                groups.append([(action, participants)])
+
+        for group in groups:
+            yield [[action] for action, _ in group]

monitoring/uss_qualifier/configurations/dev/f3548_self_contained.yaml

@@ -20,9 +20,13 @@ v1:
           flight_planners: flight_planners
           flight_planners_to_clear: flight_planners
           conflicting_flights: conflicting_flights
+          conflicting_flights_parallel: conflicting_flights_parallel
           priority_preemption_flights: conflicting_flights
+          priority_preemption_flights_parallel: conflicting_flights_parallel
           invalid_flight_intents: invalid_flight_intents
+          invalid_flight_intents_parallel: invalid_flight_intents_parallel
           non_conflicting_flights: non_conflicting_flights
+          non_conflicting_flights_parallel: non_conflicting_flights_parallel
           dss: dss
           dss_instances: dss_instances
           mock_uss: mock_uss
@@ -250,6 +254,11 @@ v1:
                   # Therefore, ground level is at roughly 93m above the WGS84 ellipsoid
                   meters_up: 93
 
+        conflicting_flights_parallel:
+          resource_type: resources.flight_planning.FlightIntentsModifier
+          dependencies:
+            base_resource: conflicting_flights
+
         # Details of flights with invalid operational intents (used in flight intent validation scenario)
         invalid_flight_intents:
           resource_type: resources.flight_planning.FlightIntentsResource
@@ -262,6 +271,12 @@ v1:
                   degrees_east: -96.7587
                   meters_up: 93
 
+        invalid_flight_intents_parallel:
+          resource_type: resources.flight_planning.FlightIntentsModifier
+          dependencies:
+            base_resource: invalid_flight_intents
+
+
         # Details of non-conflicting flights (used in data validation scenario)
         non_conflicting_flights:
           resource_type: resources.flight_planning.FlightIntentsResource
@@ -275,6 +290,11 @@ v1:
                   degrees_east: -96.7587
                   meters_up: 93
 
+        non_conflicting_flights_parallel:
+          resource_type: resources.flight_planning.FlightIntentsModifier
+          dependencies:
+            base_resource: non_conflicting_flights
+
     # How to execute a test run using this configuration
     execution:
       # Since we want to stop execution immediately if there are any unexpected failed checks, we set this parameter to

monitoring/uss_qualifier/configurations/dev/library/resources.yaml

@@ -347,6 +347,11 @@ che_conflicting_flights:
           degrees_east: 7.4774
           meters_up: 605
 
+che_conflicting_flights_parallel:
+  resource_type: resources.flight_planning.FlightIntentsModifier
+  dependencies:
+    base_resource: che_conflicting_flights
+
 che_invalid_flight_intents:
   $content_schema: monitoring/uss_qualifier/resources/definitions/ResourceDeclaration.json
   resource_type: resources.flight_planning.FlightIntentsResource
@@ -360,6 +365,11 @@ che_invalid_flight_intents:
           degrees_east: 7.4774
           meters_up: 605
 
+che_invalid_flight_intents_parallel:
+  resource_type: resources.flight_planning.FlightIntentsModifier
+  dependencies:
+    base_resource: che_invalid_flight_intents
+
 che_general_flight_auth_flights:
   $content_schema: monitoring/uss_qualifier/resources/definitions/ResourceDeclaration.json
   resource_type: resources.flight_planning.FlightIntentsResource
@@ -381,6 +391,11 @@ che_non_conflicting_flights:
           degrees_east: 7.4774
           meters_up: 605
 
+che_non_conflicting_flights_parallel:
+  resource_type: resources.flight_planning.FlightIntentsModifier
+  dependencies:
+    base_resource: che_non_conflicting_flights
+
 # ===== General flight authorization =====
 
 example_flight_check_table:

monitoring/uss_qualifier/configurations/dev/message_signing.yaml

@@ -3,8 +3,11 @@ v1:
     resources:
       resource_declarations:
         che_conflicting_flights: {$ref: 'library/resources.yaml#/che_conflicting_flights'}
+        che_conflicting_flights_parallel: {$ref: 'library/resources.yaml#/che_conflicting_flights_parallel'}
         che_invalid_flight_intents: {$ref: 'library/resources.yaml#/che_invalid_flight_intents'}
+        che_invalid_flight_intents_parallel: {$ref: 'library/resources.yaml#/che_invalid_flight_intents_parallel'}
         che_non_conflicting_flights: {$ref: 'library/resources.yaml#/che_non_conflicting_flights'}
+        che_non_conflicting_flights_parallel: {$ref: 'library/resources.yaml#/che_non_conflicting_flights_parallel'}
         che_problematically_big_area: {$ref: 'library/resources.yaml#/che_problematically_big_area'}
         che_planning_area_volume: {$ref: 'library/resources.yaml#/che_planning_area_volume'}
         che_planning_area: {$ref: 'library/resources.yaml#/che_planning_area'}
@@ -55,9 +58,13 @@ v1:
           flight_planners: flight_planners
           combination_selector: combination_selector
           conflicting_flights: che_conflicting_flights
+          conflicting_flights_parallel: che_conflicting_flights_parallel
           invalid_flight_intents: che_invalid_flight_intents
+          invalid_flight_intents_parallel: che_invalid_flight_intents_parallel
           non_conflicting_flights: che_non_conflicting_flights
+          non_conflicting_flights_parallel: che_non_conflicting_flights_parallel
           priority_preemption_flights: che_conflicting_flights
+          priority_preemption_flights_parallel: che_conflicting_flights_parallel
           dss: scd_dss
           dss_instances: scd_dss_instances
           id_generator: id_generator

monitoring/uss_qualifier/configurations/dev/uspace.yaml

@@ -5,9 +5,12 @@ v1:
       resource_declarations:
         locality_che: {$ref: 'library/resources.yaml#/locality_che'}
         che_conflicting_flights: {$ref: 'library/resources.yaml#/che_conflicting_flights'}
+        che_conflicting_flights_parallel: {$ref: 'library/resources.yaml#/che_conflicting_flights_parallel'}
         che_invalid_flight_intents: {$ref: 'library/resources.yaml#/che_invalid_flight_intents'}
+        che_invalid_flight_intents_parallel: {$ref: 'library/resources.yaml#/che_invalid_flight_intents_parallel'}
         che_invalid_flight_auth_flights: {$ref: 'library/resources.yaml#/che_invalid_flight_auth_flights'}
         che_non_conflicting_flights: {$ref: 'library/resources.yaml#/che_non_conflicting_flights'}
+        che_non_conflicting_flights_parallel: {$ref: 'library/resources.yaml#/che_non_conflicting_flights_parallel'}
         che_planning_area_volume: {$ref: 'library/resources.yaml#/che_planning_area_volume'}
         che_planning_area: {$ref: 'library/resources.yaml#/che_planning_area'}
         netrid_observation_evaluation_configuration: {$ref: 'library/resources.yaml#/netrid_observation_evaluation_configuration'}
@@ -59,10 +62,14 @@ v1:
           prod_env_version_providers: prod_env_version_providers?
 
           conflicting_flights: che_conflicting_flights
+          conflicting_flights_parallel: che_conflicting_flights_parallel
           priority_preemption_flights: che_conflicting_flights
+          priority_preemption_flights_parallel: che_conflicting_flights_parallel
           invalid_flight_intents: che_invalid_flight_intents
+          invalid_flight_intents_parallel: che_invalid_flight_intents_parallel
           invalid_flight_auth_flights: che_invalid_flight_auth_flights
           non_conflicting_flights: che_non_conflicting_flights
+          non_conflicting_flights_parallel: che_non_conflicting_flights_parallel
           flight_planners: all_flight_planners?
           mock_uss: mock_uss_instance_uss6
           mock_uss_dp: mock_uss_instance_dp
@@ -96,10 +103,14 @@ v1:
                 prod_env_version_providers: prod_env_version_providers?
 
                 conflicting_flights: conflicting_flights
+                conflicting_flights_parallel: conflicting_flights_parallel
                 priority_preemption_flights: priority_preemption_flights
+                priority_preemption_flights_parallel: priority_preemption_flights_parallel
                 invalid_flight_intents: invalid_flight_intents
+                invalid_flight_intents_parallel: invalid_flight_intents_parallel
                 invalid_flight_auth_flights: invalid_flight_auth_flights
                 non_conflicting_flights: non_conflicting_flights
+                non_conflicting_flights_parallel: non_conflicting_flights_parallel
                 flight_planners: flight_planners?
                 mock_uss: mock_uss
                 mock_uss_dp: mock_uss_dp

monitoring/uss_qualifier/configurations/dev/utm_implementation_us/definitions/baseline_a.libsonnet

@@ -23,9 +23,13 @@ function(env) {
             flight_planners: 'flight_planners',
             flight_planners_to_clear: 'flight_planners_to_clear',
             conflicting_flights: 'conflicting_flights',
+            conflicting_flights_parallel: 'conflicting_flights_parallel',
             priority_preemption_flights: 'conflicting_flights',
+            priority_preemption_flights_parallel: 'conflicting_flights_parallel',
             invalid_flight_intents: 'invalid_flight_intents',
+            invalid_flight_intents_parallel: 'invalid_flight_intents_parallel',
             non_conflicting_flights: 'non_conflicting_flights',
+            non_conflicting_flights_parallel: 'non_conflicting_flights_parallel',
             test_exclusions: 'test_exclusions',
             dss: 'dss',
             dss_instances: 'dss_instances',
@@ -205,6 +209,13 @@ function(env) {
             },
           },
 
+          conflicting_flights_parallel: {
+            resource_type: 'resources.flight_planning.FlightIntentsModifier',
+            dependencies: {
+              base_resource: 'conflicting_flights',
+            },
+          },
+
           // Details of flights with invalid operational intents (used in flight intent validation scenario)
           invalid_flight_intents: {
             resource_type: 'resources.flight_planning.FlightIntentsResource',
@@ -224,6 +235,13 @@ function(env) {
             },
           },
 
+          invalid_flight_intents_parallel: {
+            resource_type: 'resources.flight_planning.FlightIntentsModifier',
+            dependencies: {
+              base_resource: 'invalid_flight_intents',
+            },
+          },
+
           // Details of non-conflicting flights (used in data validation scenario)
           non_conflicting_flights: {
             resource_type: 'resources.flight_planning.FlightIntentsResource',
@@ -243,6 +261,13 @@ function(env) {
             },
           },
 
+          non_conflicting_flights_parallel: {
+            resource_type: 'resources.flight_planning.FlightIntentsModifier',
+            dependencies: {
+              base_resource: 'non_conflicting_flights',
+            },
+          },
+
           // Name of the system under test for which the system version should be obtained from participants who provide version information
           system_identity: {
             resource_type: 'resources.versioning.SystemIdentityResource',