From 24af3ed4e00029f01672ae9a8ec97a71def5b893 Mon Sep 17 00:00:00 2001 From: Dan Wolfson Date: Fri, 23 Oct 2026 08:42:24 -0400 Subject: [PATCH 1/3] docs(issues): ISSUE-162 Update Glossary Term resets four boolean flags to False Co-Authored-By: Claude Opus 5.5 Signed-off-by: Dan Wolfson --- PYEGERIA_ISSUES.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/PYEGERIA_ISSUES.md b/PYEGERIA_ISSUES.md index 3c267f2f..122b4792 100644 --- a/PYEGERIA_ISSUES.md +++ b/PYEGERIA_ISSUES.md @@ -286,6 +286,23 @@ they're holding real port materialization until this exists server-side --- +### ISSUE-162: Dr.Egeria "Update Glossary Term" resets isAbstractConcept / isActivityDescription / isContext / isDataValue to False whenever the document doesn't mention them + +**Layer:** Pyegeria (Dr.Egeria) · **Status:** open, logged only (no fix without the owner's word) · **Found:** 2026-10-23 by RE's question-term update batch builder, reading `md_processing/v2` (RE term batch fix round 3). Reported via the Thursday Coordinator; confirmed here in source on `main` (6.2.9), not run live. + +**What:** `TermProcessor.apply_changes` (`md_processing/v2/glossary.py:145-148`) builds the term properties with `attributes.get('Is Abstract Concept', {}).get('value', False)`, and likewise for `Is Activity Description`, `Is Context` and `Is Data Value`. On update, the body goes through `filter_update_properties` (`md_processing/v2/processors.py:1957`). For a merge update that drops only `None` values, so the `False` defaults are always sent. Every "Update Glossary Term", even one that changes only the description, therefore writes `False` to all four flags and clears any that were `True`. + +**Same pattern, likely affected (not confirmed):** `md_processing/v2/data_designer.py:54-55` (`DataValueSpecificationProcessor`) and `:547-549` (`DataClassProcessor`) default `allowsDuplicateValues` / `isNullable` to `True`. Both processors appear to build one property body for create and update, so an unrelated update would reset those to `True`. Other `.get('value', )` uses checked in `md_processing` are request options (`forLineage`, `startsWith`, ...) or create-only values, not stored properties. + +**Impact:** a term with any of the four flags set loses it on an unrelated update. RE's 96-term question update batch will check the terms before applying. + +**Proposed fix (not made):** +- Default these attributes to `None` (`.get('value')`), so a merge update omits them unless the document sets them. Keep `False` / `True` only on the create path, or leave creation to Egeria's own defaults. +- Check the Data Designer cases the same way. +- Add a micro-test: an update with no flag attributes sends none of the four keys. + +--- + ### ISSUE-161: Dr.Egeria's two entry points resolve the Egeria user differently, and the CLI fixes its credential defaults at import time **Layer:** Pyegeria · **Status:** fixed on branch `fix/issue-161-dr-egeria-credentials` (owner approved 2026-10-21), not yet released · **Found:** 2026-10-21, while checking an RE report that `dr_egeria --process` ignored `EGERIA_USER`. **That report is withdrawn; RE was the cause:** RE's own `config.py:316` defaulted its daemon identity (`EGERIA_USER_ID`) to "erinoverview", so RE handed the child process erinoverview. The CLI correctly used it. RE is removing that default. From aec2755986ca071d246a4d4505cfa38995d901ee Mon Sep 17 00:00:00 2001 From: Dan Wolfson Date: Fri, 23 Oct 2026 09:33:32 -0400 Subject: [PATCH 2/3] fix: Dr.Egeria updates no longer reset properties the document doesn't set (ISSUE-162) Update Glossary Term sent isAbstractConcept/isActivityDescription/ isContext/isDataValue = False whenever the document omitted them, and Update Data Value Specification / Data Class sent allowsDuplicateValues / isNullable = True plus empty lists/maps/0 for matchPropertyNames, matchThreshold, specificationDetails, sampleValues, dataPatterns and additionalProperties. Merge updates dropped only None, so these overwrote existing values. They now default to None (omitted on a merge update, which Egeria defaults to); the old defaults are applied only on Create, so creation is unchanged. Micro-tests: tests/micro-tests/test_issue_162_update_does_not_reset_properties.py (update cases fail on the old code). Full micro-test suite passes in a clean python:3.13-slim container. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Dan Wolfson --- md_processing/v2/data_designer.py | 57 ++++++--- md_processing/v2/glossary.py | 13 +- ...ue_162_update_does_not_reset_properties.py | 116 ++++++++++++++++++ 3 files changed, 166 insertions(+), 20 deletions(-) create mode 100644 tests/micro-tests/test_issue_162_update_does_not_reset_properties.py diff --git a/md_processing/v2/data_designer.py b/md_processing/v2/data_designer.py index de2a87f7..53646e81 100644 --- a/md_processing/v2/data_designer.py +++ b/md_processing/v2/data_designer.py @@ -17,6 +17,29 @@ from pyegeria.core.utils import body_slimmer # --- DataValueSpecificationProcessor --- + +# ISSUE-162: defaults applied only when creating a data value specification / data class. +# On update, unset attributes stay None and are dropped (Egeria's mergeUpdate defaults to +# true), so an update no longer resets these properties. +_DATA_VALUE_SPEC_CREATE_DEFAULTS = { + "matchPropertyNames": [], + "matchThreshold": 0, + "specificationDetails": {}, + "allowsDuplicateValues": True, + "isNullable": True, + "sampleValues": [], + "dataPatterns": [], + "additionalProperties": {}, +} + + +def _with_create_defaults(props: dict) -> dict: + out = dict(props) + for key, default in _DATA_VALUE_SPEC_CREATE_DEFAULTS.items(): + if key in out and out[key] is None: + out[key] = default + return out + class DataValueSpecificationProcessor(AsyncBaseCommandProcessor): """ Processor for Data Value Specification create/update with upsert logic. @@ -43,24 +66,24 @@ async def apply_changes(self) -> str: "displayName": display_name, "description": attributes.get('Description', {}).get('value'), "namespacePath": attributes.get('Namespace Path', {}).get('value'), - "matchPropertyNames": attributes.get('Match Property Names', {}).get('value', []), - "matchThreshold": attributes.get('Match Threshold', {}).get('value', 0), + "matchPropertyNames": attributes.get('Match Property Names', {}).get('value'), + "matchThreshold": attributes.get('Match Threshold', {}).get('value'), "specification": attributes.get('Specification', {}).get('value'), - "specificationDetails": attributes.get('Specification Details', {}).get('value', {}), + "specificationDetails": attributes.get('Specification Details', {}).get('value'), "dataType": attributes.get('Data Type', {}).get('value'), "units": attributes.get('Units', {}).get('value'), "absoluteUncertainty": attributes.get('Absolute Uncertainty', {}).get('value'), "relativeUncertainty": attributes.get('Relative Uncertainty', {}).get('value'), - "allowsDuplicateValues": attributes.get('Allow Duplicates', {}).get('value', True), - "isNullable": attributes.get('Is Nullable', {}).get('value', True), + "allowsDuplicateValues": attributes.get('Allow Duplicates', {}).get('value'), + "isNullable": attributes.get('Is Nullable', {}).get('value'), "defaultValue": attributes.get('Default Value', {}).get('value'), "averageValue": attributes.get('Average Value', {}).get('value'), "valueList": attributes.get('Value List', {}).get('value'), "valueRangeFrom": attributes.get('Value Range From', {}).get('value'), "valueRangeTo": attributes.get('Value Range To', {}).get('value'), - "sampleValues": attributes.get('Sample Values', {}).get('value', []), - "dataPatterns": attributes.get('Data Patterns', {}).get('value', []), - "additionalProperties": attributes.get('Additional Properties', {}).get('value', {}) + "sampleValues": attributes.get('Sample Values', {}).get('value'), + "dataPatterns": attributes.get('Data Patterns', {}).get('value'), + "additionalProperties": attributes.get('Additional Properties', {}).get('value') } # "In Data Value Specification"/"Specializes Data Value Specification" @@ -98,6 +121,7 @@ async def apply_changes(self) -> str: return await self.render_result_markdown(guid) elif verb == "Create": + props = _with_create_defaults(props) self.last_body = body = {"class": "NewElementRequestBody", "properties": props} raw_guid = await self.client.data_designer._async_create_data_value_specification(body) guid = self.extract_guid_or_raise(raw_guid, "Create Data Value Specification") @@ -539,22 +563,22 @@ async def apply_changes(self) -> str: "displayName": display_name, "description": attributes.get('Description', {}).get('value'), "namespacePath": attributes.get('Namespace', {}).get('value'), - "matchPropertyNames": attributes.get('Match Property Names', {}).get('value', []), - "matchThreshold": attributes.get('Match Threshold', {}).get('value', 0), + "matchPropertyNames": attributes.get('Match Property Names', {}).get('value'), + "matchThreshold": attributes.get('Match Threshold', {}).get('value'), "specification": attributes.get('Specification', {}).get('value'), - "specificationDetails": attributes.get('Specification Details', {}).get('value', {}), + "specificationDetails": attributes.get('Specification Details', {}).get('value'), "dataType": attributes.get('Data Type', {}).get('value'), - "allowsDuplicateValues": attributes.get('Allow Duplicate Values', {}).get('value', True), + "allowsDuplicateValues": attributes.get('Allow Duplicate Values', {}).get('value'), "isCaseSensitive": attributes.get('Is Case Sensitive', {}).get('value'), - "isNullable": attributes.get('Is Nullable', {}).get('value', True), + "isNullable": attributes.get('Is Nullable', {}).get('value'), "defaultValue": attributes.get('Default Value', {}).get('value'), "averageValue": attributes.get('Average Value', {}).get('value'), "valueList": attributes.get('Value List', {}).get('value'), "valueRangeFrom": attributes.get('Value Range From', {}).get('value'), "valueRangeTo": attributes.get('Value Range To', {}).get('value'), - "sampleValues": attributes.get('Sample Values', {}).get('value', []), - "dataPatterns": attributes.get('Data Patterns', {}).get('value', []), - "additionalProperties": attributes.get('Additional Properties', {}).get('value', {}) + "sampleValues": attributes.get('Sample Values', {}).get('value'), + "dataPatterns": attributes.get('Data Patterns', {}).get('value'), + "additionalProperties": attributes.get('Additional Properties', {}).get('value') } # 2. Relationships @@ -600,6 +624,7 @@ async def apply_changes(self) -> str: return await self.render_result_markdown(guid) elif verb == "Create": + props = _with_create_defaults(props) self.last_body = body = {"class": "NewElementRequestBody", "properties": props} raw_guid = await self.client.data_designer._async_create_data_class(body) guid = self.extract_guid_or_raise(raw_guid, "Create Data Class") diff --git a/md_processing/v2/glossary.py b/md_processing/v2/glossary.py index 1db6a5ee..f1c2e421 100644 --- a/md_processing/v2/glossary.py +++ b/md_processing/v2/glossary.py @@ -142,10 +142,10 @@ async def apply_changes(self) -> str: prop_body["user_defined_status"] = attributes.get('UserDefinedStatus', {}).get('value', None) prop_body["contextDescription"] = attributes.get('Context Description', {}).get('value', None) prop_body["contextScope"] = attributes.get('Context Scope', {}).get('value', None) - prop_body["isAbstractConcept"] = attributes.get('Is Abstract Concept', {}).get('value', False) - prop_body["isActivityDescription"] = attributes.get('Is Activity Description', {}).get('value', False) - prop_body["isContext"] = attributes.get('Is Context', {}).get('value', False) - prop_body["isDataValue"] = attributes.get('Is Data Value', {}).get('value', False) + prop_body["isAbstractConcept"] = attributes.get('Is Abstract Concept', {}).get('value') + prop_body["isActivityDescription"] = attributes.get('Is Activity Description', {}).get('value') + prop_body["isContext"] = attributes.get('Is Context', {}).get('value') + prop_body["isDataValue"] = attributes.get('Is Data Value', {}).get('value') prop_body["termActivityType"] = attributes.get('Term Activity Type', {}).get('value', None) # 2. Extract collection GUIDs @@ -193,6 +193,11 @@ async def apply_changes(self) -> str: elif verb == "Create": self.last_body = body = set_create_body(om_type or "GlossaryTerm", attributes) + # ISSUE-162: the term flags default to False only on create; on update an unset + # flag stays None so filter_update_properties drops it and Egeria keeps the value. + for flag in ("isAbstractConcept", "isActivityDescription", "isContext", "isDataValue"): + if prop_body.get(flag) is None: + prop_body[flag] = False body["properties"] = prop_body # Anchor Scope check diff --git a/tests/micro-tests/test_issue_162_update_does_not_reset_properties.py b/tests/micro-tests/test_issue_162_update_does_not_reset_properties.py new file mode 100644 index 00000000..762817a1 --- /dev/null +++ b/tests/micro-tests/test_issue_162_update_does_not_reset_properties.py @@ -0,0 +1,116 @@ +""" +ISSUE-162: an Update that doesn't mention a property must not change it. +- Update Glossary Term sent isAbstractConcept/isActivityDescription/isContext/isDataValue + = False whenever the document omitted them. +- Update Data Value Specification / Data Class sent allowsDuplicateValues/isNullable = True + and empty lists/maps/0 for several properties, so an update reset them. +Defaults now apply only on Create. No live server: the client is mocked. +""" +from unittest.mock import AsyncMock + +import pytest + +from md_processing.v2.data_designer import DataClassProcessor, DataValueSpecificationProcessor +from md_processing.v2.extraction import DrECommand +from md_processing.v2.glossary import TermProcessor + +GUID = "11111111-1111-1111-1111-111111111111" +TERM_FLAGS = ("isAbstractConcept", "isActivityDescription", "isContext", "isDataValue") +DV_DEFAULTED = { + "matchPropertyNames": [], "matchThreshold": 0, "specificationDetails": {}, + "allowsDuplicateValues": True, "isNullable": True, "sampleValues": [], + "dataPatterns": [], "additionalProperties": {}, +} + + +def _processor(cls, verb, object_type, attributes, client, om_type=None, guid=None): + cmd = DrECommand(verb=verb, object_type=object_type, attributes=attributes, raw_block=f"{verb} {object_type}") + proc = cls(client, cmd) + proc.parsed_output = {"attributes": attributes, "qualified_name": attributes["Qualified Name"]["value"], + "valid": True} + if guid: + proc.parsed_output["guid"] = guid + if om_type: + proc.get_command_spec = lambda: {"OM_TYPE": om_type} + proc.render_result_markdown = AsyncMock(return_value="# Result") + return proc + + +def _term_attrs(**extra): + attrs = {"Display Name": {"value": "T"}, "Qualified Name": {"value": "GlossaryTerm::T"}, + "Description": {"value": "new description"}} + attrs.update(extra) + return attrs + + +async def test_term_update_without_flags_sends_none_of_them(): + client = AsyncMock() + client._async_get_related_elements.return_value = [] + proc = _processor(TermProcessor, "Update", "Glossary Term", _term_attrs(), client, guid=GUID) + await proc.apply_changes() + + body = client._async_update_glossary_term.call_args.args[1] + for flag in TERM_FLAGS: + assert flag not in body["properties"], flag + assert body["properties"]["description"] == "new description" + + +async def test_term_update_sends_a_flag_the_document_sets(): + client = AsyncMock() + client._async_get_related_elements.return_value = [] + proc = _processor(TermProcessor, "Update", "Glossary Term", _term_attrs(**{"Is Context": {"value": True}}), + client, guid=GUID) + await proc.apply_changes() + + props = client._async_update_glossary_term.call_args.args[1]["properties"] + assert props["isContext"] is True + assert "isAbstractConcept" not in props + + +async def test_term_create_still_defaults_flags_to_false(): + client = AsyncMock() + client._async_create_glossary_term.return_value = GUID + proc = _processor(TermProcessor, "Create", "Glossary Term", _term_attrs(), client) + await proc.apply_changes() + + call = client._async_create_glossary_term.call_args + props = (call.kwargs.get("body") or call.args[0])["properties"] + for flag in TERM_FLAGS: + assert props[flag] is False, flag + + +def _dv_attrs(): + return {"Display Name": {"value": "D"}, "Qualified Name": {"value": "DataValueSpecification::D"}, + "Description": {"value": "changed"}} + + +@pytest.mark.parametrize("cls, om_type", [(DataValueSpecificationProcessor, "DataValueSpecification"), + (DataClassProcessor, "DataClass")]) +async def test_data_designer_update_does_not_send_defaults(cls, om_type): + client = AsyncMock() + proc = _processor(cls, "Update", "Data Value Specification", _dv_attrs(), client, om_type=om_type, guid=GUID) + proc._sync_value_spec_parent = AsyncMock() + proc._sync_all_rels = AsyncMock() + await proc.apply_changes() + + props = client.data_designer._async_update_data_value_specification.call_args.args[1]["properties"] + for key in DV_DEFAULTED: + # None is dropped by body_slimmer before sending; anything else would overwrite Egeria's value + assert props.get(key) is None, key + assert props["description"] == "changed" + + +@pytest.mark.parametrize("cls, om_type, create_method", [ + (DataValueSpecificationProcessor, "DataValueSpecification", "_async_create_data_value_specification"), + (DataClassProcessor, "DataClass", "_async_create_data_class")]) +async def test_data_designer_create_keeps_defaults(cls, om_type, create_method): + client = AsyncMock() + getattr(client.data_designer, create_method).return_value = GUID + proc = _processor(cls, "Create", "Data Value Specification", _dv_attrs(), client, om_type=om_type) + proc._sync_value_spec_parent = AsyncMock() + proc._sync_all_rels = AsyncMock() + await proc.apply_changes() + + props = getattr(client.data_designer, create_method).call_args.args[0]["properties"] + for key, default in DV_DEFAULTED.items(): + assert props[key] == default, key From 4ac8859072849c5c12cf6df577f2eae858e14056 Mon Sep 17 00:00:00 2001 From: Dan Wolfson Date: Fri, 23 Oct 2026 09:33:52 -0400 Subject: [PATCH 3/3] docs(issues): ISSUE-162 fixed (glossary term flags and Data Designer properties) Co-Authored-By: Claude Opus 5.5 Signed-off-by: Dan Wolfson --- PYEGERIA_ISSUES.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/PYEGERIA_ISSUES.md b/PYEGERIA_ISSUES.md index 122b4792..98660fc7 100644 --- a/PYEGERIA_ISSUES.md +++ b/PYEGERIA_ISSUES.md @@ -288,7 +288,7 @@ they're holding real port materialization until this exists server-side ### ISSUE-162: Dr.Egeria "Update Glossary Term" resets isAbstractConcept / isActivityDescription / isContext / isDataValue to False whenever the document doesn't mention them -**Layer:** Pyegeria (Dr.Egeria) · **Status:** open, logged only (no fix without the owner's word) · **Found:** 2026-10-23 by RE's question-term update batch builder, reading `md_processing/v2` (RE term batch fix round 3). Reported via the Thursday Coordinator; confirmed here in source on `main` (6.2.9), not run live. +**Layer:** Pyegeria (Dr.Egeria) · **Status:** fixed on branch `fix/issue-162-update-resets-booleans` (owner approved 2026-10-23), not yet released · **Found:** 2026-10-23 by RE's question-term update batch builder, reading `md_processing/v2` (RE term batch fix round 3). Reported via the Thursday Coordinator; confirmed here in source on `main` (6.2.9), not run live. **What:** `TermProcessor.apply_changes` (`md_processing/v2/glossary.py:145-148`) builds the term properties with `attributes.get('Is Abstract Concept', {}).get('value', False)`, and likewise for `Is Activity Description`, `Is Context` and `Is Data Value`. On update, the body goes through `filter_update_properties` (`md_processing/v2/processors.py:1957`). For a merge update that drops only `None` values, so the `False` defaults are always sent. Every "Update Glossary Term", even one that changes only the description, therefore writes `False` to all four flags and clears any that were `True`. @@ -296,7 +296,12 @@ they're holding real port materialization until this exists server-side **Impact:** a term with any of the four flags set loses it on an unrelated update. RE's 96-term question update batch will check the terms before applying. -**Proposed fix (not made):** +**Fix (2026-10-23):** +- **Glossary terms:** the four flags now default to `None`, so a merge update omits them unless the document sets them. Create still sets them to `False` as before. +- **Data Designer: confirmed and wider than first logged.** Both `DataValueSpecificationProcessor` and `DataClassProcessor` updates send their property body straight to `_async_update_element_body_request`, where `body_slimmer` drops only `None`. So an unrelated update also reset `matchPropertyNames`, `sampleValues` and `dataPatterns` to `[]`, `matchThreshold` to `0`, and `specificationDetails` and `additionalProperties` to `{}`, besides `allowsDuplicateValues` / `isNullable` to `True`. All of these now default to `None`, and the old defaults are applied only on Create (`_with_create_defaults`). Egeria's `mergeUpdate` defaults to `true` (`UpdateOptions.java:23`), so omitted properties keep their values. +- Tests: `tests/micro-tests/test_issue_162_update_does_not_reset_properties.py`, 7 cases. The 4 update cases fail on the old code; the 3 create cases guard the unchanged create defaults. The full micro-test suite (1,436) passes in a clean python:3.13-slim container. Not live-tested. + +**Originally proposed:** - Default these attributes to `None` (`.get('value')`), so a merge update omits them unless the document sets them. Keep `False` / `True` only on the create path, or leave creation to Egeria's own defaults. - Check the Data Designer cases the same way. - Add a micro-test: an update with no flag attributes sends none of the four keys.