From 00ba475ddf17826d27d1f4304315d41d46ae0b8d Mon Sep 17 00:00:00 2001 From: Dan Wolfson Date: Wed, 21 Oct 2026 16:41:14 -0500 Subject: [PATCH 1/2] docs(issues): ISSUE-161 Dr.Egeria entry points resolve the Egeria user differently Co-Authored-By: Claude Opus 5.5 Signed-off-by: Dan Wolfson --- PYEGERIA_ISSUES.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/PYEGERIA_ISSUES.md b/PYEGERIA_ISSUES.md index 22e2dafa..d5526cf3 100644 --- a/PYEGERIA_ISSUES.md +++ b/PYEGERIA_ISSUES.md @@ -286,6 +286,29 @@ they're holding real port materialization until this exists server-side --- +### ISSUE-161: Dr.Egeria's two entry points resolve the Egeria user differently, and the CLI fixes its credential defaults at import time + +**Layer:** Pyegeria · **Status:** open, logged only (no fix without the owner's word) · **Found:** 2026-10-21, while checking an RE report that `dr_egeria --process` ignored `EGERIA_USER`. **That report is withdrawn; RE was the cause:** RE's own `config.py:316` defaulted its daemon identity (`EGERIA_USER_ID`) to "erinoverview", so RE handed the child process erinoverview. The CLI correctly used it. RE is removing that default. + +**What was checked:** with `EGERIA_USER` set, the CLI path resolves and uses it throughout on 6.2.8: +- `config.py:423` (`os.getenv("EGERIA_USER", config.json / .env value)`) feeds `settings.User_Profile.user_name`, the `--userid` default; +- `EgeriaTech` / `BasePlatformClient` keep that `user_id`; +- `create_egeria_bearer_token()` with no arguments uses `self.user_id`; +- `_get_subclient` passes the user and token to every sub-client, and there is no token cache. +Intended precedence (CLAUDE.md): explicit `--userid` > OS env > `.env` > `config.json` > default. + +**Real issues (pyegeria):** +1. **Inconsistent resolution between entry points:** `md_processing/dr_egeria.py:93` (module constants) and `:923` (its own `__main__` path) read `os.environ.get("EGERIA_USER", "erinoverview")` / `EGERIA_USER_PASSWORD` directly, bypassing `.env` and `config.json`. `md_processing/md_processing_utils/common_md_utils.py:29-30` does the same. So `python -m md_processing.dr_egeria` and the `dr_egeria` CLI disagree whenever the user is set only in `.env` or `config.json`. +2. **Import-time defaults:** `commands/cat/dr_egeria.py:32-33` computes the `--userid` / `--user_pass` defaults when the module is imported. Code that imports it, then sets `EGERIA_USER` and invokes the command in the same process, gets the earlier user. This doesn't affect normal CLI use, where each run is a new process. +3. **Help text:** "Overrides EGERIA_USER (env var or .env file)" doesn't mention `config.json`, or that the default is fixed at import. + +**Proposed (not made):** +- Resolve credentials through `settings.User_Profile` in all three places. +- Look them up when the command runs (click `default=None`, then resolve inside the command). +- Add a micro-test that both entry points pick up a user set only in `config.json` or `.env`. + +--- + ### ISSUE-160: `AutomatedCuration`'s template builders miss placeholders Egeria's templates now use, so created elements keep literal `~{name}~` values (folder endpoint `~{directoryAddress}~` breaks folder surveys) **Layer:** Pyegeria · **Status:** partly fixed on branch `fix/issue-160-template-placeholders` (owner approved 2026-10-21); not yet merged or released. The rest is listed below for the owner. **Found:** 2026-10-21, live by the RE coordinator against Egeria `main` 05a5e982 with pyegeria 6.2.7. Two FileFolders made with `create_folder_element_from_template` had Endpoint `networkAddress` = `~{directoryAddress}~`. Egeria's Folder Survey then failed with `BASIC-FILE-CONNECTOR-404-001` "The folder named ~{directoryAddress}~ ... does not exist" (engine action `c1d924b5`). The same folders also kept `versionIdentifier` = `~{versionIdentifier}~` and `deployedImplementationType` = `~{deployedImplementationType}~` (record `~/.re-gate/repair-folder-endpoints-20261021T143415Z.json`). From 893d509bb782f918dabd920470a665e14167d385 Mon Sep 17 00:00:00 2001 From: Dan Wolfson Date: Wed, 21 Oct 2026 16:55:05 -0500 Subject: [PATCH 2/2] fix: resolve Dr.Egeria credentials at run time with one precedence chain (ISSUE-161) The dr_egeria and dr_egeria_folder CLIs fixed their --userid/--user_pass defaults when the module was imported, and md_processing/dr_egeria.py and common_md_utils.py read EGERIA_USER straight from the environment, bypassing config.json/.env. So the entry points could disagree, and a user set after import was ignored. New pyegeria.core.config.resolve_egeria_credentials(): explicit value > EGERIA_USER/EGERIA_USER_PASSWORD env (read at call time) > settings User Profile (config.json/.env) > quickstart defaults. Both CLIs default the options to None and resolve when they run; the md_processing constants and __main__ path use the same helper. Module-level names are kept. Micro-tests: tests/micro-tests/test_issue_161_dr_egeria_credentials.py (the three behaviour cases fail with the old callers). Full micro-test suite passes in a clean python:3.13-slim container. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Dan Wolfson --- PYEGERIA_ISSUES.md | 10 +- commands/cat/dr_egeria.py | 15 +-- commands/cat/dr_egeria_folder.py | 13 +-- md_processing/dr_egeria.py | 9 +- .../md_processing_utils/common_md_utils.py | 5 +- pyegeria/core/config.py | 13 +++ .../test_issue_161_dr_egeria_credentials.py | 93 +++++++++++++++++++ 7 files changed, 136 insertions(+), 22 deletions(-) create mode 100644 tests/micro-tests/test_issue_161_dr_egeria_credentials.py diff --git a/PYEGERIA_ISSUES.md b/PYEGERIA_ISSUES.md index d5526cf3..6c554acc 100644 --- a/PYEGERIA_ISSUES.md +++ b/PYEGERIA_ISSUES.md @@ -288,7 +288,7 @@ they're holding real port materialization until this exists server-side ### ISSUE-161: Dr.Egeria's two entry points resolve the Egeria user differently, and the CLI fixes its credential defaults at import time -**Layer:** Pyegeria · **Status:** open, logged only (no fix without the owner's word) · **Found:** 2026-10-21, while checking an RE report that `dr_egeria --process` ignored `EGERIA_USER`. **That report is withdrawn; RE was the cause:** RE's own `config.py:316` defaulted its daemon identity (`EGERIA_USER_ID`) to "erinoverview", so RE handed the child process erinoverview. The CLI correctly used it. RE is removing that default. +**Layer:** Pyegeria · **Status:** fixed on branch `fix/issue-161-dr-egeria-credentials` (owner approved 2026-10-21), not yet released · **Found:** 2026-10-21, while checking an RE report that `dr_egeria --process` ignored `EGERIA_USER`. **That report is withdrawn; RE was the cause:** RE's own `config.py:316` defaulted its daemon identity (`EGERIA_USER_ID`) to "erinoverview", so RE handed the child process erinoverview. The CLI correctly used it. RE is removing that default. **What was checked:** with `EGERIA_USER` set, the CLI path resolves and uses it throughout on 6.2.8: - `config.py:423` (`os.getenv("EGERIA_USER", config.json / .env value)`) feeds `settings.User_Profile.user_name`, the `--userid` default; @@ -302,7 +302,13 @@ Intended precedence (CLAUDE.md): explicit `--userid` > OS env > `.env` > `config 2. **Import-time defaults:** `commands/cat/dr_egeria.py:32-33` computes the `--userid` / `--user_pass` defaults when the module is imported. Code that imports it, then sets `EGERIA_USER` and invokes the command in the same process, gets the earlier user. This doesn't affect normal CLI use, where each run is a new process. 3. **Help text:** "Overrides EGERIA_USER (env var or .env file)" doesn't mention `config.json`, or that the default is fixed at import. -**Proposed (not made):** +**Fix (2026-10-21):** a new `pyegeria.core.config.resolve_egeria_credentials(user_id=None, user_pwd=None)` resolves, when called: an explicit value > the `EGERIA_USER` / `EGERIA_USER_PASSWORD` environment variables > the settings User Profile (`config.json` / `.env`) > "erinoverview" / "secret". +- The `dr_egeria` and `dr_egeria_folder` CLIs now default `--userid` / `--user_pass` to `None` and call it when they run. Their help text says so. +- `md_processing/dr_egeria.py` (module constants and its `__main__` path) and `common_md_utils.py` constants now use it too. The module-level `EGERIA_USER` names are kept for importers. +- Tests: `tests/micro-tests/test_issue_161_dr_egeria_credentials.py`. Five cases: precedence; env set after import, for both CLIs; config-only user; explicit flag wins. The three behaviour cases fail with the old caller code. The full micro-test suite (1,425) passes in a clean python:3.13-slim container. +- **Not changed:** other `commands/cat` tools (`list_glossaries`, `get_asset_graph`, `run_report`, `get_collection_tree`, `get_project_dependencies`, `list_user_ids`, `view_schema_tree`) still read `EGERIA_USER_PASSWORD` straight from the environment. They could adopt the helper later. + +**Originally proposed:** - Resolve credentials through `settings.User_Profile` in all three places. - Look them up when the command runs (click `default=None`, then resolve inside the command). - Add a micro-test that both entry points pick up a user set only in `config.json` or `.env`. diff --git a/commands/cat/dr_egeria.py b/commands/cat/dr_egeria.py index ac481bd3..a1b7bf6b 100644 --- a/commands/cat/dr_egeria.py +++ b/commands/cat/dr_egeria.py @@ -9,7 +9,7 @@ from rich.prompt import Prompt from pyegeria.core._exceptions import PyegeriaException, print_basic_exception -from pyegeria.core.config import settings +from pyegeria.core.config import settings, resolve_egeria_credentials from md_processing.dr_egeria import process_md_file_v2 import asyncio @@ -29,8 +29,8 @@ # User credentials: resolved via the standard pyegeria config utility, which already # honors the full precedence chain (OS env > .env file > default) — a raw # os.environ.get() here would silently miss credentials that only exist in .env. -EGERIA_USER = settings.User_Profile.user_name or "erinoverview" -EGERIA_USER_PASSWORD = settings.User_Profile.user_pwd or "secret" +# Kept for importers; the CLI itself resolves credentials when it runs (ISSUE-161). +EGERIA_USER, EGERIA_USER_PASSWORD = resolve_egeria_credentials() EGERIA_WIDTH = int(os.environ.get("EGERIA_WIDTH", settings.Environment.egeria_width or 190)) EGERIA_JUPYTER = os.environ.get("EGERIA_JUPYTER", str(settings.Environment.egeria_jupyter)).lower() in ("true", "1", "yes") @@ -48,10 +48,10 @@ help="Shortcut: execute all commands and make permanent changes in Egeria (overrides --directive)") @click.option("--server", default=EGERIA_VIEW_SERVER, help="Egeria view server to use.") @click.option("--url", default=EGERIA_VIEW_SERVER_URL, help="URL of Egeria platform to connect to") -@click.option("--userid", default=EGERIA_USER, - help="Egeria user. Overrides EGERIA_USER (env var or .env file).") -@click.option("--user_pass", default=EGERIA_USER_PASSWORD, - help="Egeria user password. Overrides EGERIA_USER_PASSWORD (env var or .env file).") +@click.option("--userid", default=None, + help="Egeria user. Defaults to EGERIA_USER (env var), then the User Profile in config.json or .env; resolved when the command runs.") +@click.option("--user_pass", default=None, + help="Egeria user password. Defaults to EGERIA_USER_PASSWORD (env var), then the User Profile in config.json or .env; resolved when the command runs.") @click.option("--parse-summary", default="none", help="When to show parse summaries", type=click.Choice(["all", "errors", "none"], case_sensitive=False)) @click.option("--attribute-logs", default="info", help="Per-attribute log verbosity", @@ -85,6 +85,7 @@ def process_markdown_file(input_file: str, output_folder: str, directive: str, try: # Instantiate the client from pyegeria import EgeriaTech + userid, user_pass = resolve_egeria_credentials(userid, user_pass) client = EgeriaTech(server, url, userid, user_pass) client.create_egeria_bearer_token() diff --git a/commands/cat/dr_egeria_folder.py b/commands/cat/dr_egeria_folder.py index bd411d0b..ae1df618 100644 --- a/commands/cat/dr_egeria_folder.py +++ b/commands/cat/dr_egeria_folder.py @@ -65,7 +65,7 @@ from rich.console import Console from rich.table import Table -from pyegeria.core.config import settings +from pyegeria.core.config import settings, resolve_egeria_credentials # Configure logging (matches commands/cat/dr_egeria.py) log_format = "{time} | {level} | {function} | {line} | {message} | {extra}" @@ -77,8 +77,8 @@ app_config = settings.Environment EGERIA_VIEW_SERVER = os.environ.get("EGERIA_VIEW_SERVER", app_config.egeria_view_server) EGERIA_VIEW_SERVER_URL = os.environ.get("EGERIA_VIEW_SERVER_URL", app_config.egeria_view_server_url) -EGERIA_USER = settings.User_Profile.user_name or "erinoverview" -EGERIA_USER_PASSWORD = settings.User_Profile.user_pwd or "secret" +# Kept for importers; the command itself resolves credentials when it runs (ISSUE-161). +EGERIA_USER, EGERIA_USER_PASSWORD = resolve_egeria_credentials() EGERIA_WIDTH = int(os.environ.get("EGERIA_WIDTH", settings.Environment.egeria_width or 190)) console = Console(width=EGERIA_WIDTH) @@ -250,9 +250,9 @@ async def run_one_file(input_file: Path, display_name: str, directive: str, clie help="Shortcut: execute all commands in every file and make permanent changes in Egeria") @click.option("--server", default=EGERIA_VIEW_SERVER, help="Egeria view server to use.") @click.option("--url", default=EGERIA_VIEW_SERVER_URL, help="URL of Egeria platform to connect to") -@click.option("--userid", default=EGERIA_USER, help="Egeria user. Overrides EGERIA_USER (env var or .env file).") -@click.option("--user_pass", default=EGERIA_USER_PASSWORD, - help="Egeria user password. Overrides EGERIA_USER_PASSWORD (env var or .env file).") +@click.option("--userid", default=None, help="Egeria user. Defaults to EGERIA_USER (env var), then the User Profile in config.json or .env; resolved when the command runs.") +@click.option("--user_pass", default=None, + help="Egeria user password. Defaults to EGERIA_USER_PASSWORD (env var), then the User Profile in config.json or .env; resolved when the command runs.") @click.option("--parse-summary", default="none", help="When to show parse summaries", type=click.Choice(["all", "errors", "none"], case_sensitive=False)) @click.option("--attribute-logs", default="info", help="Per-attribute log verbosity", @@ -279,6 +279,7 @@ def dr_egeria_folder(folder: str, directive: str, do_validate: bool, do_process: elif do_validate: directive = "validate" + userid, user_pass = resolve_egeria_credentials(userid, user_pass) usage_level = "Advanced" if advanced else "Basic" folder_path = Path(folder) diff --git a/md_processing/dr_egeria.py b/md_processing/dr_egeria.py index f63cf9df..02ad73c8 100644 --- a/md_processing/dr_egeria.py +++ b/md_processing/dr_egeria.py @@ -56,7 +56,7 @@ ) from pyegeria import settings, EgeriaTech, PyegeriaException, print_basic_exception, print_validation_error -from pyegeria.core.config import settings +from pyegeria.core.config import settings, resolve_egeria_credentials # Configure logging - module level default log_format = "{time} | {level} | {function} | {line} | {message} | {extra}" @@ -90,8 +90,8 @@ # User credentials and admin credentials are only from environment variables (not stored in config for security) EGERIA_ADMIN_USER = os.environ.get("ADMIN_USER", "garygeeke") EGERIA_ADMIN_PASSWORD = os.environ.get("ADMIN_PASSWORD", "secret") -EGERIA_USER = os.environ.get("EGERIA_USER", "erinoverview") -EGERIA_USER_PASSWORD = os.environ.get("EGERIA_USER_PASSWORD", "secret") +# Same precedence as the dr_egeria CLI: env, then config.json / .env, then defaults (ISSUE-161). +EGERIA_USER, EGERIA_USER_PASSWORD = resolve_egeria_credentials() # Legacy environment variables (deprecated, kept for backward compatibility) EGERIA_HOME_GLOSSARY_GUID = os.environ.get("EGERIA_HOME_GLOSSARY_GUID", None) @@ -920,8 +920,7 @@ def process_md_file(input_file: str, output_folder: str, directive: str, server: directive = "validate" # Run the async v2 processor - user = os.environ.get("EGERIA_USER", "erinoverview") - password = os.environ.get("EGERIA_USER_PASSWORD", "secret") + user, password = resolve_egeria_credentials() client = EgeriaTech(settings.Environment.egeria_view_server, settings.Environment.egeria_view_server_url, user_id=user) client.create_egeria_bearer_token(user, password) diff --git a/md_processing/md_processing_utils/common_md_utils.py b/md_processing/md_processing_utils/common_md_utils.py index 91ce6949..6c76c8a8 100644 --- a/md_processing/md_processing_utils/common_md_utils.py +++ b/md_processing/md_processing_utils/common_md_utils.py @@ -11,6 +11,7 @@ from rich.markdown import Markdown from pyegeria import settings, ServerClient, PyegeriaException +from pyegeria.core.config import resolve_egeria_credentials from pyegeria.core.utils import (camel_to_title_case) from pyegeria.core._globals import DEBUG_LEVEL, GovernanceDomains, resolve_enum from md_processing.md_processing_utils.message_constants import message_types @@ -26,8 +27,8 @@ EGERIA_INTEGRATION_DAEMON_URL = os.environ.get("EGERIA_INTEGRATION_DAEMON_URL", "https://localhost:9443") EGERIA_ADMIN_USER = os.environ.get("ADMIN_USER", "garygeeke") EGERIA_ADMIN_PASSWORD = os.environ.get("ADMIN_PASSWORD", "secret") -EGERIA_USER = os.environ.get("EGERIA_USER", "erinoverview") -EGERIA_USER_PASSWORD = os.environ.get("EGERIA_USER_PASSWORD", "secret") +# Same precedence as the dr_egeria CLI: env, then config.json / .env, then defaults (ISSUE-161). +EGERIA_USER, EGERIA_USER_PASSWORD = resolve_egeria_credentials() EGERIA_WIDTH = settings.Environment.egeria_width EGERIA_JUPYTER = settings.Environment.egeria_jupyter EGERIA_HOME_GLOSSARY_GUID = os.environ.get("EGERIA_HOME_GLOSSARY_GUID", None) diff --git a/pyegeria/core/config.py b/pyegeria/core/config.py index dec86cc2..b218c5c3 100644 --- a/pyegeria/core/config.py +++ b/pyegeria/core/config.py @@ -527,6 +527,19 @@ def __getattr__(self, name): settings = _LazySettings() +def resolve_egeria_credentials(user_id: Optional[str] = None, user_pwd: Optional[str] = None) -> tuple[str, str]: + """Resolve the Egeria user and password at call time (ISSUE-161). + + Precedence: an explicit value, then the EGERIA_USER / EGERIA_USER_PASSWORD environment + variables as they are now, then the User Profile from pyegeria's settings (config.json / + .env), then the quickstart defaults. Call it when a command runs, not at import, so a + variable set after import is still honoured. + """ + user = user_id or os.environ.get("EGERIA_USER") or settings.User_Profile.user_name or "erinoverview" + pwd = user_pwd or os.environ.get("EGERIA_USER_PASSWORD") or settings.User_Profile.user_pwd or "secret" + return user, pwd + + def pretty_print_config(env_file: str | None = None, safe: bool = True, to_console: bool = True) -> Dict[str, Dict[str, Any]]: """ Pretty print the current configuration and indicate the source of each value diff --git a/tests/micro-tests/test_issue_161_dr_egeria_credentials.py b/tests/micro-tests/test_issue_161_dr_egeria_credentials.py new file mode 100644 index 00000000..a24abcac --- /dev/null +++ b/tests/micro-tests/test_issue_161_dr_egeria_credentials.py @@ -0,0 +1,93 @@ +""" +ISSUE-161: Dr.Egeria's entry points resolved the Egeria user differently, and the +dr_egeria / dr_egeria_folder CLIs fixed their --userid defaults at import time. +All of them now call resolve_egeria_credentials() when they run: +explicit value > EGERIA_USER env (as it is now) > settings User Profile +(config.json / .env) > quickstart default. +No live server: EgeriaTech and the Dr.Egeria processor are replaced with fakes. +""" +import pyegeria +import pytest +from click.testing import CliRunner + +import commands.cat.dr_egeria as dr_cli +import commands.cat.dr_egeria_folder as dr_folder +from pyegeria.core.config import resolve_egeria_credentials, settings + + +class _FakeClient: + instances = [] + + def __init__(self, server, url, user, pwd=None, *args, **kwargs): + self.user, self.pwd = user, pwd + _FakeClient.instances.append(self) + + def create_egeria_bearer_token(self, *args, **kwargs): + return "token" + + +@pytest.fixture +def fakes(monkeypatch): + _FakeClient.instances = [] + monkeypatch.setattr(pyegeria, "EgeriaTech", _FakeClient) + + async def fake_process(*args, **kwargs): + return None + + async def fake_run_one_file(input_file, display_name, *args, **kwargs): + return (display_name, 1, 0, 0, "") + + monkeypatch.setattr(dr_cli, "process_md_file_v2", fake_process) + monkeypatch.setattr(dr_folder, "run_one_file", fake_run_one_file) + monkeypatch.delenv("EGERIA_USER", raising=False) + monkeypatch.delenv("EGERIA_USER_PASSWORD", raising=False) + return _FakeClient + + +def test_precedence_explicit_env_settings_default(monkeypatch): + monkeypatch.delenv("EGERIA_USER", raising=False) + monkeypatch.setattr(settings.User_Profile, "user_name", "config_user") + assert resolve_egeria_credentials()[0] == "config_user" + monkeypatch.setenv("EGERIA_USER", "env_user") + assert resolve_egeria_credentials()[0] == "env_user" + assert resolve_egeria_credentials("explicit_user")[0] == "explicit_user" + monkeypatch.delenv("EGERIA_USER") + monkeypatch.setattr(settings.User_Profile, "user_name", None) + assert resolve_egeria_credentials()[0] == "erinoverview" + + +def test_dr_egeria_cli_uses_env_set_after_import(fakes, monkeypatch, tmp_path): + doc = tmp_path / "doc.md" + doc.write_text("# empty\n") + monkeypatch.setenv("EGERIA_USER", "late_env_user") + monkeypatch.setenv("EGERIA_USER_PASSWORD", "late_pwd") + result = CliRunner().invoke(dr_cli.process_markdown_file, [str(doc), "--validate"]) + assert result.exit_code == 0, result.output + assert fakes.instances and fakes.instances[-1].user == "late_env_user" + assert fakes.instances[-1].pwd == "late_pwd" + + +def test_dr_egeria_cli_uses_config_user_when_env_unset(fakes, monkeypatch, tmp_path): + doc = tmp_path / "doc.md" + doc.write_text("# empty\n") + monkeypatch.setattr(settings.User_Profile, "user_name", "config_only_user") + result = CliRunner().invoke(dr_cli.process_markdown_file, [str(doc), "--validate"]) + assert result.exit_code == 0, result.output + assert fakes.instances[-1].user == "config_only_user" + + +def test_dr_egeria_cli_explicit_userid_wins(fakes, monkeypatch, tmp_path): + doc = tmp_path / "doc.md" + doc.write_text("# empty\n") + monkeypatch.setenv("EGERIA_USER", "env_user") + result = CliRunner().invoke(dr_cli.process_markdown_file, [str(doc), "--validate", "--userid", "flag_user"]) + assert result.exit_code == 0, result.output + assert fakes.instances[-1].user == "flag_user" + + +def test_dr_egeria_folder_uses_env_set_after_import(fakes, monkeypatch, tmp_path): + (tmp_path / "a.md").write_text("# empty\n") + monkeypatch.setenv("EGERIA_USER", "late_folder_user") + result = CliRunner().invoke(dr_folder.dr_egeria_folder, [str(tmp_path), "--validate"]) + assert result.exit_code == 0, result.output + assert fakes.instances and fakes.instances[-1].user == "late_folder_user"