ci: run the tests/dincli full-GI integration suite in a separate CI job #228

Activity

1. umermjd11

   added

   enhancementNew feature or requestNew feature or request

   on Oct 5, 2026
2. umeradl commented on Oct 8, 2026

   

   umeradl

   on Oct 8, 2026

   Member

   More actions

   Groundwork landed: PR No. 213 (7eb558d on develop, merged 2026-10-08)

   This doesn't close this issue: ci.yml is unchanged and there is still no integration job. But the harness this job will run on is now in place:

   • Isolated mode (DIN_TEST_ISOLATED=1): explicit scratch and results directories (DIN_TEST_TMPDIR, DIN_TEST_RESULTS_DIR), local-only endpoints, and a filtered subprocess environment. No developer .env is read.
   • Owned services: Anvil and an offline Kubo are started by the harness, checked for readiness, and torn down by process group, even when startup fails. Ports that are already in use are refused. Service logs are kept in the results dir, which covers the "upload logs on failure" part of the acceptance criteria.
   • Demo bootstrap: now uses configure-demo and connect-demo-wallet. The checked-in accounts.json is validated and left untouched.
   • Tests: 57 unit tests cover the harness and run in the existing not integration job.
   • Docs: Documentation/technical/testing/dincli-testing-guide.md now has a "CI smoke groundwork" section.

   Follow-ups from the PR No. 213 review that fall under this issue (checked on develop @ 7eb558d):

   1. Wallet commands: test_02_task_contracts.py (3), test_03_registration.py (5) and test_04_gi.py (8) still call system connect-wallet, 16 call sites in total. connect-wallet rejects the demo wallets that bootstrap now creates, so the suite fails past test_01 until these move to connect-demo-wallet. This probably fits with the TP-4 suite rework from task-plan-051026-2.
   2. Anvil flags are defined twice: the isolated branch in tests/dincli/conftest.py inlines the same flags as foundry/anvil.sh (--block-time 2, --code-size-limit 4294967295, …). The proposal above starts Anvil via anvil.sh, so pick one source of truth before the job lands.
   3. Compiler cache: isolated compiles run with a fresh HOME/XDG_CACHE_HOME, so the Hardhat and svm compiler caches start empty every run. The job will need network access, and will be slower, unless the runner pre-seeds or caches them.
3. sharmis-tp commented on Oct 10, 2026

   

   sharmis-tp

   on Oct 10, 2026

   Contributor

   More actions

   I'd like to take this, building on the isolated harness that landed in PR #213. My plan is a separate non-blocking integration job in ci.yml with the proposed path filters that installs Foundry, Kubo, Docker and Python 3.12, runs the suite with DIN_TEST_ISOLATED=1, uploads the dincli and anvil logs on failure, plus the doc updates in Developer/CONTRIBUTING.md and the dincli testing guide. Could you confirm nobody on the team has started it and assign it to me? The PR would target develop.

   👍React with 👍1Reacted by Dr. Abraham Nash
4. umeradl commented on Oct 11, 2026

   

   umeradl

   on Oct 11, 2026

   Member

   More actions

   Thanks @sharmis-tp. Nobody has started the CI job, so it's yours. I'm assigning you now.

   One thing about the order of work: the suite itself doesn't pass yet. Getting it to run a complete GI on the foundry contracts is task_061026_21 (TP-4 of task-plan-051026-2). That's in progress and should land around Oct 17. Until then, tests/dincli/ still deploys the task contracts from hardhat artifacts and still calls system connect-wallet in 16 places, so it fails after test_01.

   So:

   • You can build the job now: setup, path filters, DIN_TEST_ISOLATED=1, log upload and the doc updates. Keep it continue-on-error: true, and expect it to fail past test_01 until task_061026_21 merges. Please leave tests/dincli/test_0*.py alone in your PR, because that rework is part of task_061026_21 and the two would conflict.
   • Follow-ups No. 2 and No. 3 from the comment above are in your scope. For the Anvil flags, pick one source of truth: either foundry/anvil.sh or the isolated branch in tests/dincli/conftest.py. For the compiler cache, cache the svm/Hardhat compiler caches on the runner, or show they aren't needed once the suite stops using hardhat artifacts.
   • Once the suite is green, put the measured full-GI runtime in the PR description. That tells us whether to run it on every PR or only on push to develop plus nightly.
5. Santiagocetran

   added 2 commits that reference this issue on Oct 27, 2026

   ci: run the tests/dincli integration suite in a non-blocking job

   ...

   9ca6c16

   Merge pull request #242 from sharmis-tp/ci/integration-job

   ...

   954fbf8
6. umeradl commented on Oct 29, 2026

   

   umeradl

   on Oct 29, 2026

   Member

   More actions

   Rollout decision (2026-10-29): the full-GI job will not become a required check. A full GI trains 9 clients and runs auditor and aggregator containers, which is too heavy to run on every commit or push. PR No. 301 moves it out of ci.yml into a manual-only workflow, .github/workflows/full-gi.yml (workflow_dispatch), which maintainers run on demand, for example before merging a change to the GI flow. The Acceptance item about required-check rollout is replaced by this.

   Where the suite stands (found while looking at PR No. 297, handed to task_061026_21 Part D in #300 (comment)):

   • phase 1 passes with the per-proxy proxyAdmin* assertions (abd5985);
   • phase 2 passes locally with connect-demo-wallet (b929a1e);
   • phase 3 next stops because the offline IPFS node never has the din_info.json local default CIDs seeded; after that comes BL-27.

   What's left here once PR No. 301 merges: a first green manual run after Part D, with its measured runtime.
7. umeradl commented on Nov 2, 2026

   

   umeradl

   on Nov 2, 2026

   Member

   More actions

   @robertocarlous, could you run the closing check on this issue (din-close-issue 228)? I can't do it: din_peer_check.py refuses me because I wrote PRs No. 219, 229, 232 and 301, which implement it (MAINTAINING No. 5.2). Santiago wrote PR No. 296, so you're the eligible peer.

   Pointers for the check:

   • The separate job: it exists as the manual-only Full GI workflow (.github/workflows/full-gi.yml). PR No. 301 moved it out of ci.yml, and PR No. 296 (merged today) added the outcome summary.
   • Still red: the suite itself fails on develop (the fix is task_061026_21 Part D). Please check that against the issue's wording before deciding whether this closes.