Repository navigation
feat: Add flag overrides to the contract test services - #530
Draft
kinyoklion wants to merge 1 commit into
Draft
kinyoklion wants to merge 1 commit into
kinyoklion wants to merge 1 commit into
Conversation
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
September 28, 2026 20:29
021778c to
2074b74
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-contract-tests
branch
from
September 28, 2026 20:29
b5ff7c2 to
0a9c386
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
September 30, 2026 20:31
2074b74 to
1950d45
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-contract-tests
branch
from
September 30, 2026 20:31
0a9c386 to
bad30c5
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 1, 2026 23:43
1950d45 to
3e47045
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-contract-tests
branch
from
October 1, 2026 23:43
bad30c5 to
a355c3d
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 3, 2026 01:18
3e47045 to
cfb781a
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-contract-tests
branch
from
October 3, 2026 01:19
a355c3d to
dac6abe
Compare
Member
Author
|
bugbot review |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit dac6abe. Configure here.
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 13, 2026 21:02
cfb781a to
ea4dc19
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-contract-tests
branch
from
October 13, 2026 21:02
dac6abe to
3de3ec9
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 13, 2026 22:40
ea4dc19 to
3de11ab
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-contract-tests
branch
2 times, most recently
from
October 15, 2026 22:25
8d2c06c to
6190743
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 16, 2026 21:34
bb5a108 to
5e42ebc
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-contract-tests
branch
from
October 16, 2026 21:34
6190743 to
ac6bdd0
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 28, 2026 22:17
5e42ebc to
314dea2
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-contract-tests
branch
from
October 28, 2026 22:17
ac6bdd0 to
0d759ba
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 28, 2026 22:40
314dea2 to
4bace41
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-contract-tests
branch
2 times, most recently
from
October 28, 2026 22:53
1dbbf2c to
1f4ce1a
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 29, 2026 16:48
99b851f to
fac9ec8
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-contract-tests
branch
from
October 29, 2026 16:48
1f4ce1a to
d1b617b
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
October 29, 2026 17:16
fac9ec8 to
0415ec1
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-contract-tests
branch
from
October 29, 2026 17:16
d1b617b to
0d55468
Compare
kinyoklion
added a commit
to launchdarkly/sdk-test-harness
that referenced
this pull request
Nov 3, 2026
…thout the init key (#539) <!-- TERMAG_PR_BODY_BEGIN --> ## Summary Overrides only add to what the SDK reports: a flag that the override layer does not hold evaluates as it does without an override source, and the all-flags state holds every flag it holds without one, plus the overridden flags. A client that has not initialized is where SDKs broke this, and only when the store holds flags without being initialized, for example a persistent store that holds data but no init key. The flag overrides suite uses the SDK's in-memory store, which is empty until a data source provides data, so it could not tell a read of the override layer apart from a read of everything the store returns. A new `flag overrides` group in the persistent data store suite (`server_side_persistence_overrides.go`) runs when the test service has the `flag-overrides` capability. It writes flags to the store without the init key and gives the client a data source that never provides data. Each test compares a client with an override source to a client without one against the same store, so it holds whatever an SDK reports for an uninitialized store: - **a flag without an override evaluates as it does without an override source**: an override file that holds a different flag leaves the evaluation of a store flag unchanged (OVERRIDE Requirements 1.5.1 and 1.5.3). - **an empty override source leaves the all-flags state unchanged** (Requirement 1.2.2). - **overrides add to the all-flags state and make it valid**: the state holds exactly the flags of the client without a source, with override values for the overridden keys, plus the flags that exist only in the override layer, and it is valid (launchdarkly/sdk-specs#280, Requirements 1.4.1.2 and 1.4.1.3). The flag overrides suite gains **evaluate all flags returns only the override flags as a valid state** in its uninitialized client group, where the store is empty and every SDK reports only the override flags. ## Results Run with `-enable-persistence-tests` against Redis. | Test service | flag without an override | empty override source | overrides add | |---|---|---|---| | Go `v7` | fails: served from the store | fails: valid state of the store's flags | fails: the store's flags join the overrides | | Go launchdarkly/go-server-sdk#491 | passes | passes | passes | | Node `main` | passes | fails: valid state of the store's flags | passes | | Node launchdarkly/js-core#2132 first commit (`57498a9e`) | passes | passes | fails: drops a store flag the state holds without overrides | | Node launchdarkly/js-core#2132 | passes | passes | passes | | Python (top of the launchdarkly/python-server-sdk#530 stack, sync and async) | passes | passes | passes | | Ruby (top of the launchdarkly/ruby-server-sdk#454 stack) | passes | passes | passes | The Java (launchdarkly/java-core#225) and .NET (`dotnet-server-sdk-v9`) test services do not support persistent stores, so the group skips there; the new uninitialized client test passes on both. The full Redis persistent data store suite and the flag overrides suite pass against Go #491 at `30d4f216` (131 tests run). `golangci-lint` reports no issues. Go and Node fail these tests until launchdarkly/go-server-sdk#491 and launchdarkly/js-core#2132 are merged, for test services whose CI runs the persistent data store suite with this harness version. [SDK-3419](https://launchdarkly.atlassian.net/browse/SDK-3419) <!-- TERMAG_PR_BODY_END --> <!-- termag: box=i-0575d1c54853058d7 project=sdk-test-harness --> <sub>Opened from termag</sub> [SDK-3419]: https://launchdarkly.atlassian.net/browse/SDK-3419?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Adds harness coverage for **file flag overrides** when the SDK never initializes but a **persistent store already holds feature data without the `$inited` key**—a case the existing in-memory override suite could not distinguish from “read everything from the store.” > > A new **`flag overrides`** group under the server-side persistent data store suite (`server_side_persistence_overrides.go`, gated on `flag-overrides`) seeds Redis/Consul/Dynamo with flags, uses a blocking unavailable data source, and compares clients with and without override files via `uninitializedClient`. Scenarios assert: store flags not in the override file behave like no override source; an empty override file leaves **evaluate all flags** unchanged; overrides merge into all-flags state, mark it **valid**, and apply override values while retaining other store flags. > > The standalone flag-overrides **`uninitialized client`** group gains **evaluate all flags returns only the override flags as a valid state** (empty store, overrides-only valid state). The persistent suite registers the new group from `server_side_persistence_base.go`. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 09a8e02. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
Both contract test services advertise the flag-overrides capability and accept the overrides object in the SDK configuration: filePaths, and the optional duplicateKeysHandling, changeDetection, and pollIntervalMs properties, which configure the file-based override source on the FDv2 data system. An overrides object without a data system configuration is rejected, because overrides are an option of the FDv2 data system only.
kinyoklion
force-pushed
the
rlamb/overrides-python-file-source
branch
from
November 3, 2026 23:56
0415ec1 to
a8e9294
Compare
kinyoklion
force-pushed
the
rlamb/overrides-python-contract-tests
branch
from
November 3, 2026 23:56
0d55468 to
5800e36
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This is the last step of the flag overrides port described by the OVERRIDE specification. It is based on the file source branch because the phases are stacked; retarget to
feat/overridesonce that branch merges.Both contract test services (the Flask service for
LDClientand the aiohttp service forAsyncLDClient) advertise theflag-overridescapability and accept theoverridesobject in the SDK configuration as the test harness service specification describes:filePaths(required, in precedence order) and the optionalduplicateKeysHandling,changeDetection, andpollIntervalMsproperties. The values configure aFileOverrideSourceBuilderon the FDv2 data system throughConfigBuilder.overrides(...)orAsyncDataSystemConfig.override_source. Anoverridesobject without adataSystemconfiguration is rejected with an error, because overrides are an option of the FDv2 data system only.Verification with the prebuilt harness from sdk-test-harness branch
rlamb/sdk-2657/flag-overrides-tests(v3.2.0-alpha.10):-run 'flag overrides'38 ran, all passed; full suite 4809 total, 15 skipped, 4794 ran, all passed.-run 'flag overrides'38 ran, all passed; full suite 4807 total, 16 skipped, 4791 ran, all passed.The 15 tests skipped by the overrides runs are the other top-level suites excluded by the filter. The tests skipped in the full runs are those for capabilities the services do not advertise (
strongly-typed,tls:verify-peer,service-endpoints,user-type,track-hooks,wrapper, and on the async servicehook-environment-id), the long-running tests, and the persistent data store tests, all of which were skipped before this change too.SDK-3250
Note
Overview
Both contract-test services (sync Flask and async aiohttp) now advertise the
flag-overridescapability and map harnessoverridesconfig into the FDv2 data system.When
dataSystemis present, an optionaloverridesblock builds aFileOverrideSourceBuilderfromfilePathsand optionalduplicateKeysHandling,changeDetection, andpollIntervalMs—wired viadatasystem.overrides(...)on the sync client andAsyncDataSystemConfig.override_sourceon the async client.overrideswithoutdataSystemraisesValueError, since overrides are FDv2-only.No production SDK behavior changes; this is harness wiring so sdk-test-harness flag-overrides suites can run against Python.
Reviewed by Cursor Bugbot for commit dac6abe. Bugbot is set up for automated code reviews on this repo. Configure here.