Skip to content

refactor: remove SDK experiment eligibility reporting - #23

Merged
cosmos-explorer merged 1 commit into
masterfrom
refactor/remove-send-to-experiment
Nov 4, 2026
Merged

cosmos-explorer merged 1 commit into
masterfrom
refactor/remove-send-to-experiment

Conversation

@cosmos-explorer

@cosmos-explorer cosmos-explorer commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Remove sendToExperiment from evaluation event payloads and remove the SDK's experiment eligibility calculation

Validation

  • Python 3.13.3: all 129 tests pass; one existing datetime.utcnow() deprecation warning.
  • python -m flake8 . and diff whitespace checks for changed files pass.
  • Real local FeatBit service: WebSocket initialization succeeds; flag-off, rule-match, and fallthrough return expected values.
  • Repeated live validation with all five legacy field paths stripped from incoming WebSocket messages before SDK processing: values, variation IDs, and reasons match the original responses. The server itself still returns these fields.
  • Both live runs deliver evaluation events with HTTP 200 and without sendToExperiment; identify and custom metric requests also return HTTP 200. HTTP acceptance was verified, not downstream analytics storage.

Summary by CodeRabbit

  • Updates
    • Serialized feature-flag variation data no longer includes experiment-eligibility details. Flag keys, timestamps, and variation details remain available.
    • Insights behavior is unchanged: enabled insights still emit evaluation events, while disabled insights suppress them. Evaluation results, including their IDs, values, and reasons, remain unchanged.

RetriggerView in GreptileConfidence Score: 5/5

This PR appears safe to merge; no actionable issues were found.

Summary

Removes experiment eligibility calculation and sendToExperiment from evaluation events.

  • Flag evaluations keep their results without reporting experiment eligibility.

No actionable issues found. Normal rollout selection and insightsEnabled checks remain unchanged. Tests were inspected, not run during this review.

Reviews (1) · Last reviewed commit: "refactor: remove SDK experiment eligibil..." · Reviewed by Greptile

@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The SDK removes experiment eligibility calculation and storage from evaluation results. Flag event serialization no longer includes sendToExperiment. Test fixtures remove experiment eligibility fields, and tests check evaluation results and insights event behavior.

Changes: Experiment eligibility reporting

Layer / File(s) Summary
Remove eligibility from evaluation and events
fbclient/common_types.py, fbclient/evaluator.py, fbclient/event_types.py, tests/fbclient_test_data.json, tests/test_insights_enabled.py
Evaluation results no longer store experiment eligibility, and the evaluator no longer calculates it. Serialized flag variations omit sendToExperiment. Fixtures remove related fields, and tests check evaluation results and insights event emission.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Possibly related PRs

Suggested reviewers: dsun0720, deletelater

Merge Risk: Minimal · up to e6453

This change removes experiment eligibility reporting from the SDK. Flag evaluation, percentage rollout, and insights toggling are not affected, and no remaining code depends on the removed field. The one open question is how external analytics handles events without sendToExperiment; the author reports the events were accepted with HTTP 200, so the risk is low.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: removing SDK experiment eligibility reporting.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 5 functions across 3 files. (1 skipped: 1 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
Generate docstrings
  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share
Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2be58c87-8768-4e56-b6cc-60872850fa5c
Commits

Reviewing files that changed from the base of the PR and between db09e8b and e64533c.

Files selected for processing: 5
  • fbclient/common_types.py
  • fbclient/evaluator.py
  • fbclient/event_types.py
  • tests/fbclient_test_data.json
  • tests/test_insights_enabled.py
Files with no reviewable changes: 1
  • fbclient/event_types.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

@cosmos-explorer
cosmos-explorer merged commit 0fb6e07 into master Nov 4, 2026
13 checks passed
@cosmos-explorer
cosmos-explorer deleted the refactor/remove-send-to-experiment branch November 4, 2026 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant