ROX-35448: add VM fleet telemetry via ClusterMetrics - #22311
Draft
vikin91 wants to merge 7 commits into
Draft
Conversation
Define the full VirtualMachineMetrics and ScanDurationStats shape up front. Fields 3-6 (backoff, scan_duration) are declared here but will be populated in a follow-up part that depends on the retry-backoff PR. ROX-35448 Partially generated by AI. Co-authored-by: Cursor <cursoragent@cursor.com>
Capture lastAgentVersion from ResponseMeta on each successful scrape (already read but discarded). Stats() iterates vmState under s.mu to produce TrackedVMs, VMsScanned, and VersionCounts (top-20 + "other" cap, empty version bucketed as "unknown"). VMsInBackoff, AvgBackoff, MaxBackoff, ScanDuration are declared on the Stats struct but left zero-valued until Part 3, which depends on the per-VM retry backoff PR (ROX-35445). ROX-35448 Partially generated by AI. Co-authored-by: Cursor <cursoragent@cursor.com>
Add VMStatsSource interface to clustermetrics package. NewWithInterval now accepts an optional VMStatsSource; when non-nil, collectMetrics populates VirtualMachineMetrics with tracked_vms, vms_scanned, and roxagent_version_counts. sensor.go passes the VMScraper instance when VM scanning is enabled, nil otherwise. ROX-35448 Partially generated by AI. Co-authored-by: Cursor <cursoragent@cursor.com>
Declared on every Hello regardless of whether VM scanning is currently enabled, so Central can distinguish "feature off on a new Sensor" from "old Sensor that predates VM telemetry" and zero stale traits only in the former case. ROX-35448 Partially generated by AI. Co-authored-by: Cursor <cursoragent@cursor.com>
Three-way case split on (HasCapability, GetVirtualMachineMetrics): - capable + non-nil: set VM Scanning Enabled=true with counts - capable + nil: zero all VM traits (feature confirmed off) - no capability: leave existing traits untouched (old Sensor) Roxagent version counts are serialized as a sorted JSON array. UpdateSecuredClusterIdentity gains a hasVMTelemetryCap parameter; the pipeline reads it from the connection's capability set. ROX-35448 Partially generated by AI. Co-authored-by: Cursor <cursoragent@cursor.com>
Verify that VMScraper Stats() flows through collectMetrics and populates the VirtualMachineMetrics proto end-to-end. Partially generated by AI (ROX-35448). Co-authored-by: Cursor <cursoragent@cursor.com>
The cluster-metrics pipeline called HasCapability on the injector unconditionally, which panics when tests (and any nil-injector path) pass a nil MessageInjector. Treat nil as lacking the capability, and assert the full Segment trait map including empty/sorted version JSON. User request: fix the nil-injector panic and incomplete buildVMTraits assertions, then commit. Partially generated by AI.
|
Skipping CI for Draft Pull Request. |
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Central YAML (base), Organization UI (inherited) Review profile: CHILL Plan: Enterprise Run ID: ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (12)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe change adds VM telemetry metrics to cluster metrics, tracks VM agent versions, advertises Sensor capability support, and merges capability-aware VM traits into Central secured-cluster identity telemetry. ChangesVirtual machine telemetry
Estimated code review effort: 3 (Moderate) | ~30 minutes Merge Risk: ⚪ Minimal · up to This PR adds VM fleet telemetry and compatibility handling without any identified current-head correctness, availability, security, or deployment risk; no actionable merge-blocking risk remains beyond normal checks and review. Sequence Diagram(s)sequenceDiagram
participant VMScraper
participant ClusterMetrics
participant Pipeline
participant Central
VMScraper->>ClusterMetrics: Provide VM statistics
ClusterMetrics->>Pipeline: Emit cluster metrics
Pipeline->>Central: Update secured-cluster identity with capability
Central->>Central: Merge VM telemetry traits
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
Contributor
🚀 Build Images ReadyImages are ready for commit 1645e70. To use with deploy scripts: export MAIN_IMAGE_TAG=4.12.x-776-g1645e7062c |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #22311 +/- ##
==========================================
- Coverage 51.23% 51.21% -0.02%
==========================================
Files 2855 2855
Lines 178712 178796 +84
==========================================
+ Hits 91568 91579 +11
- Misses 79082 79136 +54
- Partials 8062 8081 +19
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
ACS had no per-secured-cluster signal for VM scanning: whether it is on, how many VMs Sensor is tracking, how many have been scraped, or which roxagent versions are in the fleet. That belongs on the existing cluster-metrics / Segment identity path, next to node count and CPU capacity.
Sensor's VM scraper now exposes a point-in-time
Stats()snapshot (tracked VMs, VMs with at least one successful scrape, and a version histogram capped at the top 20 versions plus anotherbucket). When VM scanning is enabled and the scraper is running, the cluster-metrics component copies that snapshot into a new optionalVirtualMachineMetricsfield onClusterMetrics.Every current Sensor advertises a
VirtualMachineTelemetrycapability on Hello, including when scanning is off. Central uses that to tell three cases apart:VM Scanning Enabled=trueand the countsRoxagent Version Countsas[]) so stale values do not lingerRoxagent version counts are sent as a sorted JSON array trait so the payload is stable.
The proto also declares backoff and trailing scan-duration fields so the message shape is complete on the wire. This PR leaves them at zero; populating them needs the per-VM retry-backoff work (ROX-35445).
The cluster-metrics pipeline treats a nil message injector as "no capability" so tests (and any nil-injector path) do not panic on
HasCapability.User-facing documentation
Internal Segment traits only; no operator-facing behavior change.
Testing and quality
Stats are only attached when
ROX_VIRTUAL_MACHINESis on and the scraper exists. The Hello capability is unconditional so Central can zero traits when scanning is off on a current Sensor.Automated testing
How I validated my change
AI-Assisted: cursor, generated scraper stats, ClusterMetrics wiring, Segment trait mapping, and the nil-injector fix; user reviewed.