perf: Refactor groupChartData to O(n) mathematical spatial indexing

[mohit-bhandari45]

Fixes #96

Summary

This PR addresses significant performance and technical debt in the timeline chart generation logic (groupChartData) on the backend API.

Previously, the time-interval aggregation algorithm relied on an O(n*t) nested iteration (executing data.filter() continuously inside an iterative loop over every sequential time bucket). For large log sets, this caused hundreds of thousands of blocking Array.prototype.filter passes within the Node event loop. Additionally, the log categories (success, warning, error) were statically hardcoded, breaking the chart if a new level was ever introduced to the schema.

Changes Made

• O(n) Spatial Indexing: Completely rewrote the bucketing algorithm to utilize a single-pass for loop over the dataset. Time arrays are pre-generated, and a log's target bucket index is now instantly targeted using lightweight division math (Math.floor(timeDifference / interval)), entirely eliminating all .filter() passes.
• Dynamic Extensibility: Removed all hardcoded "success", "error", "warning" keys. The engine now dynamically maps generation against the global LEVELS schema constant, making timeline categorization incredibly flexible out-of-the-box.
• Resolved and removed three deprecated backend // TODO comments tied to algorithmic efficiency, typings, and frontend presentation bounds.

Impact

• CPU & Latency: Massive reduction in blocking iteration for the backend API. Endpoint response times are significantly lower when dragging the latency sliders or visualizing heavy timeframes.
• Scalability: Next.js can now comfortably calculate chart time-series data for datasets multiple magnitudes larger without risking an event loop freeze.

[vercel]

@mohit-bhandari45 is attempting to deploy a commit to the OpenStatus Team on Vercel.

A member of the Team first needs to authorize it.

[mohit-bhandari45]

@mxkaske When you get time do review this pr.

[mxkaske]

@claude review

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
data-table-filters	Ready	Preview	Aug 22, 2026 5:51pm

[claude]

Claude finished @mxkaske's task in 2m 25s —— View job

---

Code Review: groupChartData O(n) refactor

• Gather context (PR diff, CLAUDE.md, helpers.ts)
• Understand the request (code review of groupChartData refactor)
• Analyze correctness of new O(n) bucketing algorithm
• Check dynamic LEVELS handling, edge cases, types
• Post review feedback

Summary

The core idea is solid: replacing the O(n·t) nested .filter() bucketing with a single-pass index computation (Math.floor(timeDiff / interval)) is a legitimate and meaningful algorithmic improvement, and switching from hardcoded success/warning/error keys to iterating the shared LEVELS constant is a good move since TimelineChartSchema's zod schema (apps/web/src/app/infinite/schema.ts:45-55) is already built dynamically off LEVELS. I traced through the boundary math (row exactly at between[0], rows beyond steps*interval but within duration, rows before/after range) and the new index-based bucketing produces the same results as the old per-bucket filter — the if (timestamps[bucketIndex]) guard correctly replicates the old out-of-range exclusion behavior.

A few issues to fix before merge:

1. Unused import — will fail lint (apps/web/src/app/infinite/api/helpers.ts:7)
addMilliseconds from date-fns is no longer used anywhere in the file after replacing the bucket-timestamp construction with startTime + i * interval (line 183), but it's still imported. This will trip @typescript-eslint/no-unused-vars in pnpm turbo lint / the pre-commit hook.

-import { addDays, addMilliseconds, differenceInMinutes } from "date-fns";
+import { addDays, differenceInMinutes } from "date-fns";

2. Type safety bypassed with a double cast (apps/web/src/app/infinite/api/helpers.ts:213)

return timestamps as unknown as TimelineChartSchema[];

timestamps is typed as Record<string, number>[], so this cast entirely opts out of TypeScript's structural check that every LEVELS key is actually present on each bucket. It works today because LEVELS.forEach (line 186) populates all keys, but nothing enforces that invariant if LEVELS or the schema changes later — the compiler would stay silent. Consider typing the bucket via a reduce over LEVELS (mirroring how timelineChartSchema itself is built in schema.ts:47-53) so the return type is checked structurally instead of forced.

3. Minor: TODO removal doesn't match PR description
The PR description says three TODOs "tied to algorithmic efficiency, typings, and frontend presentation bounds" were "resolved and removed." The efficiency one is genuinely resolved. But // TODO: use date-fns and interval to determine the format (previously on the timestamp line) was just deleted — the output is still a raw startTime + i * interval millisecond number, same as before; the formatting concern itself wasn't addressed. Not a blocker, just worth being accurate about in the description (or re-add the TODO if it's still open).

4. Nit: comment style
Comments like // 1. DYNAMIC INITIALIZATION, // THE MAGIC: ... (lines 179-181, 199-201) are more narrative than the codebase's existing terse // TODO: / // REMINDER: style, and per this repo's CLAUDE.md, comments should only explain non-obvious why, not narrate what the code does. Consider trimming these down.

5. Missing tests for a rewritten, correctness-sensitive algorithm
apps/web/src/app/infinite/api/helpers.test.ts:10-11 explicitly notes groupChartData is untested. CLAUDE.md's testing policy states "every utility function, helper, and non-trivial logic must have tests," and this PR rewrites the core bucketing logic (including the boundary/index-guard behavior discussed above). Given how easy it would be to get an off-by-one wrong here, a few unit tests (e.g. buckets align with old filter-based semantics at range boundaries, unknown steps === 0, single-row placement) would give strong regression protection for future changes.

Note on verification

I wasn't able to run pnpm turbo lint / pnpm turbo typecheck in this environment (command execution required interactive approval that wasn't available), so item #1 is based on static analysis of the import usage, not a confirmed lint run — but grepping the file confirms addMilliseconds has zero remaining call sites.

Fix unused import & type cast →
· Branch

[mohit-bhandari45]

@claude review

[mohit-bhandari45]

@claude review

[mohit-bhandari45]

@mxkaske do review this.