Skip to content

BE-921: Calculate archival temporal changes before persistence - #10125

Draft
montekki wants to merge 1 commit into
fs/be-921-batch-the-statements-in-patch-entitiesfrom
fs/be-921-plan-entity-temporal-changes
Draft

montekki wants to merge 1 commit into
fs/be-921-batch-the-statements-in-patch-entitiesfrom
fs/be-921-plan-entity-temporal-changes

Conversation

@montekki

@montekki montekki commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

🌟 Purpose

Close the live interval superseded by draft publication using the shared temporal history calculation. Delete removed rows in bulk and persist retained history through the existing bulk writer.

🔍 Changes

  1. Move the interval-closing calculation from archive_entity in entity/mod.rs into EntityHistory::close in entity/history.rs. It retains earlier transaction-time and decision-time fragments and records original rows that have no surviving fragment.
  2. Replace the inline draft and live temporal UPDATE statements in archive_entity with a call to update_entity_history. That call reuses update_entity_temporal_rows, introduced in #10063, to write the calculated intervals.
  3. Move the inline temporal DELETE into update_entity_history in entity/batch/write.rs. The new DELETE ... USING unnest(...) removes the original rows reported by history.removed() before updates and inserts run.
  4. Replace the historical-fragment INSERT in archive_entity with the shared writer's bulk insertion of history.inserted(). When an earlier transaction fragment survives a closure at the decision interval's start, reuse the original row through an update rather than deleting it and inserting that fragment.
  5. Keep the archivedById provenance update in archive_entity, after the calculated temporal history has been persisted.

Keep the archiving actor on the superseded edition. Tests cover closing an interval at its starting timestamp and existing draft-publication behavior.

Validation: 26 selected temporal, patch, draft, and read-only tests passed. Clippy, Rust formatting, embedded SQL formatting, and whitespace checks passed.

@montekki montekki self-assigned this Oct 20, 2026
@github-actions github-actions Bot added area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team labels Oct 20, 2026
@vercel

vercel Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hash Ready Ready Preview Oct 20, 2026 6:08pm UTC
petrinaut Ready Ready Preview Oct 20, 2026 6:08pm UTC
petrinaut-docs Ready Ready Preview Oct 20, 2026 6:08pm UTC
1 Skipped Deployment
Project Deployment Actions Updated
hashdotdesign-tokens Ignored Ignored Preview Oct 20, 2026 6:08pm UTC

Request Review

@codecov

codecov Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 45.45455% with 36 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.02%. Comparing base (360ecf2) to head (b3c7bf4).

Files with missing lines Patch % Lines
...src/store/postgres/knowledge/entity/batch/write.rs 0.00% 32 Missing ⚠️
...ore/src/store/postgres/knowledge/entity/history.rs 88.23% 3 Missing and 1 partial ⚠️
Additional details and impacted files
@@                                 Coverage Diff                                  @@
##           fs/be-921-batch-the-statements-in-patch-entities   #10125      +/-   ##
====================================================================================
- Coverage                                             68.03%   68.02%   -0.01%     
====================================================================================
  Files                                                  2008     2008              
  Lines                                                206521   206587      +66     
  Branches                                               8800     8801       +1     
====================================================================================
+ Hits                                                 140507   140537      +30     
- Misses                                                64344    64379      +35     
- Partials                                               1670     1671       +1     
Flag Coverage Δ
hash-graph 21.07% <ø> (ø)
hash-graph-api 39.82% <ø> (ø)
hash-graph-atlas 80.01% <ø> (ø)
hash-graph-postgres-store 32.13% <45.45%> (+0.04%) ⬆️
hashql-compiletest 28.71% <ø> (ø)
hashql-eval 81.55% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark results

hash-graph-benches – Integrations

policy_resolution_large

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2002 $$28.7 \mathrm{ms} \pm 285 \mathrm{μs}\left({\color{gray}-1.689 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.18 \mathrm{ms} \pm 22.3 \mathrm{μs}\left({\color{lightgreen}-6.125 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1002 $$13.9 \mathrm{ms} \pm 157 \mathrm{μs}\left({\color{gray}-0.296 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 3314 $$42.7 \mathrm{ms} \pm 439 \mathrm{μs}\left({\color{gray}-1.581 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$13.7 \mathrm{ms} \pm 138 \mathrm{μs}\left({\color{lightgreen}-6.617 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 1527 $$24.4 \mathrm{ms} \pm 203 \mathrm{μs}\left({\color{gray}-2.811 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 2078 $$29.2 \mathrm{ms} \pm 295 \mathrm{μs}\left({\color{gray}-3.200 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.60 \mathrm{ms} \pm 27.6 \mathrm{μs}\left({\color{lightgreen}-7.102 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 1033 $$14.9 \mathrm{ms} \pm 155 \mathrm{μs}\left({\color{gray}-2.546 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_medium

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 102 $$3.56 \mathrm{ms} \pm 26.8 \mathrm{μs}\left({\color{gray}-2.700 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.82 \mathrm{ms} \pm 22.9 \mathrm{μs}\left({\color{gray}-4.008 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 52 $$3.21 \mathrm{ms} \pm 27.1 \mathrm{μs}\left({\color{gray}1.50 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 269 $$4.97 \mathrm{ms} \pm 51.6 \mathrm{μs}\left({\color{gray}-3.650 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$3.18 \mathrm{ms} \pm 27.8 \mathrm{μs}\left({\color{lightgreen}-5.799 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 108 $$3.85 \mathrm{ms} \pm 36.8 \mathrm{μs}\left({\color{gray}-2.342 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 133 $$4.19 \mathrm{ms} \pm 44.5 \mathrm{μs}\left({\color{gray}-3.466 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.18 \mathrm{ms} \pm 21.9 \mathrm{μs}\left({\color{gray}-2.418 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 63 $$3.76 \mathrm{ms} \pm 30.7 \mathrm{μs}\left({\color{gray}-3.219 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_none

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2 $$2.57 \mathrm{ms} \pm 30.1 \mathrm{μs}\left({\color{gray}1.46 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.38 \mathrm{ms} \pm 18.7 \mathrm{μs}\left({\color{gray}-2.616 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 2 $$2.44 \mathrm{ms} \pm 21.9 \mathrm{μs}\left({\color{gray}-1.573 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 8 $$2.73 \mathrm{ms} \pm 30.1 \mathrm{μs}\left({\color{gray}-2.314 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.55 \mathrm{ms} \pm 20.6 \mathrm{μs}\left({\color{gray}-2.466 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 3 $$2.72 \mathrm{ms} \pm 22.0 \mathrm{μs}\left({\color{gray}-2.406 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_small

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 52 $$2.82 \mathrm{ms} \pm 25.3 \mathrm{μs}\left({\color{gray}-1.797 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.53 \mathrm{ms} \pm 18.7 \mathrm{μs}\left({\color{gray}-4.236 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 26 $$2.76 \mathrm{ms} \pm 23.4 \mathrm{μs}\left({\color{gray}-4.882 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 94 $$3.10 \mathrm{ms} \pm 24.4 \mathrm{μs}\left({\color{gray}-4.378 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$2.82 \mathrm{ms} \pm 22.5 \mathrm{μs}\left({\color{gray}-1.048 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 27 $$2.98 \mathrm{ms} \pm 22.9 \mathrm{μs}\left({\color{gray}-3.230 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 66 $$3.13 \mathrm{ms} \pm 29.7 \mathrm{μs}\left({\color{gray}-2.920 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.67 \mathrm{ms} \pm 20.3 \mathrm{μs}\left({\color{lightgreen}-5.214 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 29 $$3.05 \mathrm{ms} \pm 25.6 \mathrm{μs}\left({\color{gray}-2.819 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_complete

Function Value Mean Flame graphs
entity_by_id;one_depth 1 entities $$41.5 \mathrm{ms} \pm 290 \mathrm{μs}\left({\color{lightgreen}-6.953 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 10 entities $$33.2 \mathrm{ms} \pm 326 \mathrm{μs}\left({\color{gray}-4.675 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 25 entities $$36.8 \mathrm{ms} \pm 261 \mathrm{μs}\left({\color{gray}-1.806 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 5 entities $$31.4 \mathrm{ms} \pm 239 \mathrm{μs}\left({\color{gray}-0.542 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 50 entities $$44.3 \mathrm{ms} \pm 299 \mathrm{μs}\left({\color{gray}3.19 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 1 entities $$49.4 \mathrm{ms} \pm 355 \mathrm{μs}\left({\color{gray}-1.615 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 10 entities $$39.6 \mathrm{ms} \pm 270 \mathrm{μs}\left({\color{lightgreen}-5.577 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 25 entities $$87.5 \mathrm{ms} \pm 716 \mathrm{μs}\left({\color{gray}-4.097 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 5 entities $$33.6 \mathrm{ms} \pm 267 \mathrm{μs}\left({\color{gray}-3.456 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 50 entities $$285 \mathrm{ms} \pm 1.03 \mathrm{ms}\left({\color{red}8.43 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 1 entities $$10.3 \mathrm{ms} \pm 85.9 \mathrm{μs}\left({\color{gray}-4.387 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 10 entities $$10.7 \mathrm{ms} \pm 141 \mathrm{μs}\left({\color{gray}-0.991 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 25 entities $$10.5 \mathrm{ms} \pm 84.7 \mathrm{μs}\left({\color{gray}-1.038 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 5 entities $$10.2 \mathrm{ms} \pm 83.4 \mathrm{μs}\left({\color{lightgreen}-6.694 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 50 entities $$10.7 \mathrm{ms} \pm 101 \mathrm{μs}\left({\color{gray}-1.380 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$11.5 \mathrm{ms} \pm 202 \mathrm{μs}\left({\color{red}5.22 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$10.7 \mathrm{ms} \pm 104 \mathrm{μs}\left({\color{gray}-0.403 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$10.8 \mathrm{ms} \pm 108 \mathrm{μs}\left({\color{gray}-2.135 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$10.5 \mathrm{ms} \pm 77.0 \mathrm{μs}\left({\color{lightgreen}-5.902 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$11.3 \mathrm{ms} \pm 112 \mathrm{μs}\left({\color{lightgreen}-5.490 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$10.8 \mathrm{ms} \pm 101 \mathrm{μs}\left({\color{gray}-2.060 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$10.8 \mathrm{ms} \pm 82.4 \mathrm{μs}\left({\color{gray}-4.032 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$10.8 \mathrm{ms} \pm 103 \mathrm{μs}\left({\color{gray}-3.933 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$10.7 \mathrm{ms} \pm 87.8 \mathrm{μs}\left({\color{gray}-4.861 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$10.7 \mathrm{ms} \pm 102 \mathrm{μs}\left({\color{gray}-2.912 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$11.2 \mathrm{ms} \pm 105 \mathrm{μs}\left({\color{gray}-0.259 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$10.9 \mathrm{ms} \pm 92.8 \mathrm{μs}\left({\color{gray}-2.587 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$10.9 \mathrm{ms} \pm 98.4 \mathrm{μs}\left({\color{gray}-4.629 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$10.7 \mathrm{ms} \pm 83.5 \mathrm{μs}\left({\color{gray}-4.302 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba $$8.07 \mathrm{ms} \pm 88.7 \mathrm{μs}\left({\color{gray}-0.529 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property traversal_paths=0 0 $$65.2 \mathrm{ms} \pm 691 \mathrm{μs}\left({\color{gray}-2.835 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$119 \mathrm{ms} \pm 731 \mathrm{μs}\left({\color{gray}-2.920 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$72.6 \mathrm{ms} \pm 583 \mathrm{μs}\left({\color{gray}-3.157 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$82.8 \mathrm{ms} \pm 636 \mathrm{μs}\left({\color{gray}-2.642 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$91.5 \mathrm{ms} \pm 510 \mathrm{μs}\left({\color{gray}-2.825 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$97.7 \mathrm{ms} \pm 703 \mathrm{μs}\left({\color{gray}-2.653 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=0 0 $$45.6 \mathrm{ms} \pm 393 \mathrm{μs}\left({\color{gray}-2.062 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$76.9 \mathrm{ms} \pm 607 \mathrm{μs}\left({\color{gray}3.66 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$51.3 \mathrm{ms} \pm 454 \mathrm{μs}\left({\color{gray}-2.944 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$63.0 \mathrm{ms} \pm 502 \mathrm{μs}\left({\color{gray}0.255 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$64.3 \mathrm{ms} \pm 601 \mathrm{μs}\left({\color{gray}-0.681 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$65.7 \mathrm{ms} \pm 703 \mathrm{μs}\left({\color{gray}1.20 \mathrm{\%}}\right) $$

scenarios

Function Value Mean Flame graphs
full_test query-limited $$107 \mathrm{ms} \pm 1.07 \mathrm{ms}\left({\color{lightgreen}-5.438 \mathrm{\%}}\right) $$ Flame Graph
full_test query-unlimited $$117 \mathrm{ms} \pm 1.05 \mathrm{ms}\left({\color{lightgreen}-5.855 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-limited $$19.3 \mathrm{ms} \pm 183 \mathrm{μs}\left({\color{gray}-1.432 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-unlimited $$490 \mathrm{ms} \pm 1.79 \mathrm{ms}\left({\color{gray}1.12 \mathrm{\%}}\right) $$ Flame Graph

This branch was successfully deployed

3 active deployments
Preview – petrinaut-docs — b3c7bf42 Deployed Oct 20, 2026 by vercel[bot]
Preview – hash — b3c7bf42 Deployed Oct 20, 2026 by vercel[bot]
Preview – petrinaut — b3c7bf42 Deployed Oct 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

1 participant