Skip to content

BE-921: Batch entity patch writes - #10129

Draft
montekki wants to merge 1 commit into
fs/be-921-batch-entity-patch-readsfrom
fs/be-921-batch-entity-patch-writes
Draft

montekki wants to merge 1 commit into
fs/be-921-batch-entity-patch-readsfrom
fs/be-921-batch-entity-patch-writes

Conversation

@montekki

@montekki montekki commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

🌟 Purpose

Persisting drafts, type memberships, provenance, and cache entries sends separate statements for each row or edition. Write these accumulated changes in bulk alongside the edition and temporal statements already handled by the shared writer.

🔍 Changes

  1. Replace the per-draft insert loop in write_entity_patch_set in entity/batch/write.rs with one typed bulk insertion of patches.drafts.
  2. Replace the two type-membership inserts per edition in insert_entity_patch_types with one statement over edition/type pairs supplied through unnest(...). Combine direct memberships and inherited memberships, grouping inherited memberships by edition and target type to retain minimum inheritance depth.
  3. Replace the per-publication provenance update loop in update_entity_patch_provenance with UPDATE entity_ids ... FROM unnest(...) over entity IDs and provenance values.
  4. Replace the per-superseded-edition provenance update loop with one UPDATE entity_editions using ANY($1::UUID[]) and the archiving actor.
  5. Replace the per-edition cache-population loop in write_entity_patch_set with one call using every changed edition ID, including intermediate editions produced by repeated patches. Edition insertion and temporal persistence continue through the bulk writers introduced in the lower frames.

Record inherited types at their minimum inheritance depth, preserve first-publication timestamps, and retain intermediate editions returned for repeated patches.

Validation: 661 tests passed and 2 were skipped across the Postgres store, graph integration, and authorization suites before the visibility-only cleanup. All-features Clippy and formatting checks passed after the cleanup.

@montekki montekki self-assigned this Oct 20, 2026
@vercel

vercel Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hash Ready Ready Preview Oct 20, 2026 6:08pm UTC
petrinaut Ready Ready Preview Oct 20, 2026 6:08pm UTC
petrinaut-docs Ready Ready Preview Oct 20, 2026 6:08pm UTC
1 Skipped Deployment
Project Deployment Actions Updated
hashdotdesign-tokens Ignored Ignored Preview Oct 20, 2026 6:08pm UTC

Request Review

@codecov

codecov Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 54 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.95%. Comparing base (422ed20) to head (aa742b7).

Files with missing lines Patch % Lines
...src/store/postgres/knowledge/entity/batch/write.rs 0.00% 54 Missing ⚠️
Additional details and impacted files
@@                          Coverage Diff                           @@
##           fs/be-921-batch-entity-patch-reads   #10129      +/-   ##
======================================================================
- Coverage                               67.95%   67.95%   -0.01%     
======================================================================
  Files                                    2009     2009              
  Lines                                  206795   206798       +3     
  Branches                                 8801     8801              
======================================================================
+ Hits                                   140537   140538       +1     
- Misses                                  64587    64589       +2     
  Partials                                 1671     1671              
Flag Coverage Δ
hash-graph 21.07% <ø> (ø)
hash-graph-api 39.82% <ø> (ø)
hash-graph-atlas 80.01% <ø> (+<0.01%) ⬆️
hash-graph-postgres-store 31.82% <0.00%> (-0.01%) ⬇️
hashql-compiletest 28.71% <ø> (ø)
hashql-eval 81.55% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark results

hash-graph-benches – Integrations

policy_resolution_large

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2002 $$17.4 \mathrm{ms} \pm 121 \mathrm{μs}\left({\color{gray}0.064 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.19 \mathrm{ms} \pm 16.0 \mathrm{μs}\left({\color{gray}2.51 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1002 $$8.81 \mathrm{ms} \pm 91.1 \mathrm{μs}\left({\color{gray}-3.551 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 3314 $$30.5 \mathrm{ms} \pm 357 \mathrm{μs}\left({\color{gray}1.86 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$11.5 \mathrm{ms} \pm 182 \mathrm{μs}\left({\color{red}5.79 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 1527 $$18.7 \mathrm{ms} \pm 217 \mathrm{μs}\left({\color{red}6.88 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 2078 $$18.0 \mathrm{ms} \pm 160 \mathrm{μs}\left({\color{lightgreen}-5.359 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.42 \mathrm{ms} \pm 21.1 \mathrm{μs}\left({\color{gray}4.17 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 1033 $$8.76 \mathrm{ms} \pm 118 \mathrm{μs}\left({\color{lightgreen}-12.593 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_medium

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 102 $$2.32 \mathrm{ms} \pm 20.6 \mathrm{μs}\left({\color{gray}-0.152 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$1.98 \mathrm{ms} \pm 31.5 \mathrm{μs}\left({\color{gray}2.73 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 52 $$2.18 \mathrm{ms} \pm 15.9 \mathrm{μs}\left({\color{gray}1.56 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 269 $$3.21 \mathrm{ms} \pm 23.4 \mathrm{μs}\left({\color{lightgreen}-5.857 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$2.25 \mathrm{ms} \pm 17.9 \mathrm{μs}\left({\color{gray}0.290 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 108 $$2.58 \mathrm{ms} \pm 18.3 \mathrm{μs}\left({\color{gray}-0.663 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 133 $$2.70 \mathrm{ms} \pm 17.6 \mathrm{μs}\left({\color{gray}-1.342 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.14 \mathrm{ms} \pm 13.4 \mathrm{μs}\left({\color{gray}-1.851 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 63 $$2.62 \mathrm{ms} \pm 21.8 \mathrm{μs}\left({\color{gray}-4.610 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_none

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2 $$1.77 \mathrm{ms} \pm 23.2 \mathrm{μs}\left({\color{red}8.30 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$1.70 \mathrm{ms} \pm 9.74 \mathrm{μs}\left({\color{gray}-0.916 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 2 $$1.79 \mathrm{ms} \pm 11.3 \mathrm{μs}\left({\color{gray}-0.338 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 8 $$1.95 \mathrm{ms} \pm 9.26 \mathrm{μs}\left({\color{gray}2.54 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$1.84 \mathrm{ms} \pm 11.5 \mathrm{μs}\left({\color{gray}2.39 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 3 $$2.05 \mathrm{ms} \pm 19.3 \mathrm{μs}\left({\color{gray}2.75 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_small

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 52 $$1.93 \mathrm{ms} \pm 13.8 \mathrm{μs}\left({\color{gray}0.454 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$1.80 \mathrm{ms} \pm 11.3 \mathrm{μs}\left({\color{gray}3.74 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 26 $$1.91 \mathrm{ms} \pm 14.0 \mathrm{μs}\left({\color{gray}2.41 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 94 $$2.16 \mathrm{ms} \pm 13.7 \mathrm{μs}\left({\color{gray}-1.760 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$1.96 \mathrm{ms} \pm 11.3 \mathrm{μs}\left({\color{gray}-0.637 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 27 $$2.14 \mathrm{ms} \pm 25.9 \mathrm{μs}\left({\color{gray}0.373 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 66 $$2.58 \mathrm{ms} \pm 94.9 \mathrm{μs}\left({\color{red}22.0 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.05 \mathrm{ms} \pm 22.6 \mathrm{μs}\left({\color{red}6.35 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 29 $$2.29 \mathrm{ms} \pm 26.1 \mathrm{μs}\left({\color{red}10.4 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_complete

Function Value Mean Flame graphs
entity_by_id;one_depth 1 entities $$21.6 \mathrm{ms} \pm 183 \mathrm{μs}\left({\color{gray}0.143 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 10 entities $$44.0 \mathrm{ms} \pm 325 \mathrm{μs}\left({\color{gray}-1.289 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 25 entities $$24.6 \mathrm{ms} \pm 197 \mathrm{μs}\left({\color{gray}-0.367 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 5 entities $$25.0 \mathrm{ms} \pm 248 \mathrm{μs}\left({\color{gray}2.71 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 50 entities $$28.7 \mathrm{ms} \pm 236 \mathrm{μs}\left({\color{gray}1.64 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 1 entities $$22.6 \mathrm{ms} \pm 194 \mathrm{μs}\left({\color{gray}0.717 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 10 entities $$231 \mathrm{ms} \pm 1.45 \mathrm{ms}\left({\color{gray}-1.331 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 25 entities $$55.7 \mathrm{ms} \pm 700 \mathrm{μs}\left({\color{gray}-4.250 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 5 entities $$45.5 \mathrm{ms} \pm 458 \mathrm{μs}\left({\color{gray}0.488 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 50 entities $$173 \mathrm{ms} \pm 1.20 \mathrm{ms}\left({\color{red}5.55 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 1 entities $$6.62 \mathrm{ms} \pm 54.7 \mathrm{μs}\left({\color{gray}-2.251 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 10 entities $$6.74 \mathrm{ms} \pm 82.0 \mathrm{μs}\left({\color{gray}-3.289 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 25 entities $$7.00 \mathrm{ms} \pm 81.4 \mathrm{μs}\left({\color{gray}0.190 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 5 entities $$6.88 \mathrm{ms} \pm 60.3 \mathrm{μs}\left({\color{gray}0.537 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 50 entities $$7.16 \mathrm{ms} \pm 63.7 \mathrm{μs}\left({\color{gray}0.871 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$7.22 \mathrm{ms} \pm 63.7 \mathrm{μs}\left({\color{gray}4.89 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$7.04 \mathrm{ms} \pm 47.0 \mathrm{μs}\left({\color{gray}1.62 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$7.08 \mathrm{ms} \pm 56.6 \mathrm{μs}\left({\color{gray}2.05 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$7.32 \mathrm{ms} \pm 59.1 \mathrm{μs}\left({\color{gray}2.40 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$7.76 \mathrm{ms} \pm 63.7 \mathrm{μs}\left({\color{red}6.46 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$7.63 \mathrm{ms} \pm 81.6 \mathrm{μs}\left({\color{red}8.23 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$7.48 \mathrm{ms} \pm 78.7 \mathrm{μs}\left({\color{gray}1.25 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$7.22 \mathrm{ms} \pm 51.3 \mathrm{μs}\left({\color{gray}2.40 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$7.66 \mathrm{ms} \pm 72.3 \mathrm{μs}\left({\color{red}6.15 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$7.67 \mathrm{ms} \pm 107 \mathrm{μs}\left({\color{gray}1.81 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$7.57 \mathrm{ms} \pm 43.4 \mathrm{μs}\left({\color{red}6.62 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$7.34 \mathrm{ms} \pm 51.0 \mathrm{μs}\left({\color{gray}3.32 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$7.62 \mathrm{ms} \pm 96.3 \mathrm{μs}\left({\color{red}5.36 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$7.48 \mathrm{ms} \pm 100 \mathrm{μs}\left({\color{gray}2.77 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba $$5.40 \mathrm{ms} \pm 47.5 \mathrm{μs}\left({\color{gray}0.494 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property traversal_paths=0 0 $$40.0 \mathrm{ms} \pm 309 \mathrm{μs}\left({\color{gray}1.59 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$75.7 \mathrm{ms} \pm 644 \mathrm{μs}\left({\color{gray}4.96 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$47.7 \mathrm{ms} \pm 428 \mathrm{μs}\left({\color{red}9.61 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$50.1 \mathrm{ms} \pm 482 \mathrm{μs}\left({\color{gray}1.50 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$55.6 \mathrm{ms} \pm 423 \mathrm{μs}\left({\color{gray}0.158 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$60.3 \mathrm{ms} \pm 506 \mathrm{μs}\left({\color{gray}1.82 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=0 0 $$32.1 \mathrm{ms} \pm 231 \mathrm{μs}\left({\color{red}6.84 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$51.7 \mathrm{ms} \pm 421 \mathrm{μs}\left({\color{red}9.60 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$37.2 \mathrm{ms} \pm 282 \mathrm{μs}\left({\color{red}8.14 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$44.0 \mathrm{ms} \pm 335 \mathrm{μs}\left({\color{red}10.7 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$44.4 \mathrm{ms} \pm 481 \mathrm{μs}\left({\color{red}8.92 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$44.3 \mathrm{ms} \pm 326 \mathrm{μs}\left({\color{red}10.1 \mathrm{\%}}\right) $$

scenarios

Function Value Mean Flame graphs
full_test query-limited $$69.3 \mathrm{ms} \pm 512 \mathrm{μs}\left({\color{gray}-0.570 \mathrm{\%}}\right) $$ Flame Graph
full_test query-unlimited $$75.5 \mathrm{ms} \pm 579 \mathrm{μs}\left({\color{gray}-1.352 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-limited $$12.8 \mathrm{ms} \pm 225 \mathrm{μs}\left({\color{gray}-4.277 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-unlimited $$305 \mathrm{ms} \pm 1.04 \mathrm{ms}\left({\color{gray}0.957 \mathrm{\%}}\right) $$ Flame Graph

This branch was successfully deployed

3 active deployments
Preview – petrinaut-docs — aa742b7c Deployed Oct 20, 2026 by vercel[bot]
Preview – hash — aa742b7c Deployed Oct 20, 2026 by vercel[bot]
Preview – petrinaut — aa742b7c Deployed Oct 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team

Development

Successfully merging this pull request may close these issues.

1 participant