New Presplit utility for Fate table - #6471
Conversation
ctubbsii
left a comment
There was a problem hiding this comment.
Is it useful to have this utility focus exclusively on the Fate table? Would it be better to generalize it to generate splits for any table, or is this necessarily tied to the fate table for some reason?
The split generation logic is tied to the Fate table. The algorithm is modeled after getDesiredPartitions, to produce UUIDs whose string representations align with how Fate maps transaction Ids to tablet rows. I was trying to keep the work done scoped to just Fate since in original issue was asking to generate splits for the FATE table that evenly distribute FATE operations across tablets. However; I could look into creating a more general purpose split logic if that's more desirable. |
closes issue #4979
Added support for pre-splitting Fate tables via accumulo admin command.
--numSplitsoption generates split points using the same partitioning logic as inFateManager.getDesiredPartitions. The logic ensures that pre-split boundires align with how the manger expects transaction IDs to be distributed. The split utility produces raw UUID row strings which should allow it be also be used by scan_ref.--splitsFileflag writes the generated splits to a file.