refactor(storage): stream vacuum2 block gc

[dantengsky]

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Reduce peak memory usage in fuse_vacuum2 block garbage collection for very large tables.

This PR is a follow-up to the protected-segment chunking change. It avoids materializing all candidate block listing results in memory before deletion while preserving the existing vacuumed result contract.

The main memory reduction is to avoid keeping all listed block Entry values and intermediate block path vectors in memory at once. This matters for very large _b/ directories because each OpenDAL Entry carries both the object path and metadata; the metadata portion is significant and should not be treated as negligible when millions of entries are listed.

This PR intentionally preserves the existing result contract and still returns the complete list of removed files. The remaining memory pressure from retaining all removed file paths for the result set will be addressed in a follow-up PR with an explicit API/behavior design.

Changes:

• Stream block listing for non-FS object stores and stop at the same vacuum2 timestamp-derived until prefix as before.
• Filter and remove block chunks while listing, instead of building full Vec<Entry> / candidate block path vectors.
• Preserve the existing FS behavior by sorting listed entries before applying the until boundary, because FS listing order is not guaranteed.
• Reuse the existing common fuse vacuum candidate safety helper for block eligibility checks, including conservative legacy-object last_modified handling.
• Preserve the complete returned vacuumed file list.

Safety notes:

• The deletion condition remains candidate-before-gc-root AND not referenced by the protected block set.
• Index files are still removed before their data blocks for each delete chunk.
• Errors while proving legacy-object eligibility are still propagated instead of ignored by reusing the existing common vacuum logic.

Tests

• Unit Test
  • cargo test -p databend-enterprise-query vacuum_table_v2 --lib -- --nocapture
• Logic Test
  • cargo test -p databend-enterprise-query --test it storages::fuse::operations::vacuum2 -- --nocapture
• Build/Check
  • cargo check -p databend-common-storages-fuse
• Benchmark Test
• No Test - Explain why

Type of change

• Bug Fix (non-breaking change which fixes an issue)
• New Feature (non-breaking change which adds functionality)
• Breaking Change (fix or feature that could cause existing functionality not to work as expected)
• Documentation Update
• Refactoring
• Performance Improvement
• Other (please describe):

AI assistance

• AI usage: Assisted with diagnosing the vacuum2 memory path, implementing streaming block GC, reviewing conservative error handling, adding focused tests, and preparing this PR.
• Responsible human: @dantengsky
• The responsible human has read every line of this diff and can explain each change

---

This change is 

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f81aa48312

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[dantengsky]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 8e9550e75b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[dantengsky]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: ec8f22b3be

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".