Skip to content

enhance: add query_iterator to AsyncMilvusClient - #3743

Open
csy1204 wants to merge 4 commits into
milvus-io:masterfrom
csy1204:enhance/async-query-iterator
Open

enhance: add query_iterator to AsyncMilvusClient#3743
csy1204 wants to merge 4 commits into
milvus-io:masterfrom
csy1204:enhance/async-query-iterator

Conversation

@csy1204

@csy1204 csy1204 commented Aug 4, 2026

Copy link
Copy Markdown

related to: #3742

Problem

AsyncMilvusClient has no query_iterator(). Async callers who need a full filtered result set fall back to query(..., limit=N) — which has no ordering guarantee, so which N rows come back is undefined — or hand-roll primary-key cursor pagination plus mvcc-timestamp pinning in application code.

What this does

The sync QueryIterator touches the network in only four places, all through self._handler; everything else is pure logic. So instead of a parallel implementation, this splits the state machine out and drives it from both transports.

  • refactor: extract transport-agnostic _QueryIteratorBase from QueryIterator — the base holds cursor bookkeeping, expression assembly, batch/limit accounting, the result cache and the checkpoint file, and exposes _*_kwargs() builders paired with _consume_*() methods. QueryIterator becomes a thin sync driver over it. Two private helpers move to a single underscore (_setup_next_expr, _save_pk_cursor) because the subclass driver calls them and existing tests reference the mangled names.
  • enhance: add AsyncQueryIterator on top of the shared iterator base — the async driver. Setup RPCs run in a create() classmethod because __init__ cannot await. await next(), await close(), get_cursor(), and __aiter__/__anext__ for async for.
  • enhance: add query_iterator to AsyncMilvusClient — the factory, mirroring the sync one (same parameter names/order/defaults, same filter type check, same schema-cache lookup). AsyncMilvusClientSession delegates it the way the sync session does.
  • test: cover AsyncQueryIterator limits, cursors, checkpoint resume and async for
it = await client.query_iterator(collection_name="c", filter="pk > 0", batch_size=1000)
try:
    while True:
        batch = await it.next()
        if not batch:
            break
        handle(batch)
finally:
    await it.close()

# or
async for batch in it:
    handle(batch)

Behavior note

The refactor moves parameter validation ahead of the first RPC, so QueryIterator(batch_size=-1) now raises ParamError without the describe_collection call the old code made first (same for a schema dict with no is_primary field). This is the only intentional behavior difference, and it makes sync and async validate identically. It is stated in the refactor commit's message too.

How the refactor was verified

Commit 1 renames the class at the top of the file and appends a subclass, so GitHub renders it as close to a rewrite — "existing tests pass" is weak evidence for a refactor whose tests were written against the old structure. So I also compared old vs new directly with a throwaway differential harness (not part of this PR), driving both versions against identical recording handlers and diffing the full ordered sequence of describe_collection/query calls with all kwargs, plus returned batches, final cursor and checkpoint-file contents:

  • 19 scenarios, old vs new: identical except the validation-ordering difference above. Covered int/varchar/None pk expressions, the cache hit-and-refill path, limit cut-off, offset seek (full / drained-early / partial-loop), element_filter with and without an element cursor, extra rpc_options passthrough, and all six checkpoint branches (fresh, resume, one-line ParamError, unparseable ts, element-cursor resume, resume-with-residual-offset).
  • 24 scenarios, new sync vs new async: identical, zero divergence. Also covered limit=0, reduce_stop_for_best=False, the mvccTs fallback when the server returns 0, and the 100-line checkpoint-file truncation path.

Tests

tests/unit/test_async_query_iterator.py (new, no server needed) covers batch exhaustion, int and varchar PK cursor expressions, batch_size validation before any RPC, limit cut-off, checkpoint-file write / resume / removal, describe_collection called exactly once, async for parity with the next() loop, and get_cursor(). tests/unit/test_iterator_ownership.py gains an ownership assertion for the new class, and the session-delegation parametrize list in tests/unit/test_async_milvus_client.py gains a query_iterator row so cluster_id forwarding is pinned the way the sync suite pins it.

Full unit suite: 4553 passed, 3 skipped. black --check and ruff check clean.

Also ran it against a live Milvus deployment: iterating one filtered set returned 4,621 rows over 47 pages, and AsyncMilvusClient.query_iterator, async for, and the sync query_iterator all produced the identical primary-key set with no duplicates.

Scope

Deliberately limited to query_iterator, since #3483 was closed for bundling unrelated changes:

  • search_iterator async support is left for a separate PR, which is why AsyncMilvusClientSession delegates only query_iterator where the sync session delegates both.
  • __aiter__/__anext__ has no sync counterpart on purpose — it is the idiomatic async consumption protocol and six lines. I did not add __iter__ to the sync iterator.
  • Checkpoint-file I/O stays synchronous inside the async coroutines. It lives in the shared base, it is one buffered line plus a flush per batch next to a network round-trip, and moving it off-thread would mean either a thread-pool hop per batch or lifting the checkpoint logic back out of the shared base.
  • AsyncQueryIterator is not exported from the top-level pymilvus namespace, matching QueryIterator.

Happy to split, rename or adjust anything here — including the base-class extraction, if you would rather see the async driver built a different way.

@csy1204

csy1204 commented Aug 4, 2026

Copy link
Copy Markdown
Author

/assign @XuanYang-cn

For context: in #3483 you asked for that PR to be closed and split into one PR per requirement. This is the async-iterator item, on its own, with focused tests — no other changes bundled in.

@mergify

mergify Bot commented Aug 4, 2026

Copy link
Copy Markdown

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

Comment thread pymilvus/client/iterator/query_iterator.py
Comment thread pymilvus/client/iterator/query_iterator.py
Comment thread pymilvus/client/iterator/async_query_iterator.py
csy1204 added 4 commits August 5, 2026 03:15
…rator

Splits the pure state machine (cursor, expression assembly, batch/limit
accounting, result cache, checkpoint file) from the four RPC call sites so a
non-blocking driver can reuse it. Parameter validation now runs ahead of the
first RPC: constructing QueryIterator with an invalid batch_size (<0 or >
MAX_BATCH_SIZE), or with a schema dict missing is_primary, now raises before
describe_collection is ever called, whereas the old code issued that RPC
first. Sync and async now validate identically; this is a deliberate
improvement, pinned by a new test, not a side effect.

Two private helpers are renamed to a single underscore because they are now
called from the subclass driver and existing tests reference the mangled names:
__setup_next_expr -> _setup_next_expr, __save_pk_cursor -> _save_pk_cursor.

Signed-off-by: Sangyeon Cho <sang-yeon.cho@navercorp.com>
Signed-off-by: Sangyeon Cho <josang1204@gmail.com>
Drives the same state machine as QueryIterator over an async handler: setup
RPCs run in the create() classmethod because __init__ cannot await, and next()
/ close() are coroutines. Supports 'async for' through __aiter__/__anext__.

Documents that checkpointing (iterator_cp_file) writes and flushes
synchronously on the event-loop thread on every batch, and during setup/close,
so a slow or network-mounted path stalls unrelated coroutines.

Signed-off-by: Sangyeon Cho <sang-yeon.cho@navercorp.com>
Signed-off-by: Sangyeon Cho <josang1204@gmail.com>
Mirrors MilvusClient.query_iterator: same signature, same filter type check,
same schema-cache lookup. Returns an awaited AsyncQueryIterator because the
setup RPCs cannot run in __init__. AsyncMilvusClientSession delegates it the
same way the sync session does.

Documents that the iterator_cp_file checkpoint write is synchronous on the
event-loop thread on every batch, and during setup/close, and advises using
local disk or leaving checkpointing off on latency-sensitive loops.

Signed-off-by: Sangyeon Cho <sang-yeon.cho@navercorp.com>
Signed-off-by: Sangyeon Cho <josang1204@gmail.com>
… async for

Also extends TestAsyncMilvusClientSession's parametrized delegation test
with a query_iterator case, mirroring the sync suite, and strengthens
test_client_query_iterator_returns_async_iterator to assert the factory's
filter/schema keyword wiring actually reaches the query and _get_schema
calls instead of only asserting isinstance and batch output.

Signed-off-by: Sangyeon Cho <sang-yeon.cho@navercorp.com>
Signed-off-by: Sangyeon Cho <josang1204@gmail.com>
@csy1204
csy1204 force-pushed the enhance/async-query-iterator branch from 0a24c50 to 5d0fedf Compare August 4, 2026 18:20
@sre-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: csy1204
To complete the pull request process, please ask for approval from xuanyang-cn after the PR has been reviewed.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@codecov

codecov Bot commented Aug 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.96482% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.11%. Comparing base (c5e73a6) to head (5d0fedf).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
pymilvus/client/iterator/async_query_iterator.py 78.43% 11 Missing ⚠️
pymilvus/client/iterator/query_iterator.py 97.76% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3743      +/-   ##
==========================================
- Coverage   94.14%   94.11%   -0.04%     
==========================================
  Files          76       77       +1     
  Lines       15921    16012      +91     
==========================================
+ Hits        14989    15069      +80     
- Misses        932      943      +11     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mergify mergify Bot added the ci-passed label Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants