Repository navigation
taskqueue: route PushQueues to Cloud Tasks v2 (GA) - #414
Open
riddhi-shivhare wants to merge 5 commits into
Open
riddhi-shivhare wants to merge 5 commits into
riddhi-shivhare wants to merge 5 commits into
Conversation
riddhi-shivhare
force-pushed
the
taskqueue-v2-ga
branch
6 times, most recently
from
October 8, 2026 07:15
d9fb535 to
18bf8c2
Compare
When APPENGINE_USE_CLOUDTASK_PUSH_QUEUE is enabled, push queue
operations in both the root and v2 taskqueue packages are served by the
Cloud Tasks v2 API:
- Add and AddMulti use CreateTask and BatchCreateTasks.
- DeleteMulti uses BatchDeleteTasks.
- QueueStats uses v2beta3 GetQueue, since queue stats are not part of
the v2 API.
- Transactional tasks are staged with the datastore transaction and
dispatched after commit, with the existing 5-task limit.
Each staged task is a root entity, so a transaction that adds tasks must
be cross-group (XG) unless it touches no other entity group. Staged
tasks get a generated name so that a repeated dispatch is rejected as
ALREADY_EXISTS. They are dispatched before RunInTransaction returns,
while the request's API ticket is still valid. Each staged entity is
locked in a datastore transaction before dispatch, so the post-commit
path and the /_ah/cloudtask/sweep cron handler do not dispatch it
concurrently. A failed commit, including a concurrent transaction,
releases the tasks staged for it. Failures are recorded on the entity
until it is marked FAILED after 5 attempts. Each sweep reads at most
500 entities of each status, and FAILED entities are deleted after 7
days. Staged entities use the kind, property names and payload form of
the Java and Python SDKs, a JSON object {"task": T} where T is the
proto3 JSON form of the v2 Task, so a sweeper in a service written in
any of them can dispatch them. Properties written by another SDK are
ignored, and entities staged in the binary form used by earlier
versions are still dispatched.
Batch operations read the synchronous operation result and map
FailedRequests in the operation metadata back to per-task errors in
appengine.MultiError. NOT_FOUND on create maps to UNKNOWN_QUEUE. A Host
header that is a hostname of the app sets the task's service, version
and instance. Host, X-Google-* and X-AppEngine-* headers are not sent,
since Cloud Tasks sets the App Engine headers itself. AddMulti and
DeleteMulti with no tasks return without calling any API. Cloud Tasks
clients are created once and reused.
Adds cloud.google.com/go/cloudtasks v1.20.0 and unit tests in
taskqueue/cloudtask_test.go, taskqueue/cloudtask_transactional_test.go
and their v2 copies.
riddhi-shivhare
force-pushed
the
taskqueue-v2-ga
branch
from
October 8, 2026 08:32
18bf8c2 to
5242974
Compare
gajjarc
reviewed
Oct 8, 2026
gajjarc
left a comment
gajjarc
left a comment
Collaborator
There was a problem hiding this comment.
LGTM, lets wait for the tiny launch doc approval and have the public doc cl ready. Then we can safely merge.
gajjarc
approved these changes
Oct 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When APPENGINE_USE_CLOUDTASK_PUSH_QUEUE is enabled, push queue
operations in both the root and v2 taskqueue packages are served by
the Cloud Tasks v2 API:
the v2 API.
dispatched after commit, with the existing 5-task limit.
Batch operations wait on the long-running operation and map
FailedRequests back to per-task errors in appengine.MultiError. If the
operation itself fails, the error is reported for every task in the
chunk. NOT_FOUND on create maps to UNKNOWN_QUEUE. Cloud Tasks clients
are created once and reused. When the variable is not set, behavior is
unchanged.
This re-adds the Cloud Tasks backend reverted in #409
and supersedes #412, which targeted the
taskqueue-migration-preview branch.
Adds cloud.google.com/go/cloudtasks v1.20.0. The go directive in
go.mod and v2/go.mod moves to 1.26.0, and CI moves to Go 1.26.x.
Unit tests are in taskqueue/cloudtask_test.go and
v2/taskqueue/cloudtask_test.go.