From 163750c66f9591bff9a1956454ec857a9eeb9ff6 Mon Sep 17 00:00:00 2001 From: Gilles Barbier Date: Fri, 20 Nov 2026 12:05:39 +0100 Subject: [PATCH] Move reusable gRPC execution into common Rebuild the standalone gRPC extraction on current main under common/grpc and publish openworkflow-common-grpc through the common parent and root dependency management. Retain the existing Gson and Protobuf runtime alignment when leaving the implementation parent. Expose WorkflowGrpcCall and WorkflowProtobufJson using native grpc-java and Protobuf values. Keep workflow resources, expressions, model conversion, positioned errors and channel-provider policy in the existing GrpcExecutor adapter. Preserve its public constructor and result shapes, including the existing client/bidi argument convention. Add no request/response DTO hierarchy or payload cloning. Coordinate terminal delivery before cancellation callbacks, stop unfinished RPCs on local future timeouts or asynchronous completion, and release owned channels. Preserve supplier failure semantics and release raw transport references after SDK adaptation. Cover synchronous handlers, competing completions, native cancellation, deadlines, borrowed-channel reuse, status/cause/trailers and owned transport termination. Document reusable execution components as the third SDK usage in the root README, update the compiled common gRPC example and use generic engine descriptions. Keep per-message conversion, backpressure and stream resumption for separate work. The minimal common parent can be reconciled with the HTTP/OpenAPI reorganization when those changes merge. Validate 44 focused gRPC cases and the full reactor: 41 successful modules, 815 passing cases and 17 existing skips. Check repository formatting/style, common Javadocs, native dependency boundaries, BOM publication and the standalone README example. Retain validation and review logs, including intermediate timing-sensitive failures in unchanged tests. Signed-off-by: Gilles Barbier --- README.md | 10 +- common/grpc/README.md | 109 ++++ common/grpc/pom.xml | 45 ++ .../sdk/grpc/WorkflowGrpcCall.java | 249 ++++++++ .../sdk/grpc/WorkflowProtobufJson.java | 40 ++ .../sdk/grpc/WorkflowGrpcCallTest.java | 600 ++++++++++++++++++ .../sdk/grpc/WorkflowProtobufJsonTest.java | 102 +++ common/pom.xml | 14 + impl/README.md | 3 + impl/grpc/pom.xml | 4 + .../grpc/CollectionStreamObserver.java | 34 - .../sdk/impl/executors/grpc/GrpcExecutor.java | 277 ++++---- .../executors/grpc/ItemStreamObserver.java | 34 - .../executors/grpc/ModelStreamObserver.java | 54 -- .../executors/grpc/ProtobufMessageUtils.java | 10 +- impl/pom.xml | 6 - .../sdk/impl/test/grpc/GrpcUnaryTest.java | 250 ++++++++ pom.xml | 12 + 18 files changed, 1564 insertions(+), 289 deletions(-) create mode 100644 common/grpc/README.md create mode 100644 common/grpc/pom.xml create mode 100644 common/grpc/src/main/java/org/openworkflow/sdk/grpc/WorkflowGrpcCall.java create mode 100644 common/grpc/src/main/java/org/openworkflow/sdk/grpc/WorkflowProtobufJson.java create mode 100644 common/grpc/src/test/java/org/openworkflow/sdk/grpc/WorkflowGrpcCallTest.java create mode 100644 common/grpc/src/test/java/org/openworkflow/sdk/grpc/WorkflowProtobufJsonTest.java create mode 100644 common/pom.xml delete mode 100644 impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/CollectionStreamObserver.java delete mode 100644 impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/ItemStreamObserver.java delete mode 100644 impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/ModelStreamObserver.java diff --git a/README.md b/README.md index ceb6bd900..ae5c5e4c5 100644 --- a/README.md +++ b/README.md @@ -12,6 +12,14 @@ With the SDK you can: * Test your workflow definitions using the reference implementation. +## Three ways to use the SDK + +- Use the generated Java models to read and write specification documents in your own implementation. +- Run workflows with the customizable reference engine, standalone or embedded in an application. +- Reuse execution components from `common/` in another workflow engine. For example, add `org.openworkflow.sdk:openworkflow-common-grpc` to invoke a resolved gRPC operation without constructing SDK workflow contexts or adopting the SDK's data model. + +See the [common gRPC example](common/grpc/README.md). The reference implementation depends on these common components; the common components do not depend on the workflow engine. + ## Status | Latest Releases | Conformance to spec version | @@ -158,4 +166,4 @@ This project contains source code copied and modified from the following third-p * **Source:** [std-uritemplate GitHub Repository](https://github.com/std-uritemplate/std-uritemplate) * **File Copied:** `java/src/main/java/io/github/stduritemplate/StdUriTemplate.java` * **License:** Apache License 2.0 -* **Copyright:** Copyright 2024-2026 std-uritemplate contributors \ No newline at end of file +* **Copyright:** Copyright 2024-2026 std-uritemplate contributors diff --git a/common/grpc/README.md b/common/grpc/README.md new file mode 100644 index 000000000..31dcf8835 --- /dev/null +++ b/common/grpc/README.md @@ -0,0 +1,109 @@ +# Dynamic gRPC client + +`openworkflow-common-grpc` runs dynamic gRPC calls without the workflow engine, +DSL models, Jackson or generated service stubs. Its runtime dependencies are +grpc-java, Protobuf and their dependencies; the Maven parent supplies build +configuration only. Add a grpc-java transport provider, such as `grpc-netty`, +to your application. + +```xml + + org.openworkflow.sdk + openworkflow-common-grpc + 8.0.0-SNAPSHOT + +``` + +## Call from another engine + +This method can be used by an activity in another workflow engine. +The engine resolves its descriptor and JSON input before calling it. No +`WorkflowApplication` or generated service stub is needed. The equivalent call +is tested against a local server in `WorkflowGrpcCallTest`. + +```java +import com.google.protobuf.Descriptors; +import io.grpc.CallOptions; +import io.grpc.Channel; +import java.util.List; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.TimeUnit; +import org.openworkflow.sdk.grpc.WorkflowGrpcCall; +import org.openworkflow.sdk.grpc.WorkflowProtobufJson; + +public class GrpcActivity { + public static CompletableFuture> execute( + Channel channel, Descriptors.MethodDescriptor method, String json) + throws com.google.protobuf.InvalidProtocolBufferException { + var request = WorkflowProtobufJson.parse(json, method.getInputType()); + return new WorkflowGrpcCall(method).invoke( + channel, List.of(request), + CallOptions.DEFAULT.withDeadlineAfter(5, TimeUnit.SECONDS)); + } +} +``` + +Unary and server streaming accept exactly one request. Client streaming and +bidirectional streaming accept a finite list, send it in order and half-close +the request stream. Every mode returns a future of the collected response list; +client streaming normally contains one response. Calls are asynchronous and do +not wait on a future on the network path. A coroutine bridge should cancel the +**original returned future** when its coroutine is cancelled: cancelling a +future created by `thenApply` does not propagate cancellation upstream. + +## Options, errors and resource ownership + +- A caller-supplied `Channel` is never closed, including on error or cancellation. + Configure TLS, credentials, interceptors, metadata and connection pooling on + your channel. Per-call credentials, executor, deadlines and other `CallOptions` + are passed unchanged. Metadata can be attached through a channel interceptor. +- `CallOptions.DEFAULT` uses fail-fast and has no deadline. Use `withWaitForReady()` + to wait for connectivity; configure a deadline to bound that wait. Expired + deadlines fail with gRPC `DEADLINE_EXCEEDED` and cancel the transport call. +- Cancelling the returned future calls `ClientCall.cancel`. This stops the local + RPC and notifies the server; it cannot undo server work already performed. +- A local timeout applied to the original future with `orTimeout` or + `completeOnTimeout` also cancels an unfinished RPC. RPC cancellation and owned + channel cleanup are initiated before synchronous completion handlers run. +- Early completion with `complete`, `completeExceptionally` or `completeAsync` + also cancels an unfinished RPC; caller-supplied channels remain open. +- The address/port overload creates a plaintext channel per invocation and calls + `shutdown()` after transport completion, or `shutdownNow()` when the caller + abandons the RPC, including cancellation, local future timeouts and synchronous + setup failures. Shutdown is asynchronous; the library does not wait for termination. + Prefer the channel overload for TLS and reuse. +- Transport exceptions are preserved without workflow translation, including + status, cause and trailers supplied by grpc-java. Remote Java causes are not + serialized by the gRPC protocol. Setup/argument validation can throw + synchronously; errors after starting the call complete the future exceptionally. +- `WorkflowProtobufJson` uses the standard protobuf JSON parser/printer, including enum, + bytes and 64-bit number conventions; unknown fields are rejected. + +## Workflow adapter and limits + +`openworkflow-impl-grpc` retains `GrpcExecutor`, its public constructor, its builder +and `GRPC_CHANNEL_PROVIDER`. The adapter still resolves the DSL, resources, +expressions and arguments and uses the application's executor. It keeps +wait-for-ready with no additional deadline, converts results to `WorkflowModel` +and wraps transport failures in a workflow error with task position and original +cause. Retry policy belongs to the engine. Existing `.proto` loading and protoc +fallback remain there; this client accepts already built descriptors. + +The adapter preserves its existing result shapes: unary returns one model, +while all streaming modes (including client streaming) return a collection. +Its existing client/bidi request convention serializes each map entry as one +request; this extraction does not change that convention into a list-based DSL. + +Built Protobuf messages are passed directly without payload cloning. The client +snapshots the request and response lists by copying references, not messages, +to keep collection ownership explicit. + +Requests and responses are buffered in memory without a size limit. A stream +must terminate before the future delivers its collected result. There is no +incremental consumer, application backpressure contract, durable stream storage +or stream resumption after process restart. The adapter also retains raw +Protobuf responses until completion, then converts them to workflow models, +which can increase peak memory usage. Per-message conversion is left to a +separate change; this API still collects responses until completion. The SDK +result future releases its transport reference after adaptation so keeping the +converted result does not also retain the complete Protobuf response list. diff --git a/common/grpc/pom.xml b/common/grpc/pom.xml new file mode 100644 index 000000000..1742b4794 --- /dev/null +++ b/common/grpc/pom.xml @@ -0,0 +1,45 @@ + + 4.0.0 + + org.openworkflow.sdk + openworkflow-common + 8.0.0-SNAPSHOT + + openworkflow-common-grpc + Open Workflow :: gRPC Client + + + + com.google.protobuf + protobuf-java + ${version.com.google.protobuf.protobuf.java.util} + + + io.grpc + grpc-stub + + + io.grpc + grpc-protobuf + + + com.google.protobuf + protobuf-java-util + + + io.grpc + grpc-netty + test + + + org.junit.jupiter + junit-jupiter-engine + test + + + org.assertj + assertj-core + test + + + diff --git a/common/grpc/src/main/java/org/openworkflow/sdk/grpc/WorkflowGrpcCall.java b/common/grpc/src/main/java/org/openworkflow/sdk/grpc/WorkflowGrpcCall.java new file mode 100644 index 000000000..e7e671922 --- /dev/null +++ b/common/grpc/src/main/java/org/openworkflow/sdk/grpc/WorkflowGrpcCall.java @@ -0,0 +1,249 @@ +/* + * Copyright 2020-Present The Open Workflow Specification Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.openworkflow.sdk.grpc; + +import com.google.protobuf.Descriptors; +import com.google.protobuf.DynamicMessage; +import com.google.protobuf.Message; +import io.grpc.CallOptions; +import io.grpc.Channel; +import io.grpc.ClientCall; +import io.grpc.ManagedChannel; +import io.grpc.ManagedChannelBuilder; +import io.grpc.MethodDescriptor; +import io.grpc.protobuf.ProtoUtils; +import io.grpc.stub.ClientCalls; +import io.grpc.stub.StreamObserver; +import java.util.ArrayList; +import java.util.List; +import java.util.Objects; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CompletionException; +import java.util.concurrent.Executor; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.function.BooleanSupplier; +import java.util.function.Supplier; + +/** An immutable, reusable dynamic RPC. Responses are collected until the server completes. */ +public final class WorkflowGrpcCall { + private final MethodDescriptor descriptor; + + /** Builds the wire method and marshallers from an already resolved protobuf method. */ + public WorkflowGrpcCall(Descriptors.MethodDescriptor method) { + this(method.getService(), method); + } + + /** Builds a call using an explicit service descriptor, including an aliased service name. */ + public WorkflowGrpcCall( + Descriptors.ServiceDescriptor service, Descriptors.MethodDescriptor method) { + Objects.requireNonNull(service, "service"); + Objects.requireNonNull(method, "method"); + MethodDescriptor.MethodType type = + method.isClientStreaming() + ? (method.isServerStreaming() + ? MethodDescriptor.MethodType.BIDI_STREAMING + : MethodDescriptor.MethodType.CLIENT_STREAMING) + : (method.isServerStreaming() + ? MethodDescriptor.MethodType.SERVER_STREAMING + : MethodDescriptor.MethodType.UNARY); + descriptor = + MethodDescriptor.newBuilder() + .setType(type) + .setFullMethodName( + MethodDescriptor.generateFullMethodName(service.getFullName(), method.getName())) + .setRequestMarshaller( + ProtoUtils.marshaller(DynamicMessage.getDefaultInstance(method.getInputType()))) + .setResponseMarshaller( + ProtoUtils.marshaller(DynamicMessage.getDefaultInstance(method.getOutputType()))) + .build(); + } + + /** + * Invokes an RPC without owning or closing {@code channel}. Unary and server streaming require + * exactly one request; client and bidi streaming send all requests and then half-close. + * + *

The supplied options are used unchanged: DEFAULT has no deadline and uses fail-fast; + * withWaitForReady waits for connectivity, bounded only by a configured deadline or cancellation. + * Cancelling the returned future cancels the underlying ClientCall. Transport errors (including + * status, cause and trailers) complete the future exceptionally without translation. + */ + public CompletableFuture> invoke( + Channel channel, List requests, CallOptions options) { + return invoke(channel, requests, options, null); + } + + private CompletableFuture> invoke( + Channel channel, + List requests, + CallOptions options, + ManagedChannel ownedChannel) { + Objects.requireNonNull(channel, "channel"); + Objects.requireNonNull(options, "options"); + List messages = List.copyOf(requests); + if (!descriptor.getType().clientSendsOneMessage() || messages.size() == 1) { + return start(channel, messages, options, ownedChannel); + } + throw new IllegalArgumentException("Unary and server streaming calls require one request"); + } + + /** + * Uses a new plaintext channel for this invocation only. The library shuts it down on success, + * error or cancellation; no thread waits for termination. A transport provider must be on the + * classpath. Supply a configured Channel to the other overload for TLS or connection reuse. + */ + public CompletableFuture> invoke( + String address, int port, List requests, CallOptions options) { + ManagedChannel channel = ManagedChannelBuilder.forAddress(address, port).usePlaintext().build(); + try { + return invoke(channel, requests, options, channel); + } catch (RuntimeException | Error error) { + channel.shutdownNow(); + throw error; + } + } + + private CompletableFuture> start( + Channel channel, + List requests, + CallOptions options, + ManagedChannel ownedChannel) { + ClientCall call = channel.newCall(descriptor, options); + RpcFuture future = new RpcFuture(call, ownedChannel); + StreamObserver observer = + new StreamObserver<>() { + private final List responses = new ArrayList<>(); + + @Override + public void onNext(Message message) { + if (!future.terminal.get()) responses.add(message); + } + + @Override + public void onError(Throwable error) { + future.failFromTransport(error); + } + + @Override + public void onCompleted() { + future.completeFromTransport(List.copyOf(responses)); + } + }; + try { + switch (descriptor.getType()) { + case UNARY -> ClientCalls.asyncUnaryCall(call, requests.getFirst(), observer); + case SERVER_STREAMING -> + ClientCalls.asyncServerStreamingCall(call, requests.getFirst(), observer); + case CLIENT_STREAMING, BIDI_STREAMING -> { + StreamObserver sender = + descriptor.getType() == MethodDescriptor.MethodType.CLIENT_STREAMING + ? ClientCalls.asyncClientStreamingCall(call, observer) + : ClientCalls.asyncBidiStreamingCall(call, observer); + for (Message message : requests) { + if (future.isDone()) { + break; + } + sender.onNext(message); + } + if (!future.isDone()) { + sender.onCompleted(); + } + } + default -> throw new IllegalStateException("Unsupported method type"); + } + } catch (RuntimeException | Error error) { + future.completeExceptionally(error); + } + return future; + } + + /** Claim delivery before stopping the RPC, so synchronous callbacks cannot win cancellation. */ + private static final class RpcFuture extends CompletableFuture> { + private final ClientCall call; + private final ManagedChannel ownedChannel; + private final AtomicBoolean terminal = new AtomicBoolean(); + + private RpcFuture(ClientCall call, ManagedChannel ownedChannel) { + this.call = call; + this.ownedChannel = ownedChannel; + } + + private boolean completeFromTransport(List messages) { + return finish(true, null, () -> super.complete(messages)); + } + + private boolean failFromTransport(Throwable error) { + return finish(true, error, () -> super.completeExceptionally(error)); + } + + @Override + public boolean cancel(boolean mayInterruptIfRunning) { + return finish(false, null, () -> super.cancel(mayInterruptIfRunning)) || isCancelled(); + } + + @Override + public boolean complete(List messages) { + return finish(false, null, () -> super.complete(messages)); + } + + @Override + public boolean completeExceptionally(Throwable error) { + Objects.requireNonNull(error, "error"); + return finish(false, error, () -> super.completeExceptionally(error)); + } + + @Override + public CompletableFuture> completeAsync( + Supplier> supplier) { + return completeAsync(supplier, defaultExecutor()); + } + + @Override + public CompletableFuture> completeAsync( + Supplier> supplier, Executor executor) { + Objects.requireNonNull(supplier, "supplier"); + Objects.requireNonNull(executor, "executor"); + executor.execute(() -> { + if (terminal.get()) return; + try { + complete(supplier.get()); + } catch (Throwable error) { + completeExceptionally( + error instanceof CompletionException ? error : new CompletionException(error)); + } + }); + return this; + } + + private boolean finish(boolean rpcFinished, Throwable error, BooleanSupplier completion) { + if (!terminal.compareAndSet(false, true)) return false; + boolean completed; + try { + try { + if (!rpcFinished) call.cancel("RPC future completed before transport", error); + } finally { + if (ownedChannel != null) { + if (rpcFinished) ownedChannel.shutdown(); + else ownedChannel.shutdownNow(); + } + } + } finally { + // Complete only after initiating cleanup: caller handlers may block awaiting it. + completed = completion.getAsBoolean(); + } + return completed; + } + } +} diff --git a/common/grpc/src/main/java/org/openworkflow/sdk/grpc/WorkflowProtobufJson.java b/common/grpc/src/main/java/org/openworkflow/sdk/grpc/WorkflowProtobufJson.java new file mode 100644 index 000000000..c056f5693 --- /dev/null +++ b/common/grpc/src/main/java/org/openworkflow/sdk/grpc/WorkflowProtobufJson.java @@ -0,0 +1,40 @@ +/* + * Copyright 2020-Present The Open Workflow Specification Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.openworkflow.sdk.grpc; + +import com.google.protobuf.Descriptors; +import com.google.protobuf.DynamicMessage; +import com.google.protobuf.InvalidProtocolBufferException; +import com.google.protobuf.Message; +import com.google.protobuf.util.JsonFormat; + +/** JSON/Protobuf conversion independent of any workflow data model. */ +public final class WorkflowProtobufJson { + private WorkflowProtobufJson() {} + + /** Parses protobuf JSON using the supplied message descriptor. Unknown fields are rejected. */ + public static DynamicMessage parse(String json, Descriptors.Descriptor descriptor) + throws InvalidProtocolBufferException { + DynamicMessage.Builder builder = DynamicMessage.newBuilder(descriptor); + JsonFormat.parser().merge(json, builder); + return builder.build(); + } + + /** Prints protobuf JSON with the standard protobuf field, enum and numeric conventions. */ + public static String print(Message message) throws InvalidProtocolBufferException { + return JsonFormat.printer().print(message); + } +} diff --git a/common/grpc/src/test/java/org/openworkflow/sdk/grpc/WorkflowGrpcCallTest.java b/common/grpc/src/test/java/org/openworkflow/sdk/grpc/WorkflowGrpcCallTest.java new file mode 100644 index 000000000..c03853060 --- /dev/null +++ b/common/grpc/src/test/java/org/openworkflow/sdk/grpc/WorkflowGrpcCallTest.java @@ -0,0 +1,600 @@ +/* + * Copyright 2020-Present The Open Workflow Specification Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.openworkflow.sdk.grpc; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +import com.google.protobuf.DescriptorProtos; +import com.google.protobuf.Descriptors; +import com.google.protobuf.DynamicMessage; +import com.google.protobuf.Message; +import io.grpc.Attributes; +import io.grpc.CallOptions; +import io.grpc.Channel; +import io.grpc.ClientCall; +import io.grpc.Context; +import io.grpc.ManagedChannel; +import io.grpc.ManagedChannelBuilder; +import io.grpc.Metadata; +import io.grpc.MethodDescriptor; +import io.grpc.Server; +import io.grpc.ServerBuilder; +import io.grpc.ServerServiceDefinition; +import io.grpc.ServerTransportFilter; +import io.grpc.Status; +import io.grpc.StatusRuntimeException; +import io.grpc.protobuf.ProtoUtils; +import io.grpc.stub.ServerCalls; +import io.grpc.stub.StreamObserver; +import java.util.List; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.AtomicReference; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; + +/** Also serves as a compiled third-party-engine example: no workflow engine or generated stub. */ +class WorkflowGrpcCallTest { + private static final Metadata.Key DETAIL = + Metadata.Key.of("detail", Metadata.ASCII_STRING_MARSHALLER); + private static final Descriptors.ServiceDescriptor SERVICE = service(); + private final CountDownLatch started = new CountDownLatch(1); + private final CountDownLatch cancelled = new CountDownLatch(1); + private final AtomicInteger terminated = new AtomicInteger(); + private final AtomicReference receivedHeaders = new AtomicReference<>(); + private Server server; + private ManagedChannel channel; + + @BeforeEach + void setUp() throws Exception { + ServerServiceDefinition.Builder service = + ServerServiceDefinition.builder(SERVICE.getFullName()); + service.addMethod(wire("Unary"), ServerCalls.asyncUnaryCall(this::respond)); + service.addMethod( + wire("Server"), + ServerCalls.asyncServerStreamingCall((request, observer) -> { + observer.onNext(request); + observer.onNext(request); + observer.onCompleted(); + })); + service.addMethod( + wire("Client"), + ServerCalls.asyncClientStreamingCall(observer -> requests(observer, false))); + service.addMethod( + wire("Bidi"), ServerCalls.asyncBidiStreamingCall(observer -> requests(observer, true))); + server = + ServerBuilder.forPort(0) + .addService(service.build()) + .intercept( + new io.grpc.ServerInterceptor() { + @Override + public io.grpc.ServerCall.Listener interceptCall( + io.grpc.ServerCall call, + Metadata headers, + io.grpc.ServerCallHandler next) { + receivedHeaders.set(headers); + return next.startCall(call, headers); + } + }) + .addTransportFilter( + new ServerTransportFilter() { + @Override + public void transportTerminated(Attributes attributes) { + terminated.incrementAndGet(); + } + }) + .build() + .start(); + channel = + ManagedChannelBuilder.forAddress("localhost", server.getPort()).usePlaintext().build(); + } + + @AfterEach + void tearDown() throws Exception { + channel.shutdownNow(); + assertThat(channel.awaitTermination(10, TimeUnit.SECONDS)).isTrue(); + server.shutdownNow(); + assertThat(server.awaitTermination(10, TimeUnit.SECONDS)).isTrue(); + } + + @Test + void unary_shouldWorkWithoutWorkflowEngineOrGeneratedStubs() throws Exception { + assertThatThrownBy(() -> Class.forName("org.openworkflow.sdk.impl.WorkflowApplication")) + .isInstanceOf(ClassNotFoundException.class); + // A third-party engine supplies the resolved descriptor and JSON input. + Descriptors.MethodDescriptor method = SERVICE.findMethodByName("Unary"); + Message request = WorkflowProtobufJson.parse("{\"text\":\"hello\"}", method.getInputType()); + List responses = + new WorkflowGrpcCall(method) + .invoke(channel, List.of(request), CallOptions.DEFAULT) + .get(10, TimeUnit.SECONDS); + assertThat(WorkflowProtobufJson.print(responses.getFirst())).contains("hello"); + } + + @Test + void streaming_shouldCollectAllResponsesAfterHalfClose() throws Exception { + Message first = message("first"); + Message second = message("second"); + assertThat( + call("Server") + .invoke(channel, List.of(first), CallOptions.DEFAULT) + .get(10, TimeUnit.SECONDS)) + .containsExactly(first, first); + assertThat( + call("Client") + .invoke(channel, List.of(first, second), CallOptions.DEFAULT) + .get(10, TimeUnit.SECONDS)) + .containsExactly(message("first,second")); + assertThat( + call("Bidi") + .invoke(channel, List.of(first, second), CallOptions.DEFAULT) + .get(10, TimeUnit.SECONDS)) + .containsExactly(first, second); + assertThat( + call("Bidi").invoke(channel, List.of(), CallOptions.DEFAULT).get(10, TimeUnit.SECONDS)) + .isEmpty(); + } + + @Test + void errors_shouldRetainStatusAndTrailersAndLeaveChannelReusable() throws Exception { + CompletableFuture> result = + call("Unary").invoke(channel, List.of(message("error")), CallOptions.DEFAULT); + assertThatThrownBy(() -> result.get(10, TimeUnit.SECONDS)) + .isInstanceOf(ExecutionException.class) + .cause() + .isInstanceOfSatisfying( + StatusRuntimeException.class, + error -> { + assertThat(error.getStatus().getCode()).isEqualTo(Status.Code.INVALID_ARGUMENT); + assertThat(error.getStatus().getDescription()).isEqualTo("bad request"); + assertThat(error.getTrailers().get(DETAIL)).isEqualTo("preserved"); + }); + assertReusable(); + } + + @Test + void cancellation_shouldReachServerAndLeaveChannelReusable() throws Exception { + CompletableFuture> result = + call("Unary").invoke(channel, List.of(message("wait")), CallOptions.DEFAULT); + assertThat(started.await(10, TimeUnit.SECONDS)).isTrue(); + assertThat(result.cancel(true)).isTrue(); + assertThat(result.isCancelled()).isTrue(); + assertThat(cancelled.await(10, TimeUnit.SECONDS)).isTrue(); + assertReusable(); + } + + @Test + void deadline_shouldReachServerAndPreserveDeadlineStatus() throws Exception { + CompletableFuture> result = + call("Unary") + .invoke( + channel, + List.of(message("wait")), + CallOptions.DEFAULT.withDeadlineAfter(2, TimeUnit.SECONDS)); + assertThat(started.await(10, TimeUnit.SECONDS)).isTrue(); + assertThatThrownBy(() -> result.get(10, TimeUnit.SECONDS)) + .cause() + .isInstanceOfSatisfying( + StatusRuntimeException.class, + error -> + assertThat(error.getStatus().getCode()).isEqualTo(Status.Code.DEADLINE_EXCEEDED)); + assertThat(cancelled.await(10, TimeUnit.SECONDS)).isTrue(); + assertReusable(); + } + + @Test + void cancellation_shouldReachServerBeforeSynchronousCompletionHandlers() throws Exception { + var result = call("Unary").invoke(channel, List.of(message("wait")), CallOptions.DEFAULT); + assertThat(started.await(10, TimeUnit.SECONDS)).isTrue(); + var acknowledged = new java.util.concurrent.atomic.AtomicBoolean(); + result.whenComplete((value, error) -> { + try { + acknowledged.set(cancelled.await(5, TimeUnit.SECONDS)); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + } + }); + assertThat(result.cancel(true)).isTrue(); + assertThat(result.isCancelled()).isTrue(); + assertThat(acknowledged.get()).isTrue(); + assertReusable(); + } + + @Test + void ownedChannels_shouldTerminateAfterLocalFutureTimeout() throws Exception { + var result = + call("Unary") + .invoke("localhost", server.getPort(), List.of(message("wait")), CallOptions.DEFAULT); + assertThat(started.await(10, TimeUnit.SECONDS)).isTrue(); + result.orTimeout(100, TimeUnit.MILLISECONDS); + assertThatThrownBy(() -> result.get(10, TimeUnit.SECONDS)) + .hasCauseInstanceOf(java.util.concurrent.TimeoutException.class); + assertThat(cancelled.await(5, TimeUnit.SECONDS)).isTrue(); + awaitTerminated(1); + } + + @Test + void localFutureTimeout_shouldCancelOnlyTheBorrowedRpc() throws Exception { + var result = call("Unary").invoke(channel, List.of(message("wait")), CallOptions.DEFAULT); + assertThat(started.await(10, TimeUnit.SECONDS)).isTrue(); + result.orTimeout(100, TimeUnit.MILLISECONDS); + assertThatThrownBy(() -> result.get(10, TimeUnit.SECONDS)) + .hasCauseInstanceOf(java.util.concurrent.TimeoutException.class); + assertThat(cancelled.await(5, TimeUnit.SECONDS)).isTrue(); + assertReusable(); + } + + @Test + void ownedChannels_shouldTerminateAfterCompleteOnTimeout() throws Exception { + var result = + call("Unary") + .invoke("localhost", server.getPort(), List.of(message("wait")), CallOptions.DEFAULT); + assertThat(started.await(10, TimeUnit.SECONDS)).isTrue(); + result.completeOnTimeout(List.of(), 100, TimeUnit.MILLISECONDS); + assertThat(result.get(10, TimeUnit.SECONDS)).isEmpty(); + assertThat(cancelled.await(5, TimeUnit.SECONDS)).isTrue(); + awaitTerminated(1); + } + + @Test + void ownedChannels_shouldTerminateAfterAsyncCompletion() throws Exception { + var result = + call("Unary") + .invoke("localhost", server.getPort(), List.of(message("wait")), CallOptions.DEFAULT); + assertThat(started.await(10, TimeUnit.SECONDS)).isTrue(); + assertThat(result.completeAsync(List::of, Runnable::run)).isSameAs(result); + assertThat(result.get(10, TimeUnit.SECONDS)).isEmpty(); + assertThat(cancelled.await(5, TimeUnit.SECONDS)).isTrue(); + awaitTerminated(1); + } + + @Test + void asyncFailure_shouldCancelOnlyTheBorrowedRpc() throws Exception { + var result = call("Unary").invoke(channel, List.of(message("wait")), CallOptions.DEFAULT); + assertThat(started.await(10, TimeUnit.SECONDS)).isTrue(); + var failure = new IllegalStateException("fallback failed"); + result.completeAsync(() -> { + throw failure; + }); + assertThatThrownBy(() -> result.get(10, TimeUnit.SECONDS)).hasCause(failure); + assertThat(cancelled.await(5, TimeUnit.SECONDS)).isTrue(); + assertReusable(); + } + + @Test + void asyncCancellationFailure_shouldKeepTheFutureExceptionallyCompleted() throws Exception { + var result = call("Unary").invoke(channel, List.of(message("wait")), CallOptions.DEFAULT); + assertThat(started.await(10, TimeUnit.SECONDS)).isTrue(); + var failure = new java.util.concurrent.CancellationException("supplier failed"); + result.completeAsync(() -> { + throw failure; + }); + assertThatThrownBy(() -> result.get(10, TimeUnit.SECONDS)) + .isInstanceOf(ExecutionException.class) + .hasCause(failure); + assertThat(result.isCancelled()).isFalse(); + assertThat(cancelled.await(5, TimeUnit.SECONDS)).isTrue(); + assertReusable(); + } + + @Test + void options_shouldBePassedUnchangedIncludingWaitForReadyAndExecutor() throws Exception { + AtomicReference actual = new AtomicReference<>(); + Channel recording = + new Channel() { + @Override + public String authority() { + return channel.authority(); + } + + @Override + public ClientCall newCall( + MethodDescriptor method, CallOptions options) { + actual.set(options); + return channel.newCall(method, options); + } + }; + CallOptions options = + CallOptions.DEFAULT + .withWaitForReady() + .withExecutor(Runnable::run) + .withDeadlineAfter(10, TimeUnit.SECONDS); + call("Unary").invoke(recording, List.of(message("ok")), options).get(10, TimeUnit.SECONDS); + assertThat(actual.get()).isSameAs(options); + assertThat(actual.get().isWaitForReady()).isTrue(); + } + + @Test + void configuredChannel_shouldSupportInterceptorsMetadataAndPerCallCredentials() throws Exception { + Metadata headers = new Metadata(); + headers.put(DETAIL, "intercepted"); + Metadata.Key auth = Metadata.Key.of("authorization", Metadata.ASCII_STRING_MARSHALLER); + io.grpc.CallCredentials credentials = + new io.grpc.CallCredentials() { + @Override + public void applyRequestMetadata( + RequestInfo info, java.util.concurrent.Executor executor, MetadataApplier applier) { + executor.execute(() -> { + Metadata values = new Metadata(); + values.put(auth, "Bearer example"); + applier.apply(values); + }); + } + + @Override + public void thisUsesUnstableApi() {} + }; + Channel configured = + io.grpc.ClientInterceptors.intercept( + channel, io.grpc.stub.MetadataUtils.newAttachHeadersInterceptor(headers)); + call("Unary") + .invoke( + configured, + List.of(message("ok")), + CallOptions.DEFAULT.withCallCredentials(credentials)) + .get(10, TimeUnit.SECONDS); + assertThat(receivedHeaders.get().get(DETAIL)).isEqualTo("intercepted"); + assertThat(receivedHeaders.get().get(auth)).isEqualTo("Bearer example"); + assertReusable(); + } + + @Test + void ownedChannels_shouldTerminateAfterSuccessErrorAndCancellation() throws Exception { + call("Unary") + .invoke("localhost", server.getPort(), List.of(message("ok")), CallOptions.DEFAULT) + .get(10, TimeUnit.SECONDS); + awaitTerminated(1); + CompletableFuture> error = + call("Unary") + .invoke("localhost", server.getPort(), List.of(message("error")), CallOptions.DEFAULT); + assertThatThrownBy(() -> error.get(10, TimeUnit.SECONDS)) + .isInstanceOf(ExecutionException.class); + awaitTerminated(2); + CompletableFuture> pending = + call("Unary") + .invoke("localhost", server.getPort(), List.of(message("wait")), CallOptions.DEFAULT); + assertThat(started.await(10, TimeUnit.SECONDS)).isTrue(); + pending.cancel(false); + assertThat(cancelled.await(10, TimeUnit.SECONDS)).isTrue(); + awaitTerminated(3); + } + + @Test + void waitForReady_shouldWaitForConnectivityUntilDeadline() throws Exception { + int unusedPort; + try (java.net.ServerSocket reservation = new java.net.ServerSocket(0)) { + unusedPort = reservation.getLocalPort(); + } + ManagedChannel unavailable = + ManagedChannelBuilder.forAddress("localhost", unusedPort).usePlaintext().build(); + try { + CompletableFuture> failFast = + call("Unary") + .invoke( + unavailable, + List.of(message("ok")), + CallOptions.DEFAULT.withDeadlineAfter(5, TimeUnit.SECONDS)); + assertThatThrownBy(() -> failFast.get(10, TimeUnit.SECONDS)) + .cause() + .isInstanceOfSatisfying( + StatusRuntimeException.class, + error -> assertThat(error.getStatus().getCode()).isEqualTo(Status.Code.UNAVAILABLE)); + CompletableFuture> waiting = + call("Unary") + .invoke( + unavailable, + List.of(message("ok")), + CallOptions.DEFAULT + .withWaitForReady() + .withDeadlineAfter(200, TimeUnit.MILLISECONDS)); + assertThatThrownBy(() -> waiting.get(10, TimeUnit.SECONDS)) + .cause() + .isInstanceOfSatisfying( + StatusRuntimeException.class, + error -> + assertThat(error.getStatus().getCode()).isEqualTo(Status.Code.DEADLINE_EXCEEDED)); + } finally { + unavailable.shutdownNow(); + assertThat(unavailable.awaitTermination(10, TimeUnit.SECONDS)).isTrue(); + } + } + + @Test + void localErrors_shouldRetainOriginalCauseAndTrailers() { + Throwable cause = new IllegalStateException("local transport failure"); + Metadata trailers = new Metadata(); + trailers.put(DETAIL, "local"); + Channel failing = + new Channel() { + @Override + public String authority() { + return "local"; + } + + @Override + public ClientCall newCall( + MethodDescriptor method, CallOptions options) { + return new ClientCall<>() { + @Override + public void start(Listener listener, Metadata headers) { + listener.onClose(Status.INTERNAL.withCause(cause), trailers); + } + + @Override + public void request(int count) {} + + @Override + public void cancel(String message, Throwable error) {} + + @Override + public void halfClose() {} + + @Override + public void sendMessage(ReqT message) {} + }; + } + }; + var result = call("Unary").invoke(failing, List.of(message("ok")), CallOptions.DEFAULT); + assertThatThrownBy(() -> result.get(10, TimeUnit.SECONDS)) + .cause() + .isInstanceOfSatisfying( + StatusRuntimeException.class, + error -> { + assertThat(error.getCause()).isSameAs(cause); + assertThat(error.getTrailers()).isSameAs(trailers); + }); + } + + @Test + void invalidRequestCount_shouldRejectBeforeStartingCall() { + assertThatThrownBy(() -> call("Unary").invoke(channel, List.of(), CallOptions.DEFAULT)) + .isInstanceOf(IllegalArgumentException.class); + assertThatThrownBy(() -> + call("Server") + .invoke(channel, List.of(message("a"), message("b")), CallOptions.DEFAULT)) + .isInstanceOf(IllegalArgumentException.class); + } + + private void assertReusable() throws Exception { + assertThat(channel.isShutdown()).isFalse(); + for (int i = 0; i < 2; i++) { + assertThat( + call("Unary") + .invoke(channel, List.of(message("ok")), CallOptions.DEFAULT) + .get(10, TimeUnit.SECONDS)) + .containsExactly(message("ok")); + } + assertThat(channel.isShutdown()).isFalse(); + } + + private void awaitTerminated(int count) throws Exception { + long end = System.nanoTime() + TimeUnit.SECONDS.toNanos(10); + while (terminated.get() < count && System.nanoTime() < end) { + TimeUnit.MILLISECONDS.sleep(10); + } + assertThat(terminated.get()).isEqualTo(count); + } + + private void respond(Message request, StreamObserver observer) { + String text = (String) request.getField(request.getDescriptorForType().findFieldByName("text")); + if (text.equals("wait")) { + Context.current().addListener(context -> cancelled.countDown(), Runnable::run); + started.countDown(); + } else if (text.equals("error")) { + Metadata trailers = new Metadata(); + trailers.put(DETAIL, "preserved"); + observer.onError( + Status.INVALID_ARGUMENT.withDescription("bad request").asRuntimeException(trailers)); + } else { + observer.onNext(request); + observer.onCompleted(); + } + } + + private StreamObserver requests(StreamObserver observer, boolean bidi) { + StringBuilder collected = new StringBuilder(); + return new StreamObserver<>() { + @Override + public void onNext(Message value) { + if (bidi) { + observer.onNext(value); + } else { + if (!collected.isEmpty()) collected.append(','); + collected.append(value.getField(value.getDescriptorForType().findFieldByName("text"))); + } + } + + @Override + public void onError(Throwable error) {} + + @Override + public void onCompleted() { + if (!bidi) observer.onNext(message(collected.toString())); + observer.onCompleted(); + } + }; + } + + private static Message message(String text) { + Descriptors.Descriptor descriptor = SERVICE.findMethodByName("Unary").getInputType(); + return DynamicMessage.newBuilder(descriptor) + .setField(descriptor.findFieldByName("text"), text) + .build(); + } + + private static WorkflowGrpcCall call(String name) { + return new WorkflowGrpcCall(SERVICE.findMethodByName(name)); + } + + private static MethodDescriptor wire(String name) { + Descriptors.MethodDescriptor method = SERVICE.findMethodByName(name); + MethodDescriptor.MethodType type = + switch (name) { + case "Client" -> MethodDescriptor.MethodType.CLIENT_STREAMING; + case "Server" -> MethodDescriptor.MethodType.SERVER_STREAMING; + case "Bidi" -> MethodDescriptor.MethodType.BIDI_STREAMING; + default -> MethodDescriptor.MethodType.UNARY; + }; + return MethodDescriptor.newBuilder() + .setType(type) + .setFullMethodName(SERVICE.getFullName() + "/" + name) + .setRequestMarshaller( + ProtoUtils.marshaller(DynamicMessage.getDefaultInstance(method.getInputType()))) + .setResponseMarshaller( + ProtoUtils.marshaller(DynamicMessage.getDefaultInstance(method.getOutputType()))) + .build(); + } + + private static Descriptors.ServiceDescriptor service() { + DescriptorProtos.ServiceDescriptorProto.Builder service = + DescriptorProtos.ServiceDescriptorProto.newBuilder().setName("Echo"); + for (String name : List.of("Unary", "Server", "Client", "Bidi")) { + service.addMethod( + DescriptorProtos.MethodDescriptorProto.newBuilder() + .setName(name) + .setInputType(".example.Payload") + .setOutputType(".example.Payload") + .setClientStreaming(name.equals("Client") || name.equals("Bidi")) + .setServerStreaming(name.equals("Server") || name.equals("Bidi"))); + } + try { + return Descriptors.FileDescriptor.buildFrom( + DescriptorProtos.FileDescriptorProto.newBuilder() + .setName("example.proto") + .setPackage("example") + .setSyntax("proto3") + .addMessageType( + DescriptorProtos.DescriptorProto.newBuilder() + .setName("Payload") + .addField( + DescriptorProtos.FieldDescriptorProto.newBuilder() + .setName("text") + .setNumber(1) + .setType(DescriptorProtos.FieldDescriptorProto.Type.TYPE_STRING))) + .addService(service) + .build(), + new Descriptors.FileDescriptor[0]) + .findServiceByName("Echo"); + } catch (Descriptors.DescriptorValidationException error) { + throw new ExceptionInInitializerError(error); + } + } +} diff --git a/common/grpc/src/test/java/org/openworkflow/sdk/grpc/WorkflowProtobufJsonTest.java b/common/grpc/src/test/java/org/openworkflow/sdk/grpc/WorkflowProtobufJsonTest.java new file mode 100644 index 000000000..c9c1fdafe --- /dev/null +++ b/common/grpc/src/test/java/org/openworkflow/sdk/grpc/WorkflowProtobufJsonTest.java @@ -0,0 +1,102 @@ +/* + * Copyright 2020-Present The Open Workflow Specification Authors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.openworkflow.sdk.grpc; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +import com.google.protobuf.ByteString; +import com.google.protobuf.DescriptorProtos; +import com.google.protobuf.Descriptors; +import com.google.protobuf.DynamicMessage; +import com.google.protobuf.InvalidProtocolBufferException; +import com.google.protobuf.Type; +import org.junit.jupiter.api.Test; + +class WorkflowProtobufJsonTest { + @Test + void conversion_shouldPreserveNestedRepeatedEnumAndInt64Conventions() throws Exception { + String json = + """ + {"name":"example", "syntax":"SYNTAX_PROTO3", "fields":[ + {"name":"value", "kind":"TYPE_INT64", "number":1}], + "options":[{"name":"enabled", "value":{"@type":"type.googleapis.com/google.protobuf.StringValue", "value":"yes"}}]} + """; + // Any requires an explicit registry, just as the existing SDK converter does. + assertThatThrownBy(() -> WorkflowProtobufJson.parse(json, Type.getDescriptor())) + .isInstanceOf(InvalidProtocolBufferException.class); + String supported = + """ + {"name":"example", "syntax":"SYNTAX_PROTO3", "fields":[ + {"name":"value", "kind":"TYPE_INT64", "number":1}]} + """; + var message = WorkflowProtobufJson.parse(supported, Type.getDescriptor()); + Type expected = + Type.newBuilder() + .setName("example") + .setSyntax(com.google.protobuf.Syntax.SYNTAX_PROTO3) + .addFields( + com.google.protobuf.Field.newBuilder() + .setName("value") + .setKind(com.google.protobuf.Field.Kind.TYPE_INT64) + .setNumber(1)) + .build(); + assertThat(message.toByteString()).isEqualTo(expected.toByteString()); + assertThat( + WorkflowProtobufJson.parse(WorkflowProtobufJson.print(message), Type.getDescriptor())) + .isEqualTo(message); + } + + @Test + void conversion_shouldPreserveBytesAndInt64JsonRepresentation() throws Exception { + var descriptor = + Descriptors.FileDescriptor.buildFrom( + DescriptorProtos.FileDescriptorProto.newBuilder() + .setName("numbers.proto") + .setSyntax("proto3") + .addMessageType( + DescriptorProtos.DescriptorProto.newBuilder() + .setName("Data") + .addField( + DescriptorProtos.FieldDescriptorProto.newBuilder() + .setName("count") + .setNumber(1) + .setType(DescriptorProtos.FieldDescriptorProto.Type.TYPE_INT64)) + .addField( + DescriptorProtos.FieldDescriptorProto.newBuilder() + .setName("data") + .setNumber(2) + .setType( + DescriptorProtos.FieldDescriptorProto.Type.TYPE_BYTES))) + .build(), + new Descriptors.FileDescriptor[0]) + .findMessageTypeByName("Data"); + var message = + DynamicMessage.newBuilder(descriptor) + .setField(descriptor.findFieldByName("count"), Long.MAX_VALUE) + .setField(descriptor.findFieldByName("data"), ByteString.copyFromUtf8("hello")) + .build(); + String json = WorkflowProtobufJson.print(message); + assertThat(json).contains("\"9223372036854775807\"", "aGVsbG8="); + assertThat(WorkflowProtobufJson.parse(json, descriptor)).isEqualTo(message); + } + + @Test + void parser_shouldRejectUnknownFields() { + assertThatThrownBy(() -> WorkflowProtobufJson.parse("{\"unknown\":true}", Type.getDescriptor())) + .isInstanceOf(InvalidProtocolBufferException.class); + } +} diff --git a/common/pom.xml b/common/pom.xml new file mode 100644 index 000000000..1d6e59909 --- /dev/null +++ b/common/pom.xml @@ -0,0 +1,14 @@ + + 4.0.0 + + org.openworkflow.sdk + openworkflow-parent + 8.0.0-SNAPSHOT + + openworkflow-common + Open Workflow :: Common + pom + + grpc + + diff --git a/impl/README.md b/impl/README.md index 1468c9bba..c4851c837 100644 --- a/impl/README.md +++ b/impl/README.md @@ -313,3 +313,6 @@ Apache License 2.0. See `LICENSE` for details. --- *Questions or ideas? PRs and issues welcome!* + +Dynamic gRPC calls can also be used independently of the workflow engine; see the +[common gRPC component](../common/grpc/README.md) for the API, ownership and streaming limits. diff --git a/impl/grpc/pom.xml b/impl/grpc/pom.xml index 781e7b2cb..4a93fb676 100644 --- a/impl/grpc/pom.xml +++ b/impl/grpc/pom.xml @@ -9,6 +9,10 @@ Open Workflow :: Impl :: gRPC + + org.openworkflow.sdk + openworkflow-common-grpc + org.openworkflow.sdk openworkflow-impl-core diff --git a/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/CollectionStreamObserver.java b/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/CollectionStreamObserver.java deleted file mode 100644 index a8a464afa..000000000 --- a/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/CollectionStreamObserver.java +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Copyright 2020-Present The Open Workflow Specification Authors - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.openworkflow.sdk.impl.executors.grpc; - -import com.google.protobuf.Message; -import org.openworkflow.sdk.impl.TaskContext; -import org.openworkflow.sdk.impl.WorkflowContext; -import org.openworkflow.sdk.impl.WorkflowModelCollection; - -class CollectionStreamObserver extends ModelStreamObserver { - - public CollectionStreamObserver(WorkflowContext workflowContext, TaskContext taskContext) { - super(workflowContext, taskContext); - this.model = modelFactory.createCollection(); - } - - @Override - public void onNext(Message value) { - model.add(ProtobufMessageUtils.convert(value, modelFactory)); - } -} diff --git a/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/GrpcExecutor.java b/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/GrpcExecutor.java index 31102fce5..8271b7b1e 100644 --- a/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/GrpcExecutor.java +++ b/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/GrpcExecutor.java @@ -15,32 +15,36 @@ */ package org.openworkflow.sdk.impl.executors.grpc; -import com.google.protobuf.DescriptorProtos; import com.google.protobuf.Descriptors; import com.google.protobuf.DynamicMessage; import com.google.protobuf.Message; import io.grpc.CallOptions; import io.grpc.Channel; -import io.grpc.ClientCall; -import io.grpc.ManagedChannel; -import io.grpc.ManagedChannelBuilder; -import io.grpc.MethodDescriptor; -import io.grpc.MethodDescriptor.MethodType; -import io.grpc.protobuf.ProtoUtils; -import io.grpc.stub.ClientCalls; -import io.grpc.stub.StreamObserver; import java.io.IOException; +import java.util.ArrayList; +import java.util.List; import java.util.Map; +import java.util.Objects; import java.util.Optional; import java.util.concurrent.CompletableFuture; -import java.util.function.UnaryOperator; +import java.util.concurrent.CompletionException; +import java.util.concurrent.Executor; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicReference; +import java.util.function.BooleanSupplier; +import java.util.function.Supplier; +import org.openworkflow.sdk.grpc.WorkflowGrpcCall; import org.openworkflow.sdk.impl.TaskContext; import org.openworkflow.sdk.impl.WorkflowContext; +import org.openworkflow.sdk.impl.WorkflowError; +import org.openworkflow.sdk.impl.WorkflowException; import org.openworkflow.sdk.impl.WorkflowModel; import org.openworkflow.sdk.impl.WorkflowModelCollection; +import org.openworkflow.sdk.impl.WorkflowModelFactory; import org.openworkflow.sdk.impl.WorkflowValueResolver; import org.openworkflow.sdk.impl.executors.CallableTask; +/** Adapts workflow arguments, channel providers, results and errors to dynamic gRPC calls. */ public class GrpcExecutor implements CallableTask { public static final String GRPC_CHANNEL_PROVIDER = "grpcChannelProvider"; @@ -49,18 +53,7 @@ public class GrpcExecutor implements CallableTask { private final Descriptors.MethodDescriptor methodDescriptor; private final String address; private final int port; - private final GRPCOperation operation; - private final MethodDescriptor callDescriptor; - - @FunctionalInterface - private interface GRPCOperation { - CompletableFuture apply( - WorkflowContext workflowContext, - TaskContext taskContext, - Map parameters, - Descriptors.MethodDescriptor methodDescriptor, - ClientCall call); - } + private final WorkflowGrpcCall call; public GrpcExecutor( String address, @@ -72,164 +65,138 @@ public GrpcExecutor( this.port = port; this.arguments = arguments; this.methodDescriptor = methodDescriptor; - MethodType methodType = getMethodType(methodDescriptor); - this.operation = - switch (methodType) { - case CLIENT_STREAMING -> GrpcExecutor::handleClientStreaming; - case BIDI_STREAMING -> GrpcExecutor::handleBidiStreaming; - case SERVER_STREAMING -> GrpcExecutor::handleServerStreaming; - case UNARY, UNKNOWN -> GrpcExecutor::handleAsyncUnary; - }; - this.callDescriptor = - MethodDescriptor.newBuilder() - .setType(methodType) - .setFullMethodName( - MethodDescriptor.generateFullMethodName( - serviceDescriptor.getFullName(), methodDescriptor.getName())) - .setRequestMarshaller( - ProtoUtils.marshaller( - DynamicMessage.newBuilder(methodDescriptor.getInputType()).buildPartial())) - .setResponseMarshaller( - ProtoUtils.marshaller( - DynamicMessage.newBuilder(methodDescriptor.getOutputType()).buildPartial())) - .build(); + this.call = new WorkflowGrpcCall(serviceDescriptor, methodDescriptor); } @Override public CompletableFuture apply( WorkflowContext workflowContext, TaskContext taskContext, WorkflowModel input) { - return buildGrpcCallExecutor( - workflowContext, taskContext, this.arguments.apply(workflowContext, taskContext, input)); - } - - private CompletableFuture buildGrpcCallExecutor( - WorkflowContext workflowContext, TaskContext taskContext, Map arguments) { - Optional providedChannel = + Map parameters = arguments.apply(workflowContext, taskContext, input); + Optional channel = workflowContext .definition() .application() - .additionalObject(GRPC_CHANNEL_PROVIDER, workflowContext, taskContext); - Channel channel = - providedChannel.orElseGet( - () -> ManagedChannelBuilder.forAddress(address, port).usePlaintext().build()); - ClientCall call = - channel.newCall( - callDescriptor, - CallOptions.DEFAULT - .withWaitForReady() - .withExecutor(workflowContext.definition().application().executorService())); - CompletableFuture result = null; + .additionalObject(GRPC_CHANNEL_PROVIDER, workflowContext, taskContext); + List requests = new ArrayList<>(); try { - result = operation.apply(workflowContext, taskContext, arguments, methodDescriptor, call); - } finally { - if (providedChannel.isEmpty() && channel instanceof ManagedChannel managedChannel) { - if (result == null) { - managedChannel.shutdown(); - } else { - result = result.whenComplete((__, ___) -> managedChannel.shutdown()); + if (methodDescriptor.isClientStreaming()) { + // Preserve the existing DSL contract: each map entry becomes one request message. + for (Object entry : parameters.entrySet()) { + requests.add( + ProtobufMessageUtils.buildMessage( + entry, DynamicMessage.newBuilder(methodDescriptor.getInputType())) + .build()); } + } else { + requests.add(ProtobufMessageUtils.buildMessage(methodDescriptor, parameters).build()); } + } catch (IOException error) { + return CompletableFuture.failedFuture(workflowException(taskContext, error)); } + CallOptions options = + CallOptions.DEFAULT + .withWaitForReady() + .withExecutor(workflowContext.definition().application().executorService()); + CompletableFuture> transport = + channel.isPresent() + ? call.invoke(channel.get(), requests, options) + : call.invoke(address, port, requests, options); + SdkFuture result = new SdkFuture(transport); + transport + .whenComplete((messages, error) -> { + if (result.terminal.get()) return; + if (error != null) { + result.completeExceptionally(workflowException(taskContext, error)); + return; + } + try { + WorkflowModelFactory factory = + workflowContext.definition().application().modelFactory(); + if (methodDescriptor.isClientStreaming() || methodDescriptor.isServerStreaming()) { + WorkflowModelCollection collection = factory.createCollection(); + messages.forEach( + message -> collection.add(ProtobufMessageUtils.convert(message, factory))); + result.complete(collection); + } else { + result.complete( + messages.isEmpty() + ? factory.fromNull() + : ProtobufMessageUtils.convert(messages.getLast(), factory)); + } + } catch (RuntimeException conversionError) { + result.completeExceptionally(workflowException(taskContext, conversionError)); + } + }) + .whenComplete((ignored, callbackError) -> { + if (callbackError != null) result.completeExceptionally(callbackError); + }); return result; } - private static CompletableFuture handleClientStreaming( - WorkflowContext workflowContext, - TaskContext taskContext, - Map parameters, - Descriptors.MethodDescriptor methodDescriptor, - ClientCall call) { - return asyncStreamingCall( - parameters, - methodDescriptor, - responseObserver -> ClientCalls.asyncClientStreamingCall(call, responseObserver), - workflowContext, - taskContext); + private static WorkflowException workflowException(TaskContext taskContext, Throwable error) { + return new WorkflowException(WorkflowError.runtime(taskContext, error).build(), error); } - private static CompletableFuture handleBidiStreaming( - WorkflowContext workflowContext, - TaskContext taskContext, - Map parameters, - Descriptors.MethodDescriptor methodDescriptor, - ClientCall call) { - return asyncStreamingCall( - parameters, - methodDescriptor, - responseObserver -> ClientCalls.asyncBidiStreamingCall(call, responseObserver), - workflowContext, - taskContext); - } + /** + * Owns result delivery, but releases raw responses as soon as cancellation is no longer possible. + */ + private static final class SdkFuture extends CompletableFuture { + private final AtomicBoolean terminal = new AtomicBoolean(); + private final AtomicReference> upstream; - private static CompletableFuture handleServerStreaming( - WorkflowContext workflowContext, - TaskContext taskContext, - Map parameters, - Descriptors.MethodDescriptor methodDescriptor, - ClientCall call) { - ModelStreamObserver observer = - new CollectionStreamObserver(workflowContext, taskContext); - try { - ClientCalls.asyncServerStreamingCall( - call, ProtobufMessageUtils.buildMessage(methodDescriptor, parameters).build(), observer); - } catch (IOException io) { - observer.onError(io); + private SdkFuture(CompletableFuture transport) { + upstream = new AtomicReference<>(transport); } - return observer.future(); - } - private static CompletableFuture handleAsyncUnary( - WorkflowContext workflowContext, - TaskContext taskContext, - Map parameters, - Descriptors.MethodDescriptor methodDescriptor, - ClientCall call) { - ModelStreamObserver observer = - new ItemStreamObserver(workflowContext, taskContext); - try { - ClientCalls.asyncUnaryCall( - call, ProtobufMessageUtils.buildMessage(methodDescriptor, parameters).build(), observer); - } catch (IOException io) { - observer.onError(io); + @Override + public boolean cancel(boolean mayInterruptIfRunning) { + return finish(() -> super.cancel(mayInterruptIfRunning)) || isCancelled(); } - return observer.future(); - } - private static CompletableFuture asyncStreamingCall( - Map parameters, - com.google.protobuf.Descriptors.MethodDescriptor methodDescriptor, - UnaryOperator> streamObserverFunction, - WorkflowContext workflowContext, - TaskContext taskContext) { - ModelStreamObserver responseObserver = - new CollectionStreamObserver(workflowContext, taskContext); - StreamObserver requestObserver = streamObserverFunction.apply(responseObserver); - try { - for (Object entry : parameters.entrySet()) { - requestObserver.onNext( - ProtobufMessageUtils.buildMessage( - entry, DynamicMessage.newBuilder(methodDescriptor.getInputType())) - .build()); - } - requestObserver.onCompleted(); - } catch (IOException e) { - requestObserver.onError(e); + @Override + public boolean complete(WorkflowModel model) { + return finish(() -> super.complete(model)); + } + + @Override + public boolean completeExceptionally(Throwable error) { + Objects.requireNonNull(error, "error"); + return finish(() -> super.completeExceptionally(error)); + } + + @Override + public CompletableFuture completeAsync( + Supplier supplier) { + return completeAsync(supplier, defaultExecutor()); + } + + @Override + public CompletableFuture completeAsync( + Supplier supplier, Executor executor) { + Objects.requireNonNull(supplier, "supplier"); + Objects.requireNonNull(executor, "executor"); + executor.execute(() -> { + if (terminal.get()) return; + try { + complete(supplier.get()); + } catch (Throwable error) { + completeExceptionally( + error instanceof CompletionException ? error : new CompletionException(error)); + } + }); + return this; } - return responseObserver.future(); - } - private static MethodDescriptor.MethodType getMethodType( - com.google.protobuf.Descriptors.MethodDescriptor methodDesc) { - DescriptorProtos.MethodDescriptorProto methodDescProto = methodDesc.toProto(); - if (methodDescProto.getClientStreaming()) { - if (methodDescProto.getServerStreaming()) { - return MethodDescriptor.MethodType.BIDI_STREAMING; + private boolean finish(BooleanSupplier completion) { + if (!terminal.compareAndSet(false, true)) return false; + CompletableFuture pending = upstream.getAndSet(null); + boolean completed; + try { + if (pending != null && !pending.isDone()) pending.cancel(true); + } finally { + completed = completion.getAsBoolean(); } - return MethodDescriptor.MethodType.CLIENT_STREAMING; - } else if (methodDescProto.getServerStreaming()) { - return MethodDescriptor.MethodType.SERVER_STREAMING; - } else { - return MethodDescriptor.MethodType.UNARY; + return completed; } } } diff --git a/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/ItemStreamObserver.java b/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/ItemStreamObserver.java deleted file mode 100644 index 8fbaf9785..000000000 --- a/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/ItemStreamObserver.java +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Copyright 2020-Present The Open Workflow Specification Authors - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.openworkflow.sdk.impl.executors.grpc; - -import com.google.protobuf.Message; -import org.openworkflow.sdk.impl.TaskContext; -import org.openworkflow.sdk.impl.WorkflowContext; -import org.openworkflow.sdk.impl.WorkflowModel; - -class ItemStreamObserver extends ModelStreamObserver { - - public ItemStreamObserver(WorkflowContext workflowContext, TaskContext taskContext) { - super(workflowContext, taskContext); - model = modelFactory.fromNull(); - } - - @Override - public void onNext(Message value) { - model = ProtobufMessageUtils.convert(value, modelFactory); - } -} diff --git a/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/ModelStreamObserver.java b/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/ModelStreamObserver.java deleted file mode 100644 index 9528f1852..000000000 --- a/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/ModelStreamObserver.java +++ /dev/null @@ -1,54 +0,0 @@ -/* - * Copyright 2020-Present The Open Workflow Specification Authors - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.openworkflow.sdk.impl.executors.grpc; - -import com.google.protobuf.Message; -import io.grpc.stub.StreamObserver; -import java.util.concurrent.CompletableFuture; -import org.openworkflow.sdk.impl.TaskContext; -import org.openworkflow.sdk.impl.WorkflowContext; -import org.openworkflow.sdk.impl.WorkflowError; -import org.openworkflow.sdk.impl.WorkflowException; -import org.openworkflow.sdk.impl.WorkflowModel; -import org.openworkflow.sdk.impl.WorkflowModelFactory; - -abstract class ModelStreamObserver implements StreamObserver { - protected T model; - protected final WorkflowModelFactory modelFactory; - private final CompletableFuture future; - private final TaskContext taskContext; - - public ModelStreamObserver(WorkflowContext workflowContext, TaskContext taskContext) { - this.modelFactory = workflowContext.definition().application().modelFactory(); - this.taskContext = taskContext; - this.future = new CompletableFuture<>(); - } - - @Override - public void onError(Throwable t) { - future.completeExceptionally( - new WorkflowException(WorkflowError.runtime(taskContext, t).build(), t)); - } - - @Override - public void onCompleted() { - future.complete(model); - } - - public CompletableFuture future() { - return future; - } -} diff --git a/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/ProtobufMessageUtils.java b/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/ProtobufMessageUtils.java index 421b89082..5add921e7 100644 --- a/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/ProtobufMessageUtils.java +++ b/impl/grpc/src/main/java/org/openworkflow/sdk/impl/executors/grpc/ProtobufMessageUtils.java @@ -18,10 +18,10 @@ import com.google.protobuf.Descriptors; import com.google.protobuf.DynamicMessage; import com.google.protobuf.Message; -import com.google.protobuf.util.JsonFormat; import java.io.IOException; import java.io.UncheckedIOException; import java.util.Map; +import org.openworkflow.sdk.grpc.WorkflowProtobufJson; import org.openworkflow.sdk.impl.WorkflowModel; import org.openworkflow.sdk.impl.WorkflowModelFactory; import org.openworkflow.sdk.impl.jackson.JsonUtils; @@ -29,10 +29,8 @@ class ProtobufMessageUtils { static WorkflowModel convert(Message message, WorkflowModelFactory modelFactory) { - StringBuilder str = new StringBuilder(); try { - JsonFormat.printer().appendTo(message, str); - return modelFactory.from(str.toString()); + return modelFactory.from(WorkflowProtobufJson.print(message)); } catch (IOException e) { throw new UncheckedIOException("Error converting protobuf message to JSON", e); } @@ -41,7 +39,9 @@ static WorkflowModel convert(Message message, WorkflowModelFactory modelFactory) static Message.Builder buildMessage(Object object, Message.Builder builder) throws IOException { // let's use Jackson to serialize the object to string for now, although we probably need to // revisit this. - JsonFormat.parser().merge(JsonUtils.mapper().writeValueAsString(object), builder); + builder.mergeFrom( + WorkflowProtobufJson.parse( + JsonUtils.mapper().writeValueAsString(object), builder.getDescriptorForType())); return builder; } diff --git a/impl/pom.xml b/impl/pom.xml index 5a8c70f37..9960cb785 100644 --- a/impl/pom.xml +++ b/impl/pom.xml @@ -18,7 +18,6 @@ 3.7.1 25.5.5.0.3 1.5.1.Final - 2.15.0 @@ -198,11 +197,6 @@ a2a-java-sdk-client ${version.org.a2aproject.sdk} - - com.google.code.gson - gson - ${version.com.google.code.gson} - diff --git a/impl/test/src/test/java/org/openworkflow/sdk/impl/test/grpc/GrpcUnaryTest.java b/impl/test/src/test/java/org/openworkflow/sdk/impl/test/grpc/GrpcUnaryTest.java index 57e8ed5ef..22d3734dc 100644 --- a/impl/test/src/test/java/org/openworkflow/sdk/impl/test/grpc/GrpcUnaryTest.java +++ b/impl/test/src/test/java/org/openworkflow/sdk/impl/test/grpc/GrpcUnaryTest.java @@ -16,6 +16,9 @@ package org.openworkflow.sdk.impl.test.grpc; import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; import static org.openworkflow.sdk.api.WorkflowReader.readWorkflowFromClasspath; import static org.openworkflow.sdk.fluent.spec.dsl.DSL.*; @@ -32,12 +35,16 @@ import org.junit.jupiter.params.ParameterizedTest; import org.junit.jupiter.params.provider.Arguments; import org.junit.jupiter.params.provider.MethodSource; +import org.junit.jupiter.params.provider.ValueSource; import org.openworkflow.sdk.api.WorkflowReader; import org.openworkflow.sdk.api.types.Workflow; import org.openworkflow.sdk.fluent.spec.WorkflowBuilder; +import org.openworkflow.sdk.impl.TaskContext; import org.openworkflow.sdk.impl.WorkflowApplication; +import org.openworkflow.sdk.impl.WorkflowContext; import org.openworkflow.sdk.impl.WorkflowDefinition; import org.openworkflow.sdk.impl.WorkflowDefinitionId; +import org.openworkflow.sdk.impl.executors.grpc.GrpcExecutor; import org.openworkflow.sdk.impl.jackson.JsonUtils; import org.openworkflow.sdk.impl.test.grpc.handlers.PersonUnaryHandler; import org.openworkflow.sdk.impl.test.junit.DisabledIfProtocUnavailable; @@ -94,6 +101,249 @@ void grpcPerson() throws IOException { Assertions.assertThat(output).contains(Map.entry("name", "John Doe"), Map.entry("id", 891182)); } + @Test + void customChannelProvider_shouldOverrideDslEndpointAndRemainReusable() throws Exception { + io.grpc.ManagedChannel channel = + io.grpc.ManagedChannelBuilder.forAddress("localhost", server.getPort()) + .usePlaintext() + .build(); + java.util.concurrent.atomic.AtomicInteger provided = + new java.util.concurrent.atomic.AtomicInteger(); + app.close(); + app = + WorkflowApplication.builder() + .withAdditionalObject( + org.openworkflow.sdk.impl.executors.grpc.GrpcExecutor.GRPC_CHANNEL_PROVIDER, + (workflowContext, taskContext) -> { + provided.incrementAndGet(); + return channel; + }) + .build(); + try { + Workflow workflow = + WorkflowBuilder.workflow("grpc-provider", "test", "0.1.0") + .tasks( + doTasks( + call( + "greet", + grpc() + .proto("workflows-samples/grpc/proto/person.proto") + .service("Person", "unused.invalid", 1) + .method("GetPerson")))) + .build(); + WorkflowDefinition definition = app.workflowDefinition(workflow); + for (int i = 0; i < 2; i++) { + assertThat( + definition + .instance(Map.of()) + .start() + .get(10, TimeUnit.SECONDS) + .asMap() + .orElseThrow()) + .containsEntry("name", "John Doe"); + assertThat(channel.isShutdown()).isFalse(); + } + assertThat(provided.get()).isEqualTo(2); + } finally { + channel.shutdownNow(); + assertThat(channel.awaitTermination(10, TimeUnit.SECONDS)).isTrue(); + } + } + + @ParameterizedTest(name = "completion = {0}") + @ValueSource( + strings = {"cancel", "timeout", "async", "asyncFailure", "asyncCancellation", "concurrent"}) + void completion_shouldStopRpcBeforeSdkHandlers(String completion) throws Exception { + var started = new java.util.concurrent.CountDownLatch(1); + var cancelled = new java.util.concurrent.CountDownLatch(1); + Server waiting = + ServerBuilder.forPort(0) + .addService( + new PersonUnaryHandler() { + @Override + public void getPerson( + person.PersonOuterClass.GetPersonRequest request, + io.grpc.stub.StreamObserver + observer) { + io.grpc.Context.current() + .addListener(context -> cancelled.countDown(), Runnable::run); + started.countDown(); + } + }) + .build() + .start(); + var channel = + io.grpc.ManagedChannelBuilder.forAddress("localhost", waiting.getPort()) + .usePlaintext() + .build(); + var cancelling = new java.util.concurrent.CountDownLatch(1); + var releaseCancel = new java.util.concurrent.CountDownLatch(1); + io.grpc.Channel supplied = + completion.equals("concurrent") + ? new io.grpc.Channel() { + @Override + public String authority() { + return channel.authority(); + } + + @Override + public io.grpc.ClientCall newCall( + io.grpc.MethodDescriptor method, io.grpc.CallOptions options) { + return new io.grpc.ForwardingClientCall.SimpleForwardingClientCall<>( + channel.newCall(method, options)) { + @Override + public void cancel(String message, Throwable cause) { + cancelling.countDown(); + try { + if (!releaseCancel.await(10, TimeUnit.SECONDS)) + throw new AssertionError("Cancellation was never released"); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + throw new IllegalStateException(interrupted); + } + super.cancel(message, cause); + } + }; + } + } + : channel; + app.close(); + app = + WorkflowApplication.builder() + .withAdditionalObject(GrpcExecutor.GRPC_CHANNEL_PROVIDER, () -> supplied) + .build(); + try { + var workflow = mock(WorkflowContext.class); + when(workflow.definition()).thenReturn(app.workflowDefinition(getPersonCallWorkflow())); + var service = person.PersonOuterClass.getDescriptor().findServiceByName("Person"); + var executor = + new GrpcExecutor( + "unused.invalid", + 1, + (w, t, input) -> Map.of(), + service, + service.findMethodByName("GetPerson")); + var result = executor.apply(workflow, mock(TaskContext.class), app.modelFactory().fromNull()); + assertThat(started.await(10, TimeUnit.SECONDS)).isTrue(); + var acknowledged = + result.handle((value, error) -> { + try { + return cancelled.await(5, TimeUnit.SECONDS); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + return false; + } + }); + if (completion.equals("timeout")) { + result.orTimeout(100, TimeUnit.MILLISECONDS); + assertThatThrownBy(() -> result.get(10, TimeUnit.SECONDS)) + .hasCauseInstanceOf(java.util.concurrent.TimeoutException.class); + } else if (completion.equals("async")) { + result.completeAsync(() -> app.modelFactory().fromNull()); + assertThat(result.get(10, TimeUnit.SECONDS).isNull()).isTrue(); + } else if (completion.equals("asyncFailure") || completion.equals("asyncCancellation")) { + RuntimeException failure = + completion.equals("asyncCancellation") + ? new java.util.concurrent.CancellationException("fallback failed") + : new IllegalStateException("fallback failed"); + result.completeAsync( + () -> { + throw failure; + }, + Runnable::run); + assertThatThrownBy(() -> result.get(10, TimeUnit.SECONDS)) + .isInstanceOf(java.util.concurrent.ExecutionException.class) + .hasCause(failure); + assertThat(result.isCancelled()).isFalse(); + } else if (completion.equals("concurrent")) { + try (var worker = java.util.concurrent.Executors.newSingleThreadExecutor()) { + var cancellation = worker.submit(() -> result.cancel(true)); + try { + assertThat(cancelling.await(10, TimeUnit.SECONDS)).isTrue(); + assertThat(result.completeExceptionally(new java.util.concurrent.TimeoutException())) + .isFalse(); + assertThat(result.isDone()).isFalse(); + } finally { + releaseCancel.countDown(); + } + assertThat(cancellation.get(10, TimeUnit.SECONDS)).isTrue(); + assertThat(result.isCancelled()).isTrue(); + } + } else { + assertThat(result.cancel(true)).isTrue(); + assertThat(result.isCancelled()).isTrue(); + } + assertThat(acknowledged.get(10, TimeUnit.SECONDS)).isTrue(); + assertThat(channel.isShutdown()).isFalse(); + } finally { + releaseCancel.countDown(); + channel.shutdownNow(); + assertThat(channel.awaitTermination(10, TimeUnit.SECONDS)).isTrue(); + waiting.shutdownNow(); + assertThat(waiting.awaitTermination(10, TimeUnit.SECONDS)).isTrue(); + } + } + + @Test + void transportError_shouldRetainTrailersAndWorkflowTaskPosition() throws Exception { + io.grpc.Metadata.Key detail = + io.grpc.Metadata.Key.of("detail", io.grpc.Metadata.ASCII_STRING_MARSHALLER); + Server failingServer = + ServerBuilder.forPort(0) + .addService( + new PersonUnaryHandler() { + @Override + public void getPerson( + person.PersonOuterClass.GetPersonRequest request, + io.grpc.stub.StreamObserver + observer) { + io.grpc.Metadata trailers = new io.grpc.Metadata(); + trailers.put(detail, "preserved"); + observer.onError( + io.grpc.Status.NOT_FOUND + .withDescription("missing person") + .asRuntimeException(trailers)); + } + }) + .build() + .start(); + io.grpc.ManagedChannel channel = + io.grpc.ManagedChannelBuilder.forAddress("localhost", failingServer.getPort()) + .usePlaintext() + .build(); + app.close(); + app = + WorkflowApplication.builder() + .withAdditionalObject( + org.openworkflow.sdk.impl.executors.grpc.GrpcExecutor.GRPC_CHANNEL_PROVIDER, + () -> channel) + .build(); + try { + var future = app.workflowDefinition(getPersonCallWorkflow()).instance(Map.of()).start(); + org.assertj.core.api.Assertions.assertThatThrownBy(() -> future.get(10, TimeUnit.SECONDS)) + .cause() + .isInstanceOfSatisfying( + org.openworkflow.sdk.impl.WorkflowException.class, + error -> { + assertThat(error.getWorkflowError().instance()).isEqualTo("do/0/greet"); + assertThat(error.getCause()) + .isInstanceOfSatisfying( + io.grpc.StatusRuntimeException.class, + transport -> { + assertThat(transport.getStatus().getCode()) + .isEqualTo(io.grpc.Status.Code.NOT_FOUND); + assertThat(transport.getTrailers().get(detail)).isEqualTo("preserved"); + }); + }); + assertThat(channel.isShutdown()).isFalse(); + } finally { + channel.shutdownNow(); + assertThat(channel.awaitTermination(10, TimeUnit.SECONDS)).isTrue(); + failingServer.shutdownNow(); + assertThat(failingServer.awaitTermination(10, TimeUnit.SECONDS)).isTrue(); + } + } + @ParameterizedTest(name = "{0}") @MethodSource("getPersonCallSources") void testGetPersonCallDsl(String sourceName, Workflow workflow) throws IOException { diff --git a/pom.xml b/pom.xml index 29562c8ff..f897eb028 100644 --- a/pom.xml +++ b/pom.xml @@ -38,6 +38,7 @@ api + common impl types annotations @@ -83,6 +84,7 @@ 4.3.0 + 2.15.0 4.32.1 3.3.0 2.23 @@ -111,6 +113,16 @@ + + com.google.code.gson + gson + ${version.com.google.code.gson} + + + org.openworkflow.sdk + openworkflow-common-grpc + ${project.version} +