From cb308f56ecb539d350d80efb66b06c4d124d73c1 Mon Sep 17 00:00:00 2001 From: Stefan Henke Date: Fri, 9 Oct 2026 07:40:14 +0200 Subject: [PATCH 1/5] docs(java): document gen_ai agent tracing spans - java/ai.md: new 'Tracing Agents' section describing the invoke_agent / chat / execute_tool span hierarchy (incl. the inbound A2A span), the gen_ai.* span attributes, and how to enable them. - java/operating-applications/observability.md: add the Agent, ChatModel, ToolExecution and McpServer span loggers to the CAP Instrumentation table. --- java/ai.md | 43 ++++++++++++++++++++ java/operating-applications/observability.md | 4 ++ 2 files changed, 47 insertions(+) diff --git a/java/ai.md b/java/ai.md index 867df248d3..8ad6b8a5af 100644 --- a/java/ai.md +++ b/java/ai.md @@ -228,3 +228,46 @@ In CDS QL queries, elements of type `Vector` are excluded from the select list b ::: CAP Java supports multiple [vector functions](./working-with-cql/query-api.md#vector-functions) that allow you to compute vector embeddings, similarity, and distance directly in the database. + +## Tracing Agents { #ai-tracing } + +When an [agent](#agents) handles a request, CAP Java emits a hierarchy of OpenTelemetry spans that follow the request from the AI service invocation through each LLM round-trip down to the individual tool executions. The span attributes follow the [OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) (`gen_ai.*`), so traces are directly consumable by standard observability backends (for example SAP Cloud Logging, Dynatrace, or Jaeger) without custom mapping. + +The same spans are produced regardless of the underlying AI framework (LangChain4j or Spring AI). + +### Span Hierarchy + +A single agent request produces the following nested spans. When the agent is exposed via the [A2A protocol adapter](#agents), the inbound A2A request span wraps the `invoke_agent` span: + +| Span name | Span kind | Description | +| ---------------------- | ---------- | --------------------------------------------------------------------------------------------- | +| `invoke_agent ` | `INTERNAL` | One span per agent invocation, covering all LLM round-trips triggered by tool calls. | +| `chat ` | `CLIENT` | One span per LLM call. | +| `execute_tool ` | `INTERNAL` | One child span per tool call, between the LLM response requesting the tool and its result. | + +```txt +SendMessage // inbound A2A request span (SERVER), via the A2A adapter + └─ invoke_agent TravelService // gen_ai.agent.name, gen_ai.conversation.id, cds.ai.service.method + ├─ chat anthropic--claude-4.6-sonnet // gen_ai.request.model, gen_ai.usage.input_tokens/output_tokens, ... + ├─ execute_tool describe // gen_ai.tool.call.name, gen_ai.tool.call.outcome + ├─ execute_tool query + └─ chat anthropic--claude-4.6-sonnet // follow-up round-trip after the tool results +``` + +### Span Attributes + +The spans carry the following `gen_ai.*` attributes (omitted when the corresponding value is not available): + +| Span | Attributes | +| --------------- | -------------------------------------------------------------------------------------------------------------------------------------- | +| `invoke_agent` | `gen_ai.operation.name`, `gen_ai.agent.name`, `gen_ai.conversation.id`, `cds.ai.service.method`, `cds.ai.agent.outcome` | +| `chat` | `gen_ai.operation.name`, `gen_ai.provider.name`, `gen_ai.request.model`, `gen_ai.request.temperature`, `gen_ai.request.top_p`, `gen_ai.request.max_tokens`, `gen_ai.response.model`, `gen_ai.response.id`, `gen_ai.response.finish_reasons`, `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` | +| `execute_tool` | `gen_ai.operation.name`, `gen_ai.tool.call.name`, `gen_ai.tool.call.id`, `gen_ai.tool.call.outcome`, `gen_ai.tool.call.arguments`*, `gen_ai.tool.call.result`* | + +Errors are recorded on the affected span, and any tool spans still open when an invocation fails are closed and marked with outcome `error`. + +* Tool-call arguments and results are potentially sensitive and are only recorded when the [`cds.security.logPotentiallySensitive`](./developing-applications/properties#cds-security-logpotentiallysensitive) setting is enabled. By default they are omitted. + +### Enabling the Spans + +The agent spans are part of CAP's OpenTelemetry instrumentation and are created only when the OpenTelemetry Java Agent is attached and the corresponding span loggers are enabled. See [CAP Instrumentation](./operating-applications/observability#cap-instrumentation) for how to attach the Java Agent, configure OpenTelemetry export, and the full list of span loggers — including the `Agent`, `ChatModel`, and `ToolExecution` loggers for the spans described here. diff --git a/java/operating-applications/observability.md b/java/operating-applications/observability.md index d34adb8cb1..b94434d11b 100644 --- a/java/operating-applications/observability.md +++ b/java/operating-applications/observability.md @@ -504,6 +504,10 @@ By default, instrumentation for CAP-specific components is disabled, so that no | `com.sap.cds.otel.span.CQN` | `INFO` | Spans for executed CQN statement. | | `com.sap.cds.otel.span.OutboxCollector` | `INFO` | Spans for execution of the transactional outbox collector. | | `com.sap.cds.otel.span.DraftGarbageCollection` | `INFO` | Spans for execution of the draft garbage collection. | +| `com.sap.cds.otel.span.ChatModel` | `INFO` | Spans for individual LLM chat model calls (AI integration). | +| `com.sap.cds.otel.span.Agent` | `INFO` | Spans for AI agent invocations (AI integration). | +| `com.sap.cds.otel.span.ToolExecution` | `INFO` | Spans for tool executions during an agent invocation (AI integration). | +| `com.sap.cds.otel.span.McpServer` | `INFO` | Spans for inbound MCP server requests. | | `com.sap.cds.otel.span.RequestContext` | `DEBUG` | Spans for each Request Context. | | `com.sap.cds.otel.span.ChangeSetContext` | `DEBUG` | Spans for each ChangeSet Context. | | `com.sap.cds.otel.span.Emit` | `DEBUG` | Spans for dispatching events in the CAP runtime. | From a5532356c825188f38cbe4f29505241c8144de68 Mon Sep 17 00:00:00 2001 From: Stefan Henke Date: Fri, 9 Oct 2026 08:00:19 +0200 Subject: [PATCH 2/5] docs(java): refine agent tracing section Reword the intro, fold the span-enabling note under the hierarchy diagram, and keep the sensitive-data note at the end. --- java/ai.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/java/ai.md b/java/ai.md index 8ad6b8a5af..7256fcd0ac 100644 --- a/java/ai.md +++ b/java/ai.md @@ -231,7 +231,7 @@ CAP Java supports multiple [vector functions](./working-with-cql/query-api.md#ve ## Tracing Agents { #ai-tracing } -When an [agent](#agents) handles a request, CAP Java emits a hierarchy of OpenTelemetry spans that follow the request from the AI service invocation through each LLM round-trip down to the individual tool executions. The span attributes follow the [OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) (`gen_ai.*`), so traces are directly consumable by standard observability backends (for example SAP Cloud Logging, Dynatrace, or Jaeger) without custom mapping. +When a CAP Agent handles a request, CAP Java emits a hierarchy of OpenTelemetry spans that follow the request from the inbound A2A request through each LLM round-trip down to the individual tool executions. The span attributes follow the [OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) (`gen_ai.*`), so traces are directly consumable by standard observability backends (for example SAP Cloud Logging, Dynatrace, or Jaeger) without custom mapping. The same spans are produced regardless of the underlying AI framework (LangChain4j or Spring AI). @@ -254,6 +254,8 @@ SendMessage // inbound A2A request span (SERVER), v └─ chat anthropic--claude-4.6-sonnet // follow-up round-trip after the tool results ``` +The agent spans are part of CAP's OpenTelemetry instrumentation and are created only when the OpenTelemetry Java Agent is attached and the corresponding span loggers are enabled. See [CAP Instrumentation](./operating-applications/observability#cap-instrumentation) for how to attach the Java Agent, configure OpenTelemetry export, and the full list of span loggers — including the `Agent`, `ChatModel`, and `ToolExecution` loggers for the spans described here. + ### Span Attributes The spans carry the following `gen_ai.*` attributes (omitted when the corresponding value is not available): @@ -267,7 +269,3 @@ The spans carry the following `gen_ai.*` attributes (omitted when the correspond Errors are recorded on the affected span, and any tool spans still open when an invocation fails are closed and marked with outcome `error`. * Tool-call arguments and results are potentially sensitive and are only recorded when the [`cds.security.logPotentiallySensitive`](./developing-applications/properties#cds-security-logpotentiallysensitive) setting is enabled. By default they are omitted. - -### Enabling the Spans - -The agent spans are part of CAP's OpenTelemetry instrumentation and are created only when the OpenTelemetry Java Agent is attached and the corresponding span loggers are enabled. See [CAP Instrumentation](./operating-applications/observability#cap-instrumentation) for how to attach the Java Agent, configure OpenTelemetry export, and the full list of span loggers — including the `Agent`, `ChatModel`, and `ToolExecution` loggers for the spans described here. From 56bc486959889b80cf01db4ed0659729d5bf679f Mon Sep 17 00:00:00 2001 From: Stefan Henke Date: Fri, 9 Oct 2026 08:09:14 +0200 Subject: [PATCH 3/5] docs(java): don't tie agent spans to the OTel Java Agent The spans are standard OpenTelemetry spans created via GlobalOpenTelemetry; they don't require the OTel Java Agent specifically. Point to CAP Instrumentation for infrastructure setup instead. --- java/ai.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/java/ai.md b/java/ai.md index 7256fcd0ac..7973221a2d 100644 --- a/java/ai.md +++ b/java/ai.md @@ -254,7 +254,7 @@ SendMessage // inbound A2A request span (SERVER), v └─ chat anthropic--claude-4.6-sonnet // follow-up round-trip after the tool results ``` -The agent spans are part of CAP's OpenTelemetry instrumentation and are created only when the OpenTelemetry Java Agent is attached and the corresponding span loggers are enabled. See [CAP Instrumentation](./operating-applications/observability#cap-instrumentation) for how to attach the Java Agent, configure OpenTelemetry export, and the full list of span loggers — including the `Agent`, `ChatModel`, and `ToolExecution` loggers for the spans described here. +These are standard OpenTelemetry spans, emitted through CAP's OpenTelemetry instrumentation. See [CAP Instrumentation](./operating-applications/observability#cap-instrumentation) for how to set up the required infrastructure and enable the individual span loggers — including the `Agent`, `ChatModel`, and `ToolExecution` loggers for the spans described here. ### Span Attributes From 01bb3678428368a2b2663ecfe9073328a34073f7 Mon Sep 17 00:00:00 2001 From: Stefan Henke Date: Wed, 14 Oct 2026 13:17:34 +0200 Subject: [PATCH 4/5] Update java/ai.md Co-authored-by: Marc Becker --- java/ai.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/java/ai.md b/java/ai.md index 7973221a2d..993fd665ce 100644 --- a/java/ai.md +++ b/java/ai.md @@ -233,8 +233,6 @@ CAP Java supports multiple [vector functions](./working-with-cql/query-api.md#ve When a CAP Agent handles a request, CAP Java emits a hierarchy of OpenTelemetry spans that follow the request from the inbound A2A request through each LLM round-trip down to the individual tool executions. The span attributes follow the [OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) (`gen_ai.*`), so traces are directly consumable by standard observability backends (for example SAP Cloud Logging, Dynatrace, or Jaeger) without custom mapping. -The same spans are produced regardless of the underlying AI framework (LangChain4j or Spring AI). - ### Span Hierarchy A single agent request produces the following nested spans. When the agent is exposed via the [A2A protocol adapter](#agents), the inbound A2A request span wraps the `invoke_agent` span: From a774c28c01aecc0b8f8266395109711a642f8e57 Mon Sep 17 00:00:00 2001 From: Stefan Henke Date: Wed, 14 Oct 2026 13:17:45 +0200 Subject: [PATCH 5/5] Update java/ai.md Co-authored-by: Marc Becker --- java/ai.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/java/ai.md b/java/ai.md index 993fd665ce..cdd4f26274 100644 --- a/java/ai.md +++ b/java/ai.md @@ -240,7 +240,7 @@ A single agent request produces the following nested spans. When the agent is ex | Span name | Span kind | Description | | ---------------------- | ---------- | --------------------------------------------------------------------------------------------- | | `invoke_agent ` | `INTERNAL` | One span per agent invocation, covering all LLM round-trips triggered by tool calls. | -| `chat ` | `CLIENT` | One span per LLM call. | +| `chat ` | `CLIENT` | One child span per LLM call. | | `execute_tool ` | `INTERNAL` | One child span per tool call, between the LLM response requesting the tool and its result. | ```txt