Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -48,5 +48,5 @@ keywords:
- REST API
- Spring Boot
license: Apache-2.0
version: "1.2.0"
date-released: "2026-11-09"
version: "1.3.0"
date-released: "2026-11-16"
2 changes: 1 addition & 1 deletion docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Environment variable | Meaning
`TESK_API_AUTHORISATION_*` | Authorisation by group membership: `TESK_API_AUTHORISATION_GROUPS_CLAIM` (default `eduperson_entitlement`), `…_GROUP_PREFIX`, `…_GROUP_SUFFIX`, `…_DELIMITER`, `…_PARENT_GROUP`, `…_ENV_SUBGROUP`, `…_BASE_GROUP` and `…_ADMIN_SUBGROUP`; see [Access control](access-control.md#configuration) for their meanings and defaults.
`TESK_API_AUTHORISATION_LEGACY_USER_LABELS` | `true`: tasks created by earlier versions, which have only the old creator label, belong to their creators again; `false` (default): only team admins and super admins see them. See [Access control](access-control.md#task-creators). In the Helm chart of TESK: `auth.legacyUserLabels` (from chart 0.3.0).
`TESK_API_AUTHORISATION_CLAIM_*` | Authorisation without groups, by a claim of the user info; off by default. Turning it on lets everyone with one of the configured values run tasks; see [Access control](access-control.md#authorisation-without-groups) for the settings and their security implications.
`TESK_API_AUTHORISATION_SERVICE_CLIENTS_*` | Access tokens of allow-listed service clients (OAuth 2.0 client credentials grant, e.g. of a gateway), checked locally against the issuer's keys instead of at the user info endpoint; off by default. Every holder of a listed client's credentials can run tasks; see [Access control](access-control.md#tokens-of-services) for the settings and their trust implications.
`TESK_API_AUTHORISATION_SERVICE_CLIENTS_*` | Access tokens of allow-listed service clients (OAuth 2.0 client credentials grant, e.g. of a gateway), checked locally against the issuer's keys instead of at the user info endpoint; off by default. Every holder of a listed client's credentials can run tasks; see [Access control](access-control.md#tokens-of-services) for the settings and their trust implications. Since 1.3.0.
`TESK_API_SWAGGER_UI_ENABLED` | `true`: serves the [Swagger UI](api.md#swagger-ui) at `/swagger-ui/index.html` (the root path `/` redirects there) with its static resources and its configuration (`/swagger-resources`). `false` (default): serves none of them. The API description (`/v2/api-docs`) is served either way. Other values stop the app at startup. Turning it on serves the Swagger UI and its static resources publicly, also with the `auth` profile: turn it on only where it is needed. Since 1.2.0; before, the Swagger UI was always on.
`TESK_API_SWAGGER_OAUTH_*` | The OpenID Connect client of the Swagger UI, with the `auth` profile: `TESK_API_SWAGGER_OAUTH_CLIENT_ID`, `…_IMPLICIT` (default `true`: the implicit grant; `false`: the authorisation code grant without a client secret), `…_AUTHORIZATION_ENDPOINT`, `…_TOKEN_ENDPOINT` and `…_SCOPES`; see [Access control](access-control.md#built-in-client) for their meanings and defaults, and what the authorisation code grant needs.
`TESK_API_SWAGGER_OAUTH_CLIENT_SECRET` | Deprecated since 1.2.0 and ignored: the Swagger UI is a public client and gets no client secret. TESK API logs a warning at startup if it is set. Remove it from your deployment; if a client secret was configured while the Swagger UI was on (always the case before 1.2.0, with the `auth` profile), rotate that secret at your identity provider (see [Upgrading](upgrading.md#to-120)).
Expand Down
2 changes: 1 addition & 1 deletion docs/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ version tag, with
[cosign](https://docs.sigstore.dev/cosign/system_config/installation/):

```bash
cosign verify elixircloud/tesk-api:1.2.0 \
cosign verify elixircloud/tesk-api:1.3.0 \
--certificate-identity-regexp '^https://github\.com/elixir-cloud-aai/tesk-api/\.github/workflows/release\.yml@refs/tags/v' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com
```
Expand Down
45 changes: 45 additions & 0 deletions docs/releases/1.3.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
TESK API 1.3.0 brings TES 1.1 to TESK: its task fields, including `ignore_error`, its list filters and task states, and output file logs. It also brings optional access for tokens of allow-listed service clients (e.g., a TES gateway that calls TESK with its client-credentials token) and experimental support for private executor images, and it runs on Java 21 and validates tokens with the resource server of Spring Security instead of the end-of-life Spring Security OAuth. The new options are off by default, but deployments may have to check their JVM options and the order in which they upgrade TESK API and the taskmaster, and clients may notice changed answers, e.g., `CANCELING` before `CANCELED`, `400 Bad Request` for wildcard outputs without `path_prefix`, pages of at most 256 tasks from `GET /tasks` without `page_size`, and `401 Unauthorized` for tokens sent in the URL: read the [upgrade notes](https://tesk.readthedocs.io/en/latest/tesk-api/upgrading/#to-130) before you upgrade. `ignore_error` and output file logs need a taskmaster and filer of [tesk-core](https://github.com/elixir-cloud-aai/tesk-core/releases) 0.12.0 or later, deployed after this release, not before it; uploading the matches of wildcard outputs needs a taskmaster whose filer supports `path_prefix` ([tesk-core#134](https://github.com/elixir-cloud-aai/tesk-core/pull/134)).

## Breaking changes

- The image runs TESK API on Java 21 instead of Java 8; JVM options in `JAVA_OPTS` that Java 21 removed stop the JVM at startup (#162, #161)
- A single task's executors are selected by the labels `taskmaster-name` and `job-type=executor`, so that TESK API works with taskmasters that also label the filers' jobs (tesk-core 0.12.0 and later); upgrade TESK API first and such a taskmaster after the rollout, or both at once with the `Recreate` strategy (#182, #183)
- Access tokens are read only from the `Authorization: Bearer` header; an `access_token` parameter in the URL or a form is ignored (#165, #175)
- Outputs with wildcards but no `path_prefix` are refused with `400 Bad Request`, unless they go to `file:` URLs or URLs without a scheme (taken as `file:`), as TES 1.1 requires (#148, #152)
- `GET /tasks` without `page_size` returns pages of at most 256 tasks, the default of TES, instead of all tasks at once (#137, #158)
- Tasks report `CANCELING` while the pods of a cancelled task still run, and `PREEMPTED` when Kubernetes disrupted all failed executors (#144, #151)

## Features

- Optional access for tokens of allow-listed service clients (OAuth 2.0 client credentials grant), checked locally against the issuer's keys, as principals of their own that create, list, get and cancel their own tasks; settings `TESK_API_AUTHORISATION_SERVICE_CLIENTS_*` (#153, #154)
- The task fields of TES 1.1 are accepted: `outputs[].path_prefix` is passed to the taskmaster, `inputs[].streamable` is ignored, and `backend_parameters_strict: true` is refused; service info reports TES `1.1.0` (#133)
- `executors[].ignore_error: true` lets a task go on after the executor fails, with a taskmaster that supports it (tesk-core 0.12.0 and later) (#159, #174)
- `logs[0].outputs` lists the files that the outputs filer uploaded, with a taskmaster and filer that record them (tesk-core 0.12.0 and later) (#160, #176)
- `GET /tasks` filters by `state`, `tag_key` and `tag_value` (#143, #149)
- `backend_parameters` are no longer stored or returned; ignored keys are logged and listed in `system_logs`, and service info lists `tesResources_backend_parameters` (`[]`) (#145, #147)
- Experimental: private executor images, with registry credentials in the executor fields `docker_registry`, `docker_login` and `docker_pass`, passed to Kubernetes as image-pull Secrets owned by the taskmaster's job; `docker_login` and `docker_pass` go only into the Secret and are never returned, while `docker_registry` is stored and returned (#55, #141)

## Security

- Tokens are validated with the resource server of Spring Security instead of the end-of-life Spring Security OAuth; the `401` for an invalid token no longer repeats the token, malformed bearer headers are refused without a call to the user info endpoint, and calls to that endpoint time out (#165, #175)
- SnakeYAML 2 instead of 1.33 (CVE-2022-1471), with version 28 of the Kubernetes Java client (#164, #178)

## Fixes

- Inputs and outputs with `"type": null` are accepted as `FILE`, as when `type` is left out, as TES 1.1 allows (#146, #150)
- Inputs are validated without a JavaScript engine, which Java 21 lacks; the message for an input without a source now names the check (`inputs[0].urlOrContentGiven`) (#162, #161)
- Run outside a cluster with a token in its kubeconfig file, TESK API can cancel tasks: patches keep the token's `Bearer` prefix (#164, #178)
- A user info answer without `sub` gives `401` instead of `500` (#165, #175)
- A groups claim of `null` gives no group authorities instead of one named `null`, and a task that can't be serialized fails its creation with `500` instead of creating a job that can't run (#179, #184)

## Documentation

- The API docs list the differences from TES 1.1 (#137, #158)

## Maintenance

- Version 28 of the Kubernetes Java client instead of 11 (#164, #178)
- Tests cover all lines and branches, enforced by JaCoCo and Codecov (#179, #184)
- CI runs of pull requests are cancelled only by newer commits (#142)
- Dependabot labels its pull requests with `type: dependencies` only (#180, #181)
- Release preparation (#156, #155)
122 changes: 83 additions & 39 deletions docs/upgrading.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,63 @@
# Upgrading

## To the next release
## To 1.3.0

From 1.2.0, deployments may have to check their JVM options, and clients may
notice changed answers:
From 1.2.0, deployments may have to check their JVM options, the order in
which they upgrade TESK API and the taskmaster, and the taskmaster's rights;
the new options are off by default. TESK API 1.3.0 accepts the task fields,
list filters and states of TES 1.1, and clients may notice changed answers:
read all of the items below before you upgrade.

- **Java 21.** The image runs TESK API on Java 21 instead of Java 8 (see
[Deployment](deployment.md)). If you pass options to the JVM in
`JAVA_OPTS`, check that Java 21 still accepts them: options that it
removed (e.g. `-XX:MaxPermSize`, `-XX:+UseConcMarkSweepGC`,
`-XX:+UseCGroupMemoryLimitForHeap`) stop the JVM
at startup. Without `JAVA_OPTS`, nothing changes.
- **Inputs without a source.** An input with neither a `url` (that isn't
blank) nor a `content` (that isn't empty) is still refused with `400 Bad
Request`, but the message now names the check:
`Field 'inputs[0].urlOrContentGiven' URL or content required` instead of
`Field 'inputs[0]' URL or content required`. The messages are meant for
people; clients should rely on the status code, not on their text.
- **Upgrade order for filer labels.** A single task's executors are its
jobs with both labels `taskmaster-name=<task>` and `job-type=executor`
(before: `taskmaster-name` only). Taskmasters of tesk-core releases with
[tesk-core#147](https://github.com/elixir-cloud-aai/tesk-core/issues/147)
(0.12.0 and later) also give the filers' jobs the label `taskmaster-name`. Upgrade TESK API
to this release first, and set the taskmaster to such a release
(`TESK_API_TASKMASTER_IMAGE_VERSION`, see
[Configuration](configuration.md)) only once no older replica of TESK API
serves requests: in a second deployment after the rollout has finished,
or in the same one with the `Recreate` strategy. An older TESK API with
such a
taskmaster takes the inputs filer's job for the outputs filer's job on
`GET /tasks/{id}` and `POST /tasks/{id}:cancel`: a task whose taskmaster
stopped before the outputs filer after an ignored executor failure is
`COMPLETE` instead of `SYSTEM_ERROR`, and a task whose inputs filer
failed is `SYSTEM_ERROR` while its taskmaster still runs. `GET /tasks`
isn't affected. With older taskmasters, nothing changes.
- **TES 1.1 fields.** `outputs[].path_prefix`, `inputs[].streamable`,
`executors[].ignore_error` and `resources.backend_parameters_strict` are
accepted instead of being refused as unknown fields, with one exception
that is refused with `400 Bad Request`, as TESK can't do what it asks:
`backend_parameters_strict: true` with `backend_parameters` (see [TES 1.1
fields](api.md#tes-11-fields)).
- **`ignore_error`.** `executors[].ignore_error: true` is accepted rather
than refused with `400 Bad Request` (see [TES 1.1
fields](api.md#tes-11-fields)). It needs a taskmaster that goes on after
such an executor fails
([tesk-core#142](https://github.com/elixir-cloud-aai/tesk-core/issues/142)):
set the taskmaster image to such a release
(`TESK_API_TASKMASTER_IMAGE_VERSION`, see
[Configuration](configuration.md)) once this TESK API runs, not before
(see "Upgrade order for filer labels" above, as such a release may also
label the filers' jobs). An older taskmaster stops at the
failure: the task then ends `SYSTEM_ERROR`, unless the failed executor was
the last one and the task has no outputs, which completes as asked.
- **Wildcards without `path_prefix`.** An output whose `path` has wildcards
(`*`, `?`, `[`) but no `path_prefix` is refused with `400 Bad Request`,
unless its `url` is a `file:` URL or has no scheme, as TES 1.1 requires
`path_prefix` for such outputs (see [TES 1.1 fields](api.md#tes-11-fields)).
Before, such a task ran, and its output then failed or uploaded nothing
useful. Clients that send such outputs to other URLs set `path_prefix`,
e.g. to the part of `path` before its first wildcard.
e.g. to the part of `path` before its first wildcard. Uploading the
matches of a pattern needs a taskmaster whose filer supports `path_prefix`
([tesk-core#134](https://github.com/elixir-cloud-aai/tesk-core/pull/134)).
- **Output file logs.** `logs[0].outputs` lists the uploaded files (see
[Output file logs](api.md#output-file-logs)) once the taskmaster and the
filer are of a tesk-core release with
Expand All @@ -42,6 +76,30 @@ notice changed answers:
are more (see [Listing tasks](api.md#listing-tasks)). Before, it returned
all tasks at once. Clients that don't pass `page_size` must follow
`next_page_token`, or pass a `page_size` of up to 2048.
- **`CANCELING` and `PREEMPTED`.** A cancelled task is `CANCELING` while a
pod of its taskmaster, executors or outputs filer is still running, and
`CANCELED` only once they have ended; a task whose failed executors were
all disrupted by Kubernetes (preemption, eviction, node pressure, taints) is
`PREEMPTED` instead of `EXECUTOR_ERROR` (see [Getting
tasks](index.md#getting-tasks)). Clients that wait for `CANCELED` after a
cancel request wait until the pods have ended; clients that know only the
states of TES 1.0 must accept the new ones.
- **`backend_parameters`.** They are no longer stored or returned, also for
tasks created by earlier versions; with the view `FULL`, a task lists the
keys it ignored in `logs[0].system_logs` (only tasks created by 1.3.0 or
later: earlier versions recorded no ignored keys, so their tasks list
none).
Service info lists `tesResources_backend_parameters` (always `[]`); keep
it at `[]` in your own service info file, if you use one.
- **Service info.** `type.version` is `1.1.0` instead of `1.0.0`. If you
use your own file (`TESK_API_SERVICE_INFO_LOCATION`), update `type.version`
and `version` there, too.
- **Inputs without a source.** An input with neither a `url` (that isn't
blank) nor a `content` (that isn't empty) is still refused with `400 Bad
Request`, but the message now names the check:
`Field 'inputs[0].urlOrContentGiven' URL or content required` instead of
`Field 'inputs[0]' URL or content required`. The messages are meant for
people; clients should rely on the status code, not on their text.
- **Token validation.** TESK API validates access tokens with the resource
server of Spring Security instead of Spring Security OAuth, which is end
of life. Deployments need no change if
Expand All @@ -64,41 +122,14 @@ notice changed answers:
- Without a valid token, a request that accepts no JSON (e.g.
`Accept: text/plain`) gets `401` with the `WWW-Authenticate`
header and no body.
- A user whose groups claim is `null` has no group authorities; before,
they had one named `null`.
- **User info endpoint.** TESK API waits at most 10 seconds to connect to
the user info endpoint and 30 seconds for each read of its answer, and then refuses
the token (before, it waited without limit). It doesn't start if
`SECURITY_OAUTH2_RESOURCE_USER_INFO_URI` isn't an absolute `http` or
`https` URL with a host; before, it started and refused every
token.
- **`ignore_error`.** `executors[].ignore_error: true` is accepted rather
than refused with `400 Bad Request` (see [TES 1.1
fields](api.md#tes-11-fields)). It needs a taskmaster that goes on after
such an executor fails
([tesk-core#142](https://github.com/elixir-cloud-aai/tesk-core/issues/142)):
set the taskmaster image to such a release
(`TESK_API_TASKMASTER_IMAGE_VERSION`, see
[Configuration](configuration.md)) once this TESK API runs, not before
(see "Upgrade order for filer labels" below, as such a release may also
label the filers' jobs). An older taskmaster stops at the
failure: the task then ends `SYSTEM_ERROR`, unless the failed executor was
the last one and the task has no outputs, which completes as asked.
- **Upgrade order for filer labels.** A single task's executors are its
jobs with both labels `taskmaster-name=<task>` and `job-type=executor`
(before: `taskmaster-name` only). Taskmasters of tesk-core releases with
[tesk-core#147](https://github.com/elixir-cloud-aai/tesk-core/issues/147)
also give the filers' jobs the label `taskmaster-name`. Upgrade TESK API
to this release first, and set the taskmaster to such a release
(`TESK_API_TASKMASTER_IMAGE_VERSION`, see
[Configuration](configuration.md)) only once no older replica of TESK API
serves requests: in a second deployment after the rollout has finished,
or in the same one with the `Recreate` strategy. An older TESK API with
such a
taskmaster takes the inputs filer's job for the outputs filer's job on
`GET /tasks/{id}` and `POST /tasks/{id}:cancel`: a task whose taskmaster
stopped before the outputs filer after an ignored executor failure is
`COMPLETE` instead of `SYSTEM_ERROR`, and a task whose inputs filer
failed is `SYSTEM_ERROR` while its taskmaster still runs. `GET /tasks`
isn't affected. With older taskmasters, nothing changes.
- **Kubernetes client.** TESK API has version 28 of the Kubernetes Java
client instead of 11 (see [Deployment](deployment.md#kubernetes)), and
SnakeYAML 2 instead of 1.33. Nothing changes for deployments: it needs the
Expand All @@ -111,6 +142,19 @@ notice changed answers:
which Kubernetes refused. A task whose taskmaster
job has no creation time has no `creation_time`, instead of the current
time; Kubernetes sets it for every job, so this only matters for tests.
- **Tasks that can't be serialized.** If TESK API can't write a task as
JSON for its taskmaster, creating the task fails with `500 Internal
Server Error`; before, it logged the error and created a job without the
task's JSON, which could neither run nor be read back. This doesn't
happen with valid TES tasks.

New, off until you use them: tokens of allow-listed service clients
(`TESK_API_AUTHORISATION_SERVICE_CLIENTS_*`, see [Tokens of
services](access-control.md#tokens-of-services); in the Helm chart of TESK
after 0.3.0: `auth.tokenClients`) and private executor images, an experimental
workaround that needs `create` and `delete` on `secrets` (see [Private
executor images](api.md#private-executor-images) and
[Deployment](deployment.md)).

## To 1.2.0

Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

<groupId>uk.ac.ebi.tsc</groupId>
<artifactId>tesk</artifactId>
<version>1.2.0</version>
<version>1.3.0</version>
<packaging>jar</packaging>

<name>tesk</name>
Expand Down
Loading
Loading