[uniqueg]

Summary

The TESK API accepts TES 1.1 executors[].ignore_error and reports the
task's state accordingly. It needs a taskmaster with
elixir-cloud-aai/tesk-core#143 (merged, not yet released).

• ignore_error: true is no longer rejected. The converter sets the
  annotation ignore-error: "true" on the executor's job, the contract
  with the taskmaster.
• TaskStateResolver: a failed executor job with that annotation doesn't
  make the task EXECUTOR_ERROR/PREEMPTED, unless the taskmaster ended
  the job itself (activeDeadlineSeconds, or the reason
  DeadlineExceeded); then the container may never have run.
• After an ignored failure, a task with outputs (or whose stored task
  can't be read) is COMPLETE only if its outputs filer's job completed,
  otherwise SYSTEM_ERROR. This catches taskmasters that stop at the
  failure: the current default image (0.11.0), which doesn't know the
  annotation, and a new one whose failed pods didn't end in time.
• Docs: API (TES 1.1 fields, differences from TES 1.1), state rules 2 and
  4 in the index, upgrading notes.

Closes #159

Verification

• ./mvnw -B checkstyle:check spotless:check clean verify (Java 8,
  offline): BUILD SUCCESS, all unit tests and 99 integration tests pass,
  0 checkstyle violations, coverage checks pass.
• mkdocs build --strict: clean.
• New tests: TaskStateResolverTest (ignored failures, other annotation
  values, a later executor error, jobs ended by deadline, outputs filer
  required after ignored failures, missing later executors),
  TesKubernetesConverterTest (annotation, extractHasOutputs),
  TasksApiControllerTest.createTask_valid_ignoreErrorTrue,
  Tes11FieldsTest.
• Review: adversarial-reviewer and quality-reviewer, two rounds. Round
  1 found that a taskmaster stopping at an ignored failure of the last
  executor would report COMPLETE without uploaded outputs (high) or
  after a timeout (medium); fixed by the outputs filer rule above. Round 2
  found only a docs gap in rule 4, fixed.
• CI: all checks passed. The first run's codecov/patch missed two
  branches (an ignoring executor that hasn't ended, and the stored task's
  outputs read through extractStateFromK8sJobs); tests for both were
  added.
• Sourcery: 2 findings. The tests now use the literal annotation key
  (fixed). The other is declined; see below.

Declined review findings

• Sourcery said an outputs filer still running when the taskmaster's job
  completes would give SYSTEM_ERROR. The taskmaster waits for the
  outputs filer to complete before it exits, and both jobs come from one
  consistent list request, so that can't happen; answered in the thread.

• Bumping the default taskmaster image to a version with
  feat(taskmaster): go on after ignored errors tesk-core#143 needs a tesk-core release; that belongs
  to the release prep (build: prepare the 1.3.0 release #155), not this PR. Until then, an ignored failure
  of a task's last executor with outputs ends SYSTEM_ERROR, and of an
  earlier executor ends SYSTEM_ERROR because later executors are
  missing; the upgrading notes say so.

• The quality review suggested parameterizing a converter test; the class
  is JUnit 4, so it got an assertion message instead.

Open questions

None. Note for deployment: ignore_error only works once a tesk-core
release with #143 is deployed (TESK_API_TASKMASTER_IMAGE_VERSION) or the
default image is bumped.

🤖 Generated with Claude Code

Summary by Sourcery

Support TES 1.1 executor error ignoring while preserving correct task states when failures prevent execution or output delivery.

New Features:

• Support TES 1.1 executors[].ignore_error: true and propagate it to executor jobs.
• Resolve ignored executor failures according to taskmaster and outputs-filer outcomes.

Bug Fixes:

• Prevent tasks from being reported complete when ignored failures leave required outputs unavailable or later executor jobs missing.
• Treat taskmaster-ended executor jobs as real executor errors even when ignore_error is set.

Enhancements:

• Extend task state resolution to account for ignored failures, task outputs, and unreadable stored task metadata.

Documentation:

• Document ignore_error behavior, state-resolution rules, TES 1.1 differences, and taskmaster upgrade requirements.

Tests:

• Add coverage for accepted ignore_error requests, job annotations, state transitions, deadline failures, missing executors, and outputs-filer requirements.

[sourcery-ai]

Reviewer's Guide

Data structures

Define the executor-job annotation key and value used to communicate ignore_error with the taskmaster.

• src/main/java/uk/ac/ebi/tsc/tesk/k8s/constant/Constants.java L50-59

API

Allow ignore_error: true in the TES executor model and update its API description by removing the validation that rejected it.

• src/main/java/uk/ac/ebi/tsc/tesk/tes/model/TesExecutor.java L257-261, L270 (removed lines)

Implementation

Set the taskmaster contract annotation for executors that ignore errors, and supply stored-task output information when resolving task state.

• src/main/java/uk/ac/ebi/tsc/tesk/k8s/convert/TesKubernetesConverter.java L266-269, L349-368

Resolve ignored executor failures as done without failing the task, except when the taskmaster ended the job; require a completed outputs filer after an ignored failure when outputs may exist.

• src/main/java/uk/ac/ebi/tsc/tesk/k8s/convert/TaskStateResolver.java L150-167, L205-216, L244-292 (+3 small edits)

Documentation

Document ignore_error behavior and its limitations in the TES 1.1 field notes and API differences.

• docs/api.md L110-130, L152-155

Update task-state rules to account for ignored executor failures and the outputs filer requirement.

• docs/index.md L99-110

Explain the compatibility and taskmaster upgrade requirements for ignore_error.

• docs/upgrading.md L32-40

Tests

Test task-state resolution for ignored failures, deadline-ended jobs, missing or unfinished executors, and outputs-filer requirements.

• src/test/java/uk/ac/ebi/tsc/tesk/k8s/convert/TaskStateResolverTest.java L10, L28, L624-784

Test that conversion sets the ignore-error annotation and that stored-task outputs affect state resolution after an ignored failure.

• src/test/java/uk/ac/ebi/tsc/tesk/k8s/convert/TesKubernetesConverterTest.java L659-688, L862-882, L920-948

Move the ignore_error: true fixture from the invalid-task set to the valid-task set.

• src/test/resources/valid_tasks/ignore_error_true.json

Other changes

Files not grouped.

• src/test/java/uk/ac/ebi/tsc/tesk/tes/api/TasksApiControllerTest.java L224 (removed lines), L493-507
• src/test/java/uk/ac/ebi/tsc/tesk/tes/model/Tes11FieldsTest.java L63 (removed lines), L65 (removed lines), L67 (removed lines)

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.33%. Comparing base (ad32c6b) to head (da12d96).

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #174      +/-   ##
==========================================
+ Coverage   75.07%   75.33%   +0.26%     
==========================================
  Files          71       71              
  Lines        3274     3309      +35     
  Branches      386      393       +7     
==========================================
+ Hits         2458     2493      +35     
  Misses        663      663              
  Partials      153      153              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sourcery-ai]

Linked issues

Checked at commit da12d96

#159 feat: support executors[].ignore_error · 2 of 3 met

#	Requirement	Status	Notes
1	The taskmaster continues after a failed executor with ignore_error: true, and the task ends COMPLETE if the rest succeed.	🟡 Partly met	The API passes the annotation and its state resolver accounts for ignored failures, but the taskmaster behavior itself is not implemented here and still requires a tesk-core release.
2	TESK API accepts ignore_error: true and passes the flag to the taskmaster.	✅ Met	Validation no longer rejects true, and the executor job receives the ignore-error: "true" annotation.
3	Document the fields and differences in api.md.	✅ Met	Both the TES 1.1 fields and Differences from TES 1.1 sections describe the behavior and its limitations.

[sourcery-ai]

Hey - I've found 2 issues

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="src/main/java/uk/ac/ebi/tsc/tesk/k8s/convert/TaskStateResolver.java" line_range="214" />
<code_context>
+            if (allCompleted
+                    && ignoredFailure
+                    && !Boolean.FALSE.equals(hasOutputs.get())
+                    && !outputFiler.map(filer -> hasSucceeded(filer.getJob())).orElse(false)) {
+                return TesState.SYSTEM_ERROR;
+            }
</code_context>
<issue_to_address>
**Running filer reports system error**

When the taskmaster completes after an ignored executor failure while the outputs filer is still running or has not yet appeared in the API read, the new guard treats an outputs filer that is still running or not yet visible as a task failure, so a supported task that reaches taskmaster completion before its filer finishes is reported as `SYSTEM_ERROR` instead of remaining nonterminal; callers can see a failed task while valid outputs are still being produced.

Keep the task nonterminal while the outputs filer is pending or absent, and report `SYSTEM_ERROR` only when the filer has actually failed or cannot complete.
</issue_to_address>

### Comment 2
<location path="src/test/java/uk/ac/ebi/tsc/tesk/k8s/convert/TesKubernetesConverterTest.java" line_range="942" />
<code_context>
+                            0,
+                            null,
+                            User.builder("test-user-id").build());
+            String annotation = job.getMetadata().getAnnotations().get(ANN_IGNORE_ERROR_KEY);
+            assertEquals(
+                    "ignore_error: " + ignoreError,
</code_context>
<issue_to_address>
**Wrong annotation key goes unnoticed**

When `ANN_IGNORE_ERROR_KEY` has an incorrect value, when `ANN_IGNORE_ERROR_KEY` is changed from the taskmaster contract key `ignore-error`, `fromTesExecutorToK8sJob_ignoreError` reads back the same incorrect constant at line 942 and passes; `ignoreError` also writes that constant at line 783, so the resolver tests pass with the same wrong key while the taskmaster ignores the annotation.

Assert the literal key `ignore-error` in the converter test, and make the resolver test fixture use that literal contract key.

Also at `src/test/java/uk/ac/ebi/tsc/tesk/k8s/convert/TaskStateResolverTest.java:782`.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 2 findings to address first, and if the annotation or failure-state checks are wrong, a task may continue after an executor failure and the outputs filer may publish incomplete or incorrect outputs while the task is reported complete. Reverting prevents new tasks from using this behavior, but already-published outputs remain; they are bounded and can generally be identified and regenerated.

Blocking findings: src/main/java/uk/ac/ebi/tsc/tesk/k8s/convert/TaskStateResolver.java:214, src/test/java/uk/ac/ebi/tsc/tesk/k8s/convert/TesKubernetesConverterTest.java:942

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook