Skip to content

feat(flux): convert walltime to minutes/FSD for batch_system="flux" #5017

Description

@jasonb5

Summary

Flux's -t/--time-limit accepts minutes or Flux Standard Duration (FSD, RFC 23) — not HH:MM:SS. Tuolumne is currently the only E3SM machine running Flux, so this is handled as a Flux-specific code path in env_batch.py, leaving the generic format_time()/walltime_format mechanism untouched (used as-is by other Slurm/PBS-style machines).

Reference: Flux's accepted format

Per flux-submit(1):

-t, --time-limit=MINUTES|FSD
Set a time limit for the job in either minutes or Flux standard duration (RFC 23). FSD is a floating point number with a single character units suffix (s, m, h, or d). The default unit is minutes when no unit is specified.

Root cause recap

CIME/utils.py::format_time() re-arranges H/M/S fields positionally; it does not sum across units. E.g. walltime_format="%M" on 01:10:00 yields 10, not 70. This isn't a bug in format_time() itself — it's simply the wrong tool for Flux's minutes/FSD requirement, and other machines rely on its current field-rearranging behavior, so it should not be changed generically.

Proposed fix

In CIME/XML/env_batch.py::get_job_overrides() (~line 484), add a Flux-specific branch that converts the resolved walltime directly, rather than reusing the generic format_time()/walltime_format path for this scheduler:

if batch_system == "flux":
    seconds = convert_to_seconds(walltime)
    # convert to minutes and/or FSD as appropriate
    walltime = <flux-appropriate conversion>
else:
    walltime_format = self.get_value("walltime_format")
    if walltime_format:
        ...  # existing format_time() path, unchanged

Implementation should support both Flux-accepted formats:

  • Minutes (plain integer, e.g. 70)
  • FSD (floating point + unit suffix s/m/h/d, e.g. 70m, 1.17h)

Exact choice of which format to emit by default (and any rounding behavior for partial minutes) is an implementation detail to work out in the PR.

Out of scope

  • Changing the generic format_time() field-rearranging behavior used by other (non-flux) machines.
  • Rounding/ceiling-division correctness improvements to the general walltime_format path.

Validation

  • Unit tests covering batch_system="flux" walltime conversion for both minutes and FSD output paths, e.g. 01:10:0070 (minutes) and equivalent FSD form.
  • Confirm non-flux batch systems are unaffected (existing format_time()/walltime_format behavior unchanged).
  • End-to-end: submit a Tuolumne test case with env_batch.xml walltime 01:10:00; confirm the generated flux batch/flux submit -t value is valid and the enforced job time limit matches.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions