provenance-file-format

This git repository will house the definition profile for Provenance File Format (PFF).

The aim of PFF is to enable the capture of distributed provenance information during the creation of RO-Crates. The initial use-case is capture of provenance on HPC systems (SLURM), for adding standard user, project, system, metadata to the RO-Crate generated by SLURM after a job has finished.

Design Plan

1. File names: user.pff, project.pff, system.pff.
2. *.pff files will follow a LINK-ML schema (defined in this repository).
3. Environment variable (PFFPATH) which contains multiple locations which will contain relevant files. All *.pff files within that path will be ingested. PFFPATH will be added to as part of shell initialisation process.
4. A creation tool will be provided for these files. This will guide users in populating the appropriate files, and will validate the files against the LINK-ML schema.
5. SLURM plugin will scan PFFPATH locations as job runs. *.pff files found will be validated against the schema, then metadata ingested.

Schemas

• schema.yaml — entry schema (imports the fragments below)
• user.schema.yaml / project.schema.yaml / system.schema.yaml — definitions

Development

Setup (requires uv):

uv sync --group dev

Regenerating the compiled schema

schema.json is the compiled output of schema.yaml. After changing any schema, regenerate it and commit:

uv run linkml generate json-schema schema.yaml > schema.json

CI fails if schema.json is out of date.

Validating the example

uv run linkml validate -s schema.yaml -C PFF examples/example.yaml