Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ Each config's `help:` line shows a concrete example. `-c key:"'value'"` injects
- **Model / lifecycle** (`src/Model/`): a process is a graph of tasks; each task is a service implementing `TaskInterface::execute(ProcessState $state)`. `ProcessState` is the data bus passed task-to-task — `getInput()`/`setOutput()`/`setErrorOutput()`, `stop()`, plus `getContext()` and `getContextualizedOption()` (the `-c key:value` values). Optional lifecycle interfaces refine a task: `IterableTaskInterface` (emit many outputs from one input), `BlockingTaskInterface` (wait for all inputs before proceeding), `Initializable/Flushable/FinalizableTaskInterface`, and `AbstractConfigurableTask` (OptionsResolver-based options).
- **Tasks** (`src/Task/`): the generic building blocks referenced in configs — `ConstantOutputTask`, `TransformerTask`, `File/Csv/*`, `File/JsonStream/*`, `File/Xml|Yaml/*`, `Serialization/*` (normalize/denormalize/(de)serialize), `Validation/ValidatorTask`, `Debug/*` (`DebugTask`, `DieTask`, `StopwatchTask`, `ErrorForwarderTask`), `Reporting/*` (`LoggerTask`, stat counters), aggregators/iterators, and `Process/*` (run sub-processes or shell commands).
- **Transformers** (`src/Transformer/`): the value-mapping library used inside `TransformerTask` and `generic_transformers` — `MappingTransformer`, `CallbackTransformer`, `CastTransformer`, `RulesTransformer`, `Array/*`, `String/*` (implode, explode, slugify, sprintf, preg_match…), `Date/*`, `Object/*`, `Serialization/*`, `Xml/XpathEvaluatorTransformer`.
- **Registries** (`src/Registry/`): `ProcessConfigurationRegistry` and `TransformerRegistry` resolve the `@service` / transformer names used in YAML.
- **Registries** (`src/Registry/`): `ProcessConfigurationRegistry` and `TransformerRegistry` resolve the process configurations and the transformer codes used in YAML; `TaskRegistry` resolves the task codes (`task: <code>`, declared by the `code` attribute of the `cleverage.task` tag or `#[AsTask]`).

**Bridge bundles** each add a small set of tasks (`src/Task/`) plus a client/adapter concept wired in `config/services.yaml`:
- `doctrine-process-bundle` — DB & entity tasks: `Database/DatabaseReaderTask`, `DatabaseUpdaterTask`, and `EntityManager/*` (`DoctrineReaderTask`, `DoctrineWriterTask`, `DoctrineBatchWriterTask`, cleaner/detacher/refresher/remover/clearer).
Expand Down Expand Up @@ -89,7 +89,7 @@ clever_age_process:
# entrypoint_type: file|text, default values, Symfony validation constraints)
tasks:
<taskName>:
service: '@CleverAge\ProcessBundle\Task\...' # the task class, referenced as a service
task: <code> # the task code (e.g. transformer, debug, rest_request)
options: { ... }
outputs: [ <nextTask>, ... ] # edges to downstream tasks (fan-out allowed)
error_strategy: stop|skip|...
Expand Down
8 changes: 4 additions & 4 deletions config/packages/process/demo.archive.unzip.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,17 @@ clever_age_process:
target: Foo
tasks:
constant:
service: '@CleverAge\ProcessBundle\Task\ConstantOutputTask'
task: constant_output
options:
output:
filename: '%kernel.project_dir%/var/data/archive.zip'
destination: '%kernel.project_dir%/var/data/unzip_archive'
outputs: entry
entry:
service: '@CleverAge\ArchiveProcessBundle\Task\UnzipTask'
task: archive_unzip
outputs: directory
directory:
service: '@CleverAge\ProcessBundle\Task\File\InputFolderBrowserTask'
task: input_folder_browser
outputs: debug
debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
6 changes: 3 additions & 3 deletions config/packages/process/demo.archive.unzip_invalid_input.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,16 @@ clever_age_process:
target: Foo
tasks:
filename:
service: '@CleverAge\ProcessBundle\Task\ConstantOutputTask'
task: constant_output
options:
output: '%kernel.project_dir%/var/data/archive.zip'
outputs: [unzip]

unzip:
service: '@CleverAge\ArchiveProcessBundle\Task\UnzipTask'
task: archive_unzip
options:
destination: '%kernel.project_dir%/var/data/unzip_archive'
outputs: [debug]

debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
4 changes: 2 additions & 2 deletions config/packages/process/demo.archive.unzip_not_a_zip.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ clever_age_process:
target: Foo
tasks:
unzip:
service: '@CleverAge\ArchiveProcessBundle\Task\UnzipTask'
task: archive_unzip
options:
filename: '%kernel.project_dir%/var/data/sample.txt'
destination: '%kernel.project_dir%/var/data/unzip_archive'
outputs: [debug]

debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
4 changes: 2 additions & 2 deletions config/packages/process/demo.archive.unzip_write_failure.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ clever_age_process:
target: Foo
tasks:
unzip:
service: '@CleverAge\ArchiveProcessBundle\Task\UnzipTask'
task: archive_unzip
options:
filename: '%kernel.project_dir%/var/data/archive.zip'
destination: '%kernel.project_dir%/var/data/sample.txt'
outputs: [debug]

debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
4 changes: 2 additions & 2 deletions config/packages/process/demo.archive.zip.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ clever_age_process:
target: Foo
tasks:
entry:
service: '@CleverAge\ArchiveProcessBundle\Task\ZipTask'
task: archive_zip
options:
filename: '%kernel.project_dir%/var/data/zip_archive.zip'
files:
- '%kernel.project_dir%/var/data/sample.txt'
files_base_path: '%kernel.project_dir%/var/data'
outputs: debug
debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
8 changes: 4 additions & 4 deletions config/packages/process/demo.archive.zip_files_base_path.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ clever_age_process:
tasks:
# Extracts archived_file.txt into <files_base_path>/backup/<files_base_path>/
unzip:
service: '@CleverAge\ArchiveProcessBundle\Task\UnzipTask'
task: archive_unzip
options:
filename: '%kernel.project_dir%/var/data/archive.zip'
destination: 'var/data/demo_zip_files_base_path/backup/var/data'
outputs: [zip_input]

# UnzipTask outputs the destination directory (a string): replace it with the ZipTask input
zip_input:
service: '@CleverAge\ProcessBundle\Task\TransformerTask'
task: transformer
options:
transformers:
constant:
Expand All @@ -31,11 +31,11 @@ clever_age_process:

# Paths are relative to the project directory: expected entry "demo_zip_files_base_path/backup/var/data/archived_file.txt"
zip:
service: '@CleverAge\ArchiveProcessBundle\Task\ZipTask'
task: archive_zip
options:
filename: '%kernel.project_dir%/var/data/demo_zip_files_base_path.zip'
files_base_path: 'var/data'
outputs: [debug]

debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
6 changes: 3 additions & 3 deletions config/packages/process/demo.archive.zip_multiple_inputs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ clever_age_process:
target: Foo
tasks:
inputs:
service: '@CleverAge\ProcessBundle\Task\ConstantIterableOutputTask'
task: constant_iterable_output
options:
output:
- filename: '%kernel.project_dir%/var/data/demo_zip_multiple_inputs_1.zip'
Expand All @@ -22,10 +22,10 @@ clever_age_process:
outputs: [zip]

zip:
service: '@CleverAge\ArchiveProcessBundle\Task\ZipTask'
task: archive_zip
options:
files_base_path: '%kernel.project_dir%/var/data'
outputs: [debug]

debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
4 changes: 2 additions & 2 deletions config/packages/process/demo.archive.zip_relative_path.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ clever_age_process:
target: Foo
tasks:
zip:
service: '@CleverAge\ArchiveProcessBundle\Task\ZipTask'
task: archive_zip
options:
filename: '%kernel.project_dir%/var/data/demo_zip_relative_path.zip'
files: 'var/data/sample.txt'
outputs: [debug]

debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
4 changes: 2 additions & 2 deletions config/packages/process/demo.archive.zip_write_failure.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ clever_age_process:
target: Foo
tasks:
zip:
service: '@CleverAge\ArchiveProcessBundle\Task\ZipTask'
task: archive_zip
options:
filename: '%kernel.project_dir%/var/data/missing_directory/demo_zip_write_failure.zip'
files: '%kernel.project_dir%/var/data/sample.txt'
files_base_path: '%kernel.project_dir%/var/data'
outputs: [debug]

debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
Original file line number Diff line number Diff line change
Expand Up @@ -10,42 +10,42 @@ clever_age_process:
Expected: two lines, "indirect: [\"item 1\",\"item 2\",\"item 3\"]" and "no iterable: [\"constant\"]"
tasks:
iterable:
service: '@CleverAge\ProcessBundle\Task\ConstantIterableOutputTask'
task: constant_iterable_output
options:
output: [1, 2, 3]
outputs: [format]
format:
service: '@CleverAge\ProcessBundle\Task\TransformerTask'
task: transformer
options:
transformers:
sprintf: { format: 'item %s' }
outputs: [indirect_aggregate]
indirect_aggregate:
service: '@CleverAge\ProcessBundle\Task\AggregateIterableTask'
task: aggregate_iterable
outputs: [indirect_dump]
indirect_dump:
service: '@CleverAge\ProcessBundle\Task\TransformerTask'
task: transformer
options:
transformers:
callback: { callback: json_encode }
sprintf: { format: 'indirect: %s' }
outputs: [debug]

constant:
service: '@CleverAge\ProcessBundle\Task\ConstantOutputTask'
task: constant_output
options:
output: constant
outputs: [no_iterable_aggregate]
no_iterable_aggregate:
service: '@CleverAge\ProcessBundle\Task\AggregateIterableTask'
task: aggregate_iterable
outputs: [no_iterable_dump]
no_iterable_dump:
service: '@CleverAge\ProcessBundle\Task\TransformerTask'
task: transformer
options:
transformers:
callback: { callback: json_encode }
sprintf: { format: 'no iterable: %s' }
outputs: [debug]

debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
12 changes: 6 additions & 6 deletions config/packages/process/demo.blocking.two_blocking_children.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,32 +9,32 @@ clever_age_process:
Expected: two lines, "first: [1,2,3]" and "second: [1,2,3]"
tasks:
entry:
service: '@CleverAge\ProcessBundle\Task\ConstantIterableOutputTask'
task: constant_iterable_output
options:
output: [1, 2, 3]
outputs: [first_aggregate, second_aggregate]

first_aggregate:
service: '@CleverAge\ProcessBundle\Task\AggregateIterableTask'
task: aggregate_iterable
outputs: [first_dump]
first_dump:
service: '@CleverAge\ProcessBundle\Task\TransformerTask'
task: transformer
options:
transformers:
callback: { callback: json_encode }
sprintf: { format: 'first: %s' }
outputs: [debug]

second_aggregate:
service: '@CleverAge\ProcessBundle\Task\AggregateIterableTask'
task: aggregate_iterable
outputs: [second_dump]
second_dump:
service: '@CleverAge\ProcessBundle\Task\TransformerTask'
task: transformer
options:
transformers:
callback: { callback: json_encode }
sprintf: { format: 'second: %s' }
outputs: [debug]

debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
Original file line number Diff line number Diff line change
Expand Up @@ -10,26 +10,26 @@ clever_age_process:
Expected: a single line, "both: [1,2,3,4]"
tasks:
first_iterable:
service: '@CleverAge\ProcessBundle\Task\ConstantIterableOutputTask'
task: constant_iterable_output
options:
output: [1, 2]
outputs: [aggregate]
second_iterable:
service: '@CleverAge\ProcessBundle\Task\ConstantIterableOutputTask'
task: constant_iterable_output
options:
output: [3, 4]
outputs: [aggregate]

aggregate:
service: '@CleverAge\ProcessBundle\Task\AggregateIterableTask'
task: aggregate_iterable
outputs: [dump]
dump:
service: '@CleverAge\ProcessBundle\Task\TransformerTask'
task: transformer
options:
transformers:
callback: { callback: json_encode }
sprintf: { format: 'both: %s' }
outputs: [debug]

debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
6 changes: 3 additions & 3 deletions config/packages/process/demo.cache.get_invalid_input.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,16 @@ clever_age_process:
Ex: bin/console cleverage:process:execute demo.cache.get_invalid_input
tasks:
sku:
service: '@CleverAge\ProcessBundle\Task\ConstantOutputTask'
task: constant_output
options:
output: 'key1'
outputs: [ get ]

get:
service: '@CleverAge\CacheProcessBundle\Task\GetTask'
task: cache_get
options:
adapter: 'memory' # key is given by the input
outputs: [ debug ]

debug:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
12 changes: 6 additions & 6 deletions config/packages/process/demo.cache.get_on_miss.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ clever_age_process:
Ex: bin/console cleverage:process:execute demo.cache.get_on_miss
tasks:
keys:
service: '@CleverAge\ProcessBundle\Task\ConstantIterableOutputTask'
task: constant_iterable_output
options:
output:
- key: 'key1'
Expand All @@ -18,18 +18,18 @@ clever_age_process:
outputs: [ get ]

get:
service: '@CleverAge\CacheProcessBundle\Task\GetTask'
task: cache_get
options:
adapter: 'memory' # key is given by the input
on_miss: skip
outputs: [ found ]
error_outputs: [ compute ]

found:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug

compute:
service: '@CleverAge\ProcessBundle\Task\TransformerTask'
task: transformer
options:
transformers:
mapping:
Expand All @@ -44,9 +44,9 @@ clever_age_process:
outputs: [ set, computed ]

set:
service: '@CleverAge\CacheProcessBundle\Task\SetTask'
task: cache_set
options:
adapter: 'memory' # key and value are given by the input

computed:
service: '@CleverAge\ProcessBundle\Task\Debug\DebugTask'
task: debug
Loading
Loading