Repository navigation
Keep trained state across a segment; outputs carry segment and config_id - #37
Merged
Merged
Conversation
A new segment means a gap in the data, not a new model. Processors whose state is a trained model or long-run statistics (sklearn, SGD, torch, MLP, LRR/SSR, SLDA, incremental decomposition) now keep it at a segment change; RNN and the transformer decoder drop only their hidden state or cache. Outputs built fresh rather than copied from the input now carry the input's segment and a config_id derived with ConfigChain/ConfigJoin from the input's id and what the stage adds (output channels, classes, the decomposed layout), so downstream stages reset only when this stage's output configuration changes. partial_fit on the incremental decomposition and SSR goes through _prepare_state, so training messages reset state by the same rule as signal messages. Empty (end-of-stream) messages pass through the models without being predicted on.
The sampler appended every message to its buffer, so samples from before a gap (or from another stream configuration) were joined to the ones after it as if continuous. A message whose segment or config_id differs from the buffer's now starts a new buffer. An empty (end-of-stream) message no longer fails when trimming.
fit() and partial_fit() reset state directly, outside the hash check, so the first __call__ saw a new hash and rebuilt the model, discarding the training. Both now go through _prepare_state, and the processor's hash is the input feature count, the only thing the model is built for: a fit on a bare array and the first signal message agree, while a change in feature count still rebuilds the model. The output channel axis is rebuilt with the model, so it follows the new model's outputs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adopts the new stream model in ezmsg-learn (background: ezmsg-org/ezmsg-sigproc#256).
segment(an int onAxisArray) changes only when the source measured a gap: the data is no longer continuous.config_id(an int digest, 0 = undeclared) says whether the stream configuration is the same. A gap is not a new model, so a trained model should survive one.Stack: this is the bottom of two.
cboulay/config-id(narrower resets for trained models) sits on top of it. It is based ondev.What changes
_reset_segmentto keep their model or long-run statistics: sklearn, SGD, torch (simple and trainable), the old MLP, SSR/LRR, and SLDA (a classifier loaded from file). The RNN keeps its model and resets only its hidden state. The transformer decoder resets only its cache. LRR needs no override of its own: SSR keeps its fit, and the inner sigprocAffineTransformTransformeris used directly (it has no memory, so the new sigproc does not reset it at a gap).segmentand a derivedconfig_id. Outputs that are built fresh rather than copied from the input (the regressors,DecodeOutputAdapter, SGD and SLDA classifier messages, incremental decomposition, sklearn/torch/RNN/transformer/MLP heads, RefitKalman) now carry the input's segment and aconfig_idderived with sigproc'sConfigChain/ConfigJoinfrom the input's id plus what the stage adds (output channels, classes, the decomposed layout, the state dimension). An undeclared input gives an undeclared output.partial_fitgoes through_prepare_statefor the incremental decomposition and SSR/LRR, so training messages reset state by the same rule as signal messages instead of a hand-rolled hash comparison.DecodeOutputAdapter, sklearn, SGD and the RNN as empty outputs with the right channels. Before, they were predicted on (which sklearn refuses) or dropped. The incremental decomposition and SLDA already returned their empty template; it now carries segment and config_id too.fit()/partial_fit()reset state outside the hash check, so the first__call__saw a new hash and rebuilt the model, which threw the training away. Both now go through_prepare_state, andSklearnModelProcessor's hash is the input feature count, the only thing the model is built for. That narrower hash is needed for this fix (thefit()dummy message and the first real message differ in everything else), so it lands here rather than in the next PR. The output channel axis is rebuilt with the model.Behaviour changes for existing users
segmentand a new derivedconfig_id, so downstream stages reset only when this stage's output configuration changes.None. The other predictors return empty outputs where before they raised or returnedNone.SklearnModelProcessor: a model trained withfit/partial_fitbefore the first call is now used. It resets only when the input feature count changes, not when labels, key or dtype change. After such a reset, the output channel axis follows the new model.Testing
tests/unit/test_stream_contract.py(models kept across a segment for SGD, LRR, sklearn, torch and RNN; LRR keeps accumulating on a new segment; derived config_id on ALR and IPCA outputs; windowed training does not reset the decomposition; empty messages pass; SeqSeqSampler restart), and intests/unit/test_sklearn.pyfit/partial_fit before the first call and a feature-count change rebuilding the model and channels.Dependencies
ezmsg-baseproc >= 1.17.0:_reset_segment, resets decided fromconfig_id,end_of_stream()(used in tests). It requires ezmsg 3.10.0b7 (AxisArray.config_id).ezmsg-sigproc >= 3.13.0:ConfigChain/ConfigJoin, andAffineTransformTransformerkeeping its weights at a gap (LRR relies on this).pyproject.toml.