Repository navigation
fix: don't assign None when a value's group did not participate in the match - #151
Conversation
…e match A rule may match via an alternation branch that does not contain the value's named group (e.g. '^|zz'). The engine then assigned None, which appended a literal None to List values and overwrote Filldown memory, silently losing data. Values are now left unchanged when the group did not participate in the match. Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
There was a problem hiding this comment.
Consider this instead:P
for name, text in matched.groupdict().items():
if text is None:
continue # Group not part of the matched branch.
|
|
||
| If a record entry is a list then append, otherwise values are replaced. | ||
|
|
||
| A named group that did not participate in the match (e.g. the rule |
There was a problem hiding this comment.
Or rather:
"Groups that did not participate in the match leave the value unchanged."
| ' ^$boo\n' | ||
| ' ^$boo|zz -> Next.Record\n' | ||
| ' ^$hoo -> Next.Record\n\n' | ||
| 'EOF' |
There was a problem hiding this comment.
Drop 'EOF', not really needed.
| result = t.ParseText('one\nzz') | ||
| self.assertListEqual(result, [[['one'], '']]) | ||
|
|
||
| # Filldown memory is not wiped by a non-matching alternation branch. |
| 'Value hoo (tw.)\n\n' | ||
| 'Start\n' | ||
| ' ^$boo\n' | ||
| ' ^$boo|zz -> Next.Record\n\n' |
There was a problem hiding this comment.
Consider testing:
^${boo}(${hoo})?
Per review: iterate matched.groupdict() in _CheckLine and skip None
groups there, restoring the original _AssignVar body; reword the
docstring. Split the Filldown scenario into its own unittest, drop the
explicit 'EOF' states, and add the suggested '^${boo}(${hoo})?'
optional-group case.
Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
|
Thanks for the review! All suggestions applied in 3a7a3da.
Both new tests fail without the fix (the Filldown case yields |
|
Looks good. Thanks for the fixes. |
|
Thank you for the review and the merge @harro! The groupdict() form is a real improvement — glad the tests caught the distinction. On a personal note — I'm currently open to new opportunities (Full Stack (PHP, NodeJs, Python) / Data Science / Applied AI / Devtools, based in India). If there's an opening or referral you'd recommend, my inbox is open: |
Fixes #150
Summary
When a rule matches via an alternation branch that does not contain a value's named group (e.g.
^$boo|zzmatchingzz),matched.group(name)isNone._AssignVarcalledAssignVar(None)anyway, which:NonetoListvalues (output records like[['one', None], '']),Filldownmemory viaOnAssignVar, silently losing the carried-down value (and dropping the current record), andContinuerule had just assigned on the same line._AssignVarnow skips the assignment when the group did not participate in the match, leaving the value unchanged — the same state as when no rule matches.Testing
testAlternationWithoutValueGroupcovers both shapes:List: inputone/zzmust produce[[['one'], '']], not[[['one', None], '']]— fails on un-fixed HEAD at the list assertion.Filldown: inputone/zz/tw2must produce[['one', ''], ['one', 'tw2']], not[['', 'tw2']]— fails on un-fixed HEAD (first record dropped).python3 -m pytest tests/— 109 passed, 28 subtests passed, plus the 2 warnings that pre-exist on main. Same suite on un-fixed HEAD: 108 passed + the new test failing.Notes
The fix is deliberately narrow: only the non-participating-group case changes. Values that match an empty string (e.g.
(\S*)matching'') still assign normally —''is a genuine match;Nonemeans the group was not part of the branch that matched.