numpy: return empty results for intersect1d and setxor1d at size 0 - #40231
numpy: return empty results for intersect1d and setxor1d at size 0#40231VaggelisGian wants to merge 4 commits into
Conversation
jnp.intersect1d(a, b, size=0) and jnp.setxor1d(a, b, size=0) raised "ValueError: zero-size array to reduction operation max which has no identity". Root cause in both helpers: with fill_value unspecified, _intersect1d_size() and _setxor1d_size() pad unfilled slots toward vals.max()/vals.min(), and vals.max() raises when size is zero because vals itself is a zero-size array. Guard the reduction on vals.size: when the output is empty there is nothing to pad, so both functions now return an empty array of the natural result dtype, including under assume_unique=True, return_indices=True, and explicitly given fill_value (which was already handled). Deliberate limits: jnp.union1d(size=0) already returns an empty array and is untouched. jnp.setdiff1d has a related failure at size=0 that is being fixed separately via an early return, so setdiff1d is not touched here. Test Plan: python -m pytest tests/lax_numpy_setops_test.py -k "intersect or setxor or union or setdiff" -q 90 passed, 82 deselected in 16.94s JAX_ENABLE_X64=1 python -m pytest tests/lax_numpy_setops_test.py -k "intersect or setxor or union or setdiff" -q 90 passed, 86 deselected in 15.45s python -m pytest tests/lax_numpy_setops_test.py -n auto -q 172 passed in 16.99s JAX_ENABLE_X64=1 python -m pytest tests/lax_numpy_setops_test.py -n auto -q 176 passed in 12.82s Before this change the same tree failed the new size-zero tests: 9 failed, 11 passed, 152 deselected FAILED testIntersect1dSizeZero6/7/8, testSetxor1dSizeZero0/2/4/5/7/9 ValueError: zero-size array to reduction operation max which has no identity
| if fill_value is None: | ||
| vals = where(arange(len(vals)) < num_results, vals, vals.max()) | ||
| return where(arange(len(vals)) < num_results, vals, vals.min()) | ||
| if vals.size: # max() and min() are undefined for zero-size arrays. |
There was a problem hiding this comment.
It seems to me if vals is empty we should skip the fill_value where logic altogether. Something like this:
if vals.size == 0:
return vals
elif fill_value is None:
return where(arange(len(vals)) < num_results, vals, vals.min())
else:
return where(arange(len(vals)) < num_results, vals, fill_value)Same below.
There was a problem hiding this comment.
Done: both helpers now return vals early when empty. Note I kept the existing max-then-min padding for non-empty vals: the first max() where is load-bearing, it replaces the zero-filled slots of the masked gather before the min() reduction, otherwise padded entries keep the 0 fill instead of the smallest real result (caught by testIntersect1d1). ba39fb2
|
|
||
| @jtu.sample_product( | ||
| dtype=default_dtypes, | ||
| assume_unique=[False, True], |
There was a problem hiding this comment.
I don't think it's important for this case to test all parameter combinations – just a single unparameterized test with size=0 is probably fine.
Same for the intersect1d test below.
There was a problem hiding this comment.
Done: replaced both with single unparameterized tests using default int arrays.
| else: | ||
| return vals | ||
| if vals.size == 0: | ||
| # min() and max() are undefined for zero-size arrays; skip the padding. |
There was a problem hiding this comment.
This comment is misleading in the current structure – we're not doing this to avoid min/max; rather we're doing this because fill values are irrelevant in an empty array.
Same below.
There was a problem hiding this comment.
Done: both comments now say the fill values are irrelevant in an empty array. cd65c85
| def testSetxor1dSizeZero(self): | ||
| ar1 = np.array([1, 3, 5]) | ||
| ar2 = np.array([2, 3, 4]) | ||
| self.assertArraysEqual(jnp.setxor1d(ar1, ar2, size=0), |
There was a problem hiding this comment.
This will fail because assertArraysEqual defaults to asserting that dtypes match, and when X64 mode is disabled JAX will return int32 and NumPy will return int64.
Easier I think would be to just assert that the shape is as expected:
self.assertEqual(result.shape, (0,))
Same below.
There was a problem hiding this comment.
Done: both tests now assert the output shape is (0,). cd65c85
2 participants
Description
jnp.intersect1dandjnp.setxor1dwith a concretesize=0raiseValueError: zero-size array to reduction operation max which has no identity, because the_intersect1d_size/_setxor1d_sizehelpers compute their default padding viavals.max()(and.min()) on a zero-size result:The fix guards the padding reductions on
vals.size; an empty output has nothing to pad, so the arrays are returned as-is. With an explicitfill_valuethe size=0 path already worked and is unchanged; behavior at any nonzero size is bit-identical (valsalways has shape(size,), so the guard fires only at size 0).Scope note:
setdiff1dhas its own size=0 crash with a different mechanism, fixed separately (#40230);union1dis untouched.AI assistance disclosure: this fix was developed with AI coding assistance under my direction; I verified the mechanisms against NumPy oracles, ran all tests shown below, and take responsibility for the change.
Test Plan
The new
testIntersect1dSizeZero/testSetxor1dSizeZerofail on the unfixed tree exactly on the fill_value=None variants with the ValueError above (9 failed pre-fix), all pass after. A 576-case differential grid over sizes {0,1,2,3,5,10} x dtypes x assume_unique x fill_value x return_indices shows the only behavioral change is size=0 + default-fill going from raise to empty.Recent neighboring fixes to the same file: #32335, #32402, #32340. Sibling PRs: #40229 (unique empty-axis shapes), #40230 (setdiff1d size=0).