Repository navigation
[MRG] Report the half-squared L2 objective that smooth_ot_dual minimises - #914
Open
xihaian251 wants to merge 8 commits into
Open
xihaian251 wants to merge 8 commits into
xihaian251 wants to merge 8 commits into
Conversation
…imises ot.solve(..., reg_type="L2") solved <M,P> + (reg/2)||P||_F^2 -- smooth_ot_dual is called with gamma=reg and SquaredL2.Omega is half-squared -- but reported <M,P> + reg||P||_F^2, i.e. <M,P> + 2*Omega(P), which is not the optimum of the problem that was solved. The entropy branch a few lines up already reports exactly its own NegEntropy.Omega, so L2 was the lone outlier. Plans are unchanged. Because solve_bary_sample uses that value as its stopping_criterion="loss" signal, its converged barycenter can move for some tolerances and backends; the added tests assert consistency with the returned plans rather than an iteration count. Issue PythonOT#896 was reproduced independently by @anveshachuri, who also raised the coordination question that prompted this PR; no code or tests here are theirs.
added 2 commits
November 5, 2026 12:05
These docstrings are raw strings, so a double backslash survives into the rendered LaTeX, where \sum is a line break rather than \sum. The surrounding lines in the same docstrings already use single backslashes.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #914 +/- ##
==========================================
+ Coverage 97.09% 97.10% +0.01%
==========================================
Files 130 130
Lines 27229 27275 +46
==========================================
+ Hits 26437 26485 +48
+ Misses 792 790 -2 🚀 New features to boost your workflow:
|
added 2 commits
November 5, 2026 13:16
…lers Adds three parametrized cases whose expected plans and optima (17/10, 7/8, 21/8) were derived on the 2x2 transport polytope rather than read back from POT. The property test recomputes the objective from the returned plan, so a library that returned a suboptimal plan together with a value consistent with that plan would still satisfy it; an externally computed optimum does not move with the plan and catches that. Cases and this reasoning supplied by @anveshachuri in PythonOT#896. Also removes the `except NotImplementedError: pytest.skip(...)` guards. Every backend POT tests -- numpy, torch, jax and tf -- implements the L2 path and passes, so those six lines were dead defensive code added only because tf could not be run locally; they were also the entire patch-coverage gap.
The previous wording said R(T) = sum T log T for "KL" as well as "entropy". It is not: with reg_type="KL" the reported value is <M,P> + reg*KL(P|| a b^T), while "entropy" reports <M,P> + reg*sum P log P. Both names reach the same solver and return the same plan, and their values differ by the plan-independent constant reg*(sum_i a_i log a_i + sum_j b_j log b_j). Verified against the implementation rather than assumed: KL minus entropy measured 0.627741162610 against the predicted constant 0.627741162589, agreeing to the solver's own convergence tolerance.
# Conflicts: # RELEASES.md
# Conflicts: # test/test_solvers.py
Open
1 task done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Types of changes
Motivation and context / Related issue
Closes #896.
ot.solve(..., reg_type="L2")returns the correct transport plan but a value that is not theoptimum of the problem it solved. The reported scalar is off by exactly
Omega(P), i.e. a factorof two in the regularisation term.
solve_sampleandsolve_bary_sampleinherit it.Mathematical convention
POT's quadratic regularizer is half-squared everywhere it is implemented:
ot/smooth.py:223SquaredL2.Omega(T) = 0.5 * gamma * sum(T**2)ot/smooth.py:211SquaredL2.delta_Omega(X) = sum(max(X,0)**2) / (2*gamma)ot/unbalanced/_lbfgs.py:63reg_l2(G) = sum((G-c)**2) / 2ot/unbalanced/_lbfgs.py:94marg_l2(G) = reg_m1 * 0.5 * sum(...)ot/unbalanced/_mm.py:204cost = cost + reg * 0.5 * sum((G-c)**2)ot/solvers/_linear.py:531is the only place that usesreg * sum(plan**2). The decisive detail isin the same function: the entropy branch at
:489reportsreg * sum(plan*log(plan+1e-16)),which is exactly its own
NegEntropy.Omega(ot/smooth.py:204). One branch reports its ownOmega, the other reports2 * Omega.This is not a convention preference. A reported quantity that is neither the primal objective nor
the dual objective has no reading under which it is correct.
For what it is worth on the "was this deliberate?" question:
git log -S 'reg * nx.sum(plan**2)'overot/solvers.pyandot/solvers/_linear.pyreturns exactly two commits —0411ea2(#388, 2022-12-15),which introduced the expression along with the
solveAPI itself, and this one. Four years, twotouches, no correction: this reads as an error that shipped with an API rather than a convention that
was chosen and then maintained.
Root cause
solvecallssmooth_ot_dual(..., reg=reg), i.e.gamma = reg, so the plan returned minimisesThe value line then computes
<M,P> + reg*||P||**2 = <M,P> + 2*Omega(P). In dual terms, it is theaffine part of the L2 dual with the conjugate term
delta_Omegadropped — verified exactly: forM=[[0,1],[1,0]],a=b=[1/2,1/2],reg=10, solvingP_ij = (alpha_i + beta_j - M_ij)/regfor thepotentials gives
a.alpha + b.beta = (reg+2)/4 = 3, which is precisely what the code reports, whilethe full dual objective is
17/10.Changes
One production line, plus documentation and tests:
ot/solvers/_linear.py—value = value_linear + 0.5 * reg * nx.sum(plan**2).ot/solvers/_linear.py(solve,solve_sample) andot/solvers/_bary.py(solve_bary_sample) —the
reg_typedocstrings wrote ``Type of regularization :math:R``` without ever definingR`; theynow state it per name: `R(T) = sum T log T` for "entropy", `R(T) = KL(T ‖ a b^T)` for "KL", and
`R(T) = 1/2 ||T||_F^2` for "L2". The KL/entropy distinction was measured, not assumed: the two names
reach the same solver and return the same plan, and their values differ by the plan-independent
constant `reg*(sum a log a + sum b log b)` (measured `0.627741162610` against predicted
`0.627741162589`, agreeing to the solver's own tolerance).
docs/source/user_guide.rst:446— gaveOmega(gamma) = sum gamma^2(no half) in the paragraphthat, five lines later, directs the reader to
smooth_ot_dual/smooth_ot_semi_dualwithreg_type='l2', whoseOmegais half-squared. The formula and its own cited source disagreed.RELEASES.md— entry under Closed issues, including the behavioural caveat below.test/test_solvers.py— four tests: three hand-computed instances plus the property tests.Numerical behaviour
Exact values (verified independently in Wolfram Language with
Minimizeover the constrainedpolytope, in SymPy via exact KKT solve, and against live POT — all three agree):
| instance | plan |
<M,P>|||P||^2| true optimum | reported before | after ||---|---|---|---|---|---|---|
|
M=[[0,1],[1,0]],reg=10|[[3/10,1/5],[1/5,3/10]]| 2/5 | 13/50 | 17/10 | 3 | 17/10 ||
M=[[0,1],[1,0]],reg=4|[[3/8,1/8],[1/8,3/8]]| 1/4 | 5/16 | 7/8 | 3/2 | 7/8 ||
M=[[1,2],[3,4]],reg=1|1/4 * ones| 5/2 | 1/4 | 21/8 | 11/4 | 21/8 |The inflation is
reg/2 * ||P||^2, i.e.(reg^2+4)/(8*reg)for the first cost matrix — it growswithout bound as
regincreases, so this is not a rounding artifact.A regime worth knowing, because it explains why this survived: for
M=[[0,1],[1,0]]with uniform marginals, exact analysis gives the interior minimisert* = (reg+2)/(4*reg), valid only forreg > 2. Forreg <= 1both conventions share the vertex[[1/2,0],[0,1/2]], and there the reported value coincides exactly with the optimum of theno-half problem (
reg=1/8 -> 1/16,1/4 -> 1/8,1/2 -> 1/4,3/4 -> 3/8,1 -> 1/2). A usercomparing
res.valueagainst a hand-computed||P||^2objective in that regime gets bit-equalityand concludes the code is fine. It is
Falseat every testedregthat the reported value equalsthe half-squared optimum, which is the assertion that actually discriminates. The new tests therefore
use
reg=10, deliberately, not incidentally.Compatibility and behavioural impact
Stating this plainly, because it is not a no-op:
every point above and across numpy/torch/jax.
res.valuewithreg_type="L2"will see different numbers. The new number is the optimum of theproblem that was already being solved.
solve_bary_sampleresults can change. Withstopping_criterion="loss"that scalar is theBCD signal (
ot/solvers/_bary.py:149sumslist_res[k].value), so correcting it moves thestopping point. Measured on one instance (3 sources of 12/15/9 points,
n=6,reg=1.0, seed 7),comparing two installed trees differing only in this line:
tol_bary=1e-5the converged barycenter changes on numpy (7 to 13 BCDiterations), and does not change on torch (7 to 7) or jax (8 to 8);
tol_bary1e-2, 3e-3, 1e-3, 3e-4, 1e-6 and 1e-7 the result is identical on both trees;one case and loses two (differences of order 2e-5 in the true objective). So this PR should not
be read as a convergence improvement; it is a correctness fix whose barycenter side effect is
tolerance- and backend-dependent.
stopping_criterion="bary"solutions are identical before and after on all three backends;only their reported value changes.
+3.86e-2(numpy),+3.83e-2(torch) and+3.93e-2(jax) on thebarycenter instance before the change, and match the objective recomputed from the returned plans
to within
2e-9after it.How has this been tested (if it applies)
Four tests in
test/test_solvers.py, 18 cases. They deliberately do not assertres.value == <formula>, which would only restate the patch.test_solve_l2_value_hand_computed— three parametrized instances whose expected plans and optima(
17/10,7/8,21/8) were derived on the 2×2 transport polytope, outside POT. Needed because atest that evaluates the objective at the returned plan is satisfied by any self-consistent
(plan, value) pair: had the solver returned the no-half minimiser
t = 11/40with a valueconsistent with it, the property test below would recompute
137/80and pass, while this one pins17/10and fails. Suggested by @anveshachuri, who supplied the cases.atol=1e-5is calibrated from both sides: the observed convergence noise is7.3e-7on thethird instance (already present in
value_linear, so anatol=1e-9assertion would falsely failon a correct implementation), while the defect is
1.3/0.625/0.125.test_solve_l2_value_is_the_optimum— asserts the exact optimum, the exact plan, and that thevalue is a lower bound over feasible competitors. The unfixed
3.0violates the bound, so thisfails for a reason about optimality, not arithmetic agreement.
test_solve_l2_value_matches_dual_optimum— compares against the objectivesmooth_ot_dualreports for itself (
-log["res"].fun). Two independent code paths, joined by strong duality.test_solve_bary_sample_l2_value_matches_plans— recomputes the objective fromres.list_res[*].planin the test. Asserts no iteration count and no specific barycenter, soit stays valid even where the fix moves the stopping point, for both stopping criteria.
Measured on two installed trees, with provenance verified at byte level before comparing:
E:\pot911_work— branchfix-merge-bijections-plan-count, whoseot/solvers/*.pyare byte-identical to
upstream/master(90484ae), so its L2 behaviour is master's;E:\pot896_work— this branch, a fresh worktree from90484aewith the extensionsrebuilt from source;
ot/solvers/_linear.pydiffers from master in exactly the one value line(verified against an intermediate copy as 1 differing line out of 1618, with no other
.pychangedon the solvers path).
Results, by reverting only the production line and keeping the tests:
18 failed· this branch:18 passed— 4 tests × numpy/torch/jax, withno skips, so every backend ran and every result is real;
pytest test/test_solvers.py test/test_smooth.py→ 933 passed, 98 skipped; that final committouches no executable path, and CI runs the whole suite on this head —
linux3.11/3.12/3.13/3.14,linux-torch,linux-jax,linux-tf,linux-minimal,windows,macos,doctestsall pass;pre-commit run --files <changed>→ ruff (3 hooks), ruff-format and codespell all pass.Backends: numpy 2.4.6, torch 2.10.0+cpu, jax 0.12.0 (x64) locally. TensorFlow is not installed here,
so it is covered by CI — and confirmed there by reading the job log rather than trusting the green
check: on this head,
linux-tfruns all four tests under the[tf]parametrization (including thethree hand-computed instances) and reports
PASSED, as do[torch]and[jax]on their jobs. Theearlier
except NotImplementedError → pytest.skipguards were therefore dead code and have beenremoved; they were also the whole patch-coverage gap, and patch coverage is now 100% (target 50%).
Attribution
@anveshachuri reproduced this issue independently, raised the coordination question on #896 that is
why I filed this now, and supplied the three hand-derived regression cases (plans and optima) used by
test_solve_l2_value_hand_computed, together with the argument for keeping hand-computed valuesalongside the property test. Thank you — the point about anchoring to an externally computed optimum
improved the test suite, and it was right.
That is credited in the commit messages and here. Deliberately not credited as
Co-authored-by,which would assert contributed code or tests that are not in this PR. The derivations, measurements,
tolerance calibration and all code here are mine, and any error in them is mine too.
AI assistance disclosure
Parts of this analysis (exact-rational derivation, regression-test drafting, and the two-tree
numerical comparison) were produced with an AI assistant under my direction and review. Every number
in this description was reproduced on this machine against the two installed trees before being
written here, and I take responsibility for the change. Flagging it explicitly since POT has no
stated policy; say the word if you would rather this went through a different process.
PR checklist
One question for maintainers
If instead
Omega = sum gamma^2(no half) is what you want to be authoritative, the correct fix issmooth_ot_dual(..., reg=2*reg, ...), which changes every L2 plan returned since 0.9.0 andbreaks the current agreement with
ot.smooth. I deliberately did not do that: it is a versionedbehaviour change, not a bug fix. Happy to convert this if that is the decision.