Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions RELEASES.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
- Fix ordinary JAX gradients for dense `ot.emd2` and preserve the magnitude and higher-order dependence of gradients supplied to `JaxBackend.set_gradients`, including the envelope gradients in `ot.solve` (Issue #625)
- Fix `ot.semidiscrete_wasserstein2_unif_circle` pairing sorted sample locations with unsorted weights, which broke permutation and rotation invariance for nonuniform weights (Issue #921)
- Fix an out-of-bounds read in `ot.bsp.merge_bijections`: the number of plans was taken from `perms.shape[1]` (the number of samples) instead of `perms.shape[0]`, so merging fewer plans than samples read past the array and merging more silently dropped plans. A `perms` outside the documented `(n_plans, n_samples)` layout, an empty plan stack and rows that are not permutations are now rejected (PR #911, Issue #910)
- Fix the value reported by `ot.solve`, `ot.solve_sample` and `ot.solve_bary_sample` for `reg_type="L2"`: it was `<M,P> + reg*||P||_F^2`, while `smooth_ot_dual` is called with `gamma=reg` and `ot.smooth.SquaredL2.Omega` is half-squared, so the reported scalar was `<M,P> + 2*Omega(P)` and not the optimum of the problem that was solved (for `M=[[0,1],[1,0]]`, `a=b=[1/2,1/2]`, `reg=10` it returned 3.0 instead of 17/10). Transport plans are unchanged. `reg_type` docstrings and `docs/source/user_guide.rst` now state `R(T) = (1/2)||T||_F^2`. Since `solve_bary_sample` uses that value as its `stopping_criterion="loss"` signal, its converged barycenter can change for some tolerances and backends (PR #914, Issue #896)
- Fix overflow to NaN in `ot.stochastic.coordinate_grad_semi_dual` by shifting before exponentiating, as `ot.stochastic.c_transform_entropic` already does. The factor cancels in the normalisation, so results are unchanged where the old code did not overflow (PR #866, Issue #264)
- Fix `ot.bsp.compute_bspot_bijection` (and `random_state` of `ot.solve_sample(..., method="bsp")`) not being reproducible for a given `seed`: its candidate plans came from a `thread_local` `std::mt19937` seeded once per OpenMP thread, so the result depended on the call history and on the number of threads. Every thread started from the same state, so the previous version could underperform with multiple threads. Each plan now draws its own stream, derived from the seed and the plan index (PR #898, Issue #897)
- Fix duplicate function and class descriptions in generated module API pages while preserving top-level aliases and example backreferences (Issue #527)
Expand Down
2 changes: 1 addition & 1 deletion docs/source/user_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -443,7 +443,7 @@ The first general regularization term we can solve is the quadratic
regularization of the form

.. math::
\Omega(\gamma)=\sum_{i,j} \gamma_{i,j}^2
\Omega(\gamma)=\frac{1}{2}\sum_{i,j} \gamma_{i,j}^2

This regularization term has an effect similar to entropic regularization by
densifying the OT matrix, yet it keeps some sort of sparsity that is lost with
Expand Down
9 changes: 8 additions & 1 deletion ot/solvers/_bary.py
Original file line number Diff line number Diff line change
Expand Up @@ -259,7 +259,14 @@ def solve_bary_sample(
If None, then use :math:`\mathbf{c} = \mathbf{a}^{(k)} \mathbf{b}^T`.
If :math:`\texttt{reg_type}=`'entropy', then :math:`\mathbf{c} = 1_{|a^{(k)}|} 1_{|b|}^T`.
reg_type : str, optional
Type of regularization :math:`R` either "KL", "L2", "entropy", by default "KL"
Type of regularization :math:`R`, either "KL", "L2", "entropy", by default
"KL". :math:`R(T)=\sum_{i,j} T_{i,j}\log(T_{i,j})` for "entropy",
:math:`R(T)=KL(T\,\|\,\mathbf{a}\mathbf{b}^T)` for "KL" and
:math:`R(T)=\frac{1}{2}\|T\|_F^2` for "L2", so the problem solved is
:math:`\langle M, T\rangle + reg\,R(T)` and ``res.value`` is its optimum.
"KL" and "entropy" are solved by the same routine and give the same plan;
their values differ by the plan-independent constant
:math:`reg\,(\sum_i a_i\log a_i + \sum_j b_j\log b_j)`.
unbalanced : float or indexable object of length 1 or 2
Marginal relaxation term.
If it is a scalar or an indexable object of length 1,
Expand Down
27 changes: 22 additions & 5 deletions ot/solvers/_linear.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,16 @@ def solve(
If None, then use :math:`\mathbf{c} = \mathbf{a} \mathbf{b}^T`.
If :math:`\texttt{reg_type}=`'entropy', then :math:`\mathbf{c} = 1_{dim_a} 1_{dim_b}^T`.
reg_type : str, optional
Type of regularization :math:`R` either "KL", "L2", "entropy",
by default "KL". a tuple of functions can be provided for general
solver (see :any:`cg`). This is only used when ``reg!=None``.
Type of regularization :math:`R`, either "KL", "L2", "entropy", by default
"KL". :math:`R(T)=\sum_{i,j} T_{i,j}\log(T_{i,j})` for "entropy",
:math:`R(T)=KL(T\,\|\,\mathbf{a}\mathbf{b}^T)` for "KL" and
:math:`R(T)=\frac{1}{2}\|T\|_F^2` for "L2", so the problem solved is
:math:`\langle M, T\rangle + reg\,R(T)` and ``res.value`` is its optimum.
"KL" and "entropy" are solved by the same routine and give the same plan;
their values differ by the plan-independent constant
:math:`reg\,(\sum_i a_i\log a_i + \sum_j b_j\log b_j)`.
A tuple of functions can be provided for a general solver (see
:any:`cg`). This is only used when ``reg!=None``.
unbalanced : float or indexable object of length 1 or 2
Marginal relaxation term.
If it is a scalar or an indexable object of length 1,
Expand Down Expand Up @@ -528,7 +535,10 @@ def solve(
)

value_linear = nx.sum(M * plan)
value = value_linear + reg * nx.sum(plan**2)
# SquaredL2.Omega (ot/smooth.py) is half-squared and smooth_ot_dual is
# called with gamma=reg, so `plan` above minimises
# <M,P> + (reg/2)*||P||_F^2. Report that objective, not <M,P> + reg*||P||_F^2.
value = value_linear + 0.5 * reg * nx.sum(plan**2)
potentials = (log["alpha"], log["beta"])

else:
Expand Down Expand Up @@ -693,7 +703,14 @@ def solve_sample(
If None, then use :math:`\mathbf{c} = \mathbf{a} \mathbf{b}^T`.
If `reg_type='entropy'`, then :math:`\mathbf{c} = 1_{dim_a} 1_{dim_b}^T`.
reg_type : str, optional
Type of regularization :math:`R` either "KL", "L2", "entropy", by default "KL"
Type of regularization :math:`R`, either "KL", "L2", "entropy", by default
"KL". :math:`R(T)=\sum_{i,j} T_{i,j}\log(T_{i,j})` for "entropy",
:math:`R(T)=KL(T\,\|\,\mathbf{a}\mathbf{b}^T)` for "KL" and
:math:`R(T)=\frac{1}{2}\|T\|_F^2` for "L2", so the problem solved is
:math:`\langle M, T\rangle + reg\,R(T)` and ``res.value`` is its optimum.
"KL" and "entropy" are solved by the same routine and give the same plan;
their values differ by the plan-independent constant
:math:`reg\,(\sum_i a_i\log a_i + \sum_j b_j\log b_j)`.
unbalanced : float or indexable object of length 1 or 2
Marginal relaxation term.
If it is a scalar or an indexable object of length 1,
Expand Down
120 changes: 120 additions & 0 deletions test/test_solvers.py
Original file line number Diff line number Diff line change
Expand Up @@ -1235,6 +1235,126 @@ def custom_metric(X_b, X_a):
)


# Hand-computed optima for reg_type="L2". These were derived on the 2x2 transport polytope
# independently of POT: parameterise P(t) = [[t, 1/2-t], [1/2-t, t]] for 0 <= t <= 1/2, then
# minimise <M,P> + (reg/2)||P||_F^2. d^2/dt^2 = reg*4 > 0 makes the problem strictly convex, so
# the stationary point is the global minimum, and KKT holds because grad = M + reg*P is additive
# (G11+G22 == G12+G21) while every entry of P stays positive.
# M=[[0,1],[1,0]], reg=10: t*=3/10, <M,P>=2/5, ||P||^2=13/50 -> J = 17/10
# M=[[0,1],[1,0]], reg=4 : t*=3/8, <M,P>=1/4, ||P||^2=5/16 -> J = 7/8
# M=[[1,2],[3,4]], reg=1 : <M,P> = 5/2 is CONSTANT along the whole segment, so the optimum is
# set by ||P||^2 alone: t*=1/4, ||P||^2=1/4 -> J = 5/2 + 1/8 = 21/8
lst_l2_exact_cases = [
([[0.0, 1.0], [1.0, 0.0]], 10.0, [[0.3, 0.2], [0.2, 0.3]], 17.0 / 10.0),
([[0.0, 1.0], [1.0, 0.0]], 4.0, [[0.375, 0.125], [0.125, 0.375]], 7.0 / 8.0),
([[1.0, 2.0], [3.0, 4.0]], 1.0, [[0.25, 0.25], [0.25, 0.25]], 21.0 / 8.0),
]


@pytest.mark.parametrize("M,reg,plan_expected,value_expected", lst_l2_exact_cases)
def test_solve_l2_value_hand_computed(nx, M, reg, plan_expected, value_expected):
# The expected values come from the derivation above, not from any POT function, so this
# pins the optimum against a wrong plan too: a value merely consistent with whatever plan
# was returned would also match a suboptimal plan (137/80 at t=11/40 for the first case).
Mb, ab = nx.from_numpy(np.array(M), np.array([0.5, 0.5]))
res = ot.solve(Mb, ab, ab, reg=reg, reg_type="L2")

np.testing.assert_allclose(nx.to_numpy(res.plan), plan_expected, atol=1e-6)
# atol covers the BCD convergence error of smooth_ot_dual, worst observed ~8e-7 on these
# instances; the reported-value defect this guards is 0.125 to 1.3, four orders larger.
np.testing.assert_allclose(nx.to_numpy(res.value), value_expected, rtol=0.0, atol=1e-5)


def _l2_primal_objective(nx, M, plan, reg):
# <M, plan> + (reg/2)*||plan||^2: the objective that reg_type="L2" minimises, recomputed
# here so the assertion below does not reuse the solver's reporting code.
return nx.sum(M * plan) + 0.5 * reg * nx.sum(plan**2)


def test_solve_l2_value_is_the_optimum(nx):
# M=[[0,1],[1,0]] with a=b=[1/2,1/2] is small enough to solve exactly: for reg>2 the
# minimiser of <M,P> + (reg/2)||P||^2 is P=[[3/10,1/5],[1/5,3/10]], where <M,P>=2/5,
# ||P||^2=13/50 and the optimum is 17/10. reg must exceed 2: at reg<=1 both quadratic
# conventions share the vertex [[1/2,0],[0,1/2]] and no value can tell them apart.
M = np.array([[0.0, 1.0], [1.0, 0.0]])
a = np.array([0.5, 0.5])
Mb, ab = nx.from_numpy(M, a)

res = ot.solve(Mb, ab, ab, reg=10.0, reg_type="L2")
plan = nx.to_numpy(res.plan)
np.testing.assert_allclose(plan, [[0.3, 0.2], [0.2, 0.3]], atol=1e-6)

# the reported value is the exact optimum,
np.testing.assert_allclose(nx.to_numpy(res.value), 1.7, rtol=1e-6)
# the objective of the returned plan,
np.testing.assert_allclose(
nx.to_numpy(res.value),
nx.to_numpy(_l2_primal_objective(nx, Mb, res.plan, 10.0)),
rtol=1e-6,
)
# and a lower bound over feasible competitors -- the pre-fix value 3.0 violates this.
for competitor in ([[0.5, 0.0], [0.0, 0.5]], [[0.25, 0.25], [0.25, 0.25]]):
cb = nx.from_numpy(np.array(competitor))
assert nx.to_numpy(res.value) <= nx.to_numpy(
_l2_primal_objective(nx, Mb, cb, 10.0)
) + 1e-9


def test_solve_l2_value_matches_dual_optimum(nx):
# ot.solve delegates to smooth_ot_dual, which reports its own objective in log["res"]["fun"].
# Strong duality (the problem is a convex QP and the returned plan is interior) makes the two
# agree, so this compares two independent code paths rather than re-deriving a formula.
M = np.array([[0.0, 1.0], [1.0, 0.0]])
a = np.array([0.5, 0.5])
Mb, ab = nx.from_numpy(M, a)

res = ot.solve(Mb, ab, ab, reg=10.0, reg_type="L2")
_, log = ot.smooth.smooth_ot_dual(
ab, ab, Mb, 10.0, reg_type="l2", log=True, stopThr=1e-12
)
np.testing.assert_allclose(nx.to_numpy(res.value), -log["res"].fun, rtol=1e-5)


def test_solve_bary_sample_l2_value_matches_plans(nx):
# res.value must be the objective of the plans that were actually returned, under either
# stopping criterion. No iteration count and no specific barycenter is asserted, so the test
# does not track the solver's trajectory (correcting the value does move the stopping point
# for some tolerances and backends).
rng = np.random.RandomState(7)
X_list = [rng.randn(n, 2) for n in (12, 15, 9)]
a_list = [ot.utils.unif(X.shape[0]) for X in X_list]
n_bary, reg = 6, 1.0
b, w = ot.utils.unif(n_bary), ot.utils.unif(len(X_list))

Xb_list = [nx.from_numpy(X) for X in X_list]
ab_list = [nx.from_numpy(a) for a in a_list]
bb, wb = nx.from_numpy(b, w)

for stopping_criterion in ("loss", "bary"):
res = ot.solve_bary_sample(
Xb_list,
n_bary,
a_list=ab_list,
w=wb,
b=bb,
metric="sqeuclidean",
reg=reg,
reg_type="L2",
max_iter_bary=200,
# chosen so both trees stop on the same iteration: the value assertion is then the
# only thing this test varies
tol_bary=1e-4,
stopping_criterion=stopping_criterion,
)
X_b = nx.to_numpy(res.X)
objective = 0.0
for k in range(len(X_list)):
M = ot.dist(X_list[k], X_b, metric="sqeuclidean")
plan = nx.to_numpy(res.list_res[k].plan)
objective += float(w[k]) * (np.sum(M * plan) + 0.5 * reg * np.sum(plan**2))
np.testing.assert_allclose(nx.to_numpy(res.value), objective, rtol=1e-5)


@pytest.mark.parametrize("reg_type", ["entropy", "KL"])
def test_solve_jax_envelope_gradients(reg_type):
jax = pytest.importorskip("jax")
Expand Down
Loading