User-defined constrained parameterizations by bob-carpenter · Pull Request #967 · stan-dev/docs

bob-carpenter · 2026-07-16T21:23:22Z

This is a whole new chapter on user-defined constrained parameters with some challenging examples. I've tested that all the code actually works.

Submission Checklist

Builds locally
[n/a] New functions marked with <<{ since VERSION }>>
Declare copyright holder and open-source license: see below

Summary

Copyright and Licensing

Please list the copyright holder for the work you are submitting (this will be you or your assignee, such as a university or company):

Simons Foundation

By submitting this pull request, the copyright holder is agreeing to license the submitted work under the following licenses:

Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
Documentation: CC BY-ND 4.0 (https://creativecommons.org/licenses/by-nd/4.0/)

…stributions

bob-carpenter · 2026-07-17T15:28:10Z

I renamed and tagged "WIP" because I want to go over and add a little more text background and also citations to the original threads that introduced these ideas.

bob-carpenter · 2026-07-21T19:29:47Z

@WardBrian. This is now code complete, spell checked, and read over by me. Sorry, but I sort of got carried away, and as a result, it's a lot.

WardBrian

Fun read. A bunch of comments below

WardBrian · 2026-07-23T16:26:29Z

It might make sense to reassess https://mc-stan.org/docs/stan-users-guide/reparameterization.html#changes-of-variables and see if that should just be blended into this new chapter (or at least cross-referenced)

WardBrian · 2026-07-23T16:28:37Z

+  real log_sigma;
+}
+transformed parameters {
+  real<lower=0> sigma = exp(log_sigma);


It might be worth explicitly (re)-stating that the lower= syntax only performs checks in blocks other than parameters

WardBrian · 2026-07-23T16:32:31Z

+    if (cols(x) != N + binom(N, 2)) {
+      reject("cols(x) = ", cols(x),
+             "; but N + binom(N, 2) = ", N + binom(N, 2));
+    }


binom is not a function; it's choose (which is correctly used in the parameters block below)

WardBrian · 2026-07-23T16:33:12Z

+functions {
+  matrix toeplitz_matrix(vector x, int M, int N) {
+    if (num_elements(x) != M + N - 1) {
+      reject("x must have M + N - 1 elements",


I think these rejects should be fatal_errors, since the sizes will never change there is no recovery possible

WardBrian · 2026-07-23T16:34:06Z

+```stan
+functions { 
+  matrix symmetric_matrix(vector x, int N) {
+    matrix[N, N] y = lower_triangular_matrix(x, N);


Suggested change

matrix[N, N] y = lower_triangular_matrix(x, N);

matrix[N, N] y = lower_triangular_matrix(x, N); // defined as above

Or any other indication that this is not a built-in and therefore this example is not fully self-contained

WardBrian · 2026-07-23T18:11:46Z

+```  
+
+The transform itself, which takes a vector $v$ of size $\binom{N}{2}$,
+creates a skew-symmetric matrix (the `skew_symmetric` function is defined


Suggested change

creates a skew-symmetric matrix (the `skew_symmetric` function is defined

creates a skew-symmetric matrix (the `skew_symmetric_matrix` function is defined

WardBrian · 2026-07-23T18:11:53Z

+* @return The `N` x `N` rotation matrix for which `v` is the basis
+*/
+matrix special_orthogonal_jacobian(vector v, int N) {
+  matrix[N, N] S = skew_symmetric(v, N);


Suggested change

matrix[N, N] S = skew_symmetric(v, N);

matrix[N, N] S = skew_symmetric_matrix(v, N);

WardBrian · 2026-07-23T18:16:40Z

+  {
+    tuple(matrix[D, D], matrix[D, D]) phi_Gamma
+      = stationary_var1_jacobian(raw, D);
+    phi = phi_Gamma.1;
+    Gamma = phi_Gamma.2;
+  }


We have tuple unpacking

Suggested change

{

tuple(matrix[D, D], matrix[D, D]) phi_Gamma

= stationary_var1_jacobian(raw, D);

phi = phi_Gamma.1;

Gamma = phi_Gamma.2;

}

(phi, Gamma) = stationary_var1_jacobian(raw, D);

WardBrian · 2026-07-23T18:17:21Z

+covariance `Gamma`.  These need to be pulled out of the tuple.  This
+is one function returning a tuple because there is shared work.  The
+extra braces mean that `phi_Gamma` is no longer a top-level
+`transformed parameters` variable and thus will not be saved in the
+output (it would be redundant).  The model could have been written
+directly in terms of `phi_Gamma.1` and `phi_Gamma.2`, but it's easier
+to follow as written.  Note that `phi` is used in its transposed form


Which makes this explanation redundant

WardBrian · 2026-07-23T18:21:17Z

+
+## Matrices matching known margins
+
+Suppose $X$ is an $M \times N$ matrix $X$, the marginals of which are


Suggested change

Suppose $X$ is an $M \times N$ matrix $X$, the marginals of which are

Suppose $X$ is an $M \times N$ matrix, the marginals of which are

SteveBronder

Just a few small comments on the intro

SteveBronder · 2026-07-30T17:32:09Z

+$$
+p(\phi)
+= p(\theta)
+\cdot \left| \frac{\textrm{d}}{{\textrm d}\phi} \theta
+\right|,
+$$


Should you define p before this equation?

SteveBronder · 2026-07-30T17:32:19Z

+the natural logarithm.
+
+The underlying sampler works with the unconstrained variable $\phi$
+that results from transforming the constrained parameter..


Suggested change

that results from transforming the constrained parameter..

that results from transforming the constrained parameter.

SteveBronder · 2026-07-30T17:35:21Z

+prior is uniform, but improper (i.e., the integral over its support
+diverges).
+
+The way this is done is by applying a change-of-variables correction.


I would avoid the use of this.

Suggested change

The way this is done is by applying a change-of-variables correction.

Enforcing a uniform implicit prior on constrained parameters requires applying a change-of-variables correction.

SteveBronder · 2026-07-30T17:41:22Z

+existing constrained types and transforms, as that will let Stan
+automatically work out the change-of-variables.  The more general
+approach is to code a user-defined transform function and either
+adjust for then change-of-variables or not, depending on the intended


Suggested change

adjust for then change-of-variables or not, depending on the intended

adjust for the change-of-variables or not, depending on the intended

SteveBronder · 2026-07-30T17:45:34Z

+
+The underlying sampler works with the unconstrained variable $\phi$
+that results from transforming the constrained parameter..
+In order for the distribution of $\theta$ to be uniform over $(0,


Suggested change

In order for the distribution of $\theta$ to be uniform over $(0,

In order for the distribution $p(\cdot)$ of $\theta$ to be uniform over $(0,

SteveBronder · 2026-07-30T17:50:50Z

+\begin{align}
+p_\Phi(\phi) &= p_\Theta(\exp(\phi)) \cdot \left| \frac{\textrm{d}}{\textrm{d} \phi} \exp(\phi) \right|
+\\[4pt]
+&= p_\Theta(\exp(\phi)) \cdot \exp(\phi).


Suggested change

&= p_\Theta(\exp(\phi)) \cdot \exp(\phi).

&= p_\Theta(\exp(\phi)) \cdot \exp(\phi)

SteveBronder · 2026-07-30T17:58:51Z

+= p(\theta)
+\cdot \left| \frac{\textrm{d}}{{\textrm d}\phi} \theta
+\right|,
+$$


I don't love the substitution of $\phi$ for $f\left(\theta\right)$ imo I think this reads nicer as

$$ p\left(f\left(\theta\right)\right) = p(\theta) \cdot \left| \frac{\textrm{d f}}{{\textrm d}\theta} \theta \right|, $$

Though applying that through the line right after this I'm not totally understanding the equation. And what is $p_\Phi$?

$$ \begin{align} p_\Phi(\log\left(\theta\right)) &= p_\Theta\left(\exp\left(\log\left(\theta\right)\right)\right) \cdot \left| \frac{\textrm{d}}{\textrm{d} \log\left(\theta\right)} \exp\left(\log\left(\theta\right)\right) \right| \\ &= p_\Theta(\exp\left(\log\left(\theta\right)\right)) \cdot \exp\left(\log\left(\theta\right)\right) \end{align} $$

Wouldn't the last line reduce $\exp(\log(\cdot))$ so that the final line just becomes

$$ p_\Theta(\theta) \cdot \theta $$

Maybe I'm just not understanding something?

SteveBronder · 2026-07-30T19:55:05Z

+be constant and thus should be included.
+
+
+## Specialized matrices
+
+There are dozens if not hundreds of specialized matrix forms in use,
+some of the most popular of which will be described in this section.
+


Kind of a 100 G-force of a transition from change of variables to bespoke matrices

Bob Carpenter and others added 13 commits July 6, 2026 02:18

fun shapes

ac58d07

matrix marginal matching

ff097a0

Merge branch 'master' of github.com:stan-dev/docs into fun-distributions

bc56def

more sum to zero

65c5f8c

Merge branch 'master' of https://github.com/stan-dev/docs into fun-di…

21d4590

…stributions

formatting for unusual distributions

497dc11

orthonormal matrix construction

eca1bc8

orthonormal/orthogonal params

72871ba

more structured matrices

17ff2c4

Merge branch 'master' of github.com:stan-dev/docs into fun-distributions

ed54070

working corr w. zeros

cdd0f0d

Merge branch 'master' of https://github.com/stan-dev/docs into fun-di…

79791a0

…stributions

zero correlation Cholesky; formatting

3e1ab06

bob-carpenter changed the title ~~Fun distributions~~ User-defined constrained parameterizations Jul 17, 2026

bob-carpenter changed the title ~~User-defined constrained parameterizations~~ [WIP] User-defined constrained parameterizations Jul 17, 2026

Bob Carpenter and others added 5 commits July 18, 2026 16:23

stereographic corr w. zeros

eac35c7

bisoftplus as a constraint

500a6b4

typos fixed; release candidate

8799e47

bib merge conflict resolve

eafab77

residual typos

531ba3b

bob-carpenter changed the title ~~[WIP] User-defined constrained parameterizations~~ User-defined constrained parameterizations Jul 21, 2026

bob-carpenter added 2 commits July 22, 2026 15:08

typos, latex escapes

00164f5

fix overrun lines

61e66c3

WardBrian reviewed Jul 23, 2026

View reviewed changes

SteveBronder reviewed Jul 30, 2026

View reviewed changes

	matrix[N, N] y = lower_triangular_matrix(x, N);
	matrix[N, N] y = lower_triangular_matrix(x, N); // defined as above

	creates a skew-symmetric matrix (the `skew_symmetric` function is defined
	creates a skew-symmetric matrix (the `skew_symmetric_matrix` function is defined

	matrix[N, N] S = skew_symmetric(v, N);
	matrix[N, N] S = skew_symmetric_matrix(v, N);


		## Matrices matching known margins

		Suppose $X$ is an $M \times N$ matrix $X$, the marginals of which are

	Suppose $X$ is an $M \times N$ matrix $X$, the marginals of which are
	Suppose $X$ is an $M \times N$ matrix, the marginals of which are

	that results from transforming the constrained parameter..
	that results from transforming the constrained parameter.

	The way this is done is by applying a change-of-variables correction.
	Enforcing a uniform implicit prior on constrained parameters requires applying a change-of-variables correction.

	adjust for then change-of-variables or not, depending on the intended
	adjust for the change-of-variables or not, depending on the intended

	In order for the distribution of $\theta$ to be uniform over $(0,
	In order for the distribution $p(\cdot)$ of $\theta$ to be uniform over $(0,

	&= p_\Theta(\exp(\phi)) \cdot \exp(\phi).
	&= p_\Theta(\exp(\phi)) \cdot \exp(\phi)

Uh oh!

Uh oh!

Conversation

bob-carpenter commented Jul 16, 2026

Submission Checklist

Summary

Copyright and Licensing

Uh oh!

bob-carpenter commented Jul 17, 2026

Uh oh!

bob-carpenter commented Jul 21, 2026

Uh oh!

WardBrian left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SteveBronder left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SteveBronder Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SteveBronder Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

SteveBronder Jul 30, 2026 •

edited

Loading

SteveBronder Jul 30, 2026 •

edited

Loading