Ar/strict subproblems

.github/dependabot.yml

@@ -11,3 +11,9 @@ updates:
     directory: "/"
     schedule:
       interval: "weekly"
+    ignore:
+      # The v5→v7 bump silently broke coverage uploads ("Missing Head Commit"
+      # on PRs). Keep codecov-action pinned until a deliberate, verified
+      # migration — see the comment in .github/workflows/CI.yml.
+      - dependency-name: "codecov/codecov-action"
+        update-types: ["version-update:semver-major"]

.github/workflows/CI.yml

@@ -19,12 +19,13 @@ jobs:
     permissions: # needed to allow julia-actions/cache to proactively delete old caches that it has created
       actions: write
       contents: read
+      id-token: write # OIDC token for tokenless Codecov uploads (see codecov step)
     strategy:
       fail-fast: false
       matrix:
         version:
           - '1.10'
-          # - 'nightly'
+          - '1.12'
         os:
           - ubuntu-latest
         arch:
@@ -39,8 +40,22 @@ jobs:
       - uses: julia-actions/julia-buildpkg@v1
       - uses: julia-actions/julia-runtest@v1
       - uses: julia-actions/julia-processcoverage@v1
-      - uses: codecov/codecov-action@v7
+      # Pinned to v5: the dependabot bump to v7 (2026-06-25) silently broke
+      # uploads — Codecov has no commit newer than 2026-06-15, which is what
+      # produces "Missing Head Commit" on PRs. v5 is the last version verified
+      # to upload from this workflow. Before re-bumping, migrate deliberately
+      # (e.g. OIDC: `use_oidc: true` + `id-token: write` permission) and
+      # confirm a commit appears on Codecov.
+      # Authentication uses OIDC (`use_oidc` + the job's `id-token: write`
+      # permission) because the CODECOV_TOKEN secret is not set in this repo
+      # ("Token length: 0" in CI) and tokenless uploads are rejected on
+      # protected branches. OIDC requires the Codecov GitHub App to be
+      # installed for the organization; if uploads fail with an OIDC error,
+      # either install the app or set the CODECOV_TOKEN secret and replace
+      # `use_oidc` with `token: ${{ secrets.CODECOV_TOKEN }}`.
+      - uses: codecov/codecov-action@v5
         with:
           files: lcov.info
-          token: ${{ secrets.CODECOV_TOKEN }}
-          fail_ci_if_error: false
+          use_oidc: true
+          # Fail loudly: a silent upload failure hid this breakage for weeks.
+          fail_ci_if_error: true

.gitignore

@@ -55,3 +55,12 @@ plan/
 *_cuts.json
 settings.json
 *.sh
+*-backup
+*.cuts.json.stale_stronggrid
+*strong.json
+.claude/settings.local.json
+# W&B run directories and local backups of environment files
+wandb/
+*.bak_*
+# Archived SDDP cuts kept beside the canonical file (published separately)
+*.cuts.*.json

Project.toml

@@ -34,7 +34,7 @@ JLD2 = "0.6.3"
 JuMP = "1.29.4"
 MadNLP = "0.8, 0.9, 0.10"
 MadNLPGPU = "0.7, 0.8, 0.9, 0.10"
-MathOptInterface = "1.48.0"
+MathOptInterface = "1.48.0 - 1.53"
 ParametricOptInterface = "0.14.1, 0.15, 0.16"
 Statistics = "1.10, 1.11"
 Zygote = "0.6.77, 0.7"

README.md

@@ -4,13 +4,13 @@
 
 # DecisionRules.jl
 
-DecisionRules.jl is a Julia package for **training parametric decision rules through multi-stage optimization**, following the **Two-Stage General / Deep Decision Rules** (TS-GDR / TS-DDR) framework. The main use-case is multi-stage stochastic control where the feasible (closed-loop) action at each stage is obtained by solving an optimization problem (e.g., OPF, MPC), and we want to train a policy end-to-end that maps observed states and uncertainties to target trajectories.
+DecisionRules.jl is a Julia package for training parametric decision rules for multistage stochastic programs in which every stage is an optimization model, such as an optimal power flow or a model-predictive control problem. It implements two-stage generalized decision rules (TS-GDR) and their deep instantiation, two-stage deep decision rules (TS-DDR), introduced in [arXiv:2405.14973](https://arxiv.org/abs/2405.14973). A policy, from an affine rule to a deep recurrent network, maps the observed uncertainty to a target for the state at the end of every stage; an optimization problem enforces the targets against the stage models; and the multipliers of the target constraints provide the gradient with which the policy is trained. The stage models are written in JuMP, and the companion package [DecisionRulesExa.jl](https://github.com/LearningToOptimize/DecisionRulesExa.jl) trains the same policies on GPUs.
 
 ![TS-GDR / deterministic-equivalent workflow](diagram_tsddr.png)
 
-## Motivation and workflow
+## Workflow
 
-In TS-GDR, the policy does not directly output a control action. Instead, it outputs **targets** (typically a target state trajectory) that are enforced inside an optimization model through *target constraints* with slack. For a sampled uncertainty trajectory $($w_{1:T})$, the workflow is:
+In TS-GDR the policy does not output a control action. It outputs targets, typically for the state at the end of every stage, and an optimization model enforces them through target constraints: with a penalized slack in the general formulation, or as hard equalities in the strict formulation when the policy can be restricted to targets reachable in one stage. For a sampled uncertainty trajectory $w_{1:T}$, a training step is:
 
 1. sample $w_{1:T}$
 2. predict targets $\hat x_{1:T} = \pi(\cdot;\theta)$
@@ -54,22 +54,25 @@ using DecisionRules, JuMP, DiffOpt, Flux
 using SCS
 
 # 1) Build per-stage subproblems (DiffOpt-enabled) and collect:
-#    subproblems, state_params_in, state_params_out, uncertainty_sampler, uncertainties_structure
+#    subproblems, state_params_in, state_params_out, uncertainty_samples
 
 # 2) Build the deterministic equivalent over the full horizon
 det = DiffOpt.diff_model(() -> DiffOpt.diff_optimizer(SCS.Optimizer))
 
-det, uncertainties_structure_det = DecisionRules.deterministic_equivalent!(
+det, uncertainty_samples_det = DecisionRules.deterministic_equivalent!(
     det,
     subproblems,
     state_params_in,
     state_params_out,
     Float64.(initial_state),
-    uncertainties_structure,
+    uncertainty_samples,
 )
 
+# deterministic_equivalent! remaps state_params_in/state_params_out in place.
+# Copy those arrays first if you also need the original stage-wise refs later.
+
 # 3) Train a TS-DDR policy end-to-end
-num_uncertainties = length(uncertainty_sampler()[1])  # number of uncertainty components per stage
+num_uncertainties = length(uncertainty_samples[1])  # number of uncertainty components per stage
 policy = Chain(
     Dense(DecisionRules.policy_input_dim(num_uncertainties, length(initial_state)), 64, relu),
     Dense(64, length(initial_state)),
@@ -79,9 +82,9 @@ DecisionRules.train_multistage(
     policy,
     initial_state,
     det,
-    state_in_det,
-    state_out_det,
-    uncertainty_sampler;
+    state_params_in,
+    state_params_out,
+    uncertainty_samples_det;
     num_batches=100,
     num_train_per_batch=32,
     optimizer=Flux.Adam(1e-3),
@@ -97,7 +100,7 @@ Single shooting solves one optimization per stage and rolls forward using the re
 ```julia
 using DecisionRules, Flux
 
-num_uncertainties = length(uncertainty_sampler()[1])
+num_uncertainties = length(uncertainty_samples[1])
 policy = Chain(
     Dense(DecisionRules.policy_input_dim(num_uncertainties, length(initial_state)), 64, relu),
     Dense(64, length(initial_state)),
@@ -109,7 +112,7 @@ DecisionRules.train_multistage(
     subproblems,
     state_params_in,
     state_params_out,
-    uncertainty_sampler;
+    uncertainty_samples;
     num_batches=100,
     num_train_per_batch=32,
     optimizer=Flux.Adam(1e-3),
@@ -126,7 +129,7 @@ Multiple shooting partitions the horizon into windows of length `window_size`. E
 using DecisionRules, Flux, DiffOpt
 using SCS
 
-num_uncertainties = length(uncertainty_sampler()[1])
+num_uncertainties = length(uncertainty_samples[1])
 policy = Chain(
     Dense(DecisionRules.policy_input_dim(num_uncertainties, length(initial_state)), 64, relu),
     Dense(64, length(initial_state)),
@@ -149,10 +152,7 @@ DecisionRules.train_multiple_shooting(
     policy,
     initial_state,
     windows,
-    state_params_in,
-    state_params_out,
-    uncertainty_sampler;
-    window_size=24,  # e.g., 6, 24, ...
+    uncertainty_samples;
     num_batches=100,
     num_train_per_batch=32,
     optimizer=Flux.Adam(1e-3),
@@ -168,7 +168,8 @@ The training loops record metrics through a per-sample `SampleLog` cache and a p
 ```julia
 using DecisionRules, Random
 
-# Materialize a FIXED held-out evaluation set once, before training
+# Materialize a FIXED held-out evaluation set once, before training.
+# Use stage-wise subproblems and parameter refs, not DE-remapped refs.
 Random.seed!(1234)
 eval_scenarios = [DecisionRules.sample(uncertainty_samples) for _ in 1:8]
 
@@ -178,7 +179,7 @@ rollout_eval = RolloutEvaluation(
     policy_state=:realized,
 )
 
-train_multistage(policy, initial_state, det, state_in_det, state_out_det, uncertainty_sampler;
+train_multistage(policy, initial_state, det, state_params_in, state_params_out, uncertainty_samples_det;
     num_batches=100,
     record=(sample_log, iter, model) -> begin
         rollout_eval(iter, model)
@@ -202,6 +203,57 @@ Each evaluation reports (a) the rollout objective **excluding** the target-slack
 
 Per-sample debugging hooks can be attached with `SampleLog(on_sample=(s, models, log) -> ...)`; the training loop calls the hook after each sample's solve with the live JuMP model(s). The previous `record_loss=(iter, model, loss, tag) -> ...` keyword keeps working as a deprecated adapter.
 
+## Strict mode and reachable policies
+
+The standard TS-DDR target constraint uses slack:
+
+```math
+x_t + \delta_t = \hat{x}_t,
+\qquad
+\text{objective} += C_\delta \|\delta_t\|.
+```
+
+Slack makes training robust to unreachable targets, but it also makes the dual
+signal depend on the target-penalty calibration. Strict mode removes the slack:
+
+```math
+x_t = \hat{x}_t.
+```
+
+The resulting dual is the clean shadow price of imposing the target. The price
+of that cleaner signal is feasibility: every policy target must be reachable
+from the state used to condition the policy.
+
+This is automatic in the hydro strict subproblem path because each stage is
+solved sequentially and the policy receives the realized previous reservoir
+state. It is also possible in regular deterministic equivalents when the target
+trajectory is rolled out from the true initial state using a reachable policy:
+
+```math
+\hat{x}_0 = x_0,\qquad
+\hat{x}_t = \pi_\theta(w_t, \hat{x}_{t-1}),\qquad
+\hat{x}_t \in R(\hat{x}_{t-1}, w_t).
+```
+
+By induction, all targets are feasible, and the strict equalities force the
+realized trajectory to match that reachable path. See
+[`examples/HydroPowerModels`](examples/HydroPowerModels) and the
+DecisionRulesExa.jl companion for the GPU strict regular-DE implementation.
+
+The reachable map ``R(\hat x_{t-1}, w_t)`` depends on the state, and that
+dependence **must be differentiated**. Treating the interval endpoints as
+constants still trains and still lowers the loss while descending a materially
+different direction — on the hydro case, a gradient carrying 6% of the true
+magnitude and pointing 48 degrees away from it. Verify the complete actor
+gradient against finite differences before trusting any hyperparameter
+conclusion drawn on top of it.
+
+The policy helpers separate two architectural choices:
+
+- recurrent `layers` / `DR_ENCODER_LAYERS` process uncertainty history only;
+- `combiner_layers` / `DR_HEAD_LAYERS` add a nonlinear feed-forward
+  state-to-target head without recurrence over the state input.
+
 ## GPU acceleration with DecisionRulesExa.jl
 
 For large-scale problems where the inner NLP solve is the bottleneck (e.g., AC-OPF with hundreds of buses), [DecisionRulesExa.jl](https://github.com/LearningToOptimize/DecisionRulesExa.jl) provides a GPU-accelerated backend that replaces JuMP with [ExaModels.jl](https://github.com/exanauts/ExaModels.jl) and solves with [MadNLP.jl](https://github.com/MadNLP/MadNLP.jl) + CUDSS on GPU.
@@ -216,12 +268,37 @@ See the [GPU Acceleration](https://LearningToOptimize.github.io/DecisionRules.jl
 
 ## Examples and tests
 
-Examples live in `examples/`. Run tests with:
+The case studies of the paper and smaller demonstrations live in `examples/`.
+To reproduce a case study, start from [`examples/README.md`](examples/README.md),
+which says where each case lives, which engine trains which method, and what to
+expect from a run. The [documentation](https://LearningToOptimize.github.io/DecisionRules.jl/dev/)
+explains the method, the case studies and how the code is organized. Run the
+tests with:
 
 ```julia
 julia --project -e 'using Pkg; Pkg.test()'
 ```
 
+## Repository Map
+
+| Path | Purpose |
+|---|---|
+| `src/DecisionRules.jl` | Module entrypoint and exports |
+| `src/dense_multilayer_nn.jl` | MLP helpers, state-conditioned recurrent policies, nonlinear target heads |
+| `src/simulate_multistage.jl` | Stage-wise and deterministic-equivalent simulation logic |
+| `src/multiple_shooting.jl` | Windowed multiple-shooting setup, simulation, and training |
+| `src/utils.jl` | Target-parameter utilities, deficit construction, rollout evaluation |
+| `src/integer_strategies.jl` | Strategies for extracting gradients from integer/mixed-integer models |
+| `src/score_function.jl` | Score-function gradient correction for nonsmooth/integer problems |
+| `src/parameter_duals.jl` | Dual/sensitivity helpers for parameterized JuMP models |
+| `docs/src/` | Documenter.jl manual pages |
+| `examples/HydroPowerModels/` | Bolivian hydrothermal case study: case, JuMP engine, SDDP and RL baselines |
+| `examples/BatteryStorageOPF/` | Battery-storage AC-OPF case study: case construction, JuMP engine, SDDP |
+| `examples/inventory_control/` | Inventory-control example and dynamic-programming/SDDP comparisons |
+| `examples/rocket_control/` | Rocket MPC/control example |
+| `examples/Experimental/` | Research prototypes and robotics/control explorations |
+| `test/runtests.jl` | Package test suite |
+
 ## Citation
 
 If you use this package in academic work, please cite:

docs/Project.toml

@@ -1,4 +1,6 @@
 [deps]
+CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"
+ChainRulesCore = "d360d2e6-b24c-11e9-a2a3-2a2ae2dbcce4"
 DecisionRules = "47937410-f832-486f-8300-12c95b225dfc"
 DiffOpt = "930fe3bc-9c6b-11ea-2d94-6184641e85e7"
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
@@ -8,9 +10,13 @@ HiGHS = "87dc4568-4c63-4d18-b0c0-bb2238e4078b"
 Ipopt = "b6b21f68-93f8-5de0-b562-5493be1d77c9"
 JuMP = "4076af6c-e467-56ae-b986-b466b2749572"
 Literate = "98b081ad-f1c9-55d3-8b20-4c87d4299306"
+JSON = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"
 MathOptInterface = "b8f27783-ece8-5eb3-8dc8-9495eed66fee"
 Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
+SHA = "ea8e919c-243c-51af-8825-aaa63cd721ce"
+StableRNGs = "860ef19b-820b-49d6-a774-d7a799459cd3"
 Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
+Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
 
 [compat]
 Documenter = "1"

docs/make.jl

@@ -2,15 +2,16 @@ using Documenter
 using Literate
 using DecisionRules
 
-# Convert Literate.jl sources to markdown
-examples_src = joinpath(@__DIR__, "src", "examples")
-examples_out = joinpath(@__DIR__, "src", "examples")
-for file in readdir(examples_src)
-    endswith(file, ".jl") || continue
-    Literate.markdown(
-        joinpath(examples_src, file), examples_out;
-        documenter=true, credit=false,
-    )
+# Convert Literate.jl sources to markdown, in place, wherever they live: a case
+# study that owns a runnable walkthrough keeps it beside its prose rather than in
+# a separate examples pile.
+for dir in (joinpath(@__DIR__, "src", "examples"),
+            joinpath(@__DIR__, "src", "casestudies", "hydro"))
+    isdir(dir) || continue
+    for file in readdir(dir)
+        endswith(file, ".jl") || continue
+        Literate.markdown(joinpath(dir, file), dir; documenter=true, credit=false)
+    end
 end
 
 makedocs(;
@@ -20,19 +21,36 @@ makedocs(;
     format=Documenter.HTML(;
         prettyurls=get(ENV, "CI", nothing) == "true",
         canonical="https://LearningToOptimize.github.io/DecisionRules.jl",
+        size_threshold=300 * 1024,
     ),
     pages=[
         "Home" => "index.md",
-        "Algorithm" => "algorithm.md",
-        "Gradient Fallback" => "gradient_fallback.md",
-        "Uncertainty Sampling" => "sampling.md",
-        "GPU Acceleration" => "gpu_acceleration.md",
-        "Examples" => [
-            "Hydropower Scheduling" => "examples/hydro.md",
-            "Rocket Control" => "examples/rocket.md",
-            "Stochastic Lot-Sizing (Integer Variables)" => "examples/inventory.md",
+        "Part I — Theory" => [
+            "Multistage stochastic optimization" => "theory/multistage.md",
+            "The TS-DDR framework" => "algorithm.md",
+            "Stochastic dual dynamic programming" => "theory/sddp.md",
+            "Extensions: mixed gradients, critics, risk" => "theory/extensions.md",
+        ],
+        "Part II — Package Guide" => [
+            "Getting started" => "guide/getting_started.md",
+            "Uncertainty sampling" => "sampling.md",
+            "Gradient fallback" => "gradient_fallback.md",
+            "GPU acceleration" => "gpu_acceleration.md",
+            "Reading the code" => "guide/reading_the_code.md",
+            "API reference" => "api.md",
+        ],
+        "Part III — Case Studies" => [
+            "Battery-storage AC-OPF" => "casestudies/battery_storage_opf.md",
+            "Long-term hydrothermal planning" => [
+                "Overview" => "casestudies/hydro/index.md",
+                "The problem" => "casestudies/hydro/problem.md",
+                "Valuing water: two approaches" => "casestudies/hydro/method.md",
+                "Reproducing the study" => "casestudies/hydro/reproducing.md",
+                "Walkthrough" => "casestudies/hydro/walkthrough.md",
+            ],
+            "Rocket control" => "examples/rocket.md",
+            "Stochastic lot-sizing (integer variables)" => "examples/inventory.md",
         ],
-        "API Reference" => "api.md",
     ],
 )