Skip to content

Enable HTML builds and per-PR preview deploys #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mmcky merged 3 commits into from
Aug 6, 2026
Merged

Enable HTML builds and per-PR preview deploys #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5311036
Add build scaffold and HTML preview machinery
mmcky Aug 6, 2026
2bbe79c
Omit the peaceiris cname input instead of passing false
mmcky Aug 6, 2026
4e349f8
Use the defaults conda channel, not default
mmcky Aug 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
222 changes: 222 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,222 @@
name: Build HTML Preview for PR
on:
pull_request:
types: [opened, synchronize, reopened]
pull_request_target:
types: [closed]

# Ported from QuantEcon.manual, which is the only repo in the fleet serving
# /pr-N/ previews off GitHub Pages (the other translations deploy previews to
# Netlify, which needs a Netlify site and a NETLIFY_SITE_ID secret this repo
# does not have). Every job that writes to gh-pages shares one concurrency
# group with publish.yml and reap-previews.yml: peaceiris/actions-gh-pages
# force-pushes from a SHA read at job start, so two unserialised writers
# silently erase each other.
#
# Note the group only bounds the damage, it does not eliminate it: GitHub
# evicts an already-PENDING run when a newer one queues into the group, so a
# burst of merges can still drop a cleanup. That is what reap-previews.yml is
# for. Keep the locked section as short as possible so the window stays small.
jobs:
build-preview:
if: github.event.action != 'closed'
runs-on: ubuntu-latest
steps:
# No `ref:` — deliberately different from QuantEcon.manual, which pins
# the PR head. The seed/* translation branches were cut from an empty
# main and carry only `lectures/<name>.md`; the build scaffold lives on
# main. Checking out the default merge ref gives us PR content plus the
# scaffold, and previews the post-merge state, which is what a reviewer
# wants to see.
- name: Checkout (PR merge ref)
uses: actions/checkout@v4

- name: Setup Anaconda
uses: conda-incubator/setup-miniconda@v3
with:
auto-update-conda: true
auto-activate-base: true
miniconda-version: 'latest'
python-version: "3.13"
environment-file: environment.yml
activate-environment: quantecon

- name: Display Conda Environment Versions
shell: bash -l {0}
run: conda list

# `shell: bash -l {0}` is a custom shell spec, so GitHub does NOT inject
# `-eo pipefail`. Without the explicit set, a step ending in anything
# after the build reports that command's status and masks a failed build
# — this produced months of green-on-broken CI in the zh-cn edition.
- name: Prune TOC to translated lectures
shell: bash -l {0}
run: |
set -eo pipefail
python scripts/prune_toc.py lectures/_toc.yml

# No -n -W: a partial translation has unresolved cross-references into
# lectures that do not exist yet (as of this commit: writing_good_code,
# python_advanced_features, scipy, getting_started, oop_intro,
# need_for_speed, and the labels pyess_ex2 and oop_ex1). Those must stay
# warnings until the translation is complete. The fa and fr editions
# relax the same flag for the same reason.
- name: Build HTML
shell: bash -l {0}
run: |
set -eo pipefail
jb build lectures --path-output ./ --keep-going

- name: Upload Execution Reports
uses: actions/upload-artifact@v4
if: failure()
with:
name: execution-reports
path: _build/html/reports

- name: Upload preview artifact
uses: actions/upload-artifact@v4
with:
name: html-preview
path: _build/html/

deploy-preview:
needs: build-preview
# Fork PRs get the build as a status check but no deploy: the
# pull_request GITHUB_TOKEN is read-only for forks, so the push would fail
# anyway, and a fork build should never hold the gh-pages lock.
if: github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
concurrency:
group: gh-pages
cancel-in-progress: false
steps:
- name: Download preview artifact
uses: actions/download-artifact@v4
with:
name: html-preview
path: _build/html/

- name: Deploy Preview
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: _build/html/
destination_dir: pr-${{ github.event.number }}
# The `cname` input is deliberately OMITTED, not set to false.
# QuantEcon.manual passes `cname: false` intending "no CNAME", but
# YAML stringifies that to "false" and peaceiris writes a CNAME file
# containing the literal text `false` at the gh-pages ROOT — which
# GitHub then reads as a custom domain and the whole site 404s. It is
# masked there because publish.yml overwrites the root CNAME with the
# real domain; this edition has no custom domain, so nothing would
# ever correct it. Omitting the input writes no CNAME at all.
force_orphan: false

- name: Comment PR
uses: actions/github-script@v7
if: success()
with:
script: |
const prNumber = context.payload.pull_request.number;
const base = `https://quantecon.github.io/lecture-python-programming.ml/pr-${prNumber}`;
const commitSha = context.payload.pull_request.head.sha.substring(0, 7);

// Deep-link the lectures this PR translates, so the reviewer lands
// on the rendered Malayalam page rather than the landing page.
const files = await github.paginate(github.rest.pulls.listFiles, {
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber,
});

const pages = files
.filter(f => f.status !== 'removed')
.map(f => f.filename)
.filter(f => f.startsWith('lectures/') && f.endsWith('.md'))
// skip underscore files (_static, _admonition) — not built pages
.filter(f => !f.split('/').some(part => part.startsWith('_')))
.map(f => {
const rel = f.replace(/^lectures\//, '').replace(/\.md$/, '.html');
// intro.md is the TOC root, served at the preview root
const href = rel === 'intro.html' ? `${base}/` : `${base}/${rel}`;
return `- [${rel}](${href})`;
});

let body = `📖 **HTML build** - [view preview](${base}/) (${commitSha})`;
if (pages.length > 0) {
body += `\n\n**Translated pages in this PR:**\n${pages.join('\n')}`;
}

await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
body: body
});

cleanup-preview:
if: github.event.action == 'closed'
runs-on: ubuntu-latest
permissions:
contents: write
concurrency:
group: gh-pages
cancel-in-progress: false
steps:
# pull_request_target, not pull_request: a pull_request-triggered job on
# a closed fork PR gets a read-only token. This is safe only because the
# job checks out gh-pages and never executes PR-authored code — do not
# add a PR checkout or a build step here.
- name: Checkout gh-pages
uses: actions/checkout@v4
with:
ref: gh-pages
token: ${{ secrets.GITHUB_TOKEN }}
fetch-depth: 1

- name: Remove PR preview directory
run: |
set -eo pipefail
PR_DIR="pr-${{ github.event.number }}"
if [ ! -d "$PR_DIR" ]; then
echo "Preview directory $PR_DIR not found — nothing to clean up"
exit 0
fi
rm -rf "$PR_DIR"
git config user.name "github-actions[bot]"
git config user.email \
"41898282+github-actions[bot]@users.noreply.github.com"
git add .
if git diff --staged --quiet; then
echo "No changes to commit"
exit 0
fi
git commit -m "Remove preview for closed PR #${{ github.event.number }}"
for i in 1 2 3; do
if git push; then
echo "Successfully pushed cleanup changes"
exit 0
fi
echo "Push attempt $i failed, rebasing and retrying..."
git pull --rebase origin gh-pages
sleep 5
done
echo "Failed to push cleanup after 3 attempts"
exit 1

# Check the pushed branch, not the working tree — the local rm -rf
# always succeeds, so only origin/gh-pages proves the cleanup landed.
- name: Verify cleanup completion
run: |
set -eo pipefail
PR_DIR="pr-${{ github.event.number }}"
git fetch origin gh-pages
if git ls-tree -d --name-only origin/gh-pages "$PR_DIR" | grep -q .; then
echo "❌ Cleanup failed: $PR_DIR still exists on origin/gh-pages"
exit 1
fi
echo "✅ Cleanup verified: $PR_DIR is absent from origin/gh-pages"
85 changes: 85 additions & 0 deletions .github/workflows/publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
name: Build & Publish to GH Pages
on:
push:
branches:
- main

# Split into a lock-free build and a short locked deploy, which is where this
# diverges from QuantEcon.manual. There the concurrency group wraps the whole
# publish job, so the gh-pages lock is held across the conda solve and the
# Jupyter Book build (~2 minutes). GitHub evicts an already-PENDING run when a
# newer one queues into a group, so that long hold evicts queued cleanups: a
# merge train on 2026-08-02 dropped two runs in 68 seconds and left an orphan
# preview live on that site. Locking only the deploy keeps the window seconds
# long. reap-previews.yml covers what still slips through.
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup Anaconda
uses: conda-incubator/setup-miniconda@v3
with:
auto-update-conda: true
auto-activate-base: true
miniconda-version: 'latest'
python-version: "3.13"
environment-file: environment.yml
activate-environment: quantecon

- name: Display Conda Environment Versions
shell: bash -l {0}
run: conda list

# See ci.yml for why `set -eo pipefail` is explicit and why the TOC is
# pruned at build time rather than maintained in the repo.
- name: Prune TOC to translated lectures
shell: bash -l {0}
run: |
set -eo pipefail
python scripts/prune_toc.py lectures/_toc.yml

- name: Build HTML
shell: bash -l {0}
run: |
set -eo pipefail
jb build lectures --path-output ./ --keep-going

- name: Upload Execution Reports
uses: actions/upload-artifact@v4
if: failure()
with:
name: execution-reports
path: _build/html/reports

- name: Upload site artifact
uses: actions/upload-artifact@v4
with:
name: html-site
path: _build/html/

deploy:
needs: build
runs-on: ubuntu-latest
permissions:
contents: write
concurrency:
group: gh-pages
cancel-in-progress: false
steps:
- name: Download site artifact
uses: actions/download-artifact@v4
with:
name: html-site
path: _build/html/

- name: Deploy with Preview Preservation
uses: peaceiris/actions-gh-pages@v4
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: _build/html/
# keep_files is mandatory: without it every push to main wipes the
# live pr-N preview directories out from under open reviews.
keep_files: true
78 changes: 78 additions & 0 deletions .github/workflows/reap-previews.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
name: Reap Orphaned PR Previews
on:
schedule:
# Weekly, Monday 04:00 UTC — after any weekend merge activity has settled.
- cron: '0 4 * * 1'
workflow_dispatch:

# ci.yml deletes a preview when its PR closes, but that cleanup can be lost:
# GitHub evicts an already-PENDING run when a newer one queues into the shared
# gh-pages concurrency group, and cleanup only ever fires on the close event,
# so nothing retries it. QuantEcon.manual has no equivalent to this job and
# consequently serves preview directories for PRs merged months ago. This
# reconciles gh-pages against the set of open PRs and deletes the strays.
jobs:
reap:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: read
concurrency:
group: gh-pages
cancel-in-progress: false
steps:
- name: Checkout gh-pages
uses: actions/checkout@v4
with:
ref: gh-pages
token: ${{ secrets.GITHUB_TOKEN }}
fetch-depth: 1

- name: Remove previews for PRs that are no longer open
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -eo pipefail

open_prs=$(gh pr list --repo "$GITHUB_REPOSITORY" \
--state open --limit 500 --json number --jq '.[].number')
echo "Open PRs: ${open_prs:-none}"

reaped=0
while IFS= read -r dir; do
[ -n "$dir" ] || continue
number="${dir#./pr-}"
if grep -qx "$number" <<<"$open_prs"; then
echo "keep $dir (PR #$number is open)"
else
echo "reap $dir (PR #$number is not open)"
rm -rf "$dir"
reaped=$((reaped + 1))
fi
done < <(find . -maxdepth 1 -type d -name 'pr-*')

if [ "$reaped" -eq 0 ]; then
echo "No orphaned previews found"
exit 0
fi

git config user.name "github-actions[bot]"
git config user.email \
"41898282+github-actions[bot]@users.noreply.github.com"
git add .
if git diff --staged --quiet; then
echo "Nothing staged after reaping $reaped director(ies)"
exit 0
fi
git commit -m "Reap $reaped orphaned PR preview(s)"
for i in 1 2 3; do
if git push; then
echo "Reaped $reaped orphaned preview(s)"
exit 0
fi
echo "Push attempt $i failed, rebasing and retrying..."
git pull --rebase origin gh-pages
sleep 5
done
echo "Failed to push after 3 attempts"
exit 1
Loading
Loading