Skip to content

docs: add VLM training demo - #3

Open
samuelwheeler wants to merge 5 commits into
mainfrom
sww/vlm-demo-docs
Open

docs: add VLM training demo#3
samuelwheeler wants to merge 5 commits into
mainfrom
sww/vlm-demo-docs

Conversation

@samuelwheeler

@samuelwheeler samuelwheeler commented Oct 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

  • add a step-by-step Aurora tutorial for VLM projector training
  • size the PixMo-Cap subset for finite 400-step multi-node runs without data repeats
  • cover PixMo-Cap preparation, WebDataset conversion, model downloads, DDP launch, and HSDP larger-batch launch
  • add training.viz_num_examples so the demo can log 10 image-caption examples per W&B prediction table

Testing

  • cloned the PR branch into a fresh directory and ran the install path with the documented Aurora module stack
  • downloaded the OLMo 1B and SigLIP2 Base models into a fresh demo-local Hugging Face cache
  • prepared a small PixMo-Cap smoke subset and converted it to WebDataset shards
  • ran real 2-node DDP and HSDP PBS smoke jobs from the fresh clone
  • ran a 400-step DDP PBS job with 10-example W&B prediction tables
  • py_compile: src/config.py src/train.py src/training/trainer_native.py src/utils/batch_viz.py

@samuelwheeler
samuelwheeler requested a review from a team as a code owner October 13, 2026 15:41
@Sande33p Sande33p mentioned this pull request Oct 20, 2026
Open
20 of 31 tasks
Sande33p and others added 3 commits October 22, 2026 04:36
@Sande33p @claude
Add docs/tutorials index and move materials tutorial into it
d3ba92f 
Group end-to-end tutorials by application area: Markdown pages in
docs/tutorials/<area>/, notebooks next to their scripts in
applications/<area>/. Move the materials tutorial to
docs/tutorials/material_science/ and fix its relative link, list it
and the BioReason notebook in the new index and in docs/index.md, and
add docs/tutorials/ to README, CONTRIBUTING and CHANGELOG.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Sande33p @claude
Merge branch 'docs/tutorials-by-application' into sww/vlm-demo-docs
a0dc058 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Sande33p @claude
Move VLM demo into docs/tutorials/vision_language/
c42998a 
List it in the tutorials index and docs/index.md. Fix two claims in the
Monitor section: launch_aurora_web.py names the log directory after
--design, so logs land in logs/PRISM-IMAGE-ONLY-2N/, and both launch
scripts set training.viz_every_n_steps=100, not 20.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Sande33p Sande33p mentioned this pull request Oct 22, 2026
Merged
11 of 20 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@samuelwheeler @Sande33p