Skip to content

docs: add VLM training demo - #3

Open
samuelwheeler wants to merge 2 commits into
mainfrom
sww/vlm-demo-docs
Open

samuelwheeler wants to merge 2 commits into
mainfrom
sww/vlm-demo-docs

Conversation

@samuelwheeler

@samuelwheeler samuelwheeler commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • add a step-by-step Aurora tutorial for VLM projector training
  • size the PixMo-Cap subset for finite 400-step multi-node runs without data repeats
  • cover PixMo-Cap preparation, WebDataset conversion, model downloads, DDP launch, and HSDP larger-batch launch
  • add training.viz_num_examples so the demo can log 10 image-caption examples per W&B prediction table

Testing

  • cloned the PR branch into a fresh directory and ran the install path with the documented Aurora module stack
  • downloaded the OLMo 1B and SigLIP2 Base models into a fresh demo-local Hugging Face cache
  • prepared a small PixMo-Cap smoke subset and converted it to WebDataset shards
  • ran real 2-node DDP and HSDP PBS smoke jobs from the fresh clone
  • ran a 400-step DDP PBS job with 10-example W&B prediction tables
  • py_compile: src/config.py src/train.py src/training/trainer_native.py src/utils/batch_viz.py

@samuelwheeler
samuelwheeler requested a review from a team as a code owner October 13, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant