Hackathon This was my entry to Overmind’s “Small Model, Big Maths” two-day model training hackathon in San Francisco on 8 October 2026. Hackathon page (if that link no longer works: overmindlab.ai)
Final result
Submissions closed on 2026-10-07 at 17:00 PDT. My model is a light LoRA on Qwen3.5-4B, trained on day 2. Over the last day and a half no later run beat it.
Final leaderboard
overmindlab.ai high scores, captured 17:37 PDT on Oct 7, after the deadline.
Two charts that tell it
No later model beat my submission
Shorthand answers scored worse
Hover or tap a dot to see the run; click it to open the run card.
The submitted model
The model behind my score, tested once.
More pages
Training history
The six phases, from picking the base model to the last-day re-rolls, with the platform changes on the timeline.
All 52 runs
Sortable, filterable table of every run with its recipe, validation and test results. Click a row for the full card.
Mistakes & lessons
The main mistakes and what to do next time.
Overview as plain text (for screen readers, LLMs and agents)
My entry to Overmind's "Small Model, Big Maths" two-day model training hackathon in San Francisco (submissions closed 2026-10-07 17:00 PDT): a light LoRA fine-tune of Qwen3.5-4B that scored 87.2278 on GSM8K (1,212/1,319 correct, 91.89%, minus 4.66 B parameters), rank 6 on the final leaderboard.
- Final score: 87.2278 (the board shows 87.2), rank 6 on the final leaderboard
- Score formula: strict-match % on GSM8K test (n = 1,319) minus total parameters in billions
- Model: Qwen3.5-4B with a light LoRA adapter, 4.66 B parameters, all counted
- Training: one pass over the 7,473 official GSM8K training problems, LR 1e-5, 48 steps
- Test result: 1,212/1,319 correct (91.89%); untouched base model 1,198/1,319 (score 86.17)
- How it was chosen: on 500 held-out validation problems before any test use: 462/500 vs 457/500 for the base
- Training runs: 52 in total, 17 with an official test, 40 with validation
- After the submission: 16 more official tests; the best reached 1,208, none beat 1,212
- Hackathon: Overmind "Small Model, Big Maths", two days, San Francisco; https://www.overmindlab.ai/hack
- Author: Nicolas Peters Saa (board name Nodolas), https://nodolas.github.io/NicolasPetersSaa/
Final leaderboard
| Rank | Team | Score |
|---|---|---|
| 1 | Team #1 | 89.95 |
| 2 | Team #2 | 88.91 |
| 3 | Team #3 | 88.09 |
| 4 | Team #4 | 87.84 |
| 5 | Team #5 | 87.34 |
| 6 | Nodolas (me) | 87.20 |
| 7 | Team #7 | 87.07 |
| 8 | Team #8 | 86.67 |
| 9 | Team #9 | 86.45 |
What made it better
- Picking the 4B model Big enough to reason well, small enough that the size penalty did not eat the gain. (validation: 2B 328/500, 4B 419/500, 9B 434/500 (but 9.65 B params))
- Barely training it The base model was already strong; heavy fine-tuning only made it worse. (untouched 457/500 vs fine-tuned 408–420/500)
- A very light fine-tune A small learning rate and only 48 steps nudged it slightly up. This is the submission. (test 1,212/1,319 vs the base 1,198/1,319)
- Few, large steps When retraining on the new trainer, a few large updates came closest. (x7 1,208/1,319 vs 1,058/1,319 with many small steps)
What made it worse
- Training harder A higher learning rate or more steps made the answers terse and wrong. (LR 1.25e-5: 1,074/1,319)
- Overmind's trainer change After it, the same settings no longer produced the same model. (same recipe after the change: 1,112/1,319)
- Full fine-tuning Updating all the weights hurt, even at tiny learning rates. (408/500 on validation, the worst 4B)
- Other tricks Training on the test's prompt format and switching to Qwen3-4B did not help. (prompt format best 1,197/1,319; Qwen3-4B 127/200 vs 180/200)
- Counting every parameter I counted the full 4.66 B; the catalogue's 4.0 B would have ranked higher. (87.89 at 4.0 B: rank 4 instead of 6)
All of it in one file: retrospective.md · llms.txt · data/history.json