{"data":{"kind":"file","path":"README.md","version_id":"k20xyhbeqaekxdpkdny0q947","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2578,"modified_at":"2026-10-20T18:43:27.687000","content_hash":"87723f5ae556ac3f4fd8b7fbca090afb28b1a8645fe855b30244122c2bbb1cf6"},"entries":[],"content":"# gpu-train-debug\n\nDiagnose broken LLM training runs and compute the correct fix. The agent\nreads a training config, profiler numbers, and a raw log, then outputs\ndiagnosis code(s), structured intermediate calculations, and the minimal\ncorrected config fields as JSON.\n\n## v0.4.0: built to maximise RL learning signal\n\nHosted evals showed diagnosis saturating (0.947) while computed fixes lagged\n(0.42 to 0.71), so v0.4 rebalances around the failure modes:\n\n- Mix: 70% computed and interacting tasks, 20% two-fault, 10% diagnosis\n- 6 new interacting archetypes where rules compose: ZeRO sharding x memory\n  budget, checkpointing x accumulation, sequence packing x token time,\n  optimiser choice x memory footprint, accumulation x LR schedule, tensor\n  and pipeline parallelism x per-rank memory\n- Structured numeric supervision: computed tasks return intermediate values\n  (fixed memory, bytes per parameter, total tokens, steps per epoch), each\n  graded independently. Dense partial credit, still fully deterministic,\n  no free-text reasoning rewards to hack\n- Difficulty metadata on every task (tier, interaction, reasoning depth,\n  required computations) for curriculum sampling\n- Splits: train 1,200 tasks; hard 200 (no diagnosis tier); test 244\n  held out by seed. Procedural generator mints fresh variants on demand\n- Failure-driven parameter bias: large accumulation factors, big models\n  where naive memory estimates are far off, multi-epoch time estimates\n\n## Measured difficulty (hosted evals, gpt-4.1-mini)\n\n- v0.2.0 (n=30 x 3): 0.8867 overall\n- v0.3.0 (n=100 x 3): 0.8895 overall; core diagnosis 0.947,\n  computed 0.712, two-fault 0.903; hardest archetypes 0.42\n- v0.4.0 (n=100 x 3): 0.6759 overall; interacting 0.497, computed 0.643,\n  diagnosis 0.871, two-fault 0.959; 170 of 300 rollouts in the 0.2 to 0.7 band\n\n## Grader\n\nDeterministic, computed gold, no LLM judge. Tasks with intermediates:\nformat 0.10, diagnosis 0.30, intermediates 0.25, fix 0.25, minimality 0.10.\nOther tasks: format 0.10, diagnosis 0.35, fix 0.45, minimality 0.10.\nDumping every plausible fix is penalised. Cheat audit: keyword heuristic\nscores 0.147 on the v0.4 train mix and 0.112 on the hard split, gold 1.000,\nrandom 0.126. Zero runtime dependencies standalone; grades in milliseconds.\nAll data synthetic. MIT license.\n\n## Use\n\n    prime env install saloni-garg/gpu-train-debug\n\n    python3 -c \"\n    import gpu_train_debug as m, json\n    env = m.load_environment()\n    t = env.tasks[0]\n    print(len(env.tasks), env.evaluate([t['prompt']], [json.dumps(t['gold'])]))\n    \"  # 1200 [1.0]\n","encoding":"utf-8","truncated":false,"total_bytes":2578},"status":null}