SN97 v30.2 Overhaul Live; Teacher Slowdown Fixed
Share
Distil deployed a major scoring update addressing Goodhart gaming on ranking axes. The new composite.final ranking key (70% worst-3-mean + 30% weighted) replaces single-axis minimization, adds a super_teacher axis rewarding models that beat the baseline, and collapses 13 benches into 4 skill groups. Separately, a vLLM misconfiguration causing 100+ minute eval rounds was fixed; validator now targets 45-minute cycles.
- •Ranking: composite.final replaces worst() floor; super_teacher axis (10% weight) explicitly rewards exceeding Qwen3.6-35B on 16 verifiable benches.
- •Goodhart fix: negative correlation between validator math/reasoning scores and held-out GSM8K (-0.665 to -0.746 r) prompted procedural template rebalance.
- •vLLM bugs: DISTIL_VLLM_CONCURRENCY env var ignored, gpu-memory-utilization too low, no max-num-seqs cap caused OOM fallback to slow HuggingFace.
- •Next: Kimi K2.6 A/B teacher experiment, multi-GPU pod migration (8×H100), batched student forward passes for 2-3× speedup.
Distilled from 21 team messages in the official Bittensor Discord. Generated by Claude Haiku 4.5.
View original messages
- Discord message 1498924127812063313
- Discord message 1498926290206720122
- Discord message 1498944194096664596
- Discord message 1498944200203436103
- Discord message 1498944229836193884
- Discord message 1498950550958506035
- Discord message 1498951292947796149
- Discord message 1498997113542021141
- Discord message 1498997774694219869
- Discord message 1499022287939178667
- Discord message 1499028460478267413
- Discord message 1499054922816290897
- Discord message 1499083619031978126
- Discord message 1499083691660677301