Distil v32 Axis Weights Confirmed, H100 Performance Bottleneck Identified
Share
Distil published composite scoring weights via `eval_policy.json`: on-policy RKL leads at 0.39, followed by long-gen coherence (0.25) and judge probes (0.20 each), with formula `0.85 × worst_5_mean + 0.15 × weighted`. Live validator runs 24 axes including `multi_doc_synthesis` via env override. Current evaluation round progresses slowly on H100 PCIe without vLLM—reasoning axes like logic_grid take 90s per item due to uncapped 16K tokens, versus 3–5× faster on prior H200 NVL setup. v32.2 will cap problematic axes to 6K tokens.
- •Composite scoring: 0.39 on_policy_rkl, 0.25 long_gen_coherence, dethrone margin 5%
- •v32.1 uncapped most axes to 16K for chain-of-thought; probes remain tight to detect collapse
- •H100 without vLLM ~3–5× slower per benchmark, reasoning axes ~10–20× slower
- •v32.2 fixes cap logic_grid/dyval/kg to 6K to improve H100 throughput
- •87 miners submitted v32-compatible models; top KL scores 1.05–1.07, king UID 177 at 1.66 via composite
Distilled from 54 team messages in the official Bittensor Discord. Generated by Claude Haiku 4.5.
View original messages
- Discord message 1503555384303161437
- Discord message 1503555416154706082
- Discord message 1503561115089633280
- Discord message 1503561163948949616
- Discord message 1503561291598397471
- Discord message 1503561828083568730
- Discord message 1503578410650173512
- Discord message 1503579549269753906
- Discord message 1503579627858296923
- Discord message 1503592393193029772
- Discord message 1503592775453773824
- Discord message 1503619097311707250
- Discord message 1503619119940112526
- Discord message 1503629331711135947