P0 UID-rotation eval bug fixed; chat template bias identified
Share
Distil shipped a critical fix for hotkey rotation blocking new evaluations. The strict one-eval-per-registration policy was incorrectly inheriting stale composite scores from previous UID holders, locking new hotkeys out of challenger selection. Fix deployed with regression tests and three poisoned ledger entries surgically removed. Separately, team identified a P1 scoring bias: eval pipeline applies raw text to models instead of chat templates, penalizing chat-optimized models and creating divergence from real-world performance. Chat template wrapper planned before next composite schema update.
- •Hotkey rotation re-enabled: evict_stale_evaluated_uids now detects UID-hotkey mismatches and clears stale composites
- •UID 159 and two others unblocked; prior hotkey ledger entries retained as historical records
- •980/981 regression tests pass; one pre-existing failure unrelated to fix
- •Chat template bias fix planned: apply tokenizer.apply_chat_template() to v31 axes and probes before generation
Distilled from 40 team messages in the official Bittensor Discord. Generated by Claude Haiku 4.5.
View original messages
- Discord message 1506718540680658945
- Discord message 1506718605209763993
- Discord message 1506747149227134996
- Discord message 1506754354705727568
- Discord message 1506783048052772926
- Discord message 1506800828751679619
- Discord message 1506800908980453448
- Discord message 1506811802732724345
- Discord message 1506812801740701736
- Discord message 1506813487123267686
- Discord message 1506814056198312066
- Discord message 1506821595765800970
- Discord message 1506830657110937671
- Discord message 1506832126069309460