Parallel eval and scoring methodology changes coming
Share
Trajectory-RL is shipping parallel evaluation this week, reducing eval time from ~60 minutes to 30 minutes by running 2 scenarios per miner concurrently. The team also confirmed they will switch from stake-weighted to average scoring from qualified validators in the next release to reduce variance. Current top miner sits at 8.21 out of 8.6 maximum possible score.
- •Parallel eval rolling out this week; cuts eval time 50% initially
- •Next release: switch to average validator scores, dropping stake weighting
- •Eval time now ~30 minutes per full evaluation cycle
- •Three scenarios have hard caps: path-tracing, swe-bench astropy2 (0.8 each), break-filter-js-from-html unsolvable
Distilled from 12 team messages in the official Bittensor Discord. Generated by Claude Haiku 4.5.
View original messages
- Discord message 1507412742720589864
- Discord message 1507440029662974033
- Discord message 1507468801464336474
- Discord message 1507471556836720731
- Discord message 1507472374369484872
- Discord message 1507473037644009482
- Discord message 1507473478159306882
- Discord message 1507475655338627233
- Discord message 1507494003606687785
- Discord message 1507497704727646269
- Discord message 1507549905760157736
- Discord message 1507662182035361963