SN11 debates re-evaluation mechanism, reports scoring bug
Share
Trajectory-RL team discussed whether to re-evaluate top-scoring miners to prevent luck-based wins from dominating the leaderboard. Consensus emerged around re-evaluating top 5 miners, though concerns remain about evaluation queue length and benchmark depth. Separately, tao.com submissions scored zero across all scenarios; root cause under investigation.
- •Re-evaluation of top 5 miners proposed to filter lucky high scores
- •Evaluation throughput bottleneck noted; only ~20 miners evaluated daily
- •Benchmark may be converging too quickly; scenario expansion suggested
- •Tao.com scoring anomaly flagged; potential impact on future evaluations unclear
Distilled from 37 team messages in the official Bittensor Discord. Generated by Claude Haiku 4.5.
View original messages
- Discord message 1503602010421858434
- Discord message 1503723824762589355
- Discord message 1503724288388108298
- Discord message 1503726729963769977
- Discord message 1503726835882790972
- Discord message 1503740188395049201
- Discord message 1503740703044538458
- Discord message 1503741034885283900
- Discord message 1503763214293536820
- Discord message 1503763291057684743
- Discord message 1503763374268485712
- Discord message 1503766029782749254
- Discord message 1503766331298938961
- Discord message 1503766779980288040