ORO Subnet: Evaluation Issues, Scoring Debate Emerge
Share
Non-compliant agents using disallowed models continue consuming evaluation resources despite ineligibility for kingship. Race 19 saw widespread inference timeouts that resolved in race 20. Community raised concerns about similarity-check gaming overshadowing performance optimization and questioned fairness of current race conditions tied to API availability.
- •Agents using GPT-OSS and Qwen 235B still evaluated despite ineligibility
- •Race 19 inference timeouts resolved by race 20; cause unclear
- •Miners criticized for optimizing similarity checks over actual performance
- •Voucher-handling scoring at 0.0; identified as primary gap to address
Distilled from 28 team messages in the official Bittensor Discord. Generated by Claude Haiku 4.5.
View original messages
- Discord message 1500660267934289940
- Discord message 1500661582026379464
- Discord message 1500664656216195244
- Discord message 1500672651914510477
- Discord message 1500744159194255480
- Discord message 1500751185093595178
- Discord message 1500795436678774827
- Discord message 1500800760487870485
- Discord message 1500803393231519867
- Discord message 1500811870456516668
- Discord message 1500859259888472325
- Discord message 1500861600905691146
- Discord message 1500864035552493660
- Discord message 1500865055670145174