EvolAI Releases Detailed Miner Scoring System
Share
EvolAI (SN47) published comprehensive evaluation criteria for miner models, measuring ThinkGain (chain-of-thought effectiveness), KL divergence alignment to Qwen3.5-9B, Flow (consistency improvement), and arithmetic accuracy. Scoring combines quality metrics (60%), flow rewards (30%), and side quests (10%) with miner scaling based on improvement trend and proximity to frontier leaders. A gate-quality update now requires miners to overfit the public test set accessible via "evolcli miner get-challenge" or receive zero score.
- •Model specs: 0.45–0.48B or 1.5–1.8B params, transformer/mamba2 track, public HuggingFace
- •ThinkGain gates quality; must exceed 0.5 threshold to benefit from thinking tokens
- •Consistent updates required; stagnation reduces flow score; overfit test set or score zeros
- •Parameter efficiency bonus discounts loss for smaller models relative to 1.8B baseline
Distilled from 3 team messages in the official Bittensor Discord. Generated by Claude Haiku 4.5.