Cacheon restarts competition with overhauled evaluation
Share
Cacheon is restarting its competition on June 1 after refactoring evaluation logic. Speed and correctness now run on identical prompts in a single pass; TTFT and TPS metrics replaced by one end-to-end wall-time number against vLLM baseline. Spec decoding that passes correctness checks is permitted. Emissions will ramp to full rate after June 8, starting at 2% next week.
- •One-pass eval unifies speed and correctness on same prompts
- •Single speed metric: end-to-end wall time vs. vLLM baseline
- •Round artifacts now tied to block hash for transparency
- •Spec decode allowed if correctness gate passes
Distilled from 7 team messages in the official Bittensor Discord. Generated by Claude Haiku 4.5.