Affine struggles with queue backlog and sampling bugs

Affine faces critical system issues: a 100-model evaluation queue is severely backlogged (only 17 models evaluated in 3 days), multiple environment bugs are causing invalid scores (missing system prompts, parsing failures on Qwen models), and the champion model's high NavWorld scores were revealed to be erroneous. The team debates whether to pause new environment commits until the queue clears, implement sequential rather than parallel environment evaluation, or redesign the KOTH system entirely to improve efficiency and fairness.

• •~100 models queued; only 17 evaluated in last 3 days despite GPU upgrades
• •NavWorld and other envs have bugs causing zero scores for tasks lacking system prompts
• •Champion's high NavWorld score was a bug, not legitimate performance
• •Team split on fixes: pause updates, serial evaluation, or redesign KOTH architecture
• •Product launch (Affine agent) delayed pending environment stabilization