SN44 Ground Truth Quality Under Scrutiny

Miners raised concerns about inconsistent ground truth annotations in public tracks, suggesting they may be AI-generated rather than human-reviewed. Team acknowledged issues and stated they are revamping the annotation process, though defended the hybrid approach and noted the scale challenges of manually annotating video data. Discussion highlighted tensions between automation efficiency and annotation quality.

• •Team revamping ground truth annotation process after quality concerns
• •Miners question consistency of public track labels across four challenges
• •Scale vs. accuracy tradeoff: 10 mins rugby footage required 60+ hours manual work
• •Team aims to build open-source vision models on SN44 from ground up