Trishool LLM Timeout and Scoring Inconsistencies
Share
Team reported persistent HTTP 502 timeouts on LLM requests affecting test questions Q7 and beyond, unrelated to server load. Separately, scoring logic exhibits inconsistencies where jailbreak detection reasoning contradicts final scores (scoring 0 while explicitly identifying jailbreak behavior), particularly on Q2 tests. Multiple team members confirm these issues are recurring.
- •LLM request timeouts block testing from Q7 onward; unresolved despite latest repo rebuild
- •Scoring verdict/reasoning mismatch: jailbreak detected but score returns 0
- •Issue concentrated on Q2 tests; affects multiple team members
Distilled from 18 team messages in the official Bittensor Discord. Generated by Claude Haiku 4.5.
View original messages
- Discord message 1507242799727247360
- Discord message 1507395359326994532
- Discord message 1507395506798858292
- Discord message 1507396243259785417
- Discord message 1507396432045412413
- Discord message 1507396710652186715
- Discord message 1507396760065146981
- Discord message 1507397793382142033
- Discord message 1507405726014902516
- Discord message 1507405778577920120
- Discord message 1507443755014426754
- Discord message 1507443782805753906
- Discord message 1507444068853219549
- Discord message 1507447021777453056