sleepy/deep_pro_judge
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: note pi-mono vs opencode harness usage

Browse Source
This commit is contained in:
Kaloyan Nikolov
2026-04-27 19:03:52 +02:00
parent 107c805807
commit 72fa1f86fc
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+2
View File
@@ -15,6 +15,8 @@ Head-to-head evaluation of six coding LLMs across eight low-level ML kernel task
**Take every score with a grain of salt.** LLM judges can be consistent but are not infallible. The relative rankings are more useful than the exact numbers. **Take every score with a grain of salt.** LLM judges can be consistent but are not infallible. The relative rankings are more useful than the exact numbers.
**Tooling:** The first 3 challenges (KV-Cache, Fused Softmax+TopK, Layer Norm Backward) were generated using **pi-mono** as the harness. The remaining 5 challenges were generated using **opencode**.
--- ---
## TL;DR — Final Rankings ## TL;DR — Final Rankings