LLM-as-a-Verifier for DeepSeek Harness
uson1x/dsh-plugin-llm-verifier · skills
LLM-as-a-Verifier for DeepSeek Harness: continuous reward signals via select / compare / track.
★ 11GitHub stars
0Forks
2026-08-30Last updated
JavaScriptLanguage
MITLicense
Key features
- 1–20 instead of 1–5 — a finer scale separates close candidates better.
- compare can return a tie on an exactly zero margin
- Grading goes through the harness's own LLM service (ctx.llm) — same provider routing and auth as everything else, no direct API calls.
- Several repetitions (default 4) — averaging repeated grades reduces noise.
- Several criteria — small focused questions beat one big vague one.
Install command
dsh plugin --profile web add github:uson1x/dsh-plugin-llm-verifier