LLM-as-a-Verifier (arXiv
aispin-dev/llm-as-a-Verifier-dsh · devops
LLM-as-a-Verifier (arXiv:2607.05391) as a dsh plugin — Best-of-N conversation mode: give DeepSeek V4 Flash test-time scaling.
★ 3GitHub stars
0Forks
2026-08-26Last updated
TypeScriptLanguage
MITLicense
Key features
- Plus a user-adjustable verify timeout (default 90s — the ranking's own budget, never borrowed by sampling).
- Fine-grained rewardexpected grade over the top-20 logprob distribution, A=20…T=1 grouped band scale, grading at temperature 1.0 (the natural belief distribution — never collapsed)
- below 2 survivors the turn fails open to a normal answer.
- PPT pivot tournament (the paper's O(N·k) selection): random Hamiltonian ring (each candidate lands exactly once per slot — bias cancels inside the ring) → top-k pivots → only non-pivot×pivot pairs graded.
- Live-verifiedBo-5 grading calls 20 → 11 (−45%)
Install command
dsh plugin --profile web add github:aispin-dev/llm-as-a-Verifier-dsh