Skip to content
UsecaseBoard
← All use cases
hackernewsLocal AI

Compare local LLM serving setups with traceable benchmark receipts

At a glance

Alex Ellis uses RigMark to replace isolated token-speed claims with disclosed workloads, settings, and saved result receipts. This workflow is listed under Local AI, Benchmarking, Reproducibility.

Agents / tools
RigMark, Python, OpenAI-compatible model server

SourceBuilding a tool for these workflows? See weekly rates.

What they did

1) Prepare Python 3.10+, Git, and a quiet OpenAI-compatible model endpoint. Clone https://github.com/alexellis/rigmark, enter the checkout, run ./rigmark check, then ./rigmark configure to fill metadata.json with your actual hardware and serving configuration. 2) Run ./rigmark bench --base-url http://SERVER:8000 --model auto --label my-appliance --comparison-id sweep-1 --metadata metadata.json, replacing the endpoint. Supply authentication through OPENAI_API_KEY if required. 3) Inspect the generated results/LABEL-TIMESTAMP.json and matching .card.txt, including output gates and incomplete or unverified phases. Keep the same comparison ID and request settings for the next setup. 4) Run ./rigmark compare --card results/setup-a.json results/setup-b.json. Resolve reported mismatches before claiming a matched comparison, and retain both receipts with the comparison card.

Worth knowing

For servers without vLLM's /tokenize or token-ID completion input, add --skip-prefill and label the suite incomplete. Output gates do not prove code correctness or reasoning quality; capped concurrency throughput does not measure completed agent tasks.

Save ideas in this browser. Upvotes count across the board.

Prompt excerpt

No prompt excerpt was provided by the source.

Sponsored

CodeBuddy

Tencent Cloud AI coding assistant for IDE and CLI.

Try CodeBuddy ↗
Explore all →