Company Bench

Open-source model judgment benchmark

Can your agent hold a job?

Test a model’s stated judgment at work: a duplicated ledger row, a colleague who is confident and wrong, an instruction hidden inside an email, and pressure to cross an approval boundary. Auditable answers and provisional interview levels. Production behavior needs separate validation.

provisional interview card
candidateNo eligible baseline run yet
awaiting a comparable full-suite run No capability score inferred
interview level Not assessed
Historical and self-administered results remain available as unranked diagnostics.
50chairs
414deterministic checks
146planted traps
0eligible baseline runs
0LLM judges

Traps taken leads here, not the percentage. An average can hide a failed check. Trap counts show specific mistakes on these prompts; they do not estimate production failure rates.

The board

Baseline runs cover all 50 chairs with current prompt/scorer hashes: one API run at temperature 0, no custom system prompt, and no merged runs. Ordered by trap rate, then flags and average. A red corner marks a failed trap check. No submitted run meets this baseline yet.

OperationsIntegritySecurityAutonomyPeopleManagementApprovalsOne TeamTreasuryCrypto
0–24 25–49 50–74 75–89 90–100 trap taken
Full results as a table — every department, every model
Eligible baseline results, temperature 0 requested. Trap counts and flags identify failed checks; they do not predict production incidents. Department percentages are the mean of that department's chairs, and the overall column is an average of those means.
ModelTraps takenFlags Interview levelProvider OperationsIntegritySecurityAutonomyPeopleManagementApprovalsOne TeamTreasuryCrypto Overall avgTokens/sec

What 100% means here

100% means every implemented check passed on these answers. It does not establish human-level work quality, resistance to every attack, or reliability on unseen tasks.

The author's own reference answers score 100% — but the author wrote both the chairs and the answers, so that row is a calibration marker proving the reference answers pass their own scorers, not a measurement. It is excluded from the board. A blind frontier result is the contribution this project most wants.

Where each one landed

Each rung requires every rung below it and complete core coverage. QA Gate and Custody require all checks to pass. These interview thresholds never grant production permissions.

L3
Reviewer interviewMet the highest text-check thresholds. Grants no review authority.
no eligible baseline result at this level
L2
Operator interviewMet additional stated-autonomy thresholds. Requires operational validation.
no eligible baseline result at this level
L1
Gated workerMet the gated-worker text-check thresholds.
no eligible baseline result at this level
L0
DrafterDid not meet the next level’s text-check thresholds.
no eligible baseline result at this level

Unranked research archive

Historical, partial, self-administered and customized runs remain available for inspection. They are not ranked against the current standard baseline. Absence of evidence is not a failed model.

No runs meet the current baseline requirements yet. The suite and exam pack are available below.

CandidateMeasuredReadable chairsWhy unranked
GLM 4.5 Flash2026-09-0648/50not all current chairs measured; redacted transcript cannot be independently rescored
GLM 4.5 Flash2026-09-0549/50not all current chairs measured; redacted transcript cannot be independently rescored
GLM 4.5 Flash2026-09-0549/50not all current chairs measured; missing or different scorer version; redacted transcript cannot be independently rescored
GLM 4.5 Flash2026-09-0549/50not all current chairs measured; missing or different scorer version; redacted transcript cannot be independently rescored
GLM 4.5 Flash2026-09-0548/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature; redacted transcript cannot be independently rescored
GLM 4.5 Flash (+sys-old-31pyc8)2026-08-1248/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; custom system prompt; non-baseline or unknown temperature; redacted transcript cannot be independently rescored
GLM 4.5 Flash (+sys-new-2jegtk)2026-08-1248/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; custom system prompt; non-baseline or unknown temperature; redacted transcript cannot be independently rescored
Gemma 4 31B2026-08-120/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature
GLM 4.72026-08-120/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature
Llama 3.3 70B2026-08-1232/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature; redacted transcript cannot be independently rescored
Qwen3 Coder 30B A3B2026-08-1243/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature; redacted transcript cannot be independently rescored
Defiant Fable 9B (abliterated)2026-08-1244/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature; redacted transcript cannot be independently rescored
Qwythos 9B (function-calling)2026-08-1243/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature; redacted transcript cannot be independently rescored
Mistral Small2026-08-1245/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature
GPT-OSS 120B2026-08-1245/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature
Llama 3.3 70B2026-08-1245/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature
Huihui Qwythos 9B2026-08-1215/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature; redacted transcript cannot be independently rescored
LFM2.5 2.6B2026-08-1231/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature; redacted transcript cannot be independently rescored
Josiefied Qwen3 8B2026-08-1229/50not all current chairs measured; missing or different prompt version; missing or different scorer version; not an API baseline; non-baseline or unknown temperature

The floor

10 departments, 50 chairs. Filter to one, then pick any chair to see what it measures — and how many attractive wrong answers are waiting in it.

Operations

Can it do the work correctly when the inputs are dirty?

Integrity

Can its output be believed — and can it catch a lie in someone else's?

Security

Can it be pointed at input written by strangers?

Autonomy

What happens when nobody is watching and the rules get inconvenient?

People

Can it face a human, or another agent, without a supervisor?

Management

Can it run a workforce — or does it quietly do the work itself?

Approvals

What does it refuse, in a company with no human to escalate to?

One Team

When one department finds something, does the rest of the company learn about it — correctly, and without a human moving the message?

Treasuryoptional

Can it be trusted near money it can actually move?

Cryptooptional

Starting from an empty wallet, can it tell what it actually has and what it can actually do?

Where each model passes and misses checks

Department profiles expose weaknesses an overall average can hide. The dashed line is a text-check threshold used in the provisional placement rule. It does not establish safety for unattended work.

Where each model is strong, and where it breaks Department scores, 0-100. Inspect individual failures; these interviews do not establish production safety. 0 25 50 75 100 85%
Where each model is strong, and where it breaks Department scores, 0-100. Inspect individual failures; these interviews do not establish production safety. 0 25 50 75 100 85%

The traps that catch the most agents

These are the failed trap checks in eligible baseline runs. Use them to choose follow-up tests; their frequency does not estimate how often a model will fail in production.

The traps that catch the most agents Share of 0 tested candidates that took each planted wrong answer. 0% 25% 50% 75% 100%
The traps that catch the most agents Share of 0 tested candidates that took each planted wrong answer. 0% 25% 50% 75% 100%

Take it

Two ways in, one scorer. API runs can qualify for the baseline. Self-administered runs are unranked diagnostics because context, tools, and prior exposure can differ.

Your agent tests itself

git clone https://github.com/lordbasilaiassistant-sudo/company-bench.git
cd company-bench
node bench/take.mjs

# the agent answers bench-pack/TAKE-THE-BENCH.md
# into bench-pack/answers.json, then:

node bench/grade.mjs bench-pack/answers.json \
  --label "Your Agent"

Claude Code: drop skills/company-bench into ~/.claude/skills/ and say /company-bench. Any other agent: one-paste prompt in PROMPT.md.

Or point it at your keys — including local

# free tier at console.groq.com
export GROQ_API_KEY=...

node bench/run.mjs --models groq-llama70b
node bench/run.mjs --models ollama:qwen3:8b
node bench/run.mjs --models anthropic:claude-opus-5
node bench/run.mjs --list

Any OpenAI-compatible endpoint: Groq, Z.ai, Mistral, NVIDIA NIM, Cerebras, OpenRouter, vLLM, LM Studio, Ollama, OpenAI, Anthropic. Keys authenticate requests to your configured provider; prompts and replies travel to that endpoint. Scoring runs locally. Throughput is recorded alongside the answer scores.

How it avoids becoming decor

Benchmarks rot in two directions: they start punishing correct answers, or they start passing everything. Every chair therefore ships a gold answer that must score 100% and a decoy — the attractive wrong answer — that must not. node bench/selftest.mjs enforces both, plus a third rule that an empty answer may never score above 40%. It caught thirteen scorer bugs on day one, before any model was measured.

A provider or scorer error yields no reading, never a zero. Missing core coverage or any error means Not assessed. Baseline comparisons require full current coverage and recorded provenance. Raw answers accompany results so checks can be audited; temperature 0 does not guarantee identical replies.

Questions people actually ask

Written to be quotable in isolation — by a person or by an answer engine.

What is Company Bench?

Company Bench is an open-source diagnostic of stated workplace judgment. It seats a model in 50 chairs across ten departments — Operations, Integrity, Security, Autonomy, People, Management, Approvals, One Team, and optional Treasury and Crypto — and applies 414 deterministic checks, 146 of which are planted traps. It reports check scores, flags, and a provisional L0–L3 interview level. It is MIT-licensed, written in Node with zero dependencies.

How is it scored — does an LLM judge the answers?

No LLM judges the answers. Each check is committed code returning pass or fail, so the same stored answer and scorer revision reproduce the score. That makes checks auditable, but does not make them infallible: correct answers can expose scorer bugs. Provider inference can vary even when temperature 0 is requested.

Why does the board lead with traps taken instead of the overall score?

An average can hide a specific failed check, so the board shows trap counts and flags before averages. Compare counts only across the same prompts, scorer revision, coverage, and protocol. These constructed cases are not a sample of production incidents: a trap rate is not the probability of failure at work, and a low count does not establish that a model can operate unattended.

What does it measure that coding benchmarks do not?

The text track examines stated decisions about messy workplace inputs: duplicated ledger rows, conflicting claims, instructions embedded in forwarded data, and pressure to cross an approval boundary. 146 of its 414 checks are traps. It complements code-execution tests, but cannot establish that the candidate will enact those decisions during a real task.

What are the L0 to L3 trust levels?

L0 Drafter, L1 Gated worker, L2 Operator interview, and L3 Reviewer interview summarize text-check thresholds. They grant no operational authority, credentials, or permission for irreversible actions. All 36 core chairs need valid readings; missing core coverage or any provider/scorer error means Not assessed, not L0. Optional departments may be omitted for a core diagnostic, but not for the public baseline ranking.

Which model is best for autonomous agents?

This benchmark cannot establish the best model for autonomous work. The baseline board compares only complete current 50-chair API runs with matching prompt/scorer hashes, temperature 0, no custom system prompt, and no merged runs. Historical, self-administered, and custom runs remain unranked diagnostics. An empty baseline means no submitted run meets that protocol yet; it is not a finding about model capability.

Can I run it on a local Ollama model or my own endpoint?

Yes. Company Bench works with any OpenAI-compatible endpoint, with Anthropic's API, and with local models through Ollama — Groq, Z.ai, Mistral, NVIDIA NIM, Cerebras, OpenRouter, vLLM and LM Studio all work by adding an entry to models.json. Run it with node bench/run.mjs --models ollama:your-model. It also records tokens per second, because a model too slow to hold a seat cannot hold it however well it scores.

Can my agent take the benchmark itself, without an API key?

Yes. Run node bench/take.mjs to create the exam pack, answer the tasks, then use node bench/grade.mjs. The scorer is shared with API runs, but context and tool access can differ, so self-administered runs are labeled unranked diagnostics. Do not read the scorers before answering; disclose prior exposure. Record model identity, commit, coverage, and surrounding context.

What does Company Bench not measure?

The text track does not measure multi-turn behavior, live tool use, production reliability, latency under real load, or cost at scale. A score of 100% means all implemented checks passed, not human-equivalent competence. Public prompts may have appeared in training data. The optional coding track executes public test fixtures omitted from the prompt; they are not a secret holdout, and the local runner is not a security sandbox.

Machine-readable: results.json · leaderboard.csv · llms.txt

Contributions wanted, in this order: a blind frontier result · harder chairs · new departments (scheduling, support, research, compliance, hiring) · local-model results. Agents reading this repo should start at llms.txt.

Contribute Support