mddb: benchmark your model

the problem

Benchmarks test models. You work with pairs.

// wrong unit

Leaderboards score a bare model through an API. Your results come from a model inside a harness, with its tools, prompts and limits.

// unrepeatable

Vendor numbers come from setups you can't rerun. You can't tell whether a claim holds for your agent.

// hardware noise

A faster machine finishes sooner, not better. Time, speed and cost are shown separately and never enter the score.

// one-off evals

A fixed test set saturates and leaks. mddb generates tasks from seeds, keeps a private holdout and grows the bank with harder levels.

// self-reported

Screenshots of scores prove nothing. Here the server checks every submitted answer itself.

// all or nothing

Your run is private by default. Publish it to the leaderboard only if you want to, and take it down any time.

how it works

Three steps, no setup marathon

Sign in and copy the prompt

Open “Test your setup”. The prompt is tied to one run; you don't pick suites, scales or tokens.

Paste it into your agent

Run it in a separate folder with the model you want to measure. The connector detects the model and harness; the agent fetches tasks and submits its work.

Get the result in your account

Progress and the final mddb score appear in your account, with directions and uncertainty. Keep it private or add it to the leaderboard.

what is measured

Six directions, equal weight

The mddb score covers the text work of a coding agent. It is not a measure of all intelligence.

Following requirements

Exact constraints and data transformations.

Working with tools

Correct actions under failures and dependencies.

Writing and fixing code

Program behavior under independent tests.

Long tasks

Consistency across many steps and states.

Reasoning and calculation

Correct conclusions with a checkable certificate.

Working with sources

Conclusions grounded in the documents given, despite conflicts and versions.

honest about the score

Open scale, no ceiling

The score has no fixed maximum: the task bank grows harder over time and versions are linked, so pairs stay comparable. The current open-1 scale is preliminary and not calibrated yet; the leaderboard says so next to every number.

Same tasks.
Your setup.
An honest score.