AI engineering efficiency

Know which coding agents to use, where, and what they should cost

Not a vendor's leaderboard. Teev proves it against your own merged pull requests, inside your own cloud — so the decision is evidence, not instinct.

Try it now — free → Talk to us

Free to start · no card · your code and keys never leave your account

Results · run 04-1182 Illustrative — not a customer result

Codex leads — 7 of 10

70% pass rate, 2 tasks ahead of Claude Sonnet 4.5 (5 of 10).

1 gpt-5-codex ~~· codex · high~~ 7 / 10 $0.41

2 claude-sonnet-4-5 ~~· pi · medium~~ 5 / 10 $0.63

3 qwen3-coder ~~· qwen · high~~ 4 / 10 $0.02

4 kimi-k2 ~~· kimi · medium~~ 3 / 10 $0.04

Cost is per task, not per token. Every profile attempted the same ten tasks in the same sealed environment.

What you learn

Two questions. One run answers both

Every team is guessing at the same two things — what actually works, and what it ought to cost. They're usually treated as separate arguments. They're the same measurement.

Performance

Which model and harness mix actually solves your tasks?

Not which scores highest on a public leaderboard the models were trained on. Which combination of model, harness and reasoning effort resolves the kind of work your team genuinely merges — the migrations, the edge-case bugs, the fixes that touch four services.

You'll see where a profile is strong, where it falls over, and which categories of work it should never be pointed at.

You leave withThe profile to default to, the ones to route specific work toward, and the ones to stop using.

Cost

How much cheaper can you get without losing quality?

Once you know what passes, you know what you can safely stop paying for. The gap between the profile a team defaults to and the cheapest one that still clears the same tasks is usually large, and it's invisible until someone measures it on real work.

Measured per task that actually passed, not per token — because a cheap model that loops and retries is not a cheap model.

You leave withA defensible number for what your current defaults cost you, and what they'd cost done differently.

Both answers come out of the same run, on the same frozen tasks, in the same sealed environment. That's the point: you're not trading confidence for savings, you're getting the evidence to be confident about both.

What we believe

Yes, the jet is faster. You still don't take one to the office

Nobody is arguing frontier models aren't the best available. The question is whether best available is required — for a dependency bump, a null check, a flaky test, a migration shaped like the last eleven. That's a question with an answer, and almost nobody has measured it.

The default is wrong

Frontier-by-default was never a decision

It's what the tools ship with. Routine work rarely needs frontier capability, yet the most expensive model is everyone's starting point — not because anyone chose it, but because nobody had grounds to choose otherwise.

Routing is immature

One default model is not routing

Neither are keyword rules, gut feel, or a vendor shuffling its own SKUs. Real routing needs to know which profile clears which kind of work on your codebase, and that has to be measured before it can be automated.

Harnesses are underrated

The same model, two harnesses, twice the cost

Context re-feeding is usually why. Model, harness and reasoning effort together decide both what you get and what you pay — and only one of the three ever gets discussed.

Benchmarks are compromised

Public benchmarks are published, so models train on them

Every leaderboard the market watches is in the training data of the models it ranks. Your merged pull requests have never been published anywhere, which is exactly what makes them worth measuring against.

The metric is wrong

Token-maxing is not value-maxing

Cheap tokens are not cheap work. A weaker model reads more files, retries more often and loops for longer — and can cost more per merged pull request than the expensive one you avoided.

Where this goes

Models and harnesses become fungible — and invisible

Developers describe the work; a routing layer picks the rest, and every part can be swapped as the evidence changes. You can't get there without a way to keep re-measuring, which is the thing we're building.

How it works

Your merged PRs are already the test

Every merged pull request is a task with a known-good answer and a test that proves it. Teev turns that history into a private, repeatable measurement nothing has been trained on.

01 /

Find the work worth measuring

We read pull request metadata only — sizes, test files touched, linked issues — and tell you how much usable material your history holds before we look at a line of code.

02 /

Rebuild the moment before the fix

The repository exactly as it stood before the change, pinned to a recorded environment. The human fix is sealed away as the answer key.

03 /

Prove the task can grade itself

A hidden test from the real change has to fail before the fix and pass after it. If it doesn't, or if it's flaky, the task is thrown away. This is the step nobody skips and everybody underestimates.

04 /

Prove the task is fair

If the linked issue already names the root cause, the file and the fix, it isn't a test of engineering ability — it's dictation. Those tasks are flagged and don't count toward the ranking.

05 /

Run every profile against identical work

Each model, harness and effort level attempts the same tasks in the same sealed, disposable environment, under the same trial policy. A run with open network access is reported as diagnostic and refused a ranking, because the agent could have looked up the upstream fix.

06 /

Get an answer you can act on

Which profile led, by how many tasks, at what cost each — then the evidence underneath it, task by task. When profiles tie we say they tied, and separate them on cost and speed rather than inventing a winner.

Runs on your compute

We never see your code. We never see your keys

Teev runs inside your own cloud account. What crosses to us is facts about a pull request — never its contents, and never a credential.

Your cloud account

The runner

GitHub token
Model API keys
Source code
Patches, diffs, logs
Test results, artifacts

Nothing in this box leaves it

Teev

The control plane

Your account and team
Run history
Pull request metadata
Pass, fail and cost

No route into your network · no credential of yours

On the wire repo · PR number · title · author · files changed · lines added · gates failed · score · pass / fail · cost · duration

Never source · diffs · patches · issue bodies · logs · API keys · GitHub tokens

Your keys are never ours to hold

Model keys are set on your own runner, read from your own secret store, or replaced entirely by your cloud's IAM identity. GitHub issues its token directly to your runner over device flow — it never passes through us. Our API refuses credential material outright, and every deployment is checked for it.

Nothing of ours can reach into your network

Your runner polls out and asks for work. There is no inbound endpoint, no callback, no open port and no standing access for us to misuse. Outbound traffic goes to a fixed allowlist you can inspect.

You can see the diffs. We can't

When the console shows you a pull request, your browser is fetching it from GitHub, authenticated as you. The page is assembled from two sources — our metadata and GitHub's content — and only one of them is us. We link; we don't proxy.

What screening does send

To rank candidates and check a task is solvable, your runner sends linked issue text and patch context to a model provider using your own key — the same providers your engineers' coding tools already use. Your source never goes to Teev, and everything else stays on your runner.

No skin in the game

We don't have a horse in this race

We don't resell models. We don't take referral fees. We don't have a model, a harness or a partnership to protect, and we don't get paid more if the answer is expensive. If the profile that wins your comparison is the one you're already paying for, that's the answer you get — and we'll say so.

Proof

We publish our own results, including the bad ones

We run the same process against a public corpus and post all of it: the tasks, the profiles, the pass rates, the cost per task, and the runs that failed or had to be thrown out. Read it before you talk to us. If our method doesn't survive contact with evidence, you'll see that there first.

Ask for the results →

Method

The same tasks, re-run every release

Because the tasks are frozen — pinned environment, recorded digests, fixed trial policy — re-running when a new model ships costs almost nothing and produces an exact comparison rather than a fresh guess. Your evidence stops expiring every time the market moves.

What you're buying

The evidence, and the people who read it with you

The platform

Teev runs in your cloud. Point it at your repositories, build a private task set, compare every model and harness against it, and re-run the whole thing the day a new model ships. Set up yourself in an afternoon.

Deploys into your own AWS account
Public and private repositories
Model × harness × effort comparison
Cost per task, not per token
Free to start

Advisory

Evidence doesn't change behaviour on its own. We work with your platform and engineering leads to turn results into defaults, routing and habits your developers actually follow — then check, a quarter later, that the gains held and delivery didn't slip.

Efficiency diagnostic — 2 weeks, fixed scope
Proof sprint — 6–10 weeks, hands-on
Quarterly advisory — ongoing counsel

Common questions

The four we get asked every time

How many pull requests do we need?

Fewer than you'd expect, and the spread matters far more than the volume. A task set made of twenty variations of the same bug fix tells you one thing very precisely. Tasks spread across services, languages and kinds of change tell you where a cheaper profile holds and where it falls over — which is the answer you actually came for. We screen for that spread when we build it, and we'll tell you before you start whether your history has enough.

We're committed to one vendor. Is this still useful?

More useful than you'd think. Most of the spread lives inside a single vendor's line-up — which model, at what reasoning effort, through which harness. That's most of the difference in both what gets solved and what it costs, with none of the contractual friction. And when the term is up, you'll be renegotiating with evidence from your own codebase instead of a slide.

Couldn't we just build this ourselves?

You could. The hard part isn't running the models — it's building tasks that are honest. A test that fails before the fix and passes after it, every time. Flaky tasks thrown out. Environments pinned so a rerun in March means the same thing as a run in January. An issue that doesn't hand the agent the answer. A sealed network so nothing can look up the upstream commit. That's where the months go, and a measurement that skips any of it produces a number you can't act on.

Do you see our code?

No. Teev runs inside your own cloud account. We receive facts about your pull requests — repository, number, title, author, sizes, which screening gates failed — never their contents, and we hold none of your credentials. There is no field in the payload a diff could arrive in. During screening, your runner sends linked issue text and patch context to a model provider using your own key; that traffic goes from your account, not ours.

Free to start

Find out what your agents are actually worth

Set it up yourself and run your first comparison at no cost. We're early, we want the feedback, and we'd rather you saw the evidence than read our claims about it. If you'd rather we walked you through it, we'll do that too.

Try it now — free → Talk to us

What you need: a repository whose merged PRs close issues and change tests, and a cloud account to run in.