THE ROUND TABLE

I keep two windows open all day. Claude Code in one, Codex in the other. They are rivals, built by competing labs, and I use them like coworkers who happen to never speak to each other. One afternoon I wondered what would happen if I made them talk. Not to chat. To review each other's work, the way two engineers argue over a pull request, until they either agree or give up.

So I built the room where that argument happens. It is called a harness, and this is the story of what it caught, what it cost, and everything that broke on the way.

What an agent harness even is

Start with the word, because it does a lot of work and almost nobody defines it. A language model on its own does exactly one thing. Text goes in, text comes out, once. It has no memory of the last call and no way to do anything in the world. A harness is the code wrapped around that model that turns a one-shot text generator into something that takes actions, over many steps, toward a goal.

The picture that helped me: the model is the CPU, the context window is the RAM, and the harness is the operating system.

It decides what goes into the model, runs whatever the model asks for, feeds the result back, and decides when to stop. You already live inside several. Claude Code, Codex, and Cursor are all harnesses. Same models underneath, very different tools on top. The harness is where the real work happens.

Mine runs three of them at once. Codex writes the code. Claude and Gemini review it. They never speak to each other directly. The orchestrator, a plain Python script, is the only thing in the middle. It passes each agent's work to the next through files on disk. The filesystem is their shared memory. That detail matters later, because it is also the thing that keeps a reviewer honest.

My first design was wrong. It never argued.

My first version was a straight line. Gemini researches the problem. Claude writes a plan. Codex implements the plan. A hidden grader checks the result. If the grader fails, the failure goes back to Codex and it tries again.

I ran it three times, on three different problems, including a deliberately hard one with the tests hidden so nobody could cheat by reading the answers. The loop never fired. Not once. Codex got it right on the first try every single time.

It took me a while to see why, and the reason is almost boring. A good plan front-loads the correctness. By the time Gemini has looked up the exact rules and Claude has turned them into a precise spec, there is very little left for the implementer to get wrong. I had built an assembly line and called it a debate.

I built a debate and got a monologue.

The round table

So I tore out the straight line and built a round table.

It still opens the same way. Gemini researches, Claude plans. Then the loop changes. Codex writes the code. The orchestrator freezes it and shows that exact version to three judges at the same time: a hidden grader, Claude, and Gemini. The two reviewers cannot see the grader's result. They cannot see each other. Each one returns a simple verdict, approve or request changes, and every objection has to name the rule it breaks and an input that would fail.

The run only finishes when two things are both true. The grader passes, and both reviewers approve with zero blocking issues. Passing the tests is necessary but not sufficient. A human-style sign-off is required too. If anyone wants changes, the orchestrator bundles up the real objections, hands them back to Codex, and runs another round, up to four.

One choice does more than it looks like. Codex is the only agent allowed to edit files. Claude and Gemini run read-only. They physically cannot rewrite the code to their taste. They can only argue in words and hope the implementer agrees. That keeps the reviewers reviewing instead of quietly grabbing the keyboard. Every round, I also record a small thing for each reviewer: did it approve or reject, and did the hidden grader agree. That one cross-check is what tells you whether a review caught something real or just waved the code through.

The time it worked

The first time the loop actually fired, it justified the entire project.

The task was an arithmetic expression evaluator. Parse a string like -2 ** 2 and return the right number, with operator precedence, parentheses, the whole thing. I forbade the obvious cheat, no calling Python's own eval, so it had to be a real parser.

Round one, Codex wrote it, and the grader passed. Every test, green. By my objective measure the code was done. But Claude, reading that same frozen file, refused to sign off. Its objection: every number was being parsed as a float, so integer arithmetic quietly loses precision. Its failing input was 2 ** 53 + 1, a value a float rounds to the wrong answer and an integer gets exactly right. My grader compared with ==, where 4 and 4.0 are equal, so it was completely blind to this. The tests said yes. The rival in the room said not quite.

Codex took the objection, switched to integer math where it mattered, and round two it passed Gemini too. Round three, both reviewers approved, and it converged. Claude's bill for catching what my tests could not: ninety-eight cents.

The time it churned

Then I gave it a different problem, and it showed me the other face.

The task was a CSV parser. Quoted fields, escaped quotes, commas and newlines inside quotes, the real rules. Again I forbade the shortcut, no importing Python's csv module. And this time the grader passed on round one and stayed passing every single round after. The code was, by the tests, correct from the start.

It did not matter. The two reviewers spent four rounds and almost four dollars fighting over one tiny thing the spec never decided: what should the parser return for a single newline and nothing else? An empty result, or one empty record? The spec did not say. So Gemini called it a blocker. Claude called it fine. Codex changed the code to please one of them, which flipped the other to unhappy. They never both approved in the same round. They just swapped sides, round after round, until the money ran out.

It hit the four-round cap and stopped, unconverged. Correct code. Four rounds. Four dollars. No agreement.

There is no answer the tests can give you when the spec itself forgot to decide.

Pull the lever yourself

Here is the single rule that decided both outcomes, and you can pull it. The thing that mattered most was how many reviewers have to sign off. I required two. Watch what happens to the CSV disaster when you ask for only one. The data underneath these sliders is real, lifted straight from the two runs above. Nothing here is simulated.

At two reviewers, the CSV run is unsatisfiable. It burns the full four dollars and reaches no conclusion. At one, the very first round already had a reviewer who approved, so it converges immediately at a dollar seventy-one. Same code, same reviews, a third of the cost, just by loosening the gate by one. The harness's strictness is not a detail you tune at the end. It is the product.

What broke

None of this ran cleanly the first time. The honest part of any build is the part nobody screenshots.

The worst bug was the one that nearly fooled me into the wrong conclusion. For a while my reviews kept coming back "malformed," and I assumed the models were emitting broken JSON. I wrote a whole theory about markdown fences and trailing prose. Then I logged the raw output and actually looked at it. It was empty. The real error was a timeout. Claude, as the lead reviewer, was occasionally taking longer than my three-hundred-second limit and returning nothing, and my safety net was turning that silence into a fake rejection, which forced extra rounds that looked exactly like the agents disagreeing. A chunk of what I had been reading as collaboration was an infrastructure artifact. I raised the timeout, added a retry, and made a real timeout its own distinct state instead of a counterfeit objection.

The rest of the reel is shorter. macOS does not ship the timeout command I reached for first. Codex starts in a read-only sandbox and silently cannot edit a single file until you grant it workspace-write. At its default reasoning setting it spent nine thousand seven hundred tokens to answer the word "PONG." Gemini blocks on a folder-trust prompt unless you pass --skip-trust. And all three vendors report what a call cost in three different places, in three different shapes, so just adding up the bill was its own small project.

Worse than any single bug: the whole thing is wildly nondeterministic. The same calc task, run three times, took three rounds, then two, then one. Multi-agent behavior is not a fixed number you measure once. It is a distribution, and a noisy one.

What I actually learned

So, is a round table of rival AIs worth it? The honest answer turned out to be the most useful thing I learned. It depends entirely on whether there is a right answer to find.

On the calc task, there was. A real bug existed, the tests missed it, and a rival model caught it. That is cross-vendor review doing exactly what you hoped, finding the thing your own checks cannot. On the CSV task, there was not. The disagreement was about a question the spec left open, and no amount of looping could resolve what was never decided. The agents just took turns being unhappy until the money ran out.

Same harness, opposite outcomes, and the difference was not the agents. It was whether the problem had a ground truth underneath it. Which gives a rule I will actually use. When two good reviewers deadlock on an edge case, that is not a signal to keep looping. It is a signal to go pin down the spec, or write the test, and settle the question yourself. The harness cannot decide what you never decided.

The whole harness is public, every script and every task, here: github.com/Ajinkya259/round-table. It is vibe-coded with Claude, and you can read or run all of it.