---
title: "The Room That Disagreed With Me"
subtitle: "I decided which secret scanner my coding workbench would use. A blind room of two language models, scored by a third, disagreed in two minutes and named a condition. I took it before a line of the code existed."
date: 2026-10-24
events: "17.–24.10.2026"
tags: ["atlas"]
summary: "Before an AI agent builds something hard to undo, the decision goes to a room: two models from two families argue it blind, a fast judging model scores the arguments, and I decide. This is one room, R-G, where the room disagreed with me, and what the record of all the rooms says so far."
video: "https://sf3d-media.pages.dev/rooms/rooms-en.mp4"
captions: "https://sf3d.fi/video/rooms-en.vtt"
canonical: https://sf3d.fi/blog/the-room-that-disagreed
html: https://sf3d.fi/blog/the-room-that-disagreed
author: Petri Korhonen, SF3D AI Lab (Sunrise Software Oy, Tampere, Finland)
site: https://sf3d.fi (index of all articles in Markdown: https://sf3d.fi/llms.txt)
numbers: measured on the lab's own hardware unless the text marks them as held for the AMD R9700 whitepaper; the Evidence block at the end of an experimental article names the run, the date, n and the pre-registration each claim rests on
---
The video above is the article. What follows is the evidence for it: what a decision room is, what happened in room R-G on 17 October, the arithmetic behind the room's answer, and the record every room leaves behind.
## What a room is
AvoQ is the coding workbench I am building: an AI agent writes the code, and only checked code is let out. Its security gate needed a secret scanner, and on 17 October I decided which one: Betterleaks 2, from its `v2.0.0` release.
A decision like that is logged, and a room is queued for it. The room gets the facts and the options, never my choice and never a recommendation. Two models from two different families take the seats, Claude Opus 5.5 and GPT-6.1 Sol, each seals a first vote before it reads the other's, and then they argue for two rounds. They can have a web page fetched and read it as evidence. Between the rounds a third model, Jev, a fast judging model from TypeSafe, scores the arguments, not the votes.
## What happened in room R-G
The room opened at `18:50:04` and closed at `18:52:14`, Helsinki time: `2 min 10 s`. In round 1 the seats had Betterleaks' release page fetched: the newest release was `v2.0.0-rc.2`, a pre-release, and `v2.0.0` did not exist yet.
The room chose *Betterleaks 2 on a condition*: pin the release candidate and its rule file's hash together, let the gate fail closed on anything it cannot read, and move to `v2.0.0` only after an acceptance test and a Windows measurement. It chose it at a value of `0.15`, under the bar of `0.6` declared before the room ran. That low value is the room saying it is unsure.
I read the arguments and took the condition. It became row 98 of AvoQ's decisions, `41` minutes after the room closed. The gate's first commit came the next day, pinned to `2.0.0-rc.2` and to the rule file's hash. The room had changed the decision before a single line of the gate existed.
## What the record says so far
Every room leaves a record, and every decision it checks is counted. Read on 24 October, rooms had returned a result on `147` of my decisions and disagreed with `30`: about one in five. Whether the room or I was right shows only later, so each outcome is recorded as it arrives. The first pre-registered check of that record waits for `20` outcomes and has `9`.
## Why I do it this way
That is hybrid decision-making: two language models argue, a fast judging model scores, and I decide. The word *hybrid* is borrowed. Wang et al. use *hybrid thinking* for a single model that switches between reasoning and answering directly; a room keeps the two apart, as separate models with separate jobs. Without it, my decisions are slower and less complete.
## Transcript
AI writes code faster than a person can check what it decided. So before it builds something hard to undo, the decision goes to a room.
This is AvoQ, the coding workbench I am building. Its security gate needs a secret scanner. I decided: Betterleaks 2, from its version 2.0 release.
The decision is logged, and a room is queued. The question goes in blind: the facts and the options, never my choice.
This is the real room, held in Finnish. Replayed from its record: my machine seats two models from two families, Claude and GPT. Neither can see my decision. Each seals a first vote before it reads the other's.
Then two rounds of arguments. They have the release page fetched: version 2.0 does not exist yet, only a release candidate. Between the rounds Jev, a fast judging model from TypeSafe, scores the arguments, not the votes.
The room disagrees with me. Betterleaks 2, yes, but on a condition: pin the release candidate and its rule file, and let the gate fail closed. And it is unsure. The weaker seat gives it only 0.15.
I read the arguments, and I took the condition. It became row 98 of AvoQ's decisions.
The gate was built the next day, pinned exactly as the room asked. The room had changed the decision before a single line of the gate existed.
Rooms have now checked 147 of my decisions and disagreed with 30. Every outcome is recorded, so over time the system learns when to trust a room, and when to trust me.
The room added a condition my decision did not have, in two minutes. That is hybrid decision-making: two language models argue, a fast judging model scores, and I decide. Without it, my decisions are slower and less complete.
## Sources & further reading
- A. Wald, *Statistical Decision Functions* (1950): the maximin rule, choosing the action whose worst case is best.
- J. Rawls, *A Theory of Justice* (1971): the same shape applied to judging an arrangement by its worst-off position.
- S. Wang et al. (Case Western Reserve University, Michigan State University, Meta AI), *Demystifying Hybrid Thinking: Can LLMs Truly Switch Between Think and No-Think?* (arXiv 2510.12680v2): where the word *hybrid* comes from.
- J. Kwok et al. (Stanford and NVIDIA), *Contrastive Language Models: A System One Model for Fast and Generalizable Decision-Making* (2026): the kind of fast judging model Jev is, and a comparison with it.
---