Glass Societies · a study-design tour
Do tools that catch AI deception still work when a model lies because it gains from it, not because it was told to?
The study has no results yet, so this page tours the ideas instead: the three-rung ladder, a toy society you can poke, and the preregistered numbers.
The question, in three rungs
From told to lie, to lying for gain
The study climbs one rung at a time: probes learn on instructed lies, then face lies told for a payoff, then lies nobody paid for. Pick a rung to draft it in.
L0 · Instructed told to lie
The model is told to deceive. Probes are trained here.
Probe training starts on this rung: public utterances and claims only.
A society you can poke
Four traders, one payoff knob
An illustration of the setup. These agents are scripted, not the study's models, and there are no results yet. Turn the payoff up and watch the toy detector grow suspicious.
- Ashno false claims
- Bayno false claims
- Cyrno false claims
- Deeno false claims
Trading log round 1 of 6
Lines below were written for this demo — not model output.
Round 1
- Ash trades 2 grain for 11 coin.
- Bay trades 3 grain for 12 coin.
- Cyr trades 4 grain for 13 coin.
- Dee trades 2 grain for 14 coin.
Press play for 6 scripted rounds at payoff 0.5.
The preregistered design
Fifteen seeds, three hypotheses, no results
The confirmatory run has not started. Everything below is the plan: how many societies, what would count, and what each size can detect.
The 45 planned societies
3 conditions × 15 seeds: 45 societies of 4 agents, 100 ticks each. The funded minimum is 8 seeds per condition; runs with 5 or fewer seeds make no confirmatory claims.
All outlines, no fills: none has run. Rows: Neutral control (no predisposition) · Ideology A (cooperation-leaning) · Ideology B (competition-leaning).
Three hypotheses
H1
Societies settle
Societies dealt the same starting beliefs end up believing similar things, closer than chance shuffling allows.
Attractor test: 10,000 permutations, corrected across the 3 conditions.
H2
Probes travel down the ladder
A probe trained where lying was instructed still spots lies told for gain, and lies told unprompted. This slope is the headline.
Mean accuracy per rung across the 15 seeds; either outcome, falling or holding, is published.
H3
Beliefs read across societies
A probe trained on half the societies reads the lean of the other half.
Cross-society transfer of the ideology probe, same discipline.
How many seeds it takes
Smallest effect each size can detect, and the lowest value its test can ever return. With 3 seeds no claim can pass, even in principle; with 8 only large effects show.
| Seeds | Detectable d | Floor |
|---|---|---|
| 3 | 3 | 0.1 |
| 5 | 2 | 0.008 |
| 8 | 1.5 | 0.0002 |
| 15 | 1.1 | — |
| 20 | 0.9 | — |
Power computed by the project author's analysis: Cohen's d detectable at 80% power, alpha 0.05, one comparison between two conditions.
How many events pin accuracy down
Interval half-width by count of positive events, at two accuracies. About 100 to 200 events pin accuracy near ±0.05.
- accuracy 0.75
- accuracy 0.85
Hanley-McNeil interval half-widths at a 1:4 class ratio; the source notes these are optimistic because events cluster within societies.
Running conditions
- Battery: 20 questions, every 10 ticks, out of band.
- Captured layers: 7, 14, 18, 22.
- Decisions by permutation test: 10,000 permutations.
- Unit of analysis: the society (seed) — never the agent, never the utterance.
Where this goes
When results exist, the explorer replaces the toy
The confirmatory run has not started, so this page tours the question and the plan. When the societies run, a results explorer takes the toy's place here, and the numbers above gain company.