More or Less · Topics · Episode #167
Forget Independent Auditors — Make the Labs Red-Team Each Other
Clip from the full recording, 16:29–20:07 · download mp4 · watch at this point on YouTube
Dave's alternative to Dario's independent assessors: adopt the security industry's playbook and have the big labs red-team each other's frontier models before release — Sam brings the Sublime Security precedent and the secret weekly CISO calls.
Jess drills into the monitoring question from Dario’s proposal — independent assessors with deep access. Dave counters with the security industry’s actual playbook: before a new model ships, the big labs (US, China, wherever) should red-team each other’s models, the way major security organizations mutually report and fix critical bugs today. Independent auditors with employee badges sounds ‘pretty hard to coordinate and harder to trust’ — though maybe you need both. Sam adds the folklore: the top ten tech companies’ CISOs already hold an unofficial weekly call to share what they’re seeing (Jess: ‘I feel safer’), and Slow’s early investment in Sublime Security was premised on productizing that collaboration for everyone below the top ten. His caveats: at AI speed the red-teamers can’t be humans, someone has to pay for it, and he’d bet working groups already quietly exist. On Brit’s US-China question, nobody sees DeepSeek and Anthropic swapping models — ever.
Key points
- Dave's proposal: labs red-team each other's frontier models pre-release — standard practice in security, where major orgs mutually report and immediately fix critical bugs.
- Independent human auditors with badges = hard to coordinate, harder to trust; maybe you need both layers.
- Sam: the top-10 tech companies' CISOs already do an unofficial weekly intel-sharing call — Sublime Security was funded on democratizing that below the top ten.
- At model speed the testing loop can't be human — it needs an automated framework, and someone has to fund it.
- US-China mutual red-teaming: unanimous 'never gonna happen.'
Where they landed
Quotes
“The big labs, whether they're United States, China, wherever, should allow the other labs to red team their models before they go.”— Dave
“The top ten tech companies in the world, what they do is they get on a weekly call between the CISOs... they're sharing things.”— Sam
“Independent auditing with people having employee badges... sounds pretty hard to coordinate and harder to trust.”— Dave
Suggested tweets
Dave Morin's alternative to AI safety auditors: make the labs red-team each other's models before release, exactly like the security industry handles critical bugs today.
Tweet this →TIL from More or Less: the top ten tech companies' CISOs hold an unofficial weekly call to share threat intel. The AI labs will need the same — but automated.
Tweet this →All topics · Full episode #167 + transcript · Subscribe on YouTube · Spotify