More or Less · Topics · Episode #168

Sam's Theory: AI Labs Sold Us "We Must Own AGI" — Now They're Backpedaling Because They Can't Afford the Liability

Clip from the full recording, 18:06–25:08 · download mp4

Sam unpacks a Jaron Lanier idea — that technology centralizes value and socializes risk — to argue the AI labs' AGI narrative has flipped in three years from "only we can be trusted with this" to "we're just one of several ordinary businesses," exactly when the liability bill came due.

Sparked by Jessica’s comment that The Information is sorting through “interesting things” in Anthropic’s S-1, Sam goes deep on what he sees as the actual central risk in AI: liability, not capability. He revives an old Jaron Lanier framework — that a lot of technology centralizes value into one owner while radiating the risk out onto everyone else — and argues AI breaks that pattern, because the liability (hacks, harms, catastrophic misuse) is effectively infinite in a way no public company can actually absorb. His analogy: his banker father used to joke that Disney, as a public company, “is not gonna kill you because it’s too expensive for them to kill you” — but AI labs’ liability isn’t a cost line, it’s closer to “you set off some nukes.”

Sam then traces how the official AGI narrative has flipped in just a few years: the original pitch from OpenAI and (to a lesser extent) Anthropic was that AGI is so powerful and dangerous that responsible actors must be the ones who own and control it — effectively “we will not start the Manhattan Project, we will,” as Dave puts it. Now that the labs are competing as ordinary businesses among several credible peers rather than a single winner, Sam argues they’re realizing they can’t actually afford the civilizational risk they said they’d shoulder. His closing framing: the existence of LLMs isn’t in question — what’s actually uncertain is whether the capital currently being poured into them will still exist once the liability bill comes due.

Key points

  • Sam revives a Jaron Lanier framework: much of technology centralizes value into one company while radiating risk out onto everyone else; AI breaks this because the liability is effectively unbounded.
  • Sam's analogy: a public company like Disney can't afford to literally kill you because of cost, but an AI lab's liability exposure is closer to 'you set off some nukes' than a normal cost of doing business.
  • Three years ago, the industry narrative (especially from OpenAI) was 'AGI is so dangerous only we can responsibly control it' — effectively claiming sole ownership of an AGI moment.
  • Dave frames the old pitch as 'you will not start the Manhattan Project, we will' — a justification for concentrated control.
  • Sam argues the narrative has now flipped: with several credible competing labs rather than one eventual winner, the 'we must own this dangerous thing' story has quietly become 'we're a normal business competing in a market,' exactly as the liability bill comes due.
  • Sam's closing line: the real open question isn't whether LLMs exist or are useful — it's whether the capital currently being invested in them will still exist once liability gets sorted out.

Where they landed

SamThe AGI-ownership narrative was always really about liability; as labs become ordinary competitors rather than a single anointed winner, they can no longer afford the risk they claimed to responsibly shoulder.
DaveAgrees with the framing, pushes Sam on whether these companies get effectively nationalized before they can go public given the risk profile.
JessicaFlags that Anthropic's S-1 process is already surfacing exactly these unresolved liability questions.

Quotes

“You think your problem is the New York Times hack, the fact you stole the internet... it's like, no, your liability is like you set off some nukes.”— Sam
“AGI is coming. It's so powerful and dangerous that we must be the ones that controlled it and own it, because it would be too dangerous for anyone else to, and at least we're good actors.”— Sam
“You will not start the Manhattan Project, we will.”— Dave
“The question is not whether LLMs exist. It might be a question of whether society exists. It doesn't question whether LLMs exist. The question is whether the capital being put in now exists.”— Sam

Suggested tweets

Sam Lessin's theory on AI liability: Disney can't afford to literally kill you because it's too expensive. AI labs' liability isn't a cost line — it's closer to "you set off some nukes."

Tweet this →

Three years ago the pitch was: AGI is too dangerous for anyone but us to control. Now it's: we're just a normal business. Sam Lessin says that's not evolution, that's the liability bill coming due.

Tweet this →

"The question is not whether LLMs exist. It might be a question of whether society exists... the question is whether the capital being put in now exists." — Sam Lessin

Tweet this →

All topics · Full episode #168 + transcript · Subscribe on YouTube · Spotify