.svg)
.svg)
Disclaimer: I'm a Tech Founder who is actively building an AI platform in the field of observability and marketing. This article represents my personal view and direct experience and doesn’t represent my organization.
A couple of weeks ago, an essay went around about a woman who found her boyfriend's ChatGPT history while he slept on her shoulder. The chat was titled "relationship issues and uncertainty." She read it. She read all of it. She read his clinical, ranked, unedited account of why he might not be in love with her, why her body was a problem, why the cats were a problem, why she was a problem. She left him that night.
You can read the essay as a story about modern dating, about privacy, about how it feels to be rendered in someone else's unvarnished language. It is all three. But the part I cannot stop thinking about is much smaller. He did not write any of that in a journal. He did not say it to a friend, a therapist, or a priest. He typed it in clean prose into a model run by a private company, and it now sits on a server somewhere, with no legal protection of any kind, available to a subpoena.
That is the conversation we should be having about AI. It is not the one we are having, at least not enough.
The internet privacy debate of the last twenty years was a debate about metadata. Where you went, what you bought, who you knew, what you paused on. Platforms inferred your politics from your zip code, your sexuality from your watch history, your pregnancy from your shopping cart. The fight was real and we mostly lost it, but the thing being protected was at least well defined: the right not to be reconstructed from the exhaust you left behind without meaning to.
What we are doing now is not that. It is something we have never done at this scale in human history. We are typing, in full sentences, the things we used to keep in our heads.
Six percent of conversations on Claude are people asking the model what to do in their personal lives, according to research Anthropic published this week. Sam Altman, who runs a platform with nearly a billion active users, on a podcast last summer, put it more plainly than most people in his position would: people are using these products as a therapist, a life coach, a confidant, asking what they should do about relationship problems and the most personal things in their lives. He then said the part that should have made more news than it did. There is no legal privilege for any of it. If a court asks for those conversations, the company has to hand them over.
Sit with that. When you talk to a therapist, a doctor, a priest, or a lawyer, the law recognizes that the relationship requires confidentiality to function. Privilege exists because society decided, a long time ago, that some kinds of candor are too important to be discoverable.
When you talk to an AI model about your marriage, your eating disorder, your business partner, your suicidal ideation, the medication you are quietly going off, the affair you are considering, the diagnosis you have not told anyone about, none of that exists. You are not having a privileged conversation. You are creating a corporate record. It can be subpoenaed in civil litigation, requested through government surveillance authorities, exposed in a data breach, or accessed by employees of the company under whatever access controls are in place that quarter.
This is not theoretical. OpenAI is currently fighting a court order, in its lawsuit with the New York Times, that would force it to retain the chat logs of hundreds of millions of users it had previously promised to delete. The company is making the right argument. The fact that the argument has to be made at all is the problem.
The data is not the same kind of data. A search history is a list of questions you asked an index. A conversation with a model is closer to a transcript of you thinking. People do not type into ChatGPT or Claude the way they type into Google. They write in paragraphs. They explain backstory. They name people. They argue with the model when it pushes back, which means the most honest version of their reasoning, the part they would never write down anywhere else, ends up on the page. The model is engineered to receive it. It is empathetic, patient, never tired, available at three in the morning. The whole product is designed to make you say more, and then say it more clearly.
The legal regime is built for an earlier internet. HIPAA covers health information held by covered entities, which a general-purpose chatbot is not. Attorney-client privilege requires an attorney. Therapist-patient privilege requires a licensed therapist. The Fourth Amendment, under the third-party doctrine, has historically held that information you voluntarily share with a third party gets weaker constitutional protection. That is the doctrine that let the government read your bank records and your phone metadata in earlier decades. Apply it to the contents of your AI conversations and the result is a legal twilight zone in which the most intimate written record of a person's interior life sits on a private company's servers with fewer protections than the same person's text messages, emails, or doctor's notes.
Now layer on ads. On February 9, 2026, OpenAI began running ads in ChatGPT for users on the free and Go tiers. Sponsored placements at the bottom of answers, contextual to whatever you were just talking about. The company has been careful, so far. Ads do not influence the answer itself. Conversations are not sold to advertisers. Ads are not served on health, mental health, or political topics. These are the right guardrails for a launch.
They are also the high-water mark, not the floor. Internal projections shared with investors put advertising at $46 billion in revenue by 2030. That is not a number you reach by serving careful, contextual placements at the bottom of answers about CRM software. That is a number you reach by gradually expanding what counts as targetable, what counts as a sponsored answer, what counts as personalization. The same trajectory we have already watched on every other ad-supported platform that ever existed. The pattern is so well established it does not need a name. Free product, then ads, then more ads, then ads inside the thing you came for. Trust degrades on a curve and the curve does not reverse.
What will be targetable, eventually, is everything. Not because anyone in particular wants it to be, but because the unit economics of free or relatively cheap AI demand it and competitors will be doing it. The most intimate written record of a person's life, processed by a model that already understands their fears, their relationships, their financial anxieties, their medical situation, their political affiliations, their parenting struggles, will be sitting next to an ad inventory system with margins that depend on relevance and conversions. Imagine an insurer with access to that signal. Imagine a foreign adversary with access to that signal. We do not need to imagine a data breach with access to that signal, because there will be one, and then another.
This is where the conversation has to widen, because the privacy story and the democracy story are the same story.
We already ran this experiment once. Social media was sold to us as a connection technology, and it became, somewhere along the way, the substrate of our public life. We learned what that meant in stages. We learned about filter bubbles. We learned about algorithmic amplification of outrage. We learned, in 2018, that political behavior could be modeled and targeted at the individual psychographic level using data people had handed over for free. Cambridge Analytica was not a scandal because the data was unusual. It was a scandal because it was the moment we understood what the data was for. Democracies on every continent have been working through the consequences ever since, and the consequences are not finished. Polarization, demagoguery, the collapse of a shared factual baseline, the slow erosion of the idea that the other half of the country is operating in good faith. We did not lose democracy to social media. We just made it considerably harder to run one.
AI is not social media with a chatbot bolted on. It is a different category, and it is going to test democracies in ways that the social media era did not.
Start with the persuasion mechanics. An Instagram ad is a billboard you walk past. An AI conversation is a pitch tailored to you in real time, in your own conversational rhythm, responsive to your own pushback, with knowledge of whatever you have already told it about your life. The unit of persuasion has shifted from the message to the dialogue. There has never been a persuasion technology like this in human history. The closest analogue is having an unlimited supply of charismatic, infinitely patient lobbyists, each one assigned to a single voter, each one armed with that voter's medical history, marital troubles, and financial fears. We do not yet have a serious policy conversation about what that means at scale. We had a policy conversation about thirty-second TV spots. We had one about Russian troll farms buying $100,000 in Facebook ads. The next conversation is about a billion personalized dialogues a day, and we have not started it.
Then there is the framing problem. When you Google something, you know you are being shown links, ranked by an algorithm, paid for by advertisers. The frame is visible, even if the mechanics are not. When you ask a model, the answer arrives as a single fluent paragraph of apparent fact recommended actions. No clear attribution of sources. No competing voices. Just one synthesized voice, in the register of an authoritative friend or personal agent. Whatever the model includes, omits, or quietly weights becomes, for the user, the shape of the world.
The opportunity for soft influence is enormous, and almost entirely invisible. This is a problem even when no one is doing it on purpose. It is a different problem entirely once someone is.
Then surveillance exacerbates the problem. A government that can subpoena AI conversations has a tool no government in history has had. Not the Stasi, not the KGB, not any modern surveillance state. They had to install informants and tap phones, and they still only got the spoken word, fragmentarily, with effort. A future administration with a hostile attorney general and a compliant judiciary can simply ask. The data already exists, in clean prose, organized by user, indexed and searchable. The infrastructure of authoritarianism has historically been the hard part. We are casually building it as a side effect of free chatbots.
And then there is the capture problem, which is the one I think about most. Three or four labs will end up controlling the substrate through which a billion people think through their decisions. Whether to take the job. How to vote. Whether to leave the marriage. Whether the protest is worth attending. Whether the candidate is serious. Whether the article is true. Whatever editorial slant those models develop, intentionally or as drift, by training data or by safety choices or by quiet pressure from whoever happens to be in power that year, will be the slant of the entire civic discourse downstream. The First Amendment was written when "the press" meant thousands of competing pamphleteers. It does not have a clean answer for what happens when the substrate of thought itself runs through a handful of corporations, each subject to its own commercial pressures, regulatory threats, and government relationships.
Free societies depend on a private interior and an open public square. Social media degraded the public square. AI is now reaching for the private interior. Both at the same time, by the same kind of company, under the same regulatory vacuum, on a faster timeline than the last one. We can run this experiment a second time. But we should at least be honest, this time, about the experiment we are running.
Amodei, CEO of Anthropic stated his concerns clearly in his most recent Essay, the adolescence of technology. “To be clear, I think there’s a decent chance we eventually reach a point where much more significant action is warranted,” Amodei said. I would argue we are already here, in fact we have been here for a while now and we don’t want to react when it’s too late already. While some AI labs are more responsible than others or more transparent than others or do PR better than others - regardless this is an industry wide problem and a society wide risk.
The honest answer is nobody, at least not on a timeline that matches the velocity of adoption. Congress has not produced a baseline federal privacy law in twenty-five years of trying. The agencies most likely to act, the FTC and HHS, are working with statutes written before the iPhone. The judiciary will resolve the most acute cases over the next decade, one painful precedent at a time. The labs themselves are split. Some, including the one that builds the model helping me edit this sentence, have taken meaningfully more user-protective positions than others, but no lab can unilaterally create a privilege that does not exist in law.
I’m a “tech bro” founding an AI startup myself and I’m not here with a doomsday argument.
The tools are incredibly useful. People are using them because they work. The Anthropic study that surfaced the six percent number is exactly the kind of research the field needs more of, not less. The productivity and access gains are real, especially for people who could never afford the human professional in the first place.
What I am here to say is that we have not yet reckoned, as a society, with what it means that the most candid record of who we are now lives on infrastructure that was not designed to hold it and is not legally required to protect it.
The boyfriend in that essay typed his uncertainty into a system that felt private and was not. The woman read it and her relationship ended that night. Multiply that by a billion conversations a day and you have not a relationship problem. You have a civic one.
The interesting question is not whether she should have read the chat. The interesting question is why the chat existed in a form that could be read, retained, subpoenaed, breached, sold, served against, or quietly fed back into the model that built the next ad. And why, at no point in the design of any of this, was anyone forced to ask whether the legal regime governing it had any idea what was happening yet.
We are telling AI things we have never told anyone. Nobody is coming to fix that for us. The least we can do, for now, is be honest about the world we have walked into.
