People are relying more on AI assistants like ChatGPT, Claude and Gemini to help them in their daily lives whether it’s to book travel, manage personal finances, navigate a health issue or even get relationship advice.
While these AI assistants are gaining popularity because of how quickly and efficiently they can research and organize information and daily tasks, they are also consuming a large amount of personal data. In fact, to make decisions, AI assistants store voluminous amounts of personal data on each user in a black box – and users don’t have control over it.
What do these AI assistants really know about us?
Consumers may be surprised to learn that when it comes to AI assistants, deleted doesn’t mean gone. For example, OpenAI was ordered by a court to preserve chats that users had deleted, and then to hand over twenty million de-identified conversations in the New York Times case. If a human reviewer at Google touches your Gemini chat, it’s kept for up to three years even after you delete it. About 4,500 shared ChatGPT conversations ended up indexed on Google before OpenAI killed the feature.
Also, ChatGPT has been reading your entire chat history since April 2025, and it is rewriting its memories of you over time. Meta targets ads from your AI chats, and Amazon removed local voice processing from Alexa devices last year so everything goes to the cloud now.
None of these are secrets, they’re all documented. People are surprised anyway, and that surprise is the problem.
People often assume AI assistants only know what they type in the moment. In reality, depending on the product and settings, the AI assistant may have access to a much richer context layer. For example:
Image details: names, emails, tabs, notifications, file paths, locations, timestamps, faces, objects, and background information visible in screenshots or photos.
Location/context signals: approximate location, time zone, language, device context, or workspace metadata.
Behavioral preferences: how formal you like answers, what formats you prefer, what you reject, what you repeatedly ask for, and what kinds of decisions you make.
Inferred traits: likely role, industry, skill level, concerns, taste, intent, and priorities based on your prompts and files.
What are the risks?
There are real risks inherent in AI assistants having your personal data without your control over what is shared. For example, a bug in 2023 showed ChatGPT users other people’s chat titles and partial billing details, and Meta AI’s Discover feed published people’s private chats, medical information, legal trouble, and relationships, all because of a confusing share button. Researchers demonstrated a zero-click attack that made Microsoft’s Copilot leak internal data through one crafted email. In every case the user had no independent control point. The memory lived inside the company that also had the bug, the feed, the training pipeline, and the subpoena.
Other examples include the FTC and DOJ charging Amazon over Alexa’s retention of children’s voice recordings and geolocation data, including allegations that deletion requests were not fully honored. Amazon agreed to pay $25 million and change deletion practices, but people should be concerned that AI assistants are increasingly voice-first, always nearby, and used by children.
FTC cases against GoodRx, Cerebral, Monument, Flo, and others show a recurring pattern: users share health or mental-health information for care, support, or tracking, but data can flow to advertising or analytics platforms. AI assistants create the same risk if emotional, medical, or intimate conversations are reused for ads, personalization, training, or third-party tools.
We believe your context should sit with you, and apps should come ask.
Privacy vs Productivity
So how should consumers think about balancing privacy with sharing personal information to make sure the AI assistants can do their job?
Before even using an AI chatbot or assistant users should check the training defaults. For example, ChatGPT trains on consumer chats unless you opt out. Anthropic moved to the same model in September 2025, with retention up to five years. Gemini’s activity log is on by default and humans review samples of it. Meta AI, in most of the world, gives you no opt-out at all.
What to share and what to strip for personal data
A lot of people are familiar with the black and white dos-and-don’ts, but the grey area is where people get into trouble. Here are a few pointers on what to share and what to strip:
- Share the things that make it useful and are cheap to get wrong: your preferences, how you like to work, what you’re trying to do, the context of your projects. That’s where the magic is anyway.
- Keep out anything with a real failure mode: passwords and one-time codes, government ID numbers, full financials, and other people’s private information.
- You can ask an AI for help with a health question, a contract, a breakup, a work conflict, or a strategy decision, but make sure you strip it down first. Remove names, addresses, account numbers, client names, internal metrics, identifying screenshots, and anything that would hurt someone if forwarded.
Additional best practices
- Use temporary or incognito chats for sensitive one-offs.
- Open the memory settings once a month and prune what’s in there.
- Before you paste something directly into a chat or a very direct question, consider “would this hurt me or someone else if it appeared in the wrong inbox or in front of the wrong person?” If yes, anonymize it first.
- Utilize a tool like an AI passport that automatically flags potentially sensitive context.
We reject the framing that it’s a slider, privacy on one end, useful on the other. The balance is structural. Context should be reviewed before it becomes true, shared with a named app for a named purpose, logged, and revocable.
The bottom line is that people should be in control of their own data; it should be simple for users who want to share non-sensitive information among AI assistants to do that so those apps can be more tailored and productive, without exploiting sensitive, personal information.
