This summer, investor Katie Jacobs Stanton said her AI assistant, Instinct, sent an email on her behalf without consulting her first, according to TechCrunch. She described the email as innocuous. The breach of trust was not.
Next time, it could be the email that costs someone a job, a marriage or a lawsuit.
The appeal of personal AI agents is compelling. An assistant that can identify priority emails and draft responses, remember your mother-in-law’s birthday and buy a present based on your correspondence is enticing. For people stretched by demanding jobs, caregiving, or a disability, that help can be transformative. But Stanton’s experience raises an important question: when we give an assistant access to our lives, how do we retain control? Assistants such as Instinct and Meta’s Muse act for us rather than simply answering questions or making recommendations. A mistaken purchase may be refundable. An intimate disclosure cannot be taken back.
The harms are already emerging. Forbes reported that when investor Jason Yeh asked Instinct to find open dinner reservations, it booked a table instead, leaving him with a $200 cancellation fee he never approved. TechCrunch reported that one tester couldn’t get his data deleted, and another received an inbox summary hours after disconnecting her account. And in September, security researcher Patrick Wardle exposed a flaw in Muse’s Mac app that could let malware already on a computer hijack the assistant, which Meta quickly patched.
The problem now reaches our bank accounts, too. Last week, major financial institutions reported that personal AI agents were operating inside online accounts. In one case, an agent called a bank without authorization to file a complaint. JPMorgan Chase CEO Jamie Dimon said the bank would not allow unvetted “outside agents to come in inside the company”, and Schwab’s CEO said they’re actively blocking unauthorized agents. White House adviser David Sacks countered that data was shared willingly and these are “in no way cyberattacks.” Each side has a point, which is precisely the difficulty. We have not yet settled on what authorization means when the party acting is software.
The failures are also hard to see. A public record launched this week by the security firm Manifold counts 59 incidents since July 2025, forty-two of them disclosed in the past four months alone. The median incident took 70 days to come to light, and only 11 of the 59 were detected by a security control.
At EqualAI, we have long maintained that good AI governance is good business because trust will ultimately determine whether people adopt these tools at all. We recently convened senior executives who develop and deploy agents to work through a simulated crisis. The exercise exposed the same gaps repeatedly. In our mock company, without sufficient AI governance, audit trails were too thin to reconstruct what an agent had done. Vendor contracts were silent on who answers when an agent errs. Responsibility for oversight was assumed rather than assigned. These gaps were most acute where speed had outrun precaution. Those gaps shaped the roadmap that came out of the exercise.
The legal landscape is moving in the same direction. California law now prevents companies from avoiding civil liability by arguing that the AI acted autonomously. Senators Hawley and Murphy proposed extending federal anti-hacking law to developers and operators whose agents recklessly cause damage.
Those same principles should apply to the assistants people are bringing into their homes. For consumers, they translate into four commitments.
Make permission specific. An assistant should receive only the access a task requires. Permission to read your inbox should not automatically become permission to send emails in your name. Before an assistant spends money, changes who can access an account or shares sensitive data, there should be safeguards in place, which are known to users.
Assign responsibility in advance. If an assistant wrongly cancels your flight, the remedy must be clear. Companies should determine in advance of deployment who investigates an assistant’s unauthorized actions, who can halt them and how users are made whole.
Test with users’ perspective. Independent testing should confirm that assistants ask before acting, respect their limits and fully disconnect when access is revoked. Because models change constantly, testing must be continuous. Wardle’s discovery was valuable. Work like his should be the norm.
Report and study failures. Aviation became one of the safest forms of travel in part because pilots and airlines report near misses and investigators learn from them. Companies should be expected to identify an “AI incident”, preserve the records, notify those affected and correct the underlying cause, while policymakers can establish reporting requirements for serious cases.
Consumers also have a role. They need to understand permissions they’ve granted and how to revoke its access. But the greater responsibility rests with companies selling the tools to define their limits and set reasonable expectations. We shouldn’t need a cybersecurity background to use these tools safely.
We board planes without understanding how they fly because industry and regulators answer for our safety. We need a similar framework for AI assistants. If companies want our inboxes, accounts and errands, they must earn our trust by clarifying what we’re authorizing, how to regain control, and answer for mistakes their assistants make in our name.
The opinions expressed in Fortune.com commentary pieces are solely the views of their authors and do not necessarily reflect the opinions and beliefs of Fortune.