November 19, 2026 · 7 min read
AI Trusts the User
An agent can build a product in an afternoon. Deciding whom it shouldn't trust still takes a person.
Essay
Vibr was live on this site under four hours after its first commit. I've written about that speed, and I'm still proud of it. But the Vibr from that afternoon trusted everyone. It believed every request was what it looked like, every file was what it claimed to be, and every page asking it to do work was a page the designer meant to send.
That isn't a flaw in any particular model. It's the default. An agent building a product does what it's asked, for the person asking, and assumes good faith all the way down. AI trusts the user. Someone has to decide where it shouldn't.
This essay is about that someone, and why the job is more creative than it sounds.
Fast builds inherit good faith
When I describe a feature to a coding agent, I describe how it should work for the people I'm building it for: the designer who wants a prototype, the person bringing a Figma file, the agent connecting through Vibr's MCP server to build a screen. The agent builds exactly that, and builds it well. What it doesn't do on its own is imagine the person who wants something else.
That person isn't in the prompt. They're the one who words a request to slip past the content check, uploads a document that's really a zip bomb, or opens a page that quietly tries to drive someone else's Vibr window. More and more, they aren't a person at all. They're another agent, following instructions hidden in a web page, a file, or a design it was asked to read.
A harness can write a secure version of any feature you can describe. The hard part is describing the threat, and that takes imagination the prompt never had.
Every input is someone's idea
Making Vibr safe meant going back through it one door at a time and asking who else might walk through. A few of the answers:
- The draft itself. Vibr's drafts are code written by a model, so I treat them like code from a stranger. They run in a sandboxed frame with a strict content policy, never on the app's own origin, and a frame that navigates itself away is parked. The key that pays for the model never reaches the frame.
- The files people bring. Documents with macros or zip bombs are refused. A Figma file is treated as untrusted: every value is rebuilt and size-limited, pictures must be real images of a sane size and are redrawn, and its text passes the same content check as a typed request.
- The packages a draft loads. Before a custom npm package loads, Vibr pins it to a version that has been out for at least two weeks, refuses known malware, and flags lookalike names.
- Other software on the computer. Vibr's MCP server lets an agent build prototypes for you. Any page on the same computer could try to do the same, so the Vibr window asks first. Nothing runs until the designer selects Allow, and the connection checks a token, the origin, and the host.
- The model's own output. I ran prompt injection tests against Vibr's real instructions. The model refused every attempt to reveal its system prompt, but in one of four long hijack attempts it printed the attacker's text anyway. In Vibr, the person injecting is the person reading the result, so the harm stays small. I only know that because I went looking.
None of those came from the first afternoon. Each one came from sitting with the product and asking, specifically, how it could be turned against the people it's for.
Security is part of the experience
It's tempting to treat this work as a tax on the real product. I think that has it backwards. Each of those decisions is a design decision, and most of them show up in the experience.
The Allow step is the clearest example. The first version of Vibr's MCP connection had vibr.art reach into the designer's computer. Chrome asked people to allow local network access, and Safari blocked it outright. I turned the connection around: the server on the computer opens a Vibr window and hands it work inside the browser. That fixed Safari. Asking the designer to Allow before anything runs fixed the trust problem. One change made Vibr work in more browsers and made it safer, and the designer sees one clear question instead of a browser warning they don't understand.
Keys work the same way. Vibr keeps a designer's model key in their browser and sends it only to their model provider. That's a security property, and it's also the sentence that lets someone paste a key without hesitating.
The costs are real too, and they're experience costs. Vibr's first check is a short list of unambiguous words, and this week I found it refusing someone who asked for "gore-free art", and a label for an "XXX hot habanero" sauce. I fixed both the same day. Every guard has false positives, and the person who hits one didn't do anything wrong. Getting that balance right is the same craft as any other interaction: understand the people, find the edge cases, and decide what the product says when it says no.
A second opinion
A word list fails the other way too. It only knows its words, and a request can ask for something Vibr won't build without using any of them. That's why Vibr's planning step screens every request again with the designer's own model.
This week I tested a different kind of check: Jev, a fast classification model from TypeSafe. I wrote 123 test requests. Ninety should build, including a pediatric clinic, a horror game, and that hot sauce shop. Thirty-three shouldn't: some in plain words, some subtle, some in other languages, some disguised. Vibr's word list, after the fix, built all 90 and refused 18 of the 33. Jev got all 123 right, in a median of about a tenth of a second, for about two cents per thousand requests. The requests it was least sure about were the right ones to be unsure about: a nudist resort, a sex shop, and a strip club's opening hours. It let all three through.
That was enough to ship it, but not for free. Vibr had promised that a designer's words go only to the model provider they chose, and a hosted check means a second company reads every request. So the check runs through one small route on Vibr's own server. It holds the key, logs nothing it reads, and answers within a second and a half or steps aside, leaving the planning step to screen the request as before. The privacy page now says all of that, in each of Vibr's 12 languages, because a promise that changes quietly isn't a promise.
Even that needed a suspicious person. In testing, when the bot check guarding the route couldn't load its script, the request waited forever, and the draft waited with it. An ad blocker could do that to anyone. Now the check has its own timer.
And a test set I wrote myself is exactly the kind of test that flatters a model, so the next step is more requests I didn't think of.
Where the time goes
The build took an afternoon. The hardening took much longer: scanners for leaked secrets and vulnerable dependencies, static analysis, a security audit by agents following a published method, injection tests against the prompts, and then the slow part, reading every finding and deciding what was real. The static analysis findings all turned out to be noise. The audit found real gaps, and they became fixes: rate limits on image search, a cap on the error log, and a stricter rule for what a draft may import.
Agents helped with every step of that. They ran the scanners, drafted the fixes, and checked each other's audits. But they didn't decide which findings mattered, which limits to accept, or what a refusal should say. Some paths I can narrow but not close, and I've written those down as known limits instead of pretending otherwise.
That judgment, plus habits like pinning dependencies, never logging anything shaped like a key, and treating generated code as untrusted, is what secure coding looks like when an agent writes most of the code.
Distrust, designed
None of this makes me less optimistic about building with agents. It makes the job clearer. An agent will build the product you describe, quickly and well, and it will trust whoever shows up. The person's job is to imagine who else will show up, the curious, the careless, the malicious, and the automated, and to design for each of them, so the people the product is for never have to think about it.
AI trusts the user. That's what makes it so useful. Deciding where that trust ends is still our job.
Dylan