By Lynn Räbsamen, CFA | Advisory Board Member, CFA Institute | Author, Artificial Stupelligence
Security researchers just uncovered a quiet new way for attackers to slip unowned code into corporate networks: not by hacking anyone, but by letting AI agents follow instructions nobody bothered to verify. This piece explains, in plain terms, how that happens, why it already caught Fortune 500 firms, and what it means for financial institutions that are handing more and more work to AI.
When the front door opens itself
Picture a mid-sized asset manager. Nothing exotic: client portfolios, a compliance team, and a small engineering group that leans on AI coding assistants to move faster. One afternoon, an engineer asks the assistant to connect a market-data feed from a trusted, well-known vendor. Routine work. The kind nobody documents afterward.
The assistant does what it always does. It reads the vendor’s official setup notes, sees a reference to a small “helper tool,” and installs it. Nothing looks wrong. The instruction came from the vendor’s own website, over a secure connection, in a format built for exactly this purpose.
But that helper tool didn’t belong to the vendor. The link had quietly expired months ago, leaving an empty slot. And last week, a stranger claimed that slot and filled it with their own code. The assistant installed it anyway, with the full access of the engineer who ran it, inside a network that touches client data and trade instructions.
No one clicked a suspicious email. No one fell for a fake invoice. The breach walked through the front door because the firm’s own AI held it open, and the paperwork said it was fine.
In financial services, the loss is rarely the software. The loss is the data, the trades, and the trust, and none of those come with an undo button.
That scenario is hypothetical. The mechanism behind it is not.
Security researchers just proved it works, at scale, inside some of the largest companies on earth.
What the researchers actually found
A stealth startup in Israel scanned 6,214 websites belonging to defense contractors, Big Tech, and Fortune 500 firms. They were reading a humble little file most executives have never heard of, and what they found should make anyone who signs off on a technology budget put down their coffee.
The file is called llms.txt. Websites have long used a file called robots.txt to tell search engines where they may look. The AI era invented a sequel: a plain-text note a website leaves out for AI systems, summarizing what the site does and, helpfully, how to get set up with its software. Think of it as a welcome mat written specifically for robots.
The intention is reasonable. The execution, as usual, is where the money leaks out.
The researchers found 8,265 of these files. Of those, 120 pointed to software packages or web addresses that did not exist. Not “were broken.” Did not exist. The instructions cheerfully told any visiting AI to go install something no one had ever created.
An empty instruction is not a harmless one. It is an open invitation, and on the internet, invitations get answered.
How an empty slot becomes a loaded one
Here is the part that turns a typo into a threat.
When a software package name is unclaimed, anyone can claim it.
The researchers registered a few of the phantom names themselves, purely to see what would happen. Within an hour, a Fortune 500 company’s systems phoned home to their server. Over the following days, a few dozen more followed.
They had done nothing aggressive. They parked at the address the documentation pointed to and waited. The AI agents did the rest, installing code from a stranger because a file told them to.
The researchers could even name the culprits: coding assistants including Anthropic’s Claude, OpenAI’s Codex, and Nous Research’s Hermes, all faithfully following instructions no human had verified.
Now swap the curious researcher for someone with worse intentions. Same empty slot. Same trusting agent. Very different payload. That is precisely the asset-manager scenario above, and it is no longer imaginary.
clerk.com: this already happened
Consider clerk.com, a legitimate and widely used login-services company. Its file contained a one-line instruction telling AI agents to run a command that would fetch and execute a small piece of software.
The slot for that software sat empty. Someone noticed. Someone claimed it. And someone filled it with live malware.
This was not a lab exercise. On a real company’s real website, a real attacker was waiting for AI agents to arrive and follow the directions.
Clerk has since fixed the problem, and it remains unclear whether anyone was actually infected. But the trap worked exactly as designed. It did not need to break into anything. It just needed to be believed.
Why your security tools shrug
The uncomfortable truth for finance and operations leaders is that the usual defenses were built to catch a different crime.
When an AI agent installs one of these packages, everything looks routine. The instruction arrived over a secure connection, from an official website, in a format built for exactly this purpose. To your security software, it looks like a developer running an ordinary setup command from a source your systems already trust. No alarm sounds, because from where the alarm sits, nothing is wrong.
The agent cannot tell the difference between a page it is reading and a command it should obey.
Everything it reads is input, and every input is a potential order.
That single sentence is the entire vulnerability. An AI does not draw the bright line between “information” and “instruction” that a cautious professional draws by reflex. It treats the vendor’s documentation as gospel, and so, worryingly, potentially do the humans supervising it.
The old rule, quietly broken
For decades, technology security rested on a comforting distinction: data is something you look at, code is something you run, and the two do not mix without permission. AI agents have quietly erased that line.
Anything an agent can read is now something it might do.
How did the bad instructions get there in the first place? Some were old human typos, written long before AI arrived and left to rot. Others, the researchers suspect, were written by earlier AI systems that hallucinated a package name or could not tell a good instruction from a bad one. The machines now generate the documentation the next machines will obey. It is a closed loop with no adult in the room.
Artificial credulity
We were promised artificial intelligence. What we frequently get is artificial credulity: a tool that reads everything, believes most of it, and asks permission for none of it. The 227 phantom commands the researchers found were not a glitch in the system. They were the system, working exactly as designed, trusting exactly as instructed.
The good news is that this failure is boring, which means it is fixable. Verify what your agents are allowed to install. Question the documentation, even the official kind. And remember that “it came from a trusted source” is precisely the sentence every confidence trick has ever relied on.
Your AI will follow the instructions. The only question worth asking is who wrote them.
This article was partially drafted by AI and reviewed by a human.
For more insights about what AI can or cannot do, check out my book “Artificial Stupelligence: The Hilarious Truth About AI”.
Subscribe here to be the first to receive my insights.
