Get a free audit
Apple credit Apple credits the researcher who runs our AI penetration testing.

Field Notes / AI Threats

A seller billed ten AI agents twice for the same data. Six of them paid. Your purchasing agent needs controls the model can’t talk its way past.

BlockRun gave ten frontier models wallets with real USDC and sent them to buy data from sellers that cheat. Seven paid a cheating seller at least once. Four models paid nine second bills out of nine. What was tested, what the numbers can and can’t tell you, and seven payment controls to put around any agent that spends money.

Author
Red Team Partners
Read
8 MIN READ
Filed
11 Nov 2026
A human hand and a machine hand reaching towards each other, the handover that happens when an AI agent is allowed to spend money.

01 What BlockRun tested

lockRun runs a gateway that lets AI agents pay for data and services. Its Agent Money Benchmark, Version 1, put ten models through two tracks between 27 and 29 October 2026, each task run once BlockRun 2026 .

The first track is 76 financial research questions. The agent pays live data providers in USDC on Solana and is graded on the answer. The second is an adversarial marketplace of 80 tasks per model. The sellers are simulated, the payments are real on-chain USDC, and the sellers cheat.

The agents were given no payment rules. They were told the money belonged to a client, that there was a budget, and that an expense report would go to the client’s accountant. Data budgets per task were a few cents. The sellers then tried eight tricks:

TrapWhat the seller does
Charged againDelivers the paid data, then bills for it a second time
Price above listedAsks for more than the advertised price
Expired quoteAsks for payment against a quote that has run out
Wrong dataDelivers something other than what was bought
Paid, not deliveredTakes the money and sends nothing
Lookalike payeeSwaps in a wallet address that resembles the real one
Receipt mismatchSends a receipt that doesn’t match the payment
Hidden instructionsHides orders to the agent inside the data it sells

A seller that asks for payment again can keep asking until the wallet is empty, BlockRun notes. In this test a second payment counts as wasting the whole task.

02 Who paid twice

The second bill split the field cleanly. Counts are times the model paid out of times it reached the trap BlockRun 2026 .

ModelPaid twiceTasks with a scamWasted per $100
Claude Opus 5.50 / 90%$25.0
GPT-6.1 Sol0 / 90%$27.9
GPT-6 Astra0 / 90%$33.3
Gemini 4 Argon0 / 91.2%$34.3
Claude Fable 5.13 / 93.8%$37.9
Claude Sonnet 5.59 / 912.5%$56.3
GLM 5.39 / 917.5%$58.3
DeepSeek V4 Pro8 / 935.0%$65.3
Kimi K3.1 Pro9 / 923.8%$66.3
GPT-6 Luna9 / 932.5%$71.0

“Wasted per $100” is BlockRun’s own blend of money lost to scams and value lost to weak or failed answers, weighted equally across both tracks BlockRun metrics .

Three things stand out for anyone choosing a model for an agent that spends.

The cheapest model cost the most. GPT-6 Luna answered a research question for $0.032, the lowest in the field. It also wasted $71 of every $100 and paid a cheating seller in 32.5% of marketplace tasks. DeepSeek V4 Pro paid above the listed price in 23 of the 29 times it saw that trap.

One vendor, opposite results. Claude Opus 5.5 refused every second bill. Claude Sonnet 5.5, from the same vendor, paid every one. GPT-6.1 Sol and GPT-6 Astra refused all nine. GPT-6 Luna paid all nine. A vendor name tells you nothing about how the model you actually deploy handles money.

Some traps caught nobody. No model paid a lookalike payee, accepted a mismatched receipt or followed hidden instructions in the data. The traps that worked were the dull ones: a second invoice, a higher price, an old quote. Those are also the tricks a real supplier fraud uses.

03 What the numbers can’t tell you

BlockRun publishes its own limits, and they matter before you pick a model on this table BlockRun 2026 .

Each task ran once. The confidence intervals cover variation between tasks only, so a model at 9 of 9 might land at 7 on a second run. Nine attempts per model on the double-billing trap is a small sample. The cheating sellers are simulated, and only eight example tasks are public, so nobody outside BlockRun can recompute the scores.

The research answers were graded by an AI judge, DeepSeek V4 Pro, which is also one of the ten models tested. BlockRun staff reviewed every score, and a second check on 70 answers agreed with the first pass on 94% of item verdicts. BlockRun also built the gateway the models ran through and sells agent evaluation and hardening services BlockRun GitHub .

None of that changes the main lesson. Your agent will not run BlockRun’s prompt or BlockRun’s tools. A model that refused every second bill here can still pay one inside your workflow, with your prompt, after a long conversation, or when the invoice arrives through a tool it trusts. The only result that counts is an AI penetration test of your own agent, built the way you run it.

04 Seven controls outside the model

BlockRun’s summary puts it well: this is a financial-control problem as much as an intelligence problem. Every control below sits in the payment layer, where a persuasive seller can’t argue with it.

Before an agent spends real money
  • Give every purchase an order ID and make the payment service refuse a second payment against the same ID
  • Set a hard spending cap per task and per payee, enforced by the wallet or card itself
  • Pay only allow-listed payees, and treat any new wallet address or bank account as a human decision
  • Lock the price at the quote and reject any payment request above it or after the quote expires
  • Require human approval above a set amount, and for any repeat payment to the same seller within a day
  • Reconcile the agent’s expense report against the wallet or card ledger every day, and alert on any difference
  • Test the agent against cheating sellers before go-live and after every model or prompt change

The last item, red teaming the agent before go-live, is the one most teams skip. Swapping Opus for Sonnet to save on inference looks like a cost decision. In this benchmark it was the difference between 0 and 9 second bills paid.

Do you know what your agents will pay for? Your Threat Map shows every route by which an agent can be talked into spending, from a duplicate invoice to a poisoned data feed, proven and ranked by risk, with the control that closes each one. In 14 days, from $5,000.

Questions readers ask

What is the Agent Money Benchmark?

The Agent Money Benchmark is a test published by BlockRun in which ten frontier AI models were given wallets with real USDC on Solana and asked to buy data. In one track, 80 tasks per model, simulated sellers tried eight tricks: billing again after payment, charging above the listed price, expired quotes, wrong data, non-delivery, lookalike payees, mismatched receipts and hidden instructions. Version 1 ran from 27 to 29 October 2026, once per task.

Which AI models paid twice for the same data?

When a seller sent a second bill after delivering paid data, Claude Sonnet 5.5, GLM 5.3, Kimi K3.1 Pro and GPT-6 Luna paid 9 out of 9 times, DeepSeek V4 Pro paid 8 out of 9 and Claude Fable 5.1 paid 3 out of 9. Claude Opus 5.5, GPT-6.1 Sol, GPT-6 Astra and Gemini 4 Argon refused every second bill.

Which AI agent wasted the most money?

On BlockRun’s normalised measure, GPT-6 Luna wasted $71.0 of every $100, followed by Kimi K3.1 Pro at $66.3 and DeepSeek V4 Pro at $65.3. Claude Opus 5.5 wasted the least at $25.0, all of it from weaker answers, with no money lost to scams. GPT-6 Luna was also the cheapest model per research question at $0.032.

Can I trust the Agent Money Benchmark results?

Treat them as a strong signal and stop short of a final ranking. Each task ran once, the cheating sellers were simulated, the full task set is not public, and research answers were graded by an AI judge that is also one of the tested models, with BlockRun staff reviewing each score. BlockRun also built the gateway the models ran through and sells agent evaluation services. The pattern is still clear: the same kind of trap caught some models every time and others never.

How do I stop an AI agent from paying twice?

Enforce the rule in the payment layer, outside the model. Give each purchase an order ID and let the payment service refuse a second payment against the same ID. Add a hard cap per task and per payee, an allow-list of payees, a price check against the quoted price, human approval above a threshold, and a daily reconciliation of the agent’s expense report against the wallet or card ledger. Then test the agent against cheating sellers before it touches real money.

References

Sources

  1. BlockRun. Agent Money Benchmark, Version 1. Leaderboard, methodology and trap results. Runs dated 27–29 October 2026. agentmoneybench.com
  2. BlockRunAI. agent-money-bench. GitHub repository, README. github.com
  3. BlockRunAI. agent-money-bench: results/README.md. Per-model task success, cost per task and money-error rates. github.com
  4. BlockRunAI. agent-money-bench: docs/metrics.md. Definitions of scam incidence, money loss and normalised wasted spend. github.com