Among the best-performing AI models, 82.2% of test runs passed, while among the worst-performing models, 28.9% of test runs passed, the release said.
These are some of the findings of Visa’s new industry benchmark for AI shopping, Visa Agentic Commerce Benchmark, which is designed to create a roadmap for industry players to deliver better agentic shopping experiences. The test can also be used by consumers, the release said.
Visa Agentic Commerce Benchmark tests AI models’ ability to identify the needs of a consumer based on a conversation, uncover enough information to justify its decision to buy, and opt not to buy if nothing meets the customer’s requirements, per the release.
For its first round of tests, Visa put the AI models through shopping challenges that included buying a TV, a laptop or a personal care appliance, according to the release.
Visa found that the five models with the highest average cost per run passed 67.1% of test runs, while the five least expensive models passed 50.2% of runs, the release said.
The testing also found that 54% of failed runs still ended in the AI making a purchase, even though that purchase did not meet the customer’s identified requirements, or the AI did not uncover all the customer’s requirements, per the release.
“AI agents have the potential to power the next era of commerce, but trust is key to making it work,” Rubail Birwadker, global head of growth at Visa, said in the release. “Visa Agentic Commerce Benchmark is yet another way Visa is working to boost standards across the industry.”
The PYMNTS Intelligence report “Will the 2026 Shopping Season Go Agentic?” found that nearly 132 million U.S. adults have bought a product with AI’s help. However, while AI has won a place in product discovery, consumers still control the purchase.
For all PYMNTS AI coverage, subscribe to the daily AI newsletter.
