When Walmart tried to check the tech it uses to monitor prices on Amazon earlier this year, its engineers found something curious — the tech wasn’t working anymore.
Amazon’s blocking secret? Bots.
Walmart — which needs that sensitive pricing data to stay competitive with Amazon — had to spend weeks figuring out a workaround.
And Amazon is good at dual-purposing its bots — using them to seek out competitors’ pricing data and to keep competitors in the dark about their own.
“Benchmarking against Amazon is going to become hard,” said Guru Hariharan, a former Amazon manager who now sells pricing software to retailers as chief executive of Mountain View, California-based Boomerang Commerce.
Amazon, for its part, says that while it knows others are checking out its pricing with bots, it has made no changes to try and lock anyone out.
“Nothing has changed recently in how we manage bots on our site,” an Amazon spokesperson said — before noting, “we prioritize humans over bots as needed.”
Bots can slow down a website, a big motivator for retailers to block them.
Bots on the web serve a variety of purposes — but in eCommerce, they tend to be both a defensive force, keeping other spybots off a retailer’s site, and an offensive strike team out to penetrate competitors.
“CAPTCHA” — typically a distorted string of letters and numbers that humans can read but most bots can’t, is a favored method to shake off bots — but Amazon shies away from the practice because it annoys some customers.
The goal for most bots — according to experts — is to appear as human as possible while they are hunting down data.
“It is an arms race,” said Keith Anderson, a senior vice president at eCommerce analytics firm Profitero, based in Ireland. “Every week or every month, there’s some new approach from both sides.”
In the case of the Amazon lock-out, it is not clear what retailers, other than Walmart, were affected.
Recent tests by Retuers have determined that among U.S. retail chains, Amazon had by far the most sophisticated bot detection in place, both for its home page and for two popular items selected by Reuters because they change price frequently — a De’Longhi coffee maker and a Logitech webcam.
“Amazon has both the competency to detect bot traffic and the wherewithal to do something about it,” said Scott Jacobson, a former Amazon manager and now managing director of Madrona Venture Group. “That isn’t the case for most retailers.”
We’re always on the lookout for opportunities to partner with innovators and disruptors.
Learn More