How To Even Begin Measuring A Chatbot?

Bots seem to be everywhere, but are they doing what they’re supposed to be doing? How do we measure their efficiency and all the data to see if they’re really the best way of the future?

San Francisco-based Bot Metrics does just that — metrics and analysis for chatbots. And this week, it scored some funding to get those cogs moving even faster for its services and its customer base.

This week’s Chatbot Tracker touched on the hospitality space, where a bot will take your order, make you a restaurant or hotel reservation and even make sure all your travel plans go smoother.

“Existing metrics tend to favor feed-based consumption rather than chat. Everyone can make up their own rules for measuring monthly active users,” Kik’s head of data told TechCrunch back in June.

Exactly right. The challenge is to measure — though tough as it might be — how well the bots do on efficiency and truly compare that to what its human counterpart was probably doing before the bot came along.

Of course, some think the chatbot concept is just a flash in the pan; others seem to be on board for the long haul — namely, Bot Metrics.

“Every day, a new enterprise company taking about bots,” he said. “We went through a hype cycle, got into the disillusionment trough, and now, people are starting to understand it won’t do everything, but the interface is good for many things,” said Bot Metrics Cofounder Sandeep Chivukula.