What Is Multimodal AI and Why Retailers Should Care

Two breakthroughs in artificial intelligence (AI) model capabilities can help brands understand and serve their customers at a deeper level.

Advancements in natively multimodal AI and extended memory can enhance hyper-personalization, allowing businesses to create experiences for consumers that are relevant and tailored while remembering their choices for future interactions.

Multimodal AI refers to models that can understand and generate responses based on various data types — not just text, but also images, audio, video and code.

Longer memory refers to the ability of AI to retain contextual information over multiple interactions or sessions.

In April, Meta launched Llama 4’s natively multimodal models, following Google, OpenAI, and Anthropic. In the same month, OpenAI announced that ChatGPT can now remember all of a user’s chats. Meanwhile, Perplexity is beta testing its memory feature, to be released in 2025.

“AI that can process text, images, voice, and video, as well as remember customer preferences across months or years, unlocks the kind of deep personalization that customers increasingly expect and that brands have yearned to deliver for years,” Martin Baalam, CEO and co-founder of eCommerce platform Pimberly, told PYMNTS.

For instance, longer-memory AI can recall when a customer researched a holiday in April and resurface relevant offers in August, creating a seamless experience across every touchpoint, Balaam said.

Terra Higginson, principal research director at Info-Tech Research Group, shared this example of using multimodal AI.

“My Arizona backyard desperately needed shade. I didn’t know what to build, but I remembered something I’d seen while driving, so I pulled over, snapped a photo, and uploaded it into a generative AI tool,” Higginson told PYMNTS.

She asked the AI tool to turn the image into an assembly diagram and a shopping list. “Within minutes, I had a fully personalized, practical solution, built around how I think and what I needed,” Higginson said.

“This is what multimodal AI can do for customers: let them interact in the way that suits them best — voice, image, or text — and receive highly specific, contextualized responses. It meets people where they are, in the channel and format they prefer.”

Combined with long-term memory AI, the impact becomes greater, Higginson said. “These systems can remember how a customer felt last time, what they liked or didn’t like, and tailor responses accordingly. … When businesses deploy AI that truly knows who the customer is and how they prefer to interact, they unlock the ability to deliver experiences that feel uniquely personal, at scale.”

Read more: Meta Adds ‘Multimodal’ Models to Its Llama AI Stable

Human-Like Customer Service

Daniel McCarthy, associate professor of marketing at the University of Maryland, College Park, sees longer memory as an enabler of more responsive and human-like customer service.

“The value to firms comes from reliably retrieving (the right context) in a very short timeframe, so it lives inside the model’s window at inference time,” McCarthy told PYMNTS. “That’s what lets an AI agent greet a caller — or make the relevant data available to a human agent — remember the unresolved refund, and suggest the correct next step without the customer having to repeat themselves.”

Beyond call centers, McCarthy sees potential for these systems to scale personalized B2B customer support. Most B2B companies keep national account managers only for their largest clients because deep context doesn’t scale.

“It’s very expensive to have people that know particular accounts as well as key account managers often do,” McCarthy said. But “with the right AI, every account, large or small, could get a digital rep that feels almost as informed as a human key account manager.”

Consumers are eager for personalized offers, too. Consider that a whopping 89% of millennials are interested in receiving personalized offers, according to the PYMNTS Intelligence report, “Personalized Offers Are Powerful — But Too Often Off-Base,” a collaboration with Amazon Web Services.

Read more: Alibaba Cloud Launches Compact, Multimodal AI Model

Business Benefits

The enterprise-level benefits go beyond marketing and customer experience; it helps retain institutional knowledge and build resilience.

“Businesses can develop systems that ‘remember’ important decisions, past results and common issues, creating a memory that stays intact even when employees leave,” Chandrakanth Puligundla, tech lead and data analyst at Albertsons, told PYMNTS.

“These technologies can serve as living archives — a company’s brain that grows smarter over time,” Puligundla added.

In addition, in sectors like aerospace or manufacturing, multimodal AI can combine technician feedback, visual checks, and logs to deliver predictive insights with richer context, Puligundla said.

The ability to reason across modalities and remember context is crucial in compliance-heavy industries, said Jamie Allsop, managing partner, financial services, U.K., at HTEC.

“For financial institutions, that means more accurate risk modeling, proactive compliance monitoring, and customer experiences that feel intelligent, personalized and seamless,” Allsop said.

“The real opportunity lies in deploying AI that not only understands the present moment, but also remembers what came before, and uses it to shape what comes next,” according to Allsop.

Still, this vision of hyper-personalized AI raises privacy concerns. “All of the above use cases will require more customer data collected and readily available,” McCarthy warned.

“Data breaches become more of a risk with all this data floating around,” McCarthy said. “It is also more possible than ever to ‘weaponize’ these large troves of data, given how easy it is to now sort through it.”

That means brands must not only break down data silos and enrich content across channels, but also uphold robust standards for privacy and consent. As AI becomes the new front door to digital engagement, blending personalization with trust will be crucial.

See also: From Buzzwords to Bottom Lines: Understanding the AI Model Types

Read more: OpenAI Unveils ChatGPT With Enhanced User and Interaction Memory

For all PYMNTS AI coverage, subscribe to the daily AI Newsletter.