AI May No Longer Require Big Data Centers to Scale

For the past several years, the AI boom has been inseparable from a race in cloud capacity. Training large models and running inference at scale drove unprecedented capital expenditures across hyperscalers, reinforcing the idea that bigger models required bigger data centers. A growing body of research is now challenging that assumption, arguing that the infrastructure requirements of artificial intelligence have been shaped more by early architectural choices than by unavoidable technical constraints.

A recent study from Switzerland-based tech university EPFL argues that while frontier model training remains computationally intensive, many operational AI systems can be deployed without centralized hyperscale facilities. Instead, these systems can distribute workloads across existing machines, regional servers or edge environments, reducing dependency on large, centralized clusters.

How AI Became Locked Into the Data Center Model

Training large language models requires vast amounts of parallel compute, high-bandwidth networking and access to specialized accelerators. Those requirements naturally favored centralized environments optimized for scale and efficiency.

As generative AI moved from research labs into commercial deployment, inference workloads followed the same path. Enterprises adopted cloud-based AI services because they were available, scalable and tightly integrated into existing software stacks. Over time, centralized infrastructure became the default assumption for nearly all artificial intelligence use cases, regardless of their actual computational needs.

Cloud providers reinforced this trajectory through aggressive investment. According to PYMNTS reporting, OpenAI forecasts that AI-related spending will reach $115 billion through 2029, driven largely by infrastructure investments tied to training and deploying models at scale.

That spending wave has shaped enterprise expectations. AI adoption became synonymous with rising cloud bills, long-term infrastructure commitments and exposure to fluctuating pricing for compute and inference. The assumption that artificial intelligence workloads must live in hyperscale data centers went largely unchallenged.

Distributed AI and Everyday Business Workloads

The research highlights a growing mismatch between AI infrastructure and real-world enterprise use cases. These systems often rely on smaller models, repeated inference, and localized data rather than continuous access to massive, centralized models. As PYMNTS has reported, Nvidia concluded that small-language-models (SLMs) could perform 70% to 80% of enterprise tasks, leaving the most complex reasoning to large-scale systems. That two-tier structure, small for volume, large for complexity, is emerging as the most cost-effective way to operationalize AI.

By distributing computation across ordinary machines, organizations can run these workloads closer to where data is generated and decisions are made. This reduces latency, improves resilience, and lowers dependency on centralized cloud services. The approach borrows from earlier distributed computing models, updated with modern orchestration, scheduling and machine learning frameworks.

What This Means for Cloud Economics and Enterprise Strategy

If distributed AI architectures gain broader adoption, the implications extend beyond technology design. Cloud providers have built pricing and revenue models around centralized compute consumption, particularly for inference workloads that scale with usage. Shifting a portion of those workloads away from hyperscale environments could alter demand patterns across the cloud stack.

For enterprises, the shift offers greater control. Running artificial intelligence workloads on ordinary machines allows organizations to align infrastructure spending with actual operational needs rather than peak capacity assumptions. It also reduces exposure to cloud pricing volatility and capacity bottlenecks during periods of high demand as chief financial officers increasingly seek returns from AI.

The research also intersects with concerns around energy consumption and sustainability. Hyperscale data centers face increasing scrutiny over power usage and environmental impact. According to the International Energy Agency (IEA) , energy demand for data centers has grown 12% in the last five years. Making better use of existing, distributed infrastructure could ease some of those pressures without sacrificing AI capability.

None of this eliminates the role of large data centers. Frontier model training and certain high-intensity workloads will remain anchored to centralized infrastructure. But the research reframes hyperscale facilities as specialized assets rather than a universal foundation for all AI activity.

For all PYMNTS AI coverage, subscribe to the daily AI Newsletter.