As organizations race to accelerate AI adoption, the challenge has shifted from acquiring enough computing power to managing a different set of constraints. Access to electricity, cooling capacity, available land, connectivity, and the ability to maintain increasingly complex systems are now driving the conversation around AI infrastructure.
Utility providers are struggling to keep pace. In many regions, new data center projects face lengthy waits for grid connections, while local communities raise concerns about energy consumption, water usage, and environmental impact. These pressures should prompt the industry to ask a different question: if traditional data center campuses cannot scale quickly enough, where should AI infrastructure live next?
Bringing compute closer to where it’s needed
Organizations are exploring distributed infrastructure as one of the more practical concepts on the table. This model places smaller computing systems across residential or small business locations, bringing processing closer to where data is generated while also physically distributing the electrical demand. However, the industry must address several challenges before distributed residential infrastructure can reach a meaningful scale.
The first is the business model. Homeowners and small businesses would need clear incentives to host computing equipment. Beyond financial considerations, organizations would need to address operational questions, including who manages the hardware, which may vary according to local laws and zoning regulations, how teams maintain systems, and how they handle security across thousands of locations.
Connectivity is another important consideration. For distributed infrastructure to work effectively, individual systems must communicate reliably with each other and with larger cloud environments. Bandwidth limitations could become a significant factor in determining which workloads suit this approach. For example, would the bandwidth used by a server at a home degrade the homeowner’s contracted internet bandwidth?
For organizations exploring distributed AI today, the key lesson is to evaluate workloads carefully. Not every AI application requires local processing, but applications that depend on low latency or localized data may benefit from a more distributed model.
Looking below the surface
Another concept gaining attention is the use of floating or underwater data centers. Marine environments offer naturally cooler conditions than land-based facilities and could reduce cooling requirements. However, moving infrastructure into or near bodies of water does not eliminate engineering challenges. Instead, it introduces new ones.
Maintenance remains among the biggest obstacles. Engineers build traditional data centers so technicians can access and maintain or replace hardware when needed. An underwater facility requires a different operational model. Organizations would need reliable remote monitoring capabilities and carefully designed systems that minimize the need for physical intervention.
Cooling design would also require significant engineering. While surrounding water could provide thermal advantages, systems would need to account for factors such as water filtration, fluid movement, and changing environmental conditions. A cooling system for the servers would need to be similar to liquid cooling options used today. As with land-based facilities, operators would need to understand how efficiency changes over time and under different conditions.
The lesson from marine infrastructure is that organizations must weigh cooling efficiency alongside operational practicality. A deployment model is only successful if organizations can maintain and manage it throughout its lifecycle.
Reaching beyond earth
The most ambitious concept places data centers in orbit. Space-based infrastructure has attracted interest because it could provide access to continuous solar power while avoiding some of the limitations faced by terrestrial facilities.
However, orbital data centers continue to be a long-term possibility rather than an immediate commercial solution. Significant engineering challenges remain, including radiation protection, connectivity, maintenance, and the cost of transporting significant amounts of hardware into low Earth orbit.
Unlike terrestrial facilities, where technicians can access equipment relatively easily, orbital infrastructure would require systems that operate long-term without regular physical servicing. Hardware would need to withstand radiation exposure, while networking infrastructure would need to support reliable communication despite the distance from traditional internet hubs.
Orbital computing may not become mainstream in the near term, but it shows how far the industry will go to innovate as AI infrastructure requirements continue to grow.
The future of AI infrastructure will be more flexible
Distributed residential infrastructure, floating data centers, and orbital computing each address different challenges, but none represents a simple replacement for traditional data centers. Instead, the future will likely involve a combination of approaches. Large-scale facilities will continue supporting intensive AI workloads, while specialized deployments may emerge for applications that require different approaches to power, cooling, latency, or availability.
For infrastructure leaders, the most important takeaway is that location alone will not determine success. Organizations evaluating future AI deployments should focus on several foundational questions: Can the infrastructure deliver reliable performance within available power constraints? How will teams cool and maintain systems over time? Can organizations monitor and service hardware effectively?
Regardless of whether computing happens on land, at sea, or beyond Earth, success will depend on maintaining efficient, reliable, and connected systems that can support increasingly complex and varied AI workloads.
Vik Malyala is Chief Business Officer at Supermicro, where he leads business development and strategic partnerships with major technology companies worldwide.