Such a distributed computing network makes sense in that “computation for AI inference can and should be distributed at the ‘edge,’ deployed on smaller platforms closer to population centers and users,” said Benjamin Lee, a computer architect and engineer at the University of Pennsylvania, in correspondence with Ars. “The strategy could impose much smaller impacts on the grid because inference requires a few GPUs, unlike training which requires thousands of them working in concert,” he said.
However, AI inference tasks can be as varied as document question-and-answer, software code generation, and multi-turn conversations—each with different computational requirements and performance expectations, Lee cautioned. So it will be important to ensure that individual compute nodes can deliver the performance necessary for each task, along with maintaining network connectivity among the nodes.
Lee also questioned whether it’s necessary to downsize data centers to the “granularity of a few GPUs” in order to reduce their burden on the power grid. He speculated that deploying conventional 20-megawatt data centers instead of 1-gigawatt hyperscale data centers could prove similarly beneficial.
Credit:
SPAN
Then there is the issue of security. XFRA nodes spread across suburbia could become more vulnerable to certain data security threats than centralized data centers. “Many side-channel attacks require physical proximity to the machine, which data centers can guard against,” Lee said. “Distributed GPUs in individual homes are much more difficult to protect.”
Thieves may also see XFRA nodes alongside houses as a tempting target, given that the Nvidia GPUs within can each sell for around $10,000. Several comment threads on Reddit have already speculated on that possibility, with some commenters suggesting they would feel tempted to secure such compute resources for themselves as residents. “Of course, there is the risk of losing the actual hardware itself to theft,” Lee said.
Any potential benefits and complications will become more evident during SPAN’s pilot deployment phase. But at a time when Silicon Valley is currently abuzz about orbital data centers and ocean-going AI data centers, data center nodes embedded in suburbia may stand on more solid footing—at least until homeowner associations catch wind of them.