When an AI model begins training, it engages in a cycle of iterative learning, using new datasets to reinforce prior context until it converges, at which point the model achieves stable inference, and feeding it new data won’t significantly improve performance or output accuracy.
AI training is supposed to reflect the deflationary nature of technology, where advancements in function lead to reductions in cost. As models continue to develop, the expectation is that their learning environments will become more stable, leading to more efficient output generation and lower data access costs.
However, while training environments may become cheaper to manage, they may not become more affordable to power. As AI models advance in their inference capabilities, cloud providers are unsustainably hyperscaling computing resources to power AI workloads and maintain the speeds at which current flagship models execute computations.
More advanced AI will require more computational power, and data centers cannot keep outputting energy at the rates they have been to run AI stacks. More energy-efficient hardware is essential, and we need to dynamically adopt better solutions amid a growing, computationally complex AI landscape.
Understanding current AI infrastructure and its energy needs can provide context for where improvements are needed and where more energy-efficient hardware can be introduced in the AI supply chain. Let’s dive in!
ANI Infrastructure
Artificial Narrow Intelligence (ANI) is the only form of AI supported by physical infrastructure, as it is the only type of AI that exists. Generative models that aim to create new data informed by past learnings, reactive models that simply respond to immediate inputs, and limited-memory models that classify data based on past learnings are all subcategories of ANI.
ANI workloads require extensive computational power and specialized hardware to run, and the costs of these resources have increased as ANI models are trained on more comprehensive and complex datasets.
Processing Units
A fundamental part of the hardware that powers AI development is the processing unit, a component in a computer that executes instructions from programs. There are types of processing units, essential to ANI infrastructure:
CPUs are the primary processing units of computers. Designed to handle sequential tasks and small-scale computations, such as application control flows. CPUs are well-suited for executing instructions and performing straightforward calculations efficiently. They are often used in AI models that don’t require large-scale processing or complex computations (i.e., models with few parameters).
GPUs come into play to handle the more intensive AI workloads because they are designed for graphics rendering, which requires large-scale parallel processing—the execution of multiple advanced computations simultaneously. GPUs are a go-to for training deep learning models because they can handle numerous operations in parallel.
Think of a CPU as a catamaran and a GPU as a cargo ship; both have their uses. The catamaran is luxurious and fast, designed to carry only a few special passengers over short distances. In contrast, the cargo ship is massive and slow, capable of bearing the weight of thousands of tonnes across entire oceans.
CPUs perform small numbers of specialized computations sequentially and efficiently to support application logic. In contrast, GPUs are matrix-heavy and optimized for the large-scale parallel processing of thousands of complex computations.
For context, matrix-heavy refers to matrix math, an area of mathematics focused on matrices—grids of numbers that store and transform data—which are fundamental to how AI models process information. Models use matrices to store extremely complex and layered data structures that represent multiple perspectives simultaneously.
AI Models perform matrix multiplication to manipulate matrices and reference their stored data structures when learning to recognize and construct images or process unfamiliar human language patterns. Matrix-heavy processes are incredibly computationally demanding and therefore require GPUs.
TPUs are specialized processing units designed by Google to accelerate high-throughput machine learning and AI workloads. TPUs are optimized for matrix operations and can handle workloads similar to those of GPUs while consuming less energy.
Energy Needs & Materials
Now, all processing units require power to operate, measured in watts. CPUs have low power consumption, ranging from 15W to 150W, making them energy-efficient for typical single-thread computing, but wasteful when used for intensive parallel AI workloads.
GPUs consume a significant amount of power, ranging from 200W to 600W per graphics card, but they are highly efficient for training AI models and offer a sustainable performance-to-energy ratio.
TPUs are the most energy-efficient hardware optimized for AI development. Some TPUs consume around 200W, depending on the generation, offering improved energy efficiency for specific AI workloads, such as neural network training, compared to GPUs.
The core materials for any processing unit are silicon, copper, aluminum, gold, and tantalum. Silicon is a semiconductor used to build transistors in microchips. Copper is used for interconnects—internal microchip wires that connect transistors to other components.
Aluminum is used in heat sinks to dissipate heat produced by microchips and keep them cool. Small amounts of gold are used in microchip bonding wires to enable precision conductivity and prevent microchip corrosion. Tantalum is used to make capacitors that store electrical charges on microchip motherboards.
As AI models advance in their capabilities, so will the computational power needed to drive their development. Executing increasingly complex matrix computations in parallel will require higher wattages and more critical resource extraction. So, the more data an AI model is trained on, the more inferences it processes, and the more energy it consumes.
According to the International Energy Agency (IEA), global energy consumption from AI development could exceed 946 terawatt-hours (TWh) annually by 2030, surpassing today’s levels. However, estimates vary based on how quickly AI is adopted and optimized.
Navigating an Intelligent Future
With greater intelligence comes greater infrastructure. The computing power required to meet modern AI workload demands has led to the development of specialized processing hardware, ranging from CPUs and GPUs to TPUs, each playing a distinct role in accelerating matrix-intensive operations such as machine learning and neural network training.
The computational energy consumption needed to run and scale AI systems is rising. The physical and environmental costs of AI—measured not just in watt-hours, but also in mined silicon, copper, aluminum, gold, and tantalum—underscore that digital progress is deeply tied to physical materials.
As models grow in size, sophistication, and societal impact, humanity’s approach to AI development must become equally expansive: ethical, resource-conscious, and strategically placed. AI progress is not a blind sprint but a marathon that demands caution, flexibility, and a balance between technological ambition and ecological responsibility.