We often think of software as a completely digital existence, a world of "bits" entirely separate from the "atomic" world. We can download unlimited data on our phones without making them heavier; we can watch hundreds of movies without touching physical disks; we can collect hundreds of books without owning a piece of paper.
But digital infrastructure ultimately requires physical infrastructure. All this software needs some kind of computer to run. The more computing is required, the more physical infrastructure is needed. We saw this a few weeks ago when we looked at the $20 billion facilities required to manufacture modern semiconductors. We also see this in the most advanced artificial intelligence software. Creating a cutting-edge large language model requires a lot of computing, both to train the model and to run them after the model is completed. Training OpenAI's GPT-4 requires an estimated 21 billion petaFLOPs (1 petaFLOP is 10^15 floating-point operations per second). In contrast, the iPhone 12 can perform about 11 trillion floating-point operations per second (0.01 petaFLOP per second), which means if you could somehow train GPT-4 on the iPhone 12, it would take more than 60,000 years to complete. On a 100 MHz Pentium processor in 1997, only 9.2 million floating-point operations per second could be performed, theoretically requiring more than 66 billion years to train. And GPT-4 is not an exception, but part of a long-term trend where AI models are becoming larger and require more computing to create.
But of course, GPT-4 is not trained on an iPhone. It is trained in data centers, in specially designed buildings with tens of thousands of computers and the necessary supporting infrastructure. As companies compete to create their own AI models, they are building huge computing power to train and run these models. To meet the growth of AI demand, Amazon plans to invest $150 billion in data centers over the next 15 years. Meta alone plans to invest $37 billion in infrastructure and data centers in 2024, most of which is related to AI. The startup Coreweave, which provides cloud computing and computing services for AI companies, has raised billions of dollars to build its infrastructure and will build 28 data centers in 2024. The so-called "hyper-scale companies," technology companies with large computing needs such as Meta, Amazon, and Google, estimate that the data centers they plan or are developing will double their existing capacity. In cities across the country, data center construction is soaring.
But even with the surge in demand for capacity, building more data centers may become increasingly difficult. In particular, operating data centers requires a lot of electricity, and available electricity is rapidly becoming a constraint on data center construction. Nine out of the top ten U.S. utility companies list data centers as a major source of customer growth, and surveys of data center professionals list electricity availability and price as the two major factors driving data center location. With the number of data centers under construction reaching an all-time high, the problem will only get worse.
The implications of losing the competition for artificial intelligence leadership are worth considering. If the rapid development of the past few years continues, advanced artificial intelligence systems will greatly accelerate technological progress and economic growth. Powerful artificial intelligence systems are also very important for national security, enabling new types of offensive and defensive technologies. Losing the forefront of artificial intelligence development will severely weaken our national security capabilities and our ability to shape the future. And another major transformative technology invented and developed in the United States will lose to foreign competitors.
Artificial intelligence relies on the availability of stable electricity. The United States' leadership in innovating new sources of clean, stable electricity can and should be leveraged to ensure that future artificial intelligence data center construction takes place here.
Data Center Introduction
A data center is a very simple structure: a space that houses computers or other IT equipment. It can be a small cabinet with servers or several rooms in an office building, or even a large independent structure specifically built to accommodate computers.
Large computing devices have always required a specially designed space to accommodate them. When IBM launched its System/360 in 1964, it provided a 200-page physical planning manual that provided information on space and power requirements, working temperature ranges, air filtration recommendations, and all other information required for the computer to operate normally. But historically, even large computing operations could be carried out within buildings primarily used for other purposes. Even today, most "data centers" are just rooms or floors in multi-purpose buildings. According to EIA data, as of 2012, there were data centers in 97,000 buildings across the country, including offices, schools, laboratories, and warehouses. These data centers are usually about 2,000 square feet in size and occupy only 2% of the building they are in on average.
The modern data center we think of, a large building specifically built to accommodate tens of thousands of computers, is largely a post-Internet era product. Google's first "data center" was a 28-square-foot cage with 30 servers, shared with AltaVista, eBay, and Inktomi. Today, Google operates millions of servers in 37 dedicated data centers worldwide, some of which are nearly a million square feet in size. These data centers, along with thousands of others around the world, support Internet services such as web applications, streaming video, cloud storage, and artificial intelligence tools.
Large modern data centers contain tens of thousands of individual computers, which are specially designed to be stacked vertically in large racks. Racks can hold dozens of computers at a time, as well as other equipment required to operate these computers, such as network switches, power supplies, and backup batteries. The corridors inside the data center contain dozens or hundreds of racks.
The number of computer devices installed in the data center means that its power consumption is huge. The power consumption of a single computer is not large: a rack-mounted server may only consume a few hundred watts, about one-fifth the power of a hair dryer. But tens of thousands of computers together generate a huge demand. Today, large data centers may require 100 megawatts (100 million watts) or more of electricity. This is approximately the amount of electricity needed for 75,000 households or the amount of electricity required to melt 150 tons of steel in an electric arc furnace. In fact, the demand for electricity is so important that data centers are usually measured by power consumption rather than building area (CBRE's report estimates that the capacity of data centers under construction in the United States is 3,077.8 megawatts, but the specific number is unknown). Their power requirements mean that data centers need large transformers, high-capacity electrical equipment (such as switchgear), and in some cases even a new substation to connect them to power lines.
All this electricity eventually turns into heat inside the data center, which means it requires equally robust equipment to quickly dissipate the heat after it is powered on. Racks are located on raised floors, which keep them cool by drawing in large amounts of air from below and passing it through the equipment. Racks are usually arranged in alternating "hot aisles" (where hot air is expelled) and "cold aisles" (where cool air is drawn in). The hot exhaust is expelled by the data center's cooling system and recirculated after cooling. These cooling systems can be complex, with multiple heat exchange fluid "cooling loops," but almost all data centers use air to cool the IT equipment itself.
It is not surprising that these cooling systems are large-scale. The minimum amount of air required to remove one kilowatt of electricity is about 120 cubic feet per minute; for 100 megawatts of electricity, this means 12 million cubic feet per minute. The cooling system capacity of a data center cooler is thousands of times that of a home air conditioner. Even relatively small data centers will have huge air ducts, high-capacity cooling equipment, and large cooling towers. This video shows a data center with a one-million-gallon "cold battery" water tank: water is cooled at night when electricity is cheaper and used during the day to reduce the burden on the cooling system.
Due to the huge power consumption, great efforts have been made to improve the energy efficiency of data centers. A common performance indicator of data centers is the Power Usage Effectiveness (PUE), which is the ratio of the total electricity consumed by the data center to the electricity consumed by its IT equipment. The lower the ratio, the less electricity is used for other purposes besides running the computers, and the more efficient the data center is.
The PUE of data centers has been steadily declining. In 2007, the average PUE of large data centers was about 2.5: for every watt of electricity used to power the computers, 1.5 watts were used for cooling systems, backup power, or other equipment. Today, the average PUE has dropped to just over 1.5. Hyper-scale companies perform even better: Meta's average data center PUE is only 1.09, and Google's is 1.1. These improvements come from many aspects, such as more efficient components (such as uninterruptible power supply systems with lower conversion losses), better data center architecture (changing to hot aisle