The AI Data Center Power Problem And Who's Going to Solve It

 I love AI, its the perfect expert and assistant, I always wished for. I am very sure so do most of its users, yes there are some problems with it. But I am sure world's best engineers sitting in these companies will solve them.

But here's a problem 99% of coders cannot possibly solve in their cubicles, and it decides how your favorite AI behaves, power supply. Let's talk about Claude, because I am using Claude nowadays, but I have seen a very similar pattern in Google's Gemini Pro (student version) and ChatGPT. So feel free to generalize this issue

Free User Experience

If you have asked any AI model to perform complex tasks for you, or make complex decisions, most AI models take time, somewhere between a few seconds to sometimes Claude taking minutes to draft documents. Not a dealbreaker, but I wish it was faster, since I am so used to speed now. Also as a non-premium user of any of the AI models, I have also seen that they hit a limit. In many cases for premium users too !!





The Power Problem

A single NVIDIA H100 GPU used to power GPT or Claude consumes 700 Watts of power, every second, that's same as your microwave. Google's Gemini runs on a TPU, which consumes 450 Watts, same as 4 high end LED TVs. 

Guess how many of these chips are stacked together to form a small data center,100 ? 500 ? try 5000 !!!
Using basic math, the power consumption would be around 3 MW, just for the chips, add lighting an cooling costs, and congrats you are now using 5- 15 MW of power. Enough to run 12000 homes, continuously.

And in the larger ones that are specially built, that take 2-3 years to build, billions of dollars invested that you see in "AI Infrastructure" these use 100,000 GPUs, guess their power consumption..... more than 500 MW, enough to power 400,000 homes. Every single second. And AI companies will need to open dozens and dozens of them

This problem is getting worse everyday, you know why ? Because most countries DO NOT have enough power plants and power transmission cables to supply power to these mini cities.

Just to put that in perspective AI data centers are projected to consume 3% of humanity's electricity needs. In Ireland, they take more than 1/5th of the COUNTRY's electricity.

The Solution

I am sure most of you guys are aware of this problem. Here are the solutions people are planning-

1. Small Modular Reactors-  These are basically mini nuclear power plants, which can be mass manufactured, fitted and powered on very quickly. But it has the word "Nuclear" in them, to ensure they aren't mini Chernobyl, they are currently undergoing rigorous testing (as they should), extending their timeline to minimum 5 years. Considering GPT was released 4 years ago and has already affected millions of lives, another 5 years to start generating power sounds like infinity.

2. Geothermal Power - We all know this, drill the Earth to absorb the heat under the ground and spin turbines using this heat. Normal, ones take 3-4 years to start, advance technologies take 2-3 years to start generating power, faster than Nuclear, but not fast enough.

3. Gas Turbines - Wait what ? Aren't gas turbines used in airplane and jet engines ? They sure are and a "genius" thought; Hey why not take these turbines out of my private jet and use them to power my data center, guess the "genius", Elon Musk, don't believe me, check it out yourself. This "genius" , was so hungry for power (pun intended), he literally ordered gas turbines from airplane makers and put them in his data center. 
Guess what ? These turbines are designed to fly high up in the sky, so they are now through harmful gases into the neighborhood communities, and they are now fighting lawsuits (typical).

You can't wait 5 years for a data center you need a year later, and you definitely cannot be so selfish that you destroy sleep and lungs of thousands of people just because you thought Grok was a good idea 😂.

4. The Real Solution- This one is being used by data centers and household alike, you guessed it, solar and wind. A wind farm takes 1- 2 years, a solar farm 6 to 9 months. Read that again. If you want to get power in December for your data center, start putting panels in March. That's 5G fast. But you know the problem, the sun doesn't shine at night and wind doesn't blow all the time.

That's why you need batteries, giant ones, not for your phone, for the mini-grids that these AI data centers are needing today and will need for the next 10 years into the future, if future is run by AI, trust me AI is run by the sun and batteries which store its energy. Not by gas turbines which harm your lungs or nuclear reactors that will start running in 2030s. 

The sun doesn't shine for 20 hours, anywhere, so 4 hours of storage duration (lithium ion, sodium ion, lead acid) won't get us there. You need batteries, ones that are so cheap that nothing else makes sense, ones that store energy even during early sunset days (8-10 hours) to even cloudy , non windy days (24 hours). They are not a silver bullet solution, they are just iron bullets you need to kill the "Power Shortage" Monster now 😏.

I encourage all of you to do your own deep dive and research on this topic and share your knowledge and opinions with me.

Solar and batteries seem like the obvious answer, I wonder why data centers are still signing 20-year contracts with gas plants?

None of this is AI generated, so feel free to fact check me and looking forward to your views and help. 

Mayank Jately

If you want to follow this blog

Comments

Post a Comment

Popular posts from this blog

Introduction to the Blog

What's the one thing you cannot imagine modern life without ? Phones ? AI ? Social Media ? These are relatively modern inventions, commercialized in the last two decades. Let's go farther back - Cars, Trains ? Ships ? Planes ? still hardly a century old. Personally, my answer would be energy. From food you eat all the way to content you consume, everything is essentially some form of energy.  And i don't think any person reading this will need to be told that even today, most of humanity's energy comes from fossil fuels (petrol, diesel, crude oil etc), basically carbon that took 300 million years to form, which we dig up, process, and burn in seconds But energy is changing, fast and its happening at a historic pace, EVs, batteries, green hydrogen, solar, wind, hydropower, nuclear fission (hell maybe even fusion !!! ). Just to put that in perspective the amount of solar energy we deployed in the last 12 months (511 GW)is equal or may be more than the electricity demand o...
Read more

What exactly is a Redox Flow Battery ?

Image
We all know what a battery is, that same thing in almost every portable electronic device in your room has. Most batteries we use in our day to day are static batteries, i.e. once they are manufactured, their chemical composition doesn't really change. Now the chemicals used in a battery dictate everything about it, like its - 1. Cyclability  2. Efficiency 3. Safety 4. Supply chain constraints. Let me explain in a little more detail using examples - The AA and AAA batteries (chemical composition is Zinc and Manganese) normally used in T.V remotes are not rechargeable, they are single use. Once the energy inside them is used them, they are pretty much dead. They are mildly toxic, but safe to use, efficiency doesn't come into picture as they can't be recharged, energy density is ( 50-150 Wh/kg ) The Lithium Ion batteries in your phones and EVs are rechargeable (Duh !). Its of many types, i am aware of only two- a) NMC (Nickel Manganese Cobalt along with Lithium)  b) LFP (Lith...
Read more

Why Vanadium Leads Flow Batteries and Why I Didn't Build One.

Image
Search flow batteries anywhere on the internet, you'll find Vanadium's monopoly, ask any AI model about flow batteries and they'll assume vanadium and give you all the information. That's because Vanadium Redox Flow Batteries (or VRFBs in short) are already being commercialized, or at least they're trying their best.  I'll try to explain why, but before i move further, i forgot to explain the redox part in the last 2 posts. All Batteries are Redox Batteries- Batteries have 3 basic parts, positive side (cathode), negative side (anode) and a membrane or separator. So, when you connect a charger to your phone, the charger takes electrons from positive and gives it to the negative side, but if the positive and negative mix together they will neutralize each other, so they need to be separated by separator (or membrane). The word "redox" is made of two words : Reduction- gaining of electrons (by the negative side) Oxidation - loss of electrons (by the posit...
Read more