We already talked about how data hungry generative AI technologies are, both in the model creation phase as well as during fine tuning. This comes with staggering requirements in terms of computational power, which in turn means these applications are energy hungry. How much hungry? Very. Microsoft has recently posted a job for a Principal Program Manager, Nuclear Technology to build micro nuclear reactors powering their data centres (!). Let’s understand in detail why this is the case and why it matters to corporates.
Focus On: Making Energy Efficient Choices in AI
Each layer, each neuron in a deep learning model like GPT-4 performs an array of mathematical operations. Now multiply that by the millions, if not billions, of neurons. The complexity increases exponentially as these networks learn from humongous datasets. That computational burden translates into energy consumption, which is no small line item on an enterprise balance sheet, especially when you consider this is not just going to be an upfront CAPEX cost, but also an ongoing OPEX to operate the application.
That said, there are cost-mitigation strategies corporates can look at when deploying generative AI applications as part of their go to market. Let’s see the main ones.
On-Premises vs Cloud
Evaluating where to run your AI operations can make a difference. Cloud providers are aggressively optimizing for energy efficiency at a scale most individual companies can’t (how many companies can build their own nuclear reactor?), which is why I normally recommend enterprises to either licence a commercially available model or run an open source one on their cloud environment, rather than building and running it from scratch.
Algorithmic Efficiency
Leaner algorithms and more efficient neural network architectures can do the same job for less energy. Trial and error is the best approach here. Testing different models on real use cases will allow you to understand the efficiency and effectiveness of each.
Scheduled Operations
Consider non-peak hours for heavy computational tasks. Energy costs vary by demand and you can benefit from off-hours rates. This is what I did personally when playing around with open source models on my Azure and AWS cloud. Since I was just testing and learning, I did not need peak performance at all times and opted for cheaper CPUs/GPUs at off peak rates.
Diversified Energy Sources
Should you decide to run a generative AI application on premise, even without going the nuclear route like Microsoft, a diversified energy portfolio can offer cost benefits and protect against price volatility in the energy markets. It is also well known how locating your data centre in cold locations can naturally lead to lower energy costs, although you have to be careful about data sovereignty if this is important to the business.
Vendor Partnerships
Collaborate with technology providers who specialize in energy-efficient hardware. GPUs optimized for AI tasks, for instance, can make a material difference in your energy bills. It is no secret that GPUs, especially the best ones, are in scarse supply. Only by forging long term relationships with OEMs and manufacturers you can ensure a steady supply of the hardware you need.
What next?
I suggest a short checklist to ensure the energy requirements of generative AI are tackled early on in the conversation:
1. Energy Audits: Know your consumption patterns and identify energy sources to ensure these align with commercial and ESG goals.
2. C-Suite Conversations: This isn’t just an CTO issue. CFOs and CMOs need to align on the financial implications of AI-driven initiatives and measure cost-benefits.
3. Technology Scouting: Keep an eye out for emerging technologies focused on energy-efficient computing, both on the software as well as hardware side.
4. Government Grants: Some jurisdictions offer incentives for using alternative energy or implementing energy-saving technologies.
5. Data Discipline: Finally, not every task needs the might of a GPT-4. Be judicious in your AI applications.
The future isn’t just about what AI can do for you, but what costs—literal and figurative—you’re willing to bear or it is worth to bear. As always, avoiding to put the technology before the problem and focus on desired outcomes (and their business case) is the objective way to make sensible, long term decisions.
Follow me
That’s all for this week. To keep up with the latest in generative AI and its relevance to your digital transformation programs, follow me on LinkedIn or subscribe to this newsletter.