The True Cost of AI

A recent article by Chris Skinner highlights the challenge of getting best value from using Artificial Intelligence (AI) [1]. This calculation applies to organisations, such as central banks, who are starting to integrate AI into the core of what they do.

Every time you ask an AI agent a question, you’re consuming tokens. The problem is you don’t necessarily know how many tokens they’re using because AI can work in the background unseen solving the problem, particularly if you’re using agentic AI.

Traditional software costs were largely predictable. You bought a Microsoft licence, and you knew how much it would cost, whereas AI is more like electricity, cloud computing, or mobile data, where you pay according to consumption.

Based on a McKinsey survey of AI usage across industries, the rate at which AI expenditure rises is increasing rapidly. The surge in the use of tokens means that the overall AI bill is rising even as the price of individual tokens is declining. The more people use AI, the bigger the bill becomes for companies. Nvidia’s vice president of Applied Deep Learning has said that his team’s compute costs now exceed the cost of the employees using that compute. AI costs more than people. McKinsey argues that this cannot continue.