Definition
Training a model is a one-off expense for its developer; inference is the running cost every user pays on every call. For API models it is billed per token, with output tokens typically priced higher than input. For self-hosted models it is the cost of the GPUs and the people to run them.
The price of a given level of capability has fallen fast. Stanford's AI Index 2025 reports that the cost of querying a model at GPT-3.5's level fell more than 280-fold between November 2022 and October 2024. At the same time, agents, reasoning models and long contexts use far more tokens per task, so total spend often rises even as unit prices fall.