Sam Altman almost scrapped token pricing. Here’s why he backed off.

OpenAI considered dropping token-based pricing for its next model, letting developers submit a prompt and receive a price for the work, before negative customer feedback convinced the company to back away from the idea, CEO Sam Altman says.

The flat-fee pricing idea

Speaking Thursday with Stripe President John Collison on the Cheeky Pint podcast, Altman recalls that OpenAI “almost made a very — what would’ve been a bad call” to stop pricing its next model in tokens altogether, moving instead to an arrangement where a developer hands over a prompt and OpenAI hands back a price.

The billing would have worked “like AWS,” he explained, where you “just do something and we send you a bill” rather than metering the exchange token by token. Altman does not name the model in question or give a timeline for when the change was under consideration.

We’ve cued up the exchange in the video below:

Why customers pushed back

The idea ran into resistance when OpenAI tried to gauge how customers felt about receiving a price with no breakdown. “The customer response on that is so negative,” Altman recounts, with buyers telling the company, “’I need to know what I’m paying for somehow.’”

“’I need to know what I’m paying for somehow.’”

Developers already spend plenty of time tracking input and output tokens to keep costs down and estimate what they’ll spend as usage grows. Getting a price without knowing how it was calculated would only make that harder.

Paying more for clarity

What makes the admission worth a second look is that, according to Altman, the people he consulted would rather pay more to keep that visibility. He put the question to “a few people,” asking whether they would take a higher rate for the same output as long as it stayed priced in consistent tokens, and the answer, he said, was that they would “much rather you inflate the tokens and pay more,” because “I know there’s a real thing there.”

“much rather you inflate the tokens and pay more … I know there’s a real thing there.”

Developers resist a single opaque price, he reasons, because they suspect that over the long run they are “gonna get screwed somehow,” the same instinct that once had buyers paying for CPU hours instead of for a finished job.

The proposal could have had significant implications for developers building agents, whose multistep workflows already consume unpredictable amounts of compute and make spending difficult to forecast even under today’s token-based model. Altman does not explain how OpenAI would have set those per-job prices, or whether customers could have weighed one charge against another.

Tokens, a ‘fake unit’

The subject came up when Collison pressed Altman on whether tokens were the right way to measure AI usage. Altman acknowledged the limitations of token pricing but had no clear alternative to offer.

“Tokens are a kind of fake unit … they’re real in some sense, but they’re not a good unit.”

“Tokens are a kind of fake unit,” he said. “I mean, they’re real in some sense, but they’re not a good unit.” He balked at Collison’s half-serious pitch to coin an intelligence-equivalent token, comparing it to the way chipmakers still market nanometer figures that no longer correspond to any physical measurement on the chip.

Altman admits developers incur real overhead when choosing between a cheaper model and a more capable one, especially when they also have to guess how much reasoning a given task will demand. “That’s just cognitive overhead that I don’t think actually adds any value,” he said.

The exchange comes as OpenAI continues to rely on token pricing, even as Altman has argued that the cost of a given level of AI falls roughly tenfold a year. For now, token pricing remains in place, even as Altman acknowledges its shortcomings and OpenAI struggles to find a better alternative.

Source: The New Stack