Skip to main content
This page lists behavior that will surprise you. We’d rather you read it here than discover it from an invoice.

​
A cancelled OpenAI request costs the full answer

Cancel an OpenAI request partway through and you pay for the whole answer. You don’t pay only for the part you read. The same cancel on an Anthropic or Google model stops the charge where you stopped. That holds when we call Anthropic or Google ourselves. Pin a request to Bedrock or Vertex and we read it to the end, as we do for OpenAI. Cancelling still ends your connection at once, so nothing keeps streaming to your application. The generation at OpenAI is what continues, and you pay for it. OpenAI gives us no way to stop a streaming response and reports no usage for one stopped early, so we have nothing accurate to bill you for instead. We are working on an alternative, and this page changes when it lands. Your logs show a stopped request as Canceled. Its token counts are the ones the provider reported before you stopped. Two things help meanwhile. Set max_output_tokens so a cancelled request can’t run past a length you accept. Use Anthropic or Google models where stopping matters more than the model choice.

​
A stream that fails partway still starts with a success

A request can fail after the answer has started arriving. The status line already says the request succeeded, and HTTP gives no way to take that back. The failure arrives inside the stream instead. It comes as an error event, not a normal ending. Read the terminal event rather than the status code when you stream. We mark a request that failed this way as failed, in your logs and in the dashboard. So what you see there is accurate.