OpenAI Cost Tracking Middleware: Build a Logging Wrapper That Shows Every Call's Price

OpenAI cost tracking middleware is a small wrapper around your API calls that records, for every single request, how many tokens went in, how many came out, how many of those input tokens were cached, and what that request cost under the rates you configure. Put it between your code and the OpenAI client, log one line per call to a file, and you stop guessing about your bill: the next time a number on the dashboard looks wrong, you can point at the exact feature, user, or prompt that produced it.

This tutorial gives you that wrapper in Python. It is about sixty lines of code, stores no prompt or answer text, and uses the usage object OpenAI already returns on every non-streaming response. You supply your own rates, because the only rate that matters is the one on your provider's pricing page on the day you run the code. It is the piece our earlier guide to why an OpenAI bill gets so high kept pointing back to: measure first, optimise second.

Why your dashboard arrives too late

The OpenAI usage dashboard is useful, but it answers the wrong question at the wrong time. It tells you, days later, that you spent a certain amount in total. It does not tell you whether the spike on Tuesday came from the new summariser, the retry loop you added on Monday, or one user pasting a fifty-page document into a chat box. By the time you see the total, the calls that caused it are gone, and the only evidence left is a vague memory of what you shipped that week.

Per-call logging flips that around. When every request writes its own line — timestamp, feature name, model, token counts, cost — a spike becomes a query instead of a mystery. You sort the log by cost, and the top ten lines usually tell the whole story. Often the story is embarrassing in a useful way: a background job retrying a failed call five times, a system prompt that grew to thousands of tokens without anyone noticing, or a "quick" agent loop that makes forty calls where you assumed it made four. Our post on why AI agents get so expensive to run describes exactly that pattern: the per-call price looks small, and the step count is what does the damage. A wrapper is how you catch the step count.

What the wrapper records, and what it deliberately leaves out

Every OpenAI chat completion response carries a usage object. According to OpenAI's own API reference, that object reports prompt_tokens, completion_tokens, and total_tokens, with a breakdown under prompt_tokens_details that includes cached_tokens and a breakdown under completion_tokens_details that includes reasoning_tokens. Those five numbers are the whole story of a call's price, and the wrapper's job is simply to capture them before your code moves on and forgets them.

The split matters. Prompt tokens and cached tokens are not the same line on your bill: cached input is normally priced far below fresh input, so a call with a large prompt can be cheap if most of it hit the cache and expensive if none of it did. Recording only the total hides that difference, which is why this wrapper stores fresh and cached input separately. Reasoning tokens matter for the same reason: on a reasoning model, the thinking that happens before the answer is billed as output, so a short-looking answer can carry a long hidden output count. If your log only stored the visible answer length, you would never explain that bill. Our prompt caching guide explains how the cache side works in more detail; the wrapper is how you prove whether it is working for your prompts.

What the wrapper does not record is just as deliberate: no prompt text, no answer text, no user content of any kind. A cost log should be safe to keep, safe to share with a teammate, and safe to grep in a hurry. The moment it contains user data it becomes a privacy liability and a file nobody wants to open. If you later need to debug a specific expensive prompt, add a separate debug flag that logs a hash of the prompt, not the prompt itself, and keep it off by default.

The code: a wrapper you can drop in

Here is the complete wrapper. Save it as cost_tracker.py next to your app, fill in the rates from your provider's pricing page, and call track_chat() instead of calling the client directly. It appends one JSON line per call to llm_costs.jsonl, a format any log tool can read, and it returns the original response untouched, so the rest of your code does not change.

import json
import time
from datetime import datetime, timezone

from openai import OpenAI

# Fill these in from your provider's pricing page, per 1M tokens.
# The wrapper never guesses a rate: if a rate is wrong here, the
# token counts are still right and only the dollar column is off.
RATES = {
    "input_per_million": 0.0,    # fresh (uncached) input tokens
    "cached_per_million": 0.0,   # cached input tokens
    "output_per_million": 0.0,   # output tokens, including reasoning
}

client = OpenAI()  # reads OPENAI_API_KEY from the environment


def _cost_usd(usage, rates):
    prompt = usage.prompt_tokens or 0
    cached = 0
    if usage.prompt_tokens_details is not None:
        cached = usage.prompt_tokens_details.cached_tokens or 0
    fresh = max(prompt - cached, 0)
    output = usage.completion_tokens or 0
    cost = (
        fresh * rates["input_per_million"]
        + cached * rates["cached_per_million"]
        + output * rates["output_per_million"]
    ) / 1_000_000
    return fresh, cached, output, cost


def track_chat(model, messages, feature="unknown", **kwargs):
    """Drop-in replacement for client.chat.completions.create."""
    started = time.monotonic()
    response = client.chat.completions.create(
        model=model, messages=messages, **kwargs
    )
    latency_ms = round((time.monotonic() - started) * 1000)

    record = {
        "timestamp": datetime.now(timezone.utc).isoformat(),
        "feature": feature,
        "model": model,
        "latency_ms": latency_ms,
    }
    usage = getattr(response, "usage", None)
    if usage is not None:
        fresh, cached, output, cost = _cost_usd(usage, RATES)
        record.update({
            "prompt_tokens": usage.prompt_tokens,
            "fresh_input_tokens": fresh,
            "cached_input_tokens": cached,
            "output_tokens": output,
            "reasoning_tokens": (
                usage.completion_tokens_details.reasoning_tokens
                if usage.completion_tokens_details is not None
                else 0
            ),
            "cost_usd": round(cost, 6),
        })
    else:
        record["note"] = "no usage object returned"

    with open("llm_costs.jsonl", "a", encoding="utf-8") as f:
        f.write(json.dumps(record) + "\n")
    return response

Using it is one line different from a normal call: replace client.chat.completions.create(...) with track_chat(..., feature="support_reply"). The feature label is the most important parameter in the file — it is what you will group by later, so name it after your product ("support_reply", "summariser", "nightly_report"), not your code.

Two details in the code deserve a second look. First, the cost calculation subtracts cached tokens from prompt tokens to get fresh input, because prompt_tokens already includes the cached ones. Counting both would double-charge the cache. Second, the wrapper reads the reasoning count defensively, with a check for a missing details object, because not every model and endpoint populates every breakdown field. A wrapper that crashes on an unusual response is worse than no wrapper: it would take your feature down over a logging detail. Logging must never be allowed to break the call it is measuring.

Reading one log line

After a few calls, llm_costs.jsonl holds lines like this example, with the numbers shown only to illustrate the shape of a record:

{"timestamp": "2026-10-30T05:00:00+00:00", "feature": "support_reply",
 "model": "your-model-id", "latency_ms": 1840, "prompt_tokens": 2400,
 "fresh_input_tokens": 400, "cached_input_tokens": 2000,
 "output_tokens": 320, "reasoning_tokens": 0, "cost_usd": 0.0042}

Read it from right to left. The cost is the headline, but the token split is the diagnosis. Here, most of the prompt was cached, so the call stayed cheap despite a long prompt — exactly the behaviour you want from a repeated system prompt. If the same feature's lines showed cached_input_tokens near zero day after day, that would be your first fix: something is changing the prompt prefix on every call, perhaps a timestamp or a user ID injected at the top, and the cache never gets a chance to hit. That single field, watched over a week, is worth more than any pricing article, because it is measured on your traffic rather than assumed.

Two traps that make logs lie

The first trap is streaming. A streamed response does not hand you a complete usage object in the normal way: the usage arrives in a final chunk, and only if you ask for it, by setting stream_options={"include_usage": True} on the request. If you wrap a streaming call with the code above and never see usage, that is why — the wrapper did not fail, the stream simply never sent the numbers. For streamed calls, collect the chunks, keep the last chunk's usage, and log once when the stream finishes. Never log per chunk: you would count one call many times over, and your totals would be fiction.

The second trap is retries. If your app retries a failed call three times, your log should show three lines and your bill will too — the provider charges for the attempts that ran, not just the one that succeeded. That is correct behaviour for a log, but it surprises people who expect one line per user action. When a feature's cost looks too high, count its lines per user action before blaming the model. A retry loop with no backoff, or an agent loop with no step cap, will outspend any rate difference between models, and the log is the only place that pattern is visible.

Turn the log into an alert, not a spreadsheet

A log file nobody reads is decoration. The wrapper earns its keep when a tiny script reads it once a day and tells you only what changed. Start with three summaries, and nothing more: total cost per feature for the last 24 hours, the ten most expensive single calls, and any feature whose daily cost doubled compared with its seven-day average. The first tells you where the money goes, the second tells you what to fix next, and the third is the early warning that catches a runaway job before the month ends.

Keep the alert threshold simple at first, and when one fires, the log already holds the feature, model, token split, and timestamp you need to act. If you want to sanity-check the totals against your own token mix, our AI API Cost Calculator lets you plug in your own numbers and compare them with what the log reports.

That measurement habit is the real point of this tutorial. Pricing pages tell you what a token costs; only your own log tells you what your app actually spends tokens on. Build the wrapper this week, let it run for seven days without changing anything else, and read the per-feature summary before you change a single prompt or model. The numbers will almost certainly surprise you — and unlike a dashboard total, they will tell you exactly what to do next.

Usage fields in this tutorial follow OpenAI's published API reference for chat completions and its usage documentation: Completions reference and Reviewing API usage and costs. Rates are not listed here on purpose — fill in the RATES table from your provider's current pricing page, and re-check it whenever you change models.