← Blog
bedrockaiserverless

Your Bedrock bill is a payload property, not a traffic property

By , founder of CostPulse ·

Every other line on an AWS bill scales with something you can see on a dashboard: instances running, gigabytes stored, requests served. Bedrock scales with tokens — and tokens are a property of your prompts and your model's replies, not of your infrastructure. That's why inference is the spike-prone line item on modern bills: request volume can stay perfectly flat while cost triples, because the thing being metered lives inside the payload.

The bill is three dials, and you only watch one

Inference cost is roughly requests × tokens per request × price per token. Request count is the only one of those three that shows up on a typical service dashboard. The other two drift independently: tokens per request grows every time someone adds context to the prompt — more retrieved documents, longer chat history, a bigger system prompt, richer few-shot examples — and price per token jumps by an order of magnitude when a request is routed to a larger model tier. Output tokens are typically priced several times higher than input tokens, so a change that makes responses longer or more verbose moves the bill more than the same change on the input side.

Week 110K requests1.2K tokens/req · $Week 210K requests3.8K tokens/req · $$$cost = requests × tokens/request × price/token

Same request volume, one retrieval change — tokens per request is the dial that moved, and it isn't on the service dashboard.

Where the spikes actually come from

  • Context growth ships silently. A retrieval change that stuffs five documents into the prompt instead of two is invisible in code review as a cost change — it looks like a relevance improvement. It's a 2-3× multiplier on input tokens for every single request.
  • Agent loops multiply requests behind one user action. A single "request" that fans out into a chain of tool calls, retries, and re-prompts can burn dozens of model invocations. A prompt change that makes the agent loop one extra time per task raises cost by that factor across all traffic.
  • Retries re-bill the whole prompt. Timeouts and throttling retries resend the full context each attempt. A flaky downstream that triggers client-side retries bills the same tokens two or three times.
  • Model routing changes tiers. Falling back to (or quietly defaulting to) a larger model multiplies price per token — check the current per-model pricing page for the exact ratios, but the gap between tiers is typically large enough to dominate every other factor here.
  • Unbounded output. A missing or generous max_tokens cap means response length — the expensive side of the meter — is decided by the model, not by you.

Why month-end review can't catch this

A token-driven regression lands with a deploy, not with a billing cycle. If the prompt change that tripled tokens-per-request goes out on the 3rd, the damage compounds for four weeks before anyone reads a cost report. And because request volume looks normal the whole time, nothing on the operational dashboards flags it — the only signal is spend versus its own recent baseline, which is exactly the comparison invoices don't show you.

What to check first

  • CloudWatch's Bedrock namespace publishes InputTokenCount, OutputTokenCount, and Invocations per model id — graph tokens per invocation over time; that ratio moving without a matching traffic change is the smoking gun.
  • In Cost Explorer, group Bedrock spend by usage type — input and output tokens meter separately, so you can see which side of the conversation is growing.
  • Log token usage from the response metadata per request in your own application, tagged with the prompt/agent version — it's the only way to attribute a token regression to the deploy that shipped it.
  • Set max_tokens deliberately on every call path, and alarm on invocations per user action if you run agent loops.

The pattern to internalize: inference cost is a payload property, so it changes when your prompts change, not when your traffic does. Watch spend against a rolling baseline of itself, per service — it's the one view where a Tuesday deploy that tripled tokens shows up on Tuesday.

CostPulse watches for this automatically — a 7-day rolling baseline per service and region, with the alert in Slack, Teams or email within about a day of it starting. Not on the invoice a month later.

Have an AWS spend commitment? Buy through AWS Marketplace and it may count toward it, depending on your agreement. Want a full teardown instead? Book an audit — $3,500.