"What does AI cost on AWS?" has no single answer because AWS runs three fundamentally different meters under its AI services — and each one fails in a different shape. Knowing which meter you're on tells you what kind of billing surprise you're exposed to, and what kind of alerting actually catches it.
Meter 1: tokens — you pay for payload
Bedrock's on-demand pricing meters input and output tokens, priced per model, with output typically several times input. The bill is requests × tokens per request × price per token, and only the first factor appears on a traffic dashboard. Costs move when your prompts change — more retrieved context, longer histories, an extra agent loop, a quiet fallback to a larger model — not when your traffic does. Failure shape: a spike, landing the day a deploy changes the payload. What catches it: day-over-day comparison against a rolling baseline, per model — at the service level, one model's runaway hides inside another's normal day. Unit cost matters too: dollars per 1K output tokens moving while volume stays flat means the model mix changed, not the usage.
Meter 2: existence — you pay for deployed
A SageMaker real-time endpoint bills per instance-hour from deploy to delete, inference or no inference. Bedrock's provisioned throughput works the same way: model units billed by the hour (or committed by the month) whether tokens flow or not. Failure shape: a step — the charge starts the day the resource appears and runs flat until someone remembers it. It never spikes, so spike detection alone never sees it. What catches it: alerting on a service's spend the first day it appears at all, and a weekly view where a flat hosting line sits over near-zero invocations.
Meter 3: capacity floors — you pay for available
OpenSearch Serverless — the default vector store behind managed RAG features — bills compute units with a minimum that keeps running as long as any collection exists. "Serverless" does not mean scales-to-zero. Failure shape: a floor — flat, continuous, and indifferent to whether the prototype it supports was abandoned three weeks ago. What catches it: same as the step — first-appearance alerts and week-over-week drift against a trailing average, since day-over-day comparison of a flat line is permanently quiet.
Three AI meters, three failure shapes. Spike detection catches the first; the other two need first-appearance and week-over-week drift checks.
Putting the three together
- Token meters need per-model daily baselines. Watch spend per model against its own history, and watch unit cost (dollars per 1K output) for drift — the two together separate "we got more traffic" from "someone changed the prompt" from "someone changed the model."
- Existence meters and floors need appearance + drift detection. The catchable moments are the day the charge starts and the weeks it persists — a weekly digest comparing each service to its trailing average is the net that catches what daily spike detection structurally cannot.
- All three need daily delivery. Every one of these compounds daily. A monthly cost review prices the lesson at thirty days; an alert within about a day prices it at one or two.
Deeper dives on each meter: Bedrock token spikes, SageMaker idle endpoints, and the OpenSearch Serverless capacity floor.