keep cloudtrail useful without a five figure aws bill

the five figure surprise nobody budgets for

somebody in the comments called it exactly right. you turn on cloudtrail, you feel good about your logging posture, and then you turn on data events for every s3 bucket and every lambda function in the account because "more visibility is better." a month later finance is asking why the aws bill has a extra zero on it. cloudtrail isn't the problem. logging every single object read and write across your entire environment is the problem. here's how to keep the visibility without funding aws's yacht fund.

management events vs data events, know the difference

management events are the control plane stuff. someone created an iam user, someone changed a security group, someone spun up an ec2 instance. these are low volume, high signal, and cheap. cloudtrail logs your first copy of management events for free in most accounts and additional trails are still pennies. keep these on everywhere, always. this is your baseline "what changed in my account" record and there's basically no excuse to skip it.

data events are different. these are the object level actions inside services like s3 (every getobject, putobject) and lambda (every invoke). that's not "someone changed a setting," that's "someone touched a piece of data," and depending on your workload that can be thousands or millions of events per hour. cloudtrail bills data events per event delivered, so if you flip data logging on for every bucket in the account, you're paying for every single file read your app does, all day, every day.

scope data logging to what actually matters

you don't need object level logging on your static website assets bucket or your build cache bucket. you need it on the stuff that would actually hurt if it got touched without authorization: customer pii, financial records, secrets backups, anything regulated. pick the one or two buckets that are genuinely sensitive and scope data event logging to just those, using resource level selectors in your trail config instead of the "log all s3 buckets" toggle.

this is the whole trick. you get deep visibility exactly where the blast radius matters and you're not paying to log every cache hit on a public assets folder. same logic applies to lambda, if you've got a function handling payment data or auth tokens, log its invocations. if it's a function that resizes thumbnails, it can live without per-event billing.

send logs to s3, not just cloudwatch

cloudwatch logs are handy for real time alerting but ingestion and storage there cost more per gb over time than dumping logs into an s3 bucket. the standard defensive pattern is cloudtrail delivers to an s3 bucket for durable, cheap, long term storage, and you optionally forward a filtered subset to cloudwatch or a siem for alerting. don't pay cloudwatch prices to store a year of raw logs you'll rarely touch. lifecycle rules can also push older log data into glacier for even cheaper cold storage, since most of what you're keeping is for compliance and incident response, not daily reading.

set a billing alarm before you need one

even with good scoping, mistakes happen, someone adds a new bucket to the data event list and forgets it's high traffic. a billing alarm is your tripwire. set a cloudwatch billing alarm tied to estimated charges, thresholded at whatever "this is normal for us" number is, so if cloudtrail costs suddenly spike you get an email before you get a bill. this is a five minute setup that has saved more than one team from a very awkward finance meeting.

aws budgets create-budget --account-id YOUR_ACCOUNT_ID --budget file://budget.json --notifications-with-subscribers file://notify.json

the json files just define your threshold and who gets the alert. point it at your team's slack or email, not just a dashboard nobody checks.

the takeaway

security logging is supposed to protect you, not become a line item that makes leadership nervous about turning it on in the first place. keep management events on everywhere because they're cheap and they're the backbone of your audit trail. scope data events to the handful of resources where object level detail actually matters. route logs to s3 for cheap durable storage instead of leaning entirely on cloudwatch. and set a billing alarm so the first time you learn about a misconfiguration isn't when the invoice shows up. good visibility and a sane bill aren't opposites, you just have to be deliberate about where you point the expensive stuff.