An $18,000 AWS bill that should have been $12,000

The second account I want to talk about was a different kind of problem Not wasteful. Efficient, actually. A well-run engineering team who had done their homework. Good tagging. Reasonable instance sizes. Clean account structure Their bill was still too high and they couldn't figure out why $18,000 a month. For their workload it should have been closer to $12,000 I spent longer in the initial review on this one. The obvious things weren't obvious here It took me two hours to find it OpenSearch They were running a production OpenSearch cluster for application logging and search. Reasonable. The cluster was sized at three data nodes of r6g.2xlarge each. That's a serious cluster. About $2,200 a month just for those nodes I looked at the index patterns. They were retaining 14 months of log data in the hot tier. Fully indexed. Fully searchable. At all times I looked at the search queries. 98 percent of all queries were against data from the last 7 days. The other 2 percent were against data from the last 30 days. Nobody had searched data older than 30 days in the history of the cluster They were paying to keep 14 months of data in fast hot storage so that 2 percent of queries could touch data from 30 days ago We implemented Index Lifecycle Management. Hot tier for 7 days, warm tier for 30, cold tier after that with significantly cheaper storage. Reduced the node count on the cluster after the data distribution changed New OpenSearch cost: $800 a month $1,400 a month saved. No change to search functionality. Not a single query that previously worked stopped working The data was the same. The cost of keeping it was not This is the thing about well-run accounts. The easy wins are already gone. What's left is understanding the system deeply enough to find where expensive decisions were made with good intentions and incomplete information That's still findable. It just takes longer Top comments (0)