Cloud Rightsizing Without Breaking Production: gp2 to gp3, EC2 CPU Baselines and GPU Instances (G5, G6, P4, P5)

Everything in your account is sized for a peak that never comes, because at provision time nobody knows the load, doubling is free-feeling, and after an incident someone doubles again "just in case" and nobody ever walks it back. That's how fleets end up averaging 15% utilization. But rightsizing has a social failure mode that matters more than the technical one: the first resize that causes an incident ends the program. One latency regression traced to a downsize and every future recommendation gets vetoed by the team that remembers. So the job isn't "shrink things"; it's "shrink things with evidence and rollback, in risk order." Here's that order. Move 1: gp2 to gp3, the free money with one asterisk Migrating EBS volumes from gp2 to gp3 cuts the per-GB price about 20% ($0.10 to $0.08 per GB-month) and it's an in-place, online modification: no downtime, no detach, no restart. For most volumes it's strictly better, and it's the right first move for a rightsizing program because it builds trust: visible savings, zero incidents. The asterisk is performance baselines. gp3 gives a flat 3,000 IOPS and 125 MB/s regardless of size; gp2 scaled with size (3 IOPS per GB, so volumes over 1 TB had a higher baseline than gp3's default). Before migrating anything over 1 TB, or anything with known IOPS appetite, check its actual consumption (VolumeReadOps + VolumeWriteOps over two weeks) and provision gp3 IOPS/throughput to match. Provisioned extras cost money but the total still usually undercuts gp2. Two operational notes: a volume modification enters a cooldown (about six hours) before the next change, and each migration is per-volume, so script it, don't click it. Move 2: EC2 sizing on percentile baselines, never averages The classic mistake is downsizing on average CPU. Averages hide the month-end close, the Monday-morning surge, the batch window. The instance that averages 12% but hits 85% every quarter-close will page you exactly when it hurts most. The defensible method: - 90 days of history, not two weeks, so monthly and quarterly peaks exist in the data. - Size to p95/p99, not the mean: the question is "does the smaller size absorb the observed peaks with headroom", and headroom means the target's capacity at around 70-80% when your p99 lands. - One size step at a time. m5.4xlarge to m5.2xlarge, observe a full business cycle, then reconsider. Halving twice with observation beats quartering once with hope. - Mind the non-CPU dimensions. Memory isn't in default CloudWatch metrics (you need the agent), and smaller instance types also cut network ceilings and EBS bandwidth. Sub-20% sustained CPU is the trigger to investigate, not a verdict by itself. - Confidence-tier the fleet. Stateless, autoscaled, well-load-balanced services are low-risk resizes; the stateful singleton that finance runs quarter-close on is not. Do the first category in batches and the second one deliberately, with its owner in the room. Execution rails for every resize: change one dimension per change window, canary one instance before the fleet, schedule during low traffic, keep the old size one API call away, and watch p99 latency (not CPU) for a day after. CPU tells you the machine is fine; latency tells you the users are. Move 3: GPUs, where one instance equals a fleet The economics change an order of magnitude at the GPU tier. A g5.xlarge runs about $1.01 an hour ($735 a month); a p4d.24xlarge about $32.77 an hour (about $23,900 a month); p5 instances more still. One idle or oversized GPU instance is worth more than a whole rack of CPU rightsizing, and GPU fleets are routinely the least-measured part of an account. What makes GPU rightsizing different: - CPU metrics say nothing. You need GPU utilization and GPU memory: DCGM or nvidia-smi exported to CloudWatch. The commonest finding is inference services on training-class hardware: single-digit GPU utilization on an A100 (P4) doing work an L4 (G6) or A10G (G5) serves comfortably at a fraction of the price. - Family moves have a compatibility gate. G5 (A10G), G6 (L4), P4 (A100), and P5 (H100) differ in GPU architecture, so a family move means checking driver and CUDA version support for your framework build before anything else. An image pinned to an old CUDA that the newer card's minimum driver won't serve fails at boot, not gradually. Check compatibility first, then benchmark one node on the target family with production-shaped traffic, then move the fleet. - Batch and serving diverge. Training jobs want the biggest feasible box for a short time (and are schedule candidates); serving wants the smallest card meeting latency (and is a rightsize candidate). Mixing the two on one fleet guarantees one of them is wrong. The program, in one paragraph Order the work by blast radius: gp2-to-gp3 first (online, low risk, buys credibility), then percentile-based EC2 downsizing on the low-risk tier with canaries and rollback, then the GPU review where single decisions are worth thousands a month. Attach evidence to every change (the 90-day percentile chart is the argument), and measure the program by incidents caused (target: zero) as loudly as by dollars saved, because the zero is what keeps the program alive. FAQ Does migrating gp2 to gp3 cause downtime? No. It's an online, in-place volume modification while the volume stays attached and in use. The caution is performance, not availability: gp3's flat 3,000 IOPS / 125 MB/s baseline can undercut large gp2 volumes' size-scaled baseline, so check actual IOPS consumption first and provision gp3 performance to match. Also note the roughly six-hour cooldown between modifications of the same volume. What CPU threshold justifies downsizing an EC2 instance? Sustained low utilization over a long window: sub-20% CPU across 90 days is a reasonable investigation trigger. But size to percentiles (p95/p99), never averages, and confirm memory and network headroom too. Downsize one step, canary, and watch p99 latency for a full business cycle before the next step. How do I know if my GPU instances are oversized? CPU metrics won't tell you; collect GPU utilization and GPU memory via DCGM or nvidia-smi. Sustained single-digit GPU utilization on P4/P5 (A100/H100) class hardware doing inference is the classic finding, and the fix is usually a serving-class family (G5/G6). Before moving families, verify driver and CUDA compatibility for your framework build, then benchmark one node. Should I rightsize or buy Savings Plans first? Rightsize first, commit second. A commitment locks in today's shape; if you shrink the fleet 30% afterward, you've prepaid for capacity you no longer run. The order is: eliminate idle, rightsize, schedule non-production, then commit to the steady baseline that remains. What's the safest rightsizing process for production? Evidence, canary, rollback: a 90-day percentile baseline as the justification, one dimension changed per window, one instance moved before the fleet, the previous size one API call away, and p99 latency watched for at least a day. Programs die from one avoidable incident, not from moving slowly. Top comments (0)