Incident Response on AWS: From Preparation to Post-Incident Review.
DEV Community

Incident Response on AWS: From Preparation to Post-Incident Review.

How incident response differs on AWS

Incident response on AWS is faster to execute and easier to get wrong than on-premises response, because both the attacker and the responder work through the same APIs. Containment is an API call. So is destroying evidence.

Four differences drive the rest of this article:

  • Shared responsibility sets your scope. AWS secures the infrastructure. You respond to what happens with your identities, configurations, data and workloads.
  • Identity comes first. Most AWS incidents begin with valid credentials. Containing a principal usually matters more than containing a host.
  • Resources are short-lived. An instance in an Auto Scaling group or a Fargate task can disappear before you look at it. Evidence capture has to be prepared in advance and quick.
  • Everything can be automated. Any step you can run from the CLI can run from a Lambda function or a Systems Manager runbook, with the same permissions problem if you grant too much.

Two references frame the lifecycle. NIST SP 800-61 Revision 3, published in April 2025, replaced Revision 2 and maps incident response onto the six functions of the Cybersecurity Framework 2.0 (NIST). The AWS Security Incident Response Guide organises the work as preparation, operations and post-incident activity (AWS technical guide).

This article uses the operational phases responders actually work through:

  • Preparation
  • Detection and analysis
  • Containment
  • Forensic acquisition
  • Eradication and recovery
  • Post-incident activity

Commands use AWS CLI v2. Resource names and IDs are placeholders. Rehearse every command in a sandbox account before you need it in production.

Preparation

Preparation decides the outcome: every minute spent creating a role or a bucket during an incident is a minute the attacker keeps access. Build the following before you need it.

incident response architecture ยท four accounts and the flows between them

A finding in a workload account triggers the workflow in the security account, which contains the resource through the incident response role; evidence goes to the forensics account and logs are queried from the log archive.

A trust policy for the incident response role, requiring MFA and a named source role:

{ "Version" : "2012-10-17" , "Statement" : [ { "Effect" : "Allow" , "Principal" : { "AWS" : "arn:aws:iam::111122223333:role/SOC-Responder" } , "Action" : "sts:AssumeRole" , "Condition" : { "Bool" : { "aws:MultiFactorAuthPresent" : "true" } } } ] }

Give the role read access to everything, plus a short list of write actions needed for containment: changing security groups, creating snapshots, deactivating access keys, attaching policies and tagging. Do not grant delete or terminate permissions.

AWS Security Incident Response AWS Security Incident Response is a managed service worth enabling as part of preparation. It triages findings from GuardDuty and from third-party tools through Security Hub, provides a case console and gives 24/7 access to the AWS Customer Incident Response Team.

Since 21 November 2025 it has used metered pricing with the first 10,000 ingested findings per month free. It also includes an AI investigative agent that gathers evidence from CloudTrail, IAM and EC2 and builds a timeline when a case opens.

Containment by the service is off by default. Decide in advance whether you want AWS to take containment actions for you and record that decision in your playbooks.

Detection and analysis

Analysis has one goal: establish scope quickly enough to contain the right things. Answer five questions in order, and start containment as soon as the first two are clear.

Which principal or resource is involved? An access key, a role session, an instance, a bucket. Is the activity still going on? Look at the last 15 minutes before the last 15 days. What did it do? Separate read calls from calls that changed something. What else did it touch? Roles it assumed, resources it created, other accounts. How did it get in? Root cause can wait until after containment, but not until after recovery.

Set severity from impact and confidence, not from the alert's own label.

Scoping with the CLI

When you have nothing else, CloudTrail Event history answers the first questions. It covers management events for 90 days in one Region at a time.

aws cloudtrail lookup-events \
--lookup-attributes AttributeKey = AccessKeyId,AttributeValue=AKIAIOSFODNN7EXAMPLE \
--start-time 2026-11-01T00:00:00Z \
--query 'Events[].{time:EventTime,event:EventName,source:EventSource,user:Username}' \
--output table

Scoping with Athena on Security Lake

For anything beyond one Region and one key, query Security Lake. Replace the Region in the table name.

Everything a credential did, with the changes first:

SELECT time_dt , cloud . region , api . service . name AS service , api . operation , src_endpoint . ip , http_request . user_agent , api . response . error FROM amazon_security_lake_glue_db_eu_west_1 . amazon_security_lake_table_eu_west_1_cloud_trail_mgmt_2_0 WHERE time_dt > current_timestamp - interval '14' day AND actor . user . credential_uid = 'AKIAIOSFODNN7EXAMPLE' ORDER BY regexp_like ( api . operation , '^(Get|List|Describe|Head)' ) ASC , time_dt ASC ;

Roles the credential assumed , so you can follow each session :

SELECT time_dt , api . operation , src_endpoint . ip , api . request . data FROM amazon_security_lake_glue_db_eu_west_1 . amazon_security_lake_table_eu_west_1_cloud_trail_mgmt_2_0 WHERE time_dt > current_timestamp - interval '14' day AND actor . user . credential_uid = 'AKIAIOSFODNN7EXAMPLE' AND api . operation LIKE 'AssumeRole%' ;

Other principals seen from the same source addresses:

SELECT actor . user . uid AS principal , count (*) AS calls, min(time_dt) AS first_seen, max(time_dt) AS last_seen FROM amazon_security_lake_glue_db_eu_west_1.amazon_security_lake_table_eu_west_1_cloud_trail_mgmt_2_0 WHERE time_dt > current_timestamp - interval '14' day AND src_endpoint.ip IN ('198.51.100.23', '203.0.113.77') GROUP BY actor.user.uid ORDER BY calls DESC;

Amazon Detective gives the same pivots visually for a single principal or IP address. If you use AWS Security Incident Response, opening a case starts its investigative agent, which gathers the same evidence in parallel with your own work.

Build the timeline as you go Record every fact in one timeline, in UTC +8, with its source. It becomes the incident report.

Containment

Contain by removing the attacker's access without destroying evidence: isolate, never terminate. The AWS guidance groups containment into three kinds and most incidents need more than one.

IAM user access key

Deactivate the key, then deny everything on the user in case other credentials exist.

aws iam update-access-key --user-name app-user \
--access-key-id AKIAIOSFODNN7EXAMPLE --status Inactive
aws iam put-user-policy --user-name app-user --policy-name IR-DenyAll \
--policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Deny","Action":"*","Resource":"*"}]}'

Deactivate first and delete later. An inactive key can be re-enabled if the application breaks; a deleted key cannot.

IAM role sessions

Temporary credentials cannot be deleted, so deny every session issued before now.

aws iam put-role-policy --role-name AppRole --policy-name IR-RevokeOlderSessions \
--policy-document '{ "Version": "2012-10-17", "Statement": [{ "Effect": "Deny", "Action": "*", "Resource": "*", "Condition": {"DateLessThan": {"aws:TokenIssueTime": "2026-11-13T05:00:00Z"}} }] }'

This does not stop anyone who can still assume the role. Fix the trust policy or the upstream credential in the same step. Expect a short outage for legitimate workloads until they obtain fresh credentials.

EC2 instance

Tag, protect, remove from service, snapshot, then isolate.

I = i-0123456789abcdef0
aws ec2 create-tags --resources "$I" \
--tags Key = IR-Status,Value=Quarantined Key = IR-Case,Value=CASE-123
aws ec2 modify-instance-attribute --instance-id "$I" --disable-api-termination
aws autoscaling detach-instances --instance-ids "$I" \
--auto-scaling-group-name web-asg --no-should-decrement-desired-capacity
aws elbv2 deregister-targets --target-group-arn "$TG_ARN" --targets Id = "$I"
aws ec2 create-snapshots \
--instance-specification InstanceId = "$I" ,ExcludeBootVolume=false \
--description "IR CASE-123 pre-isolation"
aws ec2 modify-instance-attribute --instance-id "$I" --groups sg-0isolation0example

Changing security groups does not cut connections that are already tracked. If an established session must end immediately, add a network ACL deny entry for the instance's address as well.

Capture memory before isolating if you need Systems Manager to reach the instance, and do not stop or reboot it.

S3 bucket

aws s3api put-public-access-block --bucket example-bucket \
--public-access-block-configuration \
BlockPublicAcls = true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

Save the current bucket policy to the evidence store, then replace it with one that denies every principal except the incident response role. Check for replication rules, event notifications and access points an attacker may have added.

Lambda function

aws lambda put-function-concurrency --function-name example-fn \
--reserved-concurrent-executions 0

Setting concurrency to zero stops invocations while keeping code and configuration intact. Also check for function URLs, new triggers and new layers.

Containers

EKS: cordon the node with kubectl cordon, label the pod so a deny-all NetworkPolicy selects it, and remove it from its Service by changing its labels. Do not delete the pod.

ECS on Fargate: a task cannot be paused. Collect logs and runtime findings, then restrict its security group before you stop it.

Whole account

If you cannot scope the compromise, attach a quarantine SCP to the account that denies every action except those by the incident response role. It is blunt and causes an outage, so agree in advance who may authorise it.

Forensic acquisition

Collect evidence in order of volatility: memory first, then disk, then configuration and logs. Work on copies in the forensics account, and record a hash and a custody entry for everything you collect.

Memory

Memory holds what disk does not: running processes, injected code, network connections and keys. Capture it before isolating the instance, because Systems Manager needs network access to reach it. The alternative is an isolation security group that still allows HTTPS to your Systems Manager and S3 VPC endpoints.

Stage a capture tool such as AVML for Linux in an S3 bucket in advance, then run it with Run Command. Put the commands in a file to avoid shell quoting problems:

{ "commands" : [ "aws s3 cp s3://ir-tools-444455556666/avml /tmp/avml" , "chmod +x /tmp/avml" , "/tmp/avml /tmp/mem.lime" , "sha256sum /tmp/mem.lime | tee /tmp/mem.lime.sha256" , "aws s3 cp /tmp/mem.lime s3://ir-evidence-444455556666/CASE-123/memory/" , "aws s3 cp /tmp/mem.lime.sha256 s3://ir-evidence-444455556666/CASE-123/memory/" ] }
ws ssm send-command --instance-ids "$I" \
--document-name AWS-RunShellScript \
--comment "IR CASE-123 memory capture" \
--parameters file://capture.json

Writing the image to local disk changes the disk. If that matters for your case, snapshot the volumes first. The instance role needs permission to write, and only write, to the evidence prefix.

Disk

Snapshot every attached volume, then share the snapshots with the forensics account.

SNAP = $( aws ec2 create-snapshot --volume-id vol-0123456789abcdef0 \
--description "IR CASE-123" \
--tag-specifications 'ResourceType=snapshot,Tags=[{Key=IR-Case,Value=CASE-123}]' \
--query SnapshotId --output text )
aws ec2 wait snapshot-completed --snapshot-ids "$SNAP"
aws ec2 modify-snapshot-attribute --snapshot-id "$SNAP" \
--attribute createVolumePermission --operation-type add \
--user-ids 444455556666

A snapshot encrypted with the default AWS managed EBS key cannot be shared. Copy it with a customer managed key whose policy allows the forensics account, and share the copy.

In the forensics account, copy the snapshot, create a volume from it, attach it to an analysis instance and mount it read-only:

sudo mount -o ro,noexec,nodev /dev/nvme1n1p1 /mnt/evidence

XFS file systems also need norecovery,nouuid.

Evidence that is not on an instance

Two IAM snapshots are worth taking in every incident:

aws iam generate-credential-report
aws iam get-credential-report --query Content --output text | base64 -d > credential-report.csv
aws iam get-account-authorization-details > iam-snapshot.json

Integrity and custody

Evidence is only useful if you can show it has not changed. Store everything in an S3 bucket with Object Lock in compliance mode and versioning on. Record a SHA-256 hash at collection and verify it before analysis.

Prove CloudTrail logs are complete and unmodified with log file validation:

aws cloudtrail validate-logs \
--trail-arn arn:aws:cloudtrail:eu-west-1:111122223333:trail/org-trail \
--start-time 2026-11-01T00:00:00Z --end-time 2026-11-13T00:00:00Z

Keep a custody log from the first item collected.

Eradication and recovery

Eradication removes everything the attacker created or changed; recovery returns the workload to service from a state you trust. Do not start either until the scope from analysis is stable, or you will remove one foothold while the attacker uses another.

Find what the attacker left behind Start from the timeline: every successful write call by a compromised principal is something to reverse or verify. Then check the places persistence hides, in every Region.

A loop for running instances across all enabled Regions:

for r in $( aws ec2 describe-regions --query 'Regions[].RegionName' --output text ) ; do echo "== $r"
aws ec2 describe-instances --region "$r" \
--filters Name = instance-state-name,Values=pending,running \
--query 'Reservations[].Instances[].[InstanceId,InstanceType,LaunchTime]' \
--output text
done

IAM Access Analyzer external access findings list resource policies that grant access outside your

Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.