Day 47: Copy Requirements Before the Code, and SQS Has No Priority Feature

Neither of today's tasks has a setting for the thing it delivers. Docker has no switch that makes rebuilds fast, and SQS has no switch that makes a queue high priority. In both cases, the feature is the order you put things in.

One Docker task, one AWS task. Package a Python app as an image, then build priority queues with SQS and SNS in a CloudFormation template. The tasks come from the KodeKloud Engineer platform. This is also the last Docker task. Days 35 to 47 went from installing the engine to a real application image, and Kubernetes starts tomorrow.

Docker: Two COPY Lines, and Which One Goes First

Two COPY lines, and which one goes first:

FROM python:3.11-slim
WORKDIR /app
COPY src/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY src/ .
EXPOSE 3004
CMD ["python", "server.py"]

The obvious version copies all of src/ and then installs. It works. It also reinstalls every dependency every time you touch a line of server.py. Docker's cache guidance explains why. A change causes a rebuild for every step that follows it, so expensive steps belong near the beginning, and installing dependencies in an earlier layer means there is no need to rebuild that layer when a project file changes.

Copy the manifest on its own, install, and only then copy the code. Now editing server.py invalidates the last COPY and nothing before it.

The same logic explains --no-cache-dir, and it is a nice detail. pip's own documentation recommends leaving its cache on unless you have caching at a higher level, and it gives layered caches in container builds as the example. That is precisely this situation. The Docker layer is the cache; a pip cache sitting inside it is just weight in the image.

One thing worth knowing about that last line. CMD in exec form is a JSON array and each element is taken literally, so a stray character in "server.py" is not a syntax error. It is a different filename. The container starts, Python cannot find the file, and it exits immediately. docker logs shows it; docker ps without -a shows nothing at all.

Then the run, with the port mapping reading host first, container second:

docker run -d -p 8097:3004 --name pythonapp_nautilus nautilus/python-app curl http://localhost:8097/

CloudFormation, and the Permission That Was Not There

The AWS task was the first infrastructure-as-code task of the run: two SQS queues, an SNS topic routing by priority, and a Lambda that drains high before low, all in one template.

The first deploy rolled back:

User: ... is not authorized to perform: iam:PutRolePolicy on resource: role lambda_execution_role

The template gave the role permissions two ways. CloudFormation documents Policies as adding an inline policy embedded in the role, and ManagedPolicyArns as attaching standalone managed policies. The error named the IAM action the inline route used, PutRolePolicy. Attaching a managed policy is a different action, and this account allowed one and refused the other. The role had been created and the managed policy attached before the refusal, which is how you tell exactly which half was blocked.

Moving everything to managed policies fixed it, at a cost worth stating. The managed policies available are AmazonSQSFullAccess and AmazonSNSFullAccess, far broader than a function that needs to receive and delete messages on two queues. In an account that allows it, a separate AWS::IAM::ManagedPolicy resource keeps the permissions scoped while still attaching rather than embedding.

Two CloudFormation lessons came with the failure. validate-template had passed. The documentation is blunt that it is designed to check only the syntax of your template, not whether the property values are valid, and that to check operational validity you have to create the stack. Passing validation means the YAML parses.

And the failed stack was stuck. ROLLBACK_COMPLETE only exists after a failed stack creation, and AWS says the only operation available in that state is a delete. So the first deploy of a new stack is the one that costs a full delete-and-recreate cycle when it goes wrong. A failed update is kinder, returning to the previous working state as UPDATE_ROLLBACK_COMPLETE.

SQS: The Priority Is in the Consumer

SQS has no priority feature. The whole mechanism is two queues and a consumer that looks at one first:

response = poll_messages(os.environ['high_priority_queue'])
if response == "No more messages to poll":
    response = poll_messages(os.environ['low_priority_queue'])

What gets messages into the right queue is the SNS filter policy on each subscription:

FilterPolicy:
  priority:
    - high

AWS describes it simply: if the message attributes satisfy the filter policy, SNS sends the message to that subscriber, and otherwise it does not. Without it, SNS fan-out sends every message to every queue and the exercise means nothing.

So I checked the queue depths before invoking anything. Two and two, not four and four. That proves the routing independently of the Lambda, which matters, because four and four would mean a filter problem that no amount of debugging the function could fix.

Then four invocations:

invoke 1: "Message 'High Priority message 2' deleted"
invoke 2: "Message 'High Priority message 1' deleted"