From Playground to Caching: Learning Amazon Bedrock

This post is part of my notes from Formação AWS, a course by Henrylle Maia. It covers a class from Desafio Labs (one of the course's sections) that focuses on Amazon Bedrock. In this class, I start with the Playground to get a feel for how Bedrock behaves, and then move on to scripts to learn how to actually use the Bedrock API. To learn by doing, I created a simple chatbot that works like a clerk selling tickets for an AWS Course. It starts as a single Bash script that sends one message and prints the response, then I make it interactive so I can actually chat with it. From there, I move it to Node so I can stream the answers as they come in, and finally add a cache checkpoint so the same System Prompt doesn't get billed at full price on every message. Getting Started with Amazon Bedrock Amazon Bedrock is a fully managed AWS service that provides API access to multiple foundation models. There are several models to choose from, but for this project I'm using Anthropic's Claude Sonnet 4.5. The Cross-Region Quota When I selected my preferred model in the Playground and tried to send a prompt, I received the following error message: ThrottlingException: "Too many tokens per day, please wait before trying again." Digging into it, I discovered that there is a quota called Cross-region model inference tokens per minute for Anthropic Claude Sonnet 4.5 V1. It limits the combined input and output tokens that can be sent to that model per minute through the cross-region inference profile, across every API: Converse, ConverseStream, InvokeModel and InvokeModelWithResponseStream. Cross-region inference is a Bedrock feature that routes requests across multiple AWS Regions instead of keeping them in just one, so if one Region is busy, another can still take the request. The routing is automatic, but the throughput is still limited per model and per Region, and that limit is what I was hitting. To fix this, I went to the Service Quotas console, found that quota for Claude Sonnet 4.5, and requested an increase to 5,000,000 tokens per minute. Once the request showed up as Case Closed, I could finally start testing in the Playground. Note: It looks like every model comes with its own quota, and none of them are active on the account by default, so each one has to be requested separately. This isn't very well documented or explained by AWS. I only found out how it works by poking around Service Quotas myself. The Playground The Playground is where I can test the model's responses directly in the AWS console and easily adjust parameters like Temperature, Top P and Top K. These parameters help tune the kind of response I get from the model. Temperature is a value between 0 and 1. At 0, the model sticks to less creative options and returns a cold, mechanical response. At 1, it leans toward more creative options and may hallucinate more. Top P limits the model's vocabulary. A value of 0.5 means it only chooses from the pool of words whose probabilities add up to 50%, while 0.95 means it chooses from the pool of words that add up to 95%. So a higher value makes the response more unpredictable, as it can choose a word that had a probability of 5%, for example. Top K works a bit differently: it limits the model to the K most probable words. The higher the value, the bigger that pool of words gets for each prediction. For the tests below, I kept the default value of Top K and set the temperature to 0.7, which seems to work well for customer relations. The responses aren't rigid and cold, but they also aren't so creative that the model starts to hallucinate. Besides these sampling parameters, the Playground also lets me set a System Prompt, which gives the model the context it needs to generate the response for any user query. To see how much difference it makes, I ran two tests. For the first test, I used a very simple System Prompt, just "Act like a ticket clerk". Then I asked for tickets for the game on Sunday, and the model gave a generic response: it asked which game I meant, how many tickets I needed, my seating preference and my budget. For the second test, I used a longer, more detailed System Prompt. I told the model that it sells tickets for an AWS Training Course with AI, and that it should talk ONLY about the regular and VIP tickets, including the price and what's included in each one. This time, when I asked for tickets for the game on Sunday, the model didn't fall for it. It clarified that it only sells tickets for the AWS Training Course, not for a game, and presented the two ticket options instead. When I said I was interested in the VIP ticket, it went on to explain what's included in it. So a detailed System Prompt makes a big difference in keeping the model focused on the task I actually want it to do. Scripting with the Converse API Moving from the console to code, I wrote a simple Bash script. It sets the System Prompt, sends a single user message, and prints the model's response along with the token metrics. # V1 - Basic Chatbot with Converse API # Demonstrates only the use of System Prompt #Use inference profile (cross-region) - prefix "us." MODEL_ID="us.anthropic.claude-sonnet-4-5-20250929-v1:0" REGION="us-east-1" # System Prompt: behavior + context BEHAVIOR="Act like a ticket clerk." CONTEXT="You will sell my AWS Training Course with AI. Talk ONLY about the regular and VIP tickets. Regular ticket: $15 VIP ticket: $30 Regular: Live Event + Recording available for 2 days VIP: Same from regular + extra class + Recording available for 7 days." SYSTEM_PROMPT="$BEHAVIOR. $CONTEXT" USER_PROMPT="I want to buy a ticket for the event" echo "🤖 Ticket Selling Chatbot - V1 Basic (Converse API)" echo "===================================================" echo "" echo "📝 System Prompt: " echo "$SYSTEM_PROMPT" echo "" echo "⏳ Sending to Bedrock..." echo "" SYSTEM_JSON=$(jq -n --arg text "$SYSTEM_PROMPT" '[{"text": $text}]') # Calling Bedrock with Converse API RESPONSE=$(aws bedrock-runtime converse \ --region $REGION \ --model-id $MODEL_ID \ --system "$SYSTEM_JSON" \ --messages '[{"role":"user","content":[{"text":"'"$USER_PROMPT"'"}]}]' \ --inference-config '{"maxTokens":512,"temperature":0.7}' \ --output json) ASSISTANT_RESPONSE=$(echo $RESPONSE | jq -r '.output.message.content[0].text') echo "💳 ASSISTANT:" echo "$ASSISTANT_RESPONSE" echo "" # Metrics INPUT_TOKENS=$(echo $RESPONSE | jq -r '.usage.inputTokens') OUTPUT_TOKENS=$(echo $RESPONSE | jq -r '.usage.outputTokens') TOTAL_TOKENS=$(echo $RESPONSE | jq -r '.usage.totalTokens') echo "📊 Metrics" echo " Input Tokens: $INPUT_TOKENS" echo " Output Tokens: $OUTPUT_TOKENS" echo " Total Tokens: $TOTAL_TOKENS" As you can see in the image below, the response followed the System Prompt and used a total of only 243 tokens. For something closer to a real chatbot, I modified the script to keep accepting user prompts until the user quits. The System Prompt is also read from a file now, though I didn't change its contents for this part. #V2 - Chatbot with Converse API in Interactive Loop ... # Read System Prompt from external file SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SYSTEM_PROMPT=$(cat "$SCRIPT_DIR/system_prompt.txt") SYSTEM_JSON=$(jq -n --arg text "$SYSTEM_PROMPT" '[{text: $text}]') # Initialize empty messages history MESSAGES_JSON='[]' echo "🤖 Ticket Selling Chatbot - V2 Interactive Loop" echo "===================================================" echo "" echo "📄 System prompt carregado de: system-prompt.txt" echo "" echo "💡 Type 'quit' to exit" echo "" while true; do echo -n "👤 You: " read USER_INPUT if [[ "$USER_INPUT" == "quit" ]]; then echo "👋 Bye!" break fi MESSAGES_JSON=$(echo "$MESSAGES_JSON" | jq --arg text "$USER_INPUT" '. += [{role: "user", content: [{text: $text}]}]') # Calling Bedrock with Converse API RESPONSE=$(aws bedrock-runtime converse \ --region $REGION \ --model-id $MODEL_ID \ --system "$SYSTEM_JSON" \ --messages "$MESSAGES_JSON" \ --inference-config '{"maxTokens":512,"temperature":0.7}' \ --output json) ASSISTANT_RESPONSE=$(echo $RESPONSE | jq -r '.output.message.content[0].text') MESSAGES_JSON=$(echo "$MESSAGES_JSON" | jq --arg text "$ASSISTANT_RESPONSE" '. += [{role: "assistant", content: [{text: $text}]}]') # Show response echo "🤖 ASSISTANT:" echo "$ASSISTANT_RESPONSE" echo "" done The interactive version allows the user to ask follow-up questions within the same context. Streaming with Node Then I moved the project to JavaScript and ran it with Node. The AWS SDK for JavaScript lets me call the streaming version of the Converse API. // V3 - Streaming with SDK Node.js // Version with code streaming for real time responses import { readFileSync } from "node:fs"; import { fileURLToPath } from "node:url"; import { dirname, join } from "node:path"; import readline from "node:readline/promises"; import { stdin as input, stdout as output } from "node:process"; import { BedrockRuntimeClient, ConverseStreamCommand, } from "@aws-sdk/client-bedrock-runtime"; // Use inference profile (cross-region) - prefix "us." const MODEL_ID = "us.anthropic.claude-sonnet-4-5-20250929-v1:0"; const REGION = "us-east-1"; // Read System Prompt from external file const SCRIPT_DIR = dirname(fileURLToPath(import.meta.url)); const SYSTEM_PROMPT = readFileSync(join(SCRIPT_DIR, "complete_prompt.txt"), "utf-8"); const SYSTEM_JSON = [{ text: SYSTEM_PROMPT }]; // Initialize empty messages history let messages = []; const client = new BedrockRuntimeClient({ region: REGION, maxAttempts: 5, retryMode: "adaptive", }); const rl = readline.createInterface({ input, output }); console.log("🤖 Ticket Selling Chatbot - V3 Streaming"); console.log("==================================================="); console.log(""); console.log("📄 System prompt carregado de: complete_prompt.txt"); console.log(""); console.log("💡 Type 'quit' to exit"); console.log(""); while (true) { const userInput = await rl.question("👤 You: "); if (userInput === "quit") { console.log("👋 Bye!"); break; } messages.push({