RAG Powered Apps with Amazon Bedrock, Part 2: Automating the RAG Pipeline with Terraform

Before you start: This picks up where Part 1 left off. From part 1, you would've learned how to setup a Bedrock Knowledge Base in the console. In addition to that, you should have a general understanding of how the ingestion and query pipeline works. Introduction & Motivation I started this project with a singular goal: to build a comprehensive Terraform module that allows developers to deploy the entire infrastructure for a "Chat with PDF" application faster. When Amazon Bedrock was first unveiled in April 2023, I jumped in immediately. Like many of you, I built several proof-of-concepts (PoCs) through the AWS Console. The UI is amazing for building quick pocs, but once I moved into experimentation, I realized it would be best to quickly setup and tear down the infra. An example use case was testing if there were any cost savings in using S3 Vectors vs OpenSearch and how much cost savings exactly. None of the Terraform modules I found on GitHub (at the time) seemed to cover the end-to-end pipeline I was looking for, so I decided to build mine. I'm also big on learning so why not. What Are We Building? A couple of terraform modules to automate everything we clicked through manually in Part 1. One terraform apply brings up the full stack: - S3 Bucket: your document store. Encrypted at rest, versioning on, zero public access. - OpenSearch Serverless: the vector database. Stores the embeddings Bedrock generates during ingestion. - Bedrock Knowledge Base: orchestrates the chunking, embedding, and storage of documents, and retrieval at query time. - Ingestion Lambda: triggered automatically when you upload a file to S3. Starts a Bedrock ingestion job so documents are chunked, embedded, and indexed without ClickOps. - Query Lambda: accepts a natural language question, calls RetrieveAndGenerate , and returns an answer with source citations. Full source code + ReadMe: Bedrock Project. If you run into issues or want to extend the module, feel free to open an issue. Architecture sequenceDiagram participant User participant S3 participant IngestionLambda as Ingestion Lambda participant Bedrock participant Titan as Titan (Embeddings) participant OSS as OpenSearch Serverless participant QueryLambda as Query Lambda participant Claude Note over S3,OSS: Ingestion Phase User->>S3: Upload document S3->>IngestionLambda: S3 ObjectCreated event IngestionLambda->>Bedrock: StartIngestionJob Bedrock->>S3: Fetch document Bedrock->>Titan: Chunk + embed text Titan-->>Bedrock: Vectors Bedrock->>OSS: Store vectors + metadata Note over QueryLambda,Claude: Query Phase User->>QueryLambda: Invoke with question QueryLambda->>Bedrock: RetrieveAndGenerate Bedrock->>Titan: Embed query Titan-->>Bedrock: Query vector Bedrock->>OSS: Search for similar vectors OSS-->>Bedrock: Top matching chunks Bedrock->>Claude: Query + chunks Claude-->>QueryLambda: Answer + citations QueryLambda-->>User: Answer + source citations In Part 3 we'll put an API Gateway in front of the query Lambda. For now we're invoking it directly from the CLI. Project Structure rag-bedrock-project/ ├── main.tf ├── variables.tf ├── outputs.tf ├── backend.tf ├── terraform.tfvars.example ├── bootstrap/ └── modules/ ├── storage/ ├── opensearch/ ├── bedrock/ └── lambda/ Each module owns one piece of the infrastructure and exposes what other modules need through outputs. The root main.tf wires them together by passing outputs from one module as inputs to another. To be honest, I went back and forth on this architecture, and granted having multiple modules might be overkill. But designing this took me back to my Node.js applications days where I would put everything in a single server.js which made it difficult to debug errors. I learned about MVC which changed the way I build software. Terraform modules clicked the same way for me. One module per function. The Lambda module does not need how OpenSearch is set up. It just gets the IDs it needs through variables. As the architect, you know how each module communicates with the others. Implementation Step 1: Bootstrap Remote State First Before running terraform apply on anything, you need somewhere to store your Terraform state. Hold up? State what? Terraform state essentially tells Terraform what infrastructure already exists. Every resource it creates gets recorded in a terraform.tfstate file. Without it, Terraform can't tell what's already deployed. If your local state file is ever lost or corrupted, Terraform loses track of everything it deployed. Storing it in S3 keeps it versioned and safe. Terraform 1.10 introduced native S3 state locking so you don't need a seperate DynamoDB table. You can read more here The bootstrap/ directory sets this up. Run it once before anything else All commands use aws-vault, which stores AWS credentials in your OS keychain and injects temporary credentials at runtime. The --no-session flag skips STS session tokens, which some IAM operations reject. If you're not using aws-vault, replaceaws-vault exec YOUR_PROFILE --no-session -- with your usual credential method. aws-vault exec YOUR_PROFILE --no-session -- \ terraform -chdir=bootstrap init aws-vault exec YOUR_PROFILE --no-session -- \ terraform -chdir=bootstrap apply \ -var="project_name=my-rag" -var="environment=dev" Bootstrap creates two things: the S3 state bucket, and a scoped deployer IAM policy. You need AdministratorAccess to run bootstrap itself, because you can't use a scoped policy to create the scoped policy. Once it's done, you attach the scoped policy to your IAM user, detach AdministratorAccess , and every deploy from here runs least-privilege. After it finishes, two outputs matter: backend_config = 15 seconds left in the timeout budget let response; try { response = await client.send(command); } catch (error) { if (isRetryable(error) && hasTimeForRetry(context)) { response = await client.send(command); } else { return buildResponse(503, { error: "Service temporarily unavailable" }); } } return buildResponse(200, { answer: response.output?.text || "", citations, }); } The retry logic checks context.getRemainingTimeInMillis() before retrying. If there's less than 15 seconds left, we fail fast instead of starting a request we can't finish. Better a clean 503 than a Lambda timeout. Logging follows a strict rule: structured JSON, no PII. We log request IDs, durations, and error types, but never the actual query or response content: function log(level, requestId, message) { console.log(JSON.stringify({ level, timestamp: new Date().toISOString(), requestId: requestId || undefined, message, })); } Security: Lambda dependencies are locked in package-lock.json . No floating version ranges that could pull in a compromised package on the next deploy. Step 4: Deploy The Bedrock module needs the AOSS collection endpoint. The OpenSearch module needs the Bedrock KB role ARN. Both modules need something the other creates. On a fresh deploy, Terraform can't resolve both at once. Fix: Create the collection first in a separate targeted apply, so the endpoint exists by the time the full apply runs. The README has the complete deployment reference. The short version: # Step 1: create the OSS collection first aws-vault exec YOUR_PROFILE --no-session -- terraform apply -target=module.opensearch # Step 2: deploy everything else aws-vault exec YOUR_PROFILE --no-session -- terraform apply Go make coffee. AOSS takes about 10 minutes to spin up. You only need to do this once. After that first run, the endpoint is stored in state and every subsequent deploy is just terraform apply . One thing worth knowing if you're using aws-vault: some IAM operations reject session tokens that aws-vault generates by default. If you hit an InvalidClientTokenId error mid-apply, make sure you're using --no-session . Step 5: Test It Upload, check status, and invoke Upload a document. The ingestion Lambda fires automatically on upload. No manual trigger needed: aws-vault exec YOUR_PROFILE --no-session -- \ aws s3 cp ./my-document.pdf \ s3://$(terraform output -raw document_bucket_name)/ Check ingestion status and wait for COMPLETE before querying: aws-vault exec YOUR_PROFILE --no-session -- \ aws bedrock-agent list-ingestion-jobs \ --knowledge-base-id $(terraform output -raw knowledge_base_id) \ --region us-east-1 Then invoke the query Lambda: aws-vault exec YOUR_PROFILE --no-session -- \ aws lambda invoke \ --function-name $(terraform output -raw query_function_name) \ --payload '{"body":"{"query":"What is the document about?"}","requestContext":{"requestId":"test-1"},"headers":{}}' \ --cli-binary-format raw-in-base64-out \ /tmp/response.json && cat /tmp/response.json The payload wraps the query in a body field because the Lambda parses event.body.query . A successful response looks like: { "statusCode": 200, "headers": { "Content-Type": "application/json" }, "body": "{"answer":"The document covers...","citations":[{"text":"...","sources":[{"uri":"s3://my-rag-dev-documents/my-document.pdf"}]}]}" } If you're getting empty answers, ingestion likely isn't done yet. Wait for the status to show COMPLETE and try again. Step 6: Cleanup Before you run terraform destroy , there's one thing to sort out. The data source has a data_deletion_policy that defaults to DELETE . When Terraform tears down the stack, it tries to clean up vectors from OpenSearch as part of deleting the data source. If the collection is also being destroyed in the same apply, Bedrock can't reach it and the deletion gets stuck. Set it to RETAIN first, apply, then destroy: # modules/bedrock/main.tf resource "aws_bedrockagent_data_source" "s3" { name = "${var.config.environment}-${var.config.project_name}-s3-source" knowledge_base_id = aws_bedrockagent_knowledge_base.main.id data_deletion_policy = "RETAIN" # set this before destroying # ... rest of config } You could also use the console to do this aws-vault exec YOUR_PROFILE --no-session -- terraform apply Full cleanup: tear down the stack aws-vault exec YOUR_