Test Coding Agent Pull Requests with Argo CD and Signadot
DEV Community

Test Coding Agent Pull Requests with Argo CD and Signadot

Read this tutorial on Signadot docs.

Coding agents open pull requests faster than anyone can test them by hand. If you already run Argo CD, it can give each of those pull requests a preview of its own, and the Argo CD preview environments article explains why a Signadot sandbox beats a full copy of the app for that.

This tutorial builds the whole loop. A coding agent changes HotROD, a demo ride-sharing app, and opens a pull request. An Argo CD ApplicationSet turns the pull request into an Application whose hooks create a sandbox for the changed route service, run the pull request's Playwright suite against it, and delete the sandbox when the pull request closes. Argo CD Notifications reports the result as an argocd/preview check.

When the check fails, the agent reads why through the read-only Argo CD MCP server, pushes a fix and waits for the next result.

Time: about 40 minutes. Example files: signadot/examples/argocd-tutorial

Tip: What the agent can and can't do

The agent pushes commits and reads Argo CD. It needs no kubeconfig and no Signadot API key, and the repository's Claude Code settings block kubectl, argocd and signadot. It can't sync or change anything in Argo CD, and its Argo CD account sees only preview Applications. The preview is its only way to test end to end.

How it works

Each piece does one job:

  • GitHub Actions - Builds ghcr.io/<owner>/hotrod:<head sha> for every commit on a pull request labeled preview.
  • ApplicationSet - Creates one Application, hotrod-pr-<number>, for every open pull request with the preview label, and deletes it when the pull request closes.
  • Preview chart - A ConfigMap with the sandbox specification and three hook Jobs: Sync creates or updates the sandbox, PostSync runs the tests, PostDelete deletes the sandbox.
  • Signadot - The sandbox runs the pull request's image in place of route for requests that carry its routing key. Everything else comes from the baseline.
  • Argo CD Notifications - Sets the argocd/preview commit status: pending when a sync starts, then success or failure.
  • Argo CD MCP server - Lets the agent read the Application's status, events and hook logs, and nothing else.

Argo CD doesn't build anything here. It does what it already does for the baseline, reconciling what Git says should exist, plus one post-deploy test. The Argo CD integration guide compares this with letting CI run the tests.

The preview chart lives in the examples repository, not in HotROD, so an agent's pull request can't change how its own preview is created or tested. It controls only its code and its Playwright tests.

Prerequisites

  • A Kubernetes cluster with the Signadot Operator installed. Docker Desktop, kind or minikube works.
  • The Signadot CLI, logged in, with a local connection for the cluster in ~/.signadot/config.yaml:
local:
  connections:
    - cluster: <your-signadot-cluster>
      kubeContext: <your-kube-context>
      type: PortForward
  • A Signadot API key for the preview hooks, from a service account under Admin → Service Accounts in the dashboard. See API Keys.
  • The argocd, gh and kubectl CLIs, and Node.js for npx.
  • Claude Code, or another coding agent that supports MCP.

The steps below pin Argo CD v3.5.6 and argocd-mcp 0.9.0.

Step 1: Fork HotROD and add the preview files

The agent works in your fork of HotROD, so its pull requests and images are yours. Clone the examples repository next to it and copy the preview files into the fork:

export OWNER=<your-github-user> # lowercase
git clone https://github.com/signadot/examples.git
gh repo fork signadot/hotrod --clone
cd hotrod
gh repo set-default $OWNER/hotrod
cp -R ../examples/argocd-tutorial/hotrod/. .
git rm -q .github/workflows/build.yml
git add -A && git commit -m "Add Argo CD previews" && git push origin main
gh label create preview

gh repo fork --clone makes the upstream repository the default for gh. Without gh repo set-default, the agent's gh pr create would open its pull request on signadot/hotrod, where your ApplicationSet never sees it.

HotROD's own build.yml needs Signadot's secrets, so the fork drops it.

The copied files:

  • .github/workflows/preview-image.yaml - Builds and pushes the image for each commit on a preview pull request.
  • AGENTS.md and CLAUDE.md - Tell the agent how previews work and which MCP tools to call when one fails. CLAUDE.md imports AGENTS.md.
  • .mcp.json - Starts the Argo CD MCP server with MCP_READ_ONLY=true.
  • .claude/settings.json - Denies Claude Code kubectl, argocd and signadot.

The workflow triggers on labeled, synchronize and reopened. GitHub sends a labeled event for labels set when a pull request is opened, so each commit is built once:

hotrod/.github/workflows/preview-image.yaml (excerpt)

on:
  pull_request:
    types: [labeled, synchronize, reopened]
jobs:
  build:
    if: >-
      github.event.action == 'labeled' && github.event.label.name == 'preview' ||
      github.event.action != 'labeled' && contains(github.event.pull_request.labels.*.name, 'preview')

Open your fork's Actions tab and enable workflows. GitHub disables them on new forks.

Step 2: Install Argo CD and add two secrets

kubectl create namespace argocd
kubectl apply -n argocd --server-side --force-conflicts \
  -f https://raw.githubusercontent.com/argoproj/argo-cd/v3.5.6/manifests/install.yaml
kubectl create namespace hotrod-previews
kubectl -n hotrod-previews create secret generic signadot \
  --from-literal=SIGNADOT_ORG=<your-org> --from-literal=SIGNADOT_API_KEY=<your-api-key>
kubectl -n argocd patch secret argocd-notifications-secret \
  -p '{"stringData": {"github-token": "<your-github-token>"}}'

The preview's hook Jobs run in hotrod-previews and read the Signadot key from the signadot secret. The GitHub token lets the ApplicationSet list your pull requests and Argo CD Notifications set commit statuses. Create a fine-grained token for your fork with Commit statuses: Read and write and Pull requests: Read.

Step 3: Deploy the baseline and the ApplicationSet

export CLUSTER=<your-signadot-cluster>
cd ../examples/argocd-tutorial
sed -i.orig -e "s/<owner>/$OWNER/g" -e "s/<cluster>/$CLUSTER/g" argocd/*.yaml
kubectl apply --server-side --field-manager=argocd-tutorial -f argocd/
kubectl -n argocd get applications
NAME    SYNC STATUS   HEALTH STATUS
hotrod  Synced         Healthy

This applies three files: hotrod.yaml deploys HotROD from your fork's main branch, with the Signadot DevMesh sidecar, into the hotrod namespace. If HotROD already runs there, Argo CD adopts it. previews.yaml holds the previews AppProject and the ApplicationSet. config.yaml adds the agent's account and the notification settings.

The separate field manager keeps the defaults the Argo CD install put in argocd-cm; with the default kubectl manager, this apply would remove them.

The ApplicationSet polls GitHub every minute, because no webhook can reach a local cluster. Each Application renders the preview chart with the pull request's number and head commit:

argocd/previews.yaml (excerpt)

generators:
  - pullRequest:
      github:
        owner: <owner>
        repo: hotrod
        labels:
          - preview
        tokenRef:
          secretName: argocd-notifications-secret
          key: github-token
      requeueAfterSeconds: 60
template:
  metadata:
    name: hotrod-pr-{{ .number }}
  spec:
    project: previews
    source:
      repoURL: https://github.com/signadot/examples.git
      targetRevision: main
      path: argocd-tutorial/preview
      helm:
        valuesObject:
          pr: "{{ .number }}"
          headSha: "{{ .head_sha }}"
          image: ghcr.io/<owner>/hotrod:{{ .head_sha }}
    # ...
    syncPolicy:
      automated:
        prune: true
      retry:
        limit: 0

Two settings matter for agent feedback: retry.limit: 0. Argo CD 3.5 retries a failed automated sync five times by default, which delays a failing result by several minutes. Here a failed preview reports at once, and the agent's next push is the retry.

The sandbox specification is a ConfigMap. Argo CD doesn't count hooks when it compares an Application with Git. The ConfigMap changes with every commit, so each push makes the Application OutOfSync, and the automated sync reruns the hooks.

The sandbox specification forks only route and runs the pull request's image:

preview/templates/sandbox.yaml (rendered)

name: hotrod-pr-323
spec:
  cluster: <your-signadot-cluster>
  description: HotROD pull request 323
  ttl:
    duration: 3d
    offsetFrom: updatedAt
  forks:
    - forkOf:
        kind: Deployment
        namespace: hotrod
        name: route
      customizations:
        images:
          - image: ghcr.io/<owner>/hotrod:<head sha>
  defaultRouteGroup:
    endpoints:
      - name: frontend
        target: http://frontend.hotrod.svc:8080

Keep the description the same for every commit: Signadot rejects updates that change it. The TTL is a backstop in case the PostDelete hook never runs.

The hooks:

  • Sync (hotrod-pr-<number>-sandbox). Container wait-for-image uses crane to wait until CI has pushed the image. Container signadot then runs signadot sandbox apply and waits for the sandbox to be ready.
  • PostSync (hotrod-pr-<number>-e2e). Init container routing-key looks up the sandbox's routing key. It is the only container in this Job that holds the Signadot API key, and it fails the Job if no key comes back, so the tests never run against the baseline by mistake. Container e2e fetches the pull request's head commit and runs npm run e2e:playwright with SIGNADOT_ROUTING_KEY set.
  • PostDelete (hotrod-pr-<number>-cleanup). Runs signadot sandbox delete after the ApplicationSet deletes the Application.

The previews AppProject allows only ConfigMaps, Jobs and Pods, and only in hotrod-previews. Pods are listed so that Argo CD shows the hook Jobs' Pods, which is where it reads their logs.

Step 4: Give the agent read-only access

signadot local connect
argocd login argocd-server.argocd.svc --insecure --username admin \
  --password "$(argocd admin initial-password -n argocd | head -1)"
export ARGOCD_API_TOKEN=$(argocd account generate-token --account agent --expires-in 24h)
argocd logout argocd-server.argocd.svc --insecure

signadot local connect makes in-cluster addresses such as argocd-server.argocd.svc resolve on your machine. The Argo CD UI is at https://argocd-server.argocd.svc. Logging out keeps your admin session out of the shell the agent runs in.

The agent account comes from config.yaml. It has an API key and no login, and its RBAC allows reading preview Applications and their logs:

argocd/config.yaml (excerpt)

accounts.agent: apiKey
---
policy.csv: |
  p, agent, applications, get, previews/*, allow
  p, agent, logs, get, previews/*, allow

.mcp.json passes the token to the MCP server and turns on read-only mode, which removes the create, update, delete, sync and resource-action tools:

hotrod/.mcp.json

{
  "mcpServers": {
    "argocd": {
      "command": "npx",
      "args": ["-y", "argocd-mcp@0.9.0", "stdio"],
      "env": {
        "ARGOCD_BASE_URL": "${ARGOCD_BASE_URL:-https://argocd-server.argocd.svc}",
        "ARGOCD_API_TOKEN": "${ARGOCD_API_TOKEN}",
        "MCP_READ_ONLY": "true",
        "NODE_TLS_REJECT_UNAUTHORIZED": "0"
      }
    }
  }
}

This Argo CD uses a self-signed certificate, so --insecure and NODE_TLS_REJECT_UNAUTHORIZED=0 skip verification. Drop both once Argo CD has a trusted certificate.

Step 5: Hand the agent the task

cd ../../hotrod
claude

Approve the argocd MCP server from .mcp.json, then give it the task from prompt.md: In the route service, make the "Resolving routes" notification show the ETA that route computed, for example "Resolving routes (ETA 12m30s)". Open a pull request for it and don't stop until its preview passes. Then tell me the PR URL, the preview URL and what you checked.

In our run, Claude Code found the notification in services/route/server.go and the Playwright test that asserts its text, and changed both:

  • Body:"Resolving routes", + Body:fmt.Sprintf("Resolving routes (ETA %s)", eta),
  • ).toHaveText("Resolving routes") + ).toHaveText(/^Resolving routes \(ETA (\d+h)?(\d+m)?\d+s\)$/);

It opened the pull request with gh pr create --label preview and waited with gh pr checks --watch. Within a minute, the ApplicationSet created hotrod-pr-323 and the check turned pending.

The Sync hook waited for the image:

waiting for CI to push ghcr.io/<owner>/hotrod:<head sha>: Error: GET https://ghcr.io/v2/<owner>/hotrod/manifests/<head sha>: MANIFEST_UNKNOWN: manifest unknown

The image build took under two minutes. Then the Sync hook created the sandbox and the PostSync hook ran the pull request's tests against it:

Testing 6651b6c22905f7543e15ef683033570a444ff956 in sandbox hotrod-pr-323 (routing key ttdg1bpt6cgcj)
Running 2 tests using 1 worker
✓1 [chromium] › basic.spec.ts:4:6 › Request a ride › check route resolve routes (2.7s)
✓2 [chromium] › basic.spec.ts:17:6 › Request a ride › check driver series (2.5s)
2 passed (6.7s)

About two and a half minutes after the push, argocd/preview passed. The agent then called get_application_workload_logs on the e2e Job to confirm the run tested its own commit, and put the preview URL in the pull request description.

The first image build creates the package ghcr.io/<owner>/hotrod. If GitHub makes it private, the Sync hook keeps logging DENIED after the build succeeds. HotROD is open source, so make the package public in its Package settings, and the preview continues on its own. Keep images of your own services private; see Use it with your own services.

Step 6: Watch the agent fix a failing preview

A passing first try doesn't show the loop's real value, so break the preview the way a teammate might. driver gives each route call a one-second deadline (services/route/client.go). Once the agent is done, in a second terminal in hotrod, with the agent's branch checked out, push a commit that makes route slower than that:

perl -pi -e 's/50 \* time\.Millisecond/1500 * time.Millisecond/' pkg/config/route.go
git commit -am "route: simulate a slower routing engine" && git push

The change builds and passes unit tests. In the preview, driver gives up on every route lookup, no ride gets a driver, and both Playwright tests fail:

Error: Timed out 5000ms waiting for expect(locator).toHaveText(expected)
Locator: locator('//div[p[2
Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.