2B Gemma 4 Deployment with Cloud Run, NVIDIA L4, MCP SDK 2.x, and Claude Code
DEV Community

2B Gemma 4 Deployment with Cloud Run, NVIDIA L4, MCP SDK 2.x, and Claude Code

2B Gemma 4 Deployment with Cloud Run, NVIDIA L4, MCP SDK 2.x, and Claude Code

This article provides a step-by-step deployment guide for Gemma 4 E2B to a Cloud Run hosted GPU-enabled system. A suite of Python MCP tools is built to simplify management of the vLLM hosted deployment with Claude Code.

What is this project trying to do?

This project is a DevOps/SRE assistant for a Gemma 4 model served by vLLM on Cloud Run with an NVIDIA L4 GPU. A single-file Python MCP server provides tools to stage the weights, deploy the service, check its health, benchmark it, and tear it down. Cloud Run is the serverless option. There is no VM to provision and no driver to install: the service scales to zero when idle, and one gcloud command attaches the GPU.

Where do I start?

The strategy for starting MCP development for model management is an incremental step-by-step approach. First, the basic development environment is set up with the required system variables and a working Claude Code configuration. Then, the Python MCP server is brought up over stdio and validated with Claude Code in the local environment. That server then stages the model, deploys it to Cloud Run, and drives validation and a benchmark sweep against the live endpoint.

Prerequisites

  • Python 3.10 or newer - mcp 2.x declares Requires-Python >=3.10
  • Claude Code installed and working
  • The Google Cloud SDK, logged in, with application default credentials
  • A Google Cloud project with Cloud Run GPU access in your region (this one runs in us-east4)
  • A GCS bucket named <project>-bucket for the model weights

Setup the Basic Environment

  1. Clone the repository and switch to the Cloud Run directory:
    cd ~
git clone https://github.com/xbill9/gemma4-dev
cd gemma4-dev/gpu-2B-cloudrun-devops-agent
  2. Run init.sh once. It checks your gcloud login and application default credentials, asks for a project ID, installs the Python requirements, enables the Cloud Run, Secret Manager and related APIs, and grants the default compute service account its roles.
    source init.sh
  3. If your session times out or you need to reset your variables, run set_env.sh:
    source set_env.sh

Model Management Tool with MCP Stdio Transport

One of the key features that the MCP libraries provide is abstracting various transport methods. The tool implementation is the same no matter which transport the MCP client uses to connect. The simplest transport is stdio - the client launches the server as a local process and talks to it over stdin and stdout.

Wait - Why MCPServer and not FastMCP?

Nothing in the repository changed. A fresh install did. requirements.txt listed mcp with no version bound, so the next pip install resolved the 2.x line, and the server stopped importing: python3 -c "from mcp.server.fastmcp import FastMCP" raise ModuleNotFoundError(_MESSAGE, name=__name__)

Running the Python Code

The project can be linted:

make lint

All checks passed!

Test the Protocol by Hand

Unit tests call Python. A client speaks JSON-RPC over stdio, so test that too. Hold stdin open with sleep - with a bare printf pipe the server sees end-of-input and exits after answering only initialize:

printf '%s\n' \
'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"0"}}}' \
'{"jsonrpc":"2.0","method":"notifications/initialized"}' \
'{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}' \
'{"jsonrpc":"2.0","id":3,"method":"resources/list","params":{}}' \
| python3 server.py 2>/dev/null

Validation with Claude Code

Check the connection from Claude Code to the local server:

claude mcp get cloudrun-devops

If the server failed at startup earlier in the session, reconnect it from /mcp or start a new session to pick up the fixed code.

Model Lifecycle Management via MCP

The MCP tools cover the whole lifecycle of the Cloud Run deployment. Every tool is prefixed cloudrun_. Abridged output of cloudrun_get_help:

cloudrun_get_help

The server is running in CLOUD RUN mode targeting NVIDIA L4 GPU in region us-east4.

Stage the Model Weights

Cloud Run mounts the bucket read-only at /mnt/models through GCS FUSE, so the weights go to GCS once. Download to a real disk rather than /tmp, which on this host is a RAM-backed tmpfs smaller than the model:

hf download google/gemma-4-E2B-it --local-dir ~/hf-downloads/gemma-4-E2B-it
gcloud storage rsync ~/hf-downloads/gemma-4-E2B-it gs://aisprint-491218-bucket/gemma-4-E2B-it

Deploy to Cloud Run

The deploy-vllm target in the Makefile is the single source of truth for the vLLM and Cloud Run flags. The ones that matter most:

make deploy

Checking System Status

The status can be checked with an MCP tool:

cloudrun_get_system_status

The GPU Cloud Run System Status is 🟢 Online (https://gpu-2b-l4-devops-agent-289270257791.us-east4.run.app).

Next Step

Use cloudrun_query_gemma4 to interact with the model.

Cross Check The Deployed Model

Ask vLLM what it loaded:

curl -s -H "Authorization: Bearer $(gcloud auth print-identity-token)" \
https://gpu-2b-l4-devops-agent-289270257791.us-east4.run.app/v1/models | python3 -m json.tool
Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.