From Zero to Green Build: How I Built a Jenkins + DevPod CI Pipeline with Mise (+ 10 Real Errors That Taught Me Everything)

A complete hands-on guide to building an isolated, reproducible CI pipeline with Jenkins Controller, SSH-connected DevPod agents, and Mise runtime management. Continuous Integration (CI) setups frequently suffer from a classic problem: the "Works on My Machine" dilemma. Local development happens in nicely customized containers or modern toolchains, while CI build servers either drift into giant "snowflakes" full of globally installed packages, or run bloated custom VMs that take forever to configure and maintain. In this project, I set out to build a modern, isolated, and fully reproducible CI lab environment: - Jenkins Controller handles scheduling and orchestration. - DevPod (Containers via SSH) provides isolated, reproducible build agents. - Mise acts as a declarative polyglot toolchain manager ( mise.toml ) to deterministically manage runtimes (Python, Java 21, Node.js, and Jenkins CLI) without polluting global paths. Along the way, things did not work on the first try. I hit subtle shell differences, monorepo path traps, SSH agent quirks, and environment activation edge cases. Here is the complete walkthrough of how the architecture works, how every file is configured, and the 10 real-world errors that taught me foundational DevOps lessons. 🏗️ Architecture & High-Level Flow The goal was simple: separate the orchestrator from the execution environment. The Jenkins Controller should never execute heavy compilation or test tasks directly; it should delegate build execution to an ephemeral DevPod container agent over SSH. flowchart TD subgraph GitHub["GitHub Repository (alanvarghese-dev/lab)"] Repo["Source Code & Jenkinsfile"] end subgraph Jenkins_Master["Jenkins Controller"] JC["Jenkins Controller\n(Scheduler & Orchestrator)"] end subgraph DevPod_Agent["DevPod Build Agent (Container)"] Agent["Jenkins Agent Node\n(Label: 'devpod')"] Mise["Mise Runtime Manager\n(mise.toml)"] Python["Python Runtime\n(Latest via Mise)"] Pytest["pytest\n(Test Suite Runner)"] App["Application Build\n(app.py / test_app.py)"] end Repo -->|SCM Trigger / Poll| JC JC -->|SSH Connection| Agent Agent --> Mise Mise --> Python Python --> Pytest Pytest --> App Complete End-to-End Flow GitHub (alanvarghese-dev/lab/jenkins-pipeline) ↓ Jenkins Controller (Orchestrates Job & evaluates Jenkinsfile) ↓ [SSH] DevPod Jenkins Agent (Container with label: 'devpod') ↓ Mise (Resolves toolchain from mise.toml) ↓ Python & pip (Installs dependencies from requirements.txt) ↓ pytest (Executes unit test suite) ↓ Build Status: SUCCESS ✅ 🛠️ The Tech Stack | Component | Technology | Purpose | |---|---|---| | CI Orchestrator | Jenkins | Job orchestration, pipeline stage execution, reporting | | Build Agent | DevPod (Docker) | Isolated, disposable containerized build execution environment | | Base Image | mcr.microsoft.com/devcontainers/base:ubuntu-24.04 | Clean, standardized Linux environment | | Tool Version Manager | Mise | Polyglot runtime versioning (Python, Java 21, Node.js, Jenkins CLI) | | App Runtime & Tests | Python 3 + pytest | Application code (app.py ) & unit tests (test_app.py ) | | Transport | SSH | Controller-to-Agent communication channel | 📁 Repository Structure The project lives in a monorepo setup (alanvarghese-dev/lab ) designed to house multiple DevOps learning projects side by side: lab/ └── jenkins-pipeline/ ├── .devcontainer.json # Devcontainer build definition ├── Dockerfile # DevPod agent image with Mise ├── Jenkinsfile # Declarative Jenkins CI pipeline ├── mise.toml # Tool versions (Java, Python, Node, CLI) ├── app.py # Sample Python application ├── test_app.py # Pytest test suite ├── requirements.txt # Python dependencies (pytest) └── README.md # Project documentation ⚙️ Step-by-Step Configuration 1. Devcontainer Agent Image (Dockerfile & .devcontainer.json ) To build our DevPod agent with Mise pre-baked, we copy the binary directly from the official Mise multi-stage Docker image: # Dockerfile FROM mcr.microsoft.com/devcontainers/base:ubuntu-24.04 COPY --from=jdxcode/mise /usr/local/bin/mise /usr/local/bin/ # Activate mise in interactive shells RUN echo 'eval "$(mise activate bash)"' >> /home/vscode/.bashrc && \ echo 'eval "$(mise activate zsh)"' >> /home/vscode/.zshrc And configure the .devcontainer.json : { "build": { "context": "..", "dockerfile": "Dockerfile" } } 2. Declarative Toolchain (mise.toml ) Instead of installing packages globally via apt-get or installing random pythons across system directories, we declare all required tooling in mise.toml : [tools] "aqua:jenkins-zh/jenkins-cli" = "latest" python = "latest" java = "21" node = "latest" Because this file is checked into version control, any developer opening the container with DevPod or any CI agent running Jenkins gets the exact same runtime versions. 3. The Application & Pytest Suite Keep the application minimal so you can focus 100% on pipeline predictability: # app.py def hello(): return "Hello from Jenkins + Devpod!" if name == "main": print(hello()) # test_app.py from app import hello def test_hello(): assert hello() == "Hello from Jenkins + Devpod!" # requirements.txt pytest 4. The Declarative Jenkins Pipeline (Jenkinsfile ) Here is the complete Jenkinsfile . Notice two critical patterns: - dir('jenkins-pipeline') : Because this is a monorepo, we explicitly change into the project directory so Mise can findmise.toml . - mise exec python -- ... : We avoid brittle shell activation scripts and let Mise directly wrap command execution. pipeline { agent { label 'devpod' } stages { stage('Checkout') { steps { checkout scm } } stage('Setup Python') { steps { dir('jenkins-pipeline') { sh ''' set -e echo "Project directory:" && pwd echo "Mise configuration:" && ls -la mise.toml mise install echo "Python version:" mise exec python -- python --version ''' } } } stage('Install Dependencies') { steps { dir('jenkins-pipeline') { sh ''' set -e mise exec python -- python -m pip install -r requirements.txt ''' } } } stage('Test') { steps { dir('jenkins-pipeline') { sh ''' set -e mise exec python -- python -m pytest ''' } } } } post { success { echo 'Python CI build successful! 🚀' } failure { echo 'Python CI build failed! ❌' } } } 💥 10 Real Errors Encountered & What They Taught Me Building CI pipelines is 10% writing code and 90% debugging subtle environment mismatches. Here are the 10 errors I hit during this lab and the lessons behind them: 1. DevPod Container Created, but Features Failed - Symptom: The container was created in Docker, but DevPod reported connection and tunneling errors. - Lesson: Container creation and DevPod connectivity are distinct stages. Always ensure: Container Running ➔DevPod SSH Access ➔Shell Available before troubleshooting build tools. 2. "No SSH session / exited" - Symptom: DevPod SSH tunneling abruptly dropped. - Lesson: This was an infrastructure/connection issue, not a Jenkins or Python failure. Once devpod ssh succeeded reliably from the terminal, the underlying transport was proven stable. 3. Dotfiles Were Not Applied - Symptom: The agent container spun up without customized shell dotfiles. - Lesson: DevPod’s automated dotfiles mechanism requires setup scripts to be located in conventional locations. 4. "Setup not found" - Symptom: DevPod searched for bootstrap files ( install.sh ,bootstrap.sh ,setup.sh ) and couldn't find them. - Lesson: Automation tools depend on strict conventions. Always verify standard entry points when integrating dotfiles repositories into devcontainers. 5. GitHub SSH Authentication Confusion - Symptom: Output showed: Hi username! You've successfully authenticated, but GitHub does not provide shell access. - Lesson: This is actually a success message from GitHub confirming that your SSH public key is valid. GitHub explicitly denies interactive PTY shells, so don't mistake this for an authentication failure! 6. java: command not found (The Non-Interactive Shell Trap) - Symptom: Java 21 was installed inside DevPod, but Jenkins reported java: command not found . - Lesson: Installed on disk ≠ Available to the Jenkins process. When you log in via devpod ssh , your.zshrc /.bashrc loads your PATH. Jenkins launches a non-interactive/bin/sh shell that skips interactive rc files. Interactive Login: .zshrc / .bashrc loaded ➔ PATH configured ➔ Tools work Jenkins Non-Login: /bin/sh launched ➔ Minimal PATH ➔ Command not found 7. "Python is installed but not activated" - Symptom: Mise reported that Python was installed on the machine, but not active. - Lesson: Mise relies on discovering a valid mise.toml in the current directory or parent tree. If you are in a directory without a config, Mise will not activate the tool by default. 8. Syntax error: "(" unexpected on eval "$(mise activate bash)" - Symptom: Attempting to evaluate Mise activation inside a Jenkins sh step blew up with a syntax error. - Lesson: Jenkins defaults to /bin/sh (often Debian/Ubuntudash ), which does not support certain Bash/Zsh process substitution or subshell syntax. - Solution: Instead of hacking shell startup scripts, use mise exec -- directly in pipeline steps! 9. Mise Could Not Find Python in Monorepo Root - Symptom: mise exec -- python --version failed in Jenkins. - Root Cause: Jenkins checked out the entire monorepo ( lab/ ), and executed commands at the root. Themise.toml was insidelab/jenkins-pipeline/ . - Fix: Use Jenkins' dir('jenkins-pipeline') { ... } block so the working directory matches wheremise.toml lives. 10. Git "Does not have a commit checked out" - Symptom: Git commands failed unexpectedly inside the agent. - Root Cause: Commands ran from /workspaces instead of/workspaces/jenkins-pipeline . - Lesson: Always verify your working directory before running Git operations: pwd && git status . 🔍 The 5-Step CI Troubleshooting Checklist Whenever your Jenkins build fails with a "command not found" or tool version issue, run through this checklist before touching code: flowchart TD A["1. Is the tool installed?\n(mise ls)"] -->|Yes| B["2. Where is t