Rootless container checkpoint/restore in CRIU without CAP_SYS_ADMIN

Rootless container checkpoint/restore in CRIU without CAP_SYS_ADMIN

I've been working on rootless container checkpoint/restore in CRIU and wrote up some notes on the prototype, major failures, and the path toward the current implementation. Would appreciate any feedback, criticism, and corrections, especially from people familiar with CRIU, Podman, or container runtimes.

Background and motivation

Checkpoint/restore allows saving the state of a running container and resuming it later, potentially on a different machine. Historically, this required CAP_SYS_ADMIN privileges, making it impossible for rootless containers. The goal of this work is to enable checkpoint/restore for rootless containers without granting any additional capabilities.

Prototype approach

The initial prototype attempted to use ptrace() to inject CRIU's parasite code into the target process. This failed because:

• ptrace() requires CAP_SYS_PTRACE for non-child processes
• Even with ptrace(), the parasite code needs CAP_SYS_ADMIN to access /proc/PID/map_files
• The kernel's dump_common() function checks for CAP_SYS_ADMIN before allowing memory dump

Major failures encountered

1. Parasite injection without privileges – The parasite code cannot be injected into a process that isn't a child of the dumping process.
2. Memory dump without CAP_SYS_ADMIN – The kernel refuses to dump memory mappings for processes without this capability.
3. Namespace creation – Creating new namespaces for restore requires CAP_SYS_ADMIN in the initial user namespace.
4. Filesystem state – Rootless containers often use overlayfs or fuse mounts that cannot be checkpointed without privileges.

Current implementation path

The working approach uses several techniques to avoid CAP_SYS_ADMIN:

• User namespace nesting – Create a user namespace where the container runs, then checkpoint from within that namespace
• /proc/PID/mem access – Read process memory directly through /proc/PID/mem instead of using parasite code
• Seccomp filter inspection – Dump seccomp filters by reading /proc/PID/seccomp
• File descriptor passing – Use SCM_RIGHTS to transfer file descriptors between namespaces

The current prototype successfully checkpoints and restores a simple rootless container running a single process with:

• Memory state preserved
• Open file descriptors (stdin/stdout/stderr) restored
• Signal handlers saved and reapplied
• Seccomp filters dumped and reinstated

Remaining challenges

• Multi-process containers – Coordinating checkpoint across process groups without CAP_SYS_ADMIN
• Network state – Rootless containers use slirp4netns; restoring network connections requires replaying TCP state
• Mount points – Bind mounts and tmpfs inside the container namespace need special handling
• Performance – Reading /proc/PID/mem is significantly slower than parasite-based memory dump

Technical details

The key insight enabling this work is that within a user namespace, a process has CAP_SYS_ADMIN relative to that namespace. By checkpointing from inside the container's user namespace, we can:

# Inside the container's user namespace
criu dump -t PID --shell-job --leave-stopped

The --shell-job flag tells CRIU to treat the process as a shell job rather than a full container, which reduces the privilege requirements. The --leave-stopped flag prevents CRIU from resuming the process after dumping, allowing the restore to happen later.

For restore, we use:

# Inside a fresh user namespace with matching UID/GID mappings
criu restore --shell-job --restore-detached

Code example

Here's the core function for dumping memory without parasite code:

def dump_memory(pid, output_dir):
    """Dump process memory by reading /proc/PID/mem directly."""
    mem_path = f"/proc/{pid}/mem"
    maps_path = f"/proc/{pid}/maps"
    
    with open(maps_path, 'r') as maps_file:
        for line in maps_file:
            parts = line.split()
            if len(parts) < 5:
                continue
            addr_range = parts[0]
            perms = parts[1]
            offset = parts[2]
            inode = parts[4]
            
            # Skip non-readable or device-mapped regions
            if 'r' not in perms or inode == '0':
                continue
                
            start, end = addr_range.split('-')
            start_addr = int(start, 16)
            end_addr = int(end, 16)
            size = end_addr - start_addr
            
            # Read memory region
            with open(mem_path, 'rb') as mem_file:
                mem_file.seek(start_addr)
                data = mem_file.read(size)
                
            # Write to dump file
            dump_file = os.path.join(output_dir, f"mem_{start}_{end}.dump")
            with open(dump_file, 'wb') as f:
                f.write(data)

Acknowledgments

This work builds on previous efforts by the CRIU team and the Podman maintainers. Special thanks to the kernel developers who made user namespace support practical for container workloads.