Linux Kernel Modules That Explain How Podman Really Works

Preparing an Environment to Compile Linux Kernel Modules

1. Clone the repository inside the Killercoda Ubuntu environment:

git clone https://github.com/sergioarmgpl/kernel-modules.git
cd kernel-modules

2. Install the dependencies needed to build kernel modules (kernel headers, compiler, make, etc.):

chmod +x ins_dep.sh
./ins_dep.sh

This script takes care of installing the Linux kernel headers that match the running kernel (linux-headers-$(uname -r)) plus the basic build toolchain, which is required because kernel modules must be compiled against the exact kernel version you are running.

Installing Podman on Ubuntu

Since the Killercoda Ubuntu playground doesn't ship with Podman preinstalled, install it from the official Ubuntu repositories:

sudo apt-get update
sudo apt-get install -y podman

Verify the installation:

podman --version

You should see an output similar to:

podman version 4.9.3

Podman is daemonless: it doesn't need a long-running background service running as root to manage containers. Instead, it launches each container as a regular child process of the CLI itself, which fits nicely with what we are going to observe later with module3 when we inspect the kernel's process table.

Module 1: Kernel Debug Messages with dmesg

module1 is the classic "Hello World" of Linux kernel programming. Its only job is to register init and exit functions and print a message to the kernel ring buffer (accessible with dmesg) when the module is loaded and unloaded, using macros like printk(KERN_INFO ...).

1. Compile the module:

cd module1
make all

2. Load the module into the running kernel:

sudo insmod module1.ko

3. Show the debug message printed by the module using the kernel ring buffer:

dmesg | tail -n 10

You should see an entry similar to:

[ 1234.567890] Module 1: Hello, Kernel! Module loaded successfully.

4. Once you are done, unload the module and clean the build:

sudo rmmod module1
make clean

dmesg shows you both the load message (printed inside the module's init function) and, after rmmod, the unload message (printed inside the exit function). This is the most basic proof that a piece of your own code is now running as part of the Linux kernel.

Module 2: Interacting with a Character Device

module2 goes one step further than module1: instead of only printing a message, it registers a character device in the kernel. A character device is one of the ways the kernel exposes functionality to user space as if it were a regular file, implementing callback functions such as open, read, write, and release.

When the module is loaded, it creates a device node (for example /dev/module2 or similar, depending on the major/minor number it registers), and every time a user-space process interacts with that file, the corresponding callback inside the module gets executed.

1. Compile and load the module:

cd ../module2
make all
sudo insmod module2.ko

2. Check that the device was created and look at the assigned major number:

dmesg | tail -n 10
cat /proc/devices | grep module2
ls -l /dev/module2

3. Write to the device created by the module:

echo "Hello from user space" | sudo tee /dev/module2

You can also read back from it to see how the module responds to the data it received:

sudo cat /dev/module2

4. Explaining the kernel messages of module 2:

Every time you load the module, write to the device, or read from it, module2 prints tracing messages through printk, which you can inspect with:

dmesg | tail -n 20

These messages typically show:

• A message when the device is opened, confirming a process obtained a file descriptor to /dev/module2.
• A message when data is written, usually including how many bytes were received and, in some implementations, echoing back the content that was sent.
• A message when the device is read, showing how many bytes were copied back to user space.
• A message when the device is released (closed).

This illustrates one of the core ideas of an Operating System: user-space processes never touch hardware or kernel memory directly; they go through system calls (open, read, write, close), and the kernel module is the piece of code that decides what happens on the other side of that system call.

5. Clean up:

sudo rmmod module2
make clean

Module 3: Kernel Process Information and Containers

1. Before loading the module, create some containers so we have real workloads running on the system. Create an nginx, a redis, and a mongo container using Podman:

podman run -d --name web nginx
podman run -d --name cache redis
podman run -d --name db mongo

2. List the containers using Podman:

podman ps

You should see the three containers (web, cache, db) with their container IDs, image names, and status.

3. List the same workloads from the Operating System's point of view, using standard process commands. Since every container is ultimately just one (or more) Linux processes running in isolated namespaces and cgroups, you can find them with:

ps -ef | grep -E "nginx|redis|mongo"
pstree -p | grep -E "nginx|redis|mongod"
top

This shows that containers are not "magic": nginx, redis-server, and mongod show up as regular PIDs on the host, the same way any other Linux process would. This is especially visible with Podman, since it is daemonless and runs each container as a child process of the conmon/podman process tree itself, rather than hiding it behind a separate background service.

4. Now compile and load module3:

cd ../module3
make all
sudo insmod module3.ko

5. Inspect the information the module loads into the kernel log:

dmesg | tail -n 30

Explaining what module 3 loads and how it relates to the previous commands:

module3 walks the kernel's internal task list (the same in-kernel data structure that backs commands like ps and top) and prints information about the currently running processes: PID, process name (comm), and often parent PID or state.

In other words, it is doing at the kernel level exactly what ps -ef does at the user-space level, except it reads the data directly from kernel structures (such as task_struct, traversed with helpers like for_each_process()) instead of going through /proc.

This is why the output of dmesg after loading module3 will include entries for nginx, redis-server, mongod, their conmon parents, and other system processes: they are all regular entries in the kernel's process table, the exact same list that podman ps indirectly depends on (Podman only adds a mapping between container IDs/names and the PIDs and namespaces the kernel is already tracking, without needing a central daemon to do so).

Putting it all together:

• podman ps shows you the container abstraction: names, images, ports.
• ps / pstree / top show you the OS process abstraction: PIDs, parents, CPU/memory usage.
• module3 shows you the kernel data structures that make both of the previous views possible in the first place.

6. Clean up everything:

sudo rmmod module3
make clean
podman rm -f web cache db

References

• https://killercoda.com/playgrounds/scenario/ubuntu
• https://github.com/sergioarmgpl/kernel-modules
• https://github.com/sergioarmgpl/kernel-modules/blob/main/ins_dep.sh
• https://podman.io/