Stop Guessing systemd Service APIs: Practical varlinkctl on Linux

You need one clean answer from a local service: resolve a name, describe the host, extend a PCR, or talk to a custom tool that already speaks JSON on stdin/stdout. The usual options are brittle: scrape systemctl text, write a one-off D-Bus client, or invent yet another Unix-socket protocol. Modern systemd components expose Varlink interfaces instead. varlinkctl is the operator-facing client for those interfaces: discover what a service implements, print the interface IDL, call methods with JSON arguments, stream multi-reply methods, tunnel calls over SSH, and even wrap a sandboxed command as a Varlink service. This guide is operational. Commands and behavior come from varlinkctl(1) , the UAPI.20 Varlink IPC specification, and the Debian/unstable systemd man pages for the services used in the labs (notably systemd-resolved ). What Varlink is (and is not) | Piece | Job | |---|---| | Varlink | JSON method calls over a stream (typically AF_UNIX ), with a typed interface definition language | | UAPI.20 | The UAPI Group spec consolidating protocol + transport bindings | varlinkctl | CLI to introspect and invoke Varlink services (systemd 255+) | busctl / D-Bus | Parallel IPC stack; still widely used, different wire format and tooling | systemctl text | Human UI - fine for shells, awkward as a stable machine API | Varlink messages are plain JSON objects. On stream transports each message is NUL-terminated. Services describe themselves at runtime, so clients can list interfaces and fetch the IDL without a separate schema package. You do not need to replace every D-Bus workflow. Use varlinkctl when the component already speaks Varlink (resolved, hostnamed, many systemd helpers) or when you want a JSON-friendly, socket-activated service of your own. Prerequisites command -v varlinkctl varlinkctl --version # Added in systemd 255; several commands below need 257-262 features. # Labs that call resolved need the stub/service running: systemctl is-active systemd-resolved.service ls -l /run/systemd/resolve/io.systemd.Resolve Privileged examples (PCR extend, some hostnamed paths) need root or an authorized policy. DNS resolve calls as a normal user usually work when resolved is active and the socket is accessible. Address forms you will actually use varlinkctl accepts several service address syntaxes (from varlinkctl(1) ): | Form | Meaning | |---|---| unix:/run/path.sock or /run/path.sock | Connect to an AF_UNIX stream socket | @name after unix: | Abstract-namespace socket | exec:/usr/lib/systemd/tool | Fork the binary and speak Varlink on the passed socket | ssh-unix:host:/run/... | OpenSSH ≥ 9.4 path to a remote AF_UNIX socket | ssh-exec:host:cmdline | Run a remote command and speak Varlink on its stdio | Relative ./socket or ./binary | Local path forms (must start with / or ./ ) | Convenience: a bare absolute socket path or executable path is enough when the target is local. Lab 1 - Inventory a live service (resolved) # General metadata varlinkctl info /run/systemd/resolve/io.systemd.Resolve # Interface names only varlinkctl list-interfaces /run/systemd/resolve/io.systemd.Resolve # Method names (list-methods added in 257) varlinkctl list-methods /run/systemd/resolve/io.systemd.Resolve # Full IDL for the primary interface varlinkctl introspect /run/systemd/resolve/io.systemd.Resolve io.systemd.Resolve Typical info fields include vendor, product, version, URL, and the interface list (io.systemd , io.systemd.Resolve , org.varlink.service , …). Pretty JSON for scripts: varlinkctl info /run/systemd/resolve/io.systemd.Resolve -j varlinkctl list-methods /run/systemd/resolve/io.systemd.Resolve --json=short -j is “pretty when interactive, short when piped”; --json=pretty|short forces the mode. Lab 2 - Call a method with JSON arguments Resolve a hostname through resolved’s ResolveHostname method (example shape from varlinkctl(1) ): varlinkctl call \ /run/systemd/resolve/io.systemd.Resolve \ io.systemd.Resolve.ResolveHostname \ '{"name":"systemd.io","family":2}' \ -j Notes that prevent foot-guns: - Method names are fully qualified: interface.Method . - Parameters are a JSON object. Use {} for empty input. - If you omit the arguments parameter, varlinkctl reads JSON from STDIN. - Replies are JSON objects on STDOUT - pipe to jq when you want fields only. varlinkctl call \ /run/systemd/resolve/io.systemd.Resolve \ io.systemd.Resolve.ResolveHostname \ '{"name":"systemd.io","family":2}' \ --json=short \ | jq -r '.addresses[]? | "(.family) (.address)"' family: 2 is AF_INET in the usual Linux numbering; adjust if you want IPv6 (10 / AF_INET6 ) or leave the field out when the interface allows defaults (check the IDL from introspect ). Lab 3 - Multi-reply methods, collection, and oneway Some methods stream updates or enumerate objects. Flags from varlinkctl(1) : # Expect a sequence of replies (JSON-SEQ). Default call timeout is still 45s. varlinkctl call --more ADDRESS INTERFACE.Method '{"...":"..."}' # Same idea, but keep listening (timeout disabled) - shortcut -E varlinkctl call -E ADDRESS INTERFACE.Method '{}' # Gather every reply into one JSON array varlinkctl call --collect ADDRESS INTERFACE.Method '{}' # Fire-and-forget (no reply expected) varlinkctl call --oneway ADDRESS INTERFACE.Method '{"...":"..."}' Timeout control: # Default 45s; disable for long subscriptions varlinkctl call --more --timeout=infinity ADDRESS INTERFACE.Method '{}' # Treat a specific Varlink error as success (257+) varlinkctl call \ --graceful=org.varlink.service.InvalidParameter \ ADDRESS INTERFACE.Method '{"experimental":true}' Use --more with a sane timeout for finite enumerations; use -E / --timeout=infinity only for true subscriptions so a stuck peer cannot hang a cron job forever. Lab 4 - Exec targets and helper binaries Not every Varlink peer is a long-running daemon socket. Some tools speak Varlink when executed. The man page demonstrates systemd-pcrextend : # Inspect the executable as a Varlink service (requires privileges for real PCR ops) sudo varlinkctl info /usr/lib/systemd/systemd-pcrextend sudo varlinkctl introspect /usr/lib/systemd/systemd-pcrextend io.systemd.PCRExtend # Example method shape from the man page - only on systems where PCR extend is appropriate # sudo varlinkctl call /usr/lib/systemd/systemd-pcrextend \ # io.systemd.PCRExtend.Extend '{"pcr":15,"text":"foobar"}' exec: form is equivalent when you want to be explicit: sudo varlinkctl info exec:/usr/lib/systemd/systemd-pcrextend Treat PCR/measurement labs as maintenance-window work on hosts where measured boot policy is intentional. Do not extend PCRs on production machines as a casual test. Lab 5 - Remote calls over SSH When the remote side has OpenSSH 9.4+ (for ssh-unix: ) and the socket path exists: # Talk to hostnamed on a remote machine via its AF_UNIX socket # varlinkctl call ssh-unix:somehost:/run/systemd/io.systemd.Hostname \ # io.systemd.Hostname.Describe '{}' -j # Or run a Varlink-capable binary on the remote stdio path # varlinkctl call ssh-exec:somehost:systemd-creds \ # org.varlink.service.GetInfo '{}' -j This is useful for fleet introspection without installing a custom agent: SSH provides transport and auth; Varlink provides a typed method call. Abstract-namespace sockets are not supported over ssh-unix: (filesystem path sockets only). Lab 6 - Registry and socket discovery (260+) Newer systemd builds keep well-known entrypoints under /run/varlink/registry/ : # System registry (default) varlinkctl list-registry # Per-user registry when applicable varlinkctl list-registry --user ls -l /run/varlink/registry/ 2>/dev/null list-sockets (262+) enumerates listening AF_UNIX stream sockets marked as Varlink entrypoints via the user.varlink=entrypoint xattr (needs kernel support for xattrs on socket inodes; documented as Linux 7.0+ in the man page). Prefer list-registry on typical current distro kernels if list-sockets is unavailable. Lab 7 - Serve a sandboxed stdio tool as Varlink (261+) varlinkctl serve turns a command that speaks a protocol on stdio into a socket-activated Varlink service. On upgrade, the client’s connection is handed to the command. The man page’s decompressor example: /etc/systemd/system/varlink-decompress-xz.socket [Socket] ListenStream=/run/varlink/registry/com.example.Decompress.XZ [Install] WantedBy=sockets.target /etc/systemd/system/varlink-decompress-xz.service [Service] ExecStart=varlinkctl serve com.example.Decompress.XZ xz -d DynamicUser=yes PrivateNetwork=yes ProtectSystem=strict ProtectHome=yes NoNewPrivileges=yes SystemCallFilter=~@privileged @resources MemoryMax=256M Enable and call: sudo systemctl daemon-reload sudo systemctl enable --now varlink-decompress-xz.socket echo "hello" | xz | varlinkctl call --upgrade \ unix:/run/varlink/registry/com.example.Decompress.XZ \ com.example.Decompress.XZ '{}' # expected stdout: hello Quick test without unit files (ephemeral listen socket): systemd-socket-activate -l /tmp/decompress.sock -- \ varlinkctl serve com.example.Decompress.XZ xz -d & echo "hello" | xz | varlinkctl call --upgrade \ unix:/tmp/decompress.sock \ com.example.Decompress.XZ '{}' Why this pattern matters: - The heavy lifting stays in a forked child with systemd sandboxing ( ProtectSystem= ,MemoryMax= , …). - Clients discover a stable method name instead of ad-hoc FIFO paths. - --upgrade moves from JSON control plane to raw stream payload once the call succeeds. - Pair with --exec on the client when the reply includes file descriptors ($LISTEN_FDS hand-off; 258+). Lab 8 - Validate IDL before you ship an interface cat > /tmp/com.example.Echo.varlink (message: string) error EmptyMessage () EOF varlinkctl validate-idl /tmp/com.example.Echo.varlink # prints the definition with syntax highlighting on success Catch naming and structure mistakes early; the wire protocol expects the same IDL shape services return from org.varlink.service introspection. Operator cheat sheet | Goal | Command pattern | |---|---| | Who am I talking to? | varlinkctl info A