dature: type-safe configuration through plain dataclasses
DEV Community

dature: type-safe configuration through plain dataclasses

From pains to a spec

A deployment complains that port in production is 9090, while defaults.yaml says 8080. Was it override.yaml, an environment variable, or a Kubernetes ConfigMap? Without the right tooling, finding out can eat half a working day.

I've been writing Python since 2017, and I've worked with configuration in every form: INI files in legacy projects, environment variables in Docker containers, YAML with templates in Helm, pydantic models in fresh services, Vault and Consul in bigger infrastructure.

In the previous two articles I traced how we got to the way we handle configuration today and why no existing Python library covers all of it, which left me and my team unsatisfied. This one is about what I built in response.

TL;DR dature loads config from many sources into your own stdlib @dataclass, with no base class and no framework. Every value remembers where it came from, so errors name the file and line, and a debug report shows each field's origin. Merging is configurable per field, and sources can be conditional (when=) or depend on each other. It's not a replacement for everything: if you need tight FastAPI dependency injection, pydantic-settings is a more direct path.

Six pains became the specification:

  • merging individual fields from different sources;
  • no way to tell where an invalid value came from;
  • validators wanted both inside the type and outside it;
  • secret masking doesn't work everywhere;
  • adding your own source should be easy;
  • it must be a library, not a framework.

My library, dature, grew out of these. If tight integration with FastAPI dependency injection is critical for you, pydantic-settings gives a more direct path. dature has its own niche; it doesn't displace everything else, it closes specific pains that my team and I ran into.

dature's philosophy

dature is a library. Its only job is to load data from arbitrary sources into a standard-library @dataclass and return a ready object. It doesn't capture your entry point, doesn't change the working directory, doesn't manage logging.

The dataclass is the single source of truth. Type hints describe the structure, the types, and the constraints. There's no need to duplicate definitions in YAML files, separate validators, or Pydantic models.

from dataclasses import dataclass
import dature

@dataclass
class Config:
    host: str
    port: int
    debug: bool = False

config = dature.load(
    dature.Yaml12Source(file="config.yaml"),
    schema=Config
)
# isinstance(config, Config) → True
# Full IDE and mypy support, __post_init__ works

Loading from any source while keeping provenance

The main idea of dature is not to lose a value's origin after parsing. For every field it's known which source and which line it came from, and that is available both in validation errors and in the debug report.

The list of supported sources is, as you'd expect, broad: eight file formats with auto-detection by extension: YAML 1.1/1.2, JSON, JSON5, TOML 1.0/1.1, INI, .env; environment variables; Docker secrets; Vault, Consul, etcd, ZooKeeper, AWS SSM, AWS Secrets Manager, Azure App Configuration, Azure Key Vault, GCP Secret Manager; argparse CLI; a custom CLI source.

One call, several sources:

from dataclasses import dataclass
import dature

@dataclass
class Config:
    host: str
    port: int
    secret_key: str
    debug: bool = False

config = dature.load(
    dature.Yaml12Source(file="defaults.yaml"),
    dature.Toml11Source(file="config.toml", skip_if_missing=True),
    dature.EnvFileSource(file=".env", skip_if_broken=True),
    dature.EnvSource(prefix="APP_"),  # environment variables with the APP_ prefix
    schema=Config,
)

skip_if_broken=True means that if a file is invalid, the source is silently skipped. skip_if_missing=True means that if a file doesn't exist, the source is silently skipped. This lets you describe the chain "defaults → local overrides → environment variables" without checking for file existence by hand.

The format is detected automatically from the extension. If you need a non-standard loader, you write your own Source.

Merge strategies

Four global strategies decide how to resolve conflicts between sources:

  • last_wins: the last source overrides the earlier ones (the default);
  • first_wins: the first source takes priority;
  • first_found: take the first source that isn't broken and ignore the rest;
  • raise_on_conflict: an error if two sources set different values.

You can set separate rules for individual fields. For example, a list of servers from different sources should be combined rather than replaced:

from dataclasses import dataclass
import dature

@dataclass
class Config:
    servers: list[str]
    port: int

config = dature.load(
    dature.Yaml12Source(file="defaults.yaml"),
    dature.Yaml12Source(file="local.yaml", skip_if_broken=True),
    dature.EnvSource(prefix="APP_"),
    field_merges={
        dature.F[Config].servers: "append_unique"
    },
    schema=Config,
)

Field groups let you enforce consistency: if one field of a group is overridden in a new source, the rest must be overridden too:

import dature

dature.load(
    dature.Yaml12Source(file="defaults.yaml"),
    dature.Yaml12Source(file="override.yaml"),
    field_groups=(
        (dature.F[Config].db_host, dature.F[Config].db_port, dature.F[Config].db_name),
    ),
    schema=Config,
)
# If override.yaml changes db_host but not db_port → error

Validation

dature uses Annotated for built-in validators:

from dataclasses import dataclass
from typing import Annotated
import dature

@dataclass
class Config:
    port: Annotated[int, (dature.V >= 1) & (dature.V <= 65535)]
    host: Annotated[str, dature.V.len() >= 1]
    log_level: Annotated[str, dature.V.matches(r"^(DEBUG|INFO|WARNING|ERROR)$")]

The validator lives right in the field's type, and that matters. When you add a new field to the dataclass, you physically can't forget its constraints: they're part of the signature.

Validators can also be supplied separately, through a Source, when you want to reuse one schema with different rules or move the checks into another module:

import dature

config = dature.load(
    dature.Yaml12Source(
        file="config.yaml",
        validators={
            dature.F[Config].port: (dature.V >= 1) & (dature.V <= 65535),
        },
    ),
    schema=Config,
)

For checks across fields there are root validators:

import dature

def check_ssl(config: Config) -> bool:
    return not config.use_ssl or config.cert_path is not None

config = dature.load(
    dature.Yaml12Source(
        file="config.yaml",
        root_validators=(
            dature.V.root(func=check_ssl, error_message="cert_path required when use_ssl is True"),
        ),
    ),
    schema=Config,
)

The standard __post_init__ works too, because dature returns an ordinary dataclass.

When validation fails, the error points at the exact place:

Config loading errors (1)
[port] Must be greater than 0
├── port: -1
│   ^^
└── FILE 'config.yaml', line 2

The message has everything you need to fix it in a minute: the file path, the line number, the value itself, and carets under the offending piece. That's the difference that's worth an hour of debugging in production logs.

Secrets and debugging

Fields of type SecretStr are masked automatically in errors and on output:

from dataclasses import dataclass
import dature
from dature.fields import SecretStr

@dataclass
class Config:
    database_url: str
    api_key: SecretStr

config = dature.load(
    dature.Yaml12Source(file="config.yaml"),
    schema=Config
)
print(config.api_key)  # SecretStr('**********')
print(config.api_key.get_secret_value())  # the real value

Masking works on three levels:

  • By type: SecretStr, PaymentCardNumber.
  • By field name: by default, fields containing the substrings password, passwd, secret, token, key, auth, credential, uri, url, connection_string are masked. The list is configurable via MaskingConfig.secret_field_names.
  • By heuristic: bigram analysis. The string is lowercased and split into bigrams, and the share of rare bigrams is compared against a threshold (heuristic_threshold, 0.5 by default). If more than half are rare, the string is considered a randomly generated token and is masked. The minimum string length for analysis is 8 characters (min_heuristic_length). The heuristic requires the optional random_string_detector dependency; without it, only the first two levels work.

Debug mode shows where each value came from:

import dature

config = dature.load(
    dature.Yaml12Source(file="defaults.yaml"),
    dature.EnvSource(prefix="APP_"),
    schema=Config,
    debug=True,
    masking_mode="secrets_only",  # so ordinary values appear in the report
)

report = dature.load_report(config)
for origin in report.field_origins:
    print(f"{origin.key} = {origin.value} ← {origin.source_file or origin.source_loader_type}")
# host = 'localhost' ← defaults.yaml
# port = 9090 ← env
# debug = False ← defaults.yaml

Debug mode is especially useful when a config is assembled from several sources and it's unclear which one dominates a given field: debug=True shows each value's path from source to final object.

Custom fields and sources

dature works with your own types and formats. Registering a loader for a new type:

from dataclasses import dataclass
import dature

@dataclass(frozen=True, slots=True)
class Rgb:
    r: int
    g: int
    b: int

def rgb_from_string(value: str) -> Rgb:
    parts = value.split(",")
    return Rgb(r=int(parts[0]), g=int(parts[1]), b=int(parts[2]))

@dataclass
class AppConfig:
    name: str
    color: Rgb

config = dature.load(
    dature.Yaml12Source(
        file=SOURCES_DIR / "custom_type_common.yaml",
        type_loaders={Rgb: rgb_from_string},  # applies only to this file source
    ),
    schema=AppConfig,
)

A custom format is a subclass of FileSource:

import xml.etree.ElementTree as ET
from dataclasses import dataclass
from pathlib import Path
import dature
from adaptix import Provider, loader
from dature.loaders import bool_loader, float_from_string
from dature.sources.base import FileSource
from dature.types import FileOrStream, JSONValue

@dataclass(kw_only=True, repr=False)
class XmlSource(FileSource):
    format_name = "xml"

    def _load_file(self, path: FileOrStream) -> JSONValue:
        if not isinstance(path, Path):
            msg = "XmlSource only supports file paths"
            raise TypeError(msg)
        tree = ET.parse(path)  # noqa: S314
        root = tree.getroot()
        return {child.tag: child.text or "" for child in root}

    def format_loaders(self) -> list[Provider]:
        return [
            loader(bool, bool_loader),
            loader(float, float_from_string),
        ]

@dataclass
class Config:
    host: str
    port: int
    debug: bool

config = dature.load(
    XmlSource(file="custom_loader.xml"),
    schema=Config,
)

Real-world scenarios where dature helps

So the features don't hang in the air, here are several typical scenarios. In these situations you normally end up writing a lot of boilerplate, while with dature we get by in a few lines.

Scenario: merge lists, replace everything else. An application config: common defaults for CORS origins, plus extra ones for each environment. The behavior you want: lists are combined, while database.host is replaced by source priority.

Before, you had to write a custom SettingsSource in pydantic-settings, override _field_is_complex, and check that deep-merge is enabled for the specific field. That came to twenty lines of infrastructure code for a single business scenario.

With dature it's one field_merges line:

config = dature.load(
    dature.Yaml12Source(file="defaults.yaml"),
    dature.Yaml12Source(file=f"config.{env}.yaml", skip_if_broken=True),
    dature.EnvSource(prefix="APP_"),
    field_merges={
        dature.F[Config].cors_origins: "append_unique",
    },
    schema=Config,
)

Everything else is last_wins by default, and no custom source class is needed.

Scenario: the config itself answers where each field came from. The classic production detective story. A deployment complains that port in production is 9090 while defaults.yaml says 8080. You need to quickly find the override's source: override.yaml, environment variables, or a Kubernetes ConfigMap. So the config should not only hold the final value but also show its origin.

Without a suitable tool, the detective work starts with a trip into the container: env | grep APP_, check mounted files, reread docker-compose. Understanding the origin of a single value can sometimes eat half a working day.

With dature:

config = dature.load(
    dature.Yaml12Source(file="defaults.yaml"),
    dature.Yaml12Source(file="override.yaml", skip_if_broken=True),
    dature.EnvSource(prefix="APP_"),
    schema=Config,
    debug=True,
)

report = dature.load_report(config)
for origin in report.field_origins:
    print(f"{origin.key} = {origin.value} ← {origin.source_file or origin.source_loader_type}")

The output immediately shows port = 9090 ← env. The value came from an environment variable, and the case is closed in a minute rather than half a day of digging through files in a container. It's especially useful in CI: you can run load(..., debug=True) as part of a test stage and verify where each field is supposed to come from in production.

Scenario: a custom source for an internal configuration service. A large company has its own configuration service: a REST endpoint that returns settings for the requesting module. Nobody is going to replace it with "ordinary Vault," and without dature the options are to load it manually and glue it to the other sources by hand, or to write a custom SettingsSource in pydantic-settings with all that entails.

With dature, a custom source is a subclass of Source with one _load method:

from dataclasses import dataclass
import requests
import dature
from dature.sources.base import So
Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.