From Side Project to Production Tool: What 8 Releases of tfdrift Taught Me About Building DevOps Tooling

The features that actually mattered

I shipped a lot of features. Some of them turned out to be essential. Others were nice-to-haves that nobody asked for. Here's what landed, in order of how much they actually mattered to users.

Incremental scanning changed everything

This was the most recent addition and probably the most impactful. Before incremental scanning, every tfdrift scan and every tfdrift watch cycle re-scanned every workspace from scratch. For a repo with 20 workspaces, that meant 20 terraform plan calls even if nothing had changed since the last scan.

Now tfdrift maintains a local cache that tracks file checksums for each workspace. If the .tf files, .tfvars files, and Terraform state haven't changed since the last clean scan, the workspace is skipped entirely.

# First scan - checks everything
tfdrift scan --path ./infrastructure
Scanned 20 workspaces in 45s

# Second scan - skips unchanged workspaces
tfdrift scan --path ./infrastructure
Scanned 3 workspaces (17 cached) in 8s

For watch mode, this is transformative. Instead of hammering your cloud provider's APIs every 30 minutes with 20 plan calls, it only re-checks workspaces where files actually changed. The cache lives at ~/.tfdrift/cache.json and can be cleared with tfdrift scan --no-cache.

Parallel scanning was the obvious optimization

Sequential scanning was the first bottleneck anyone hit. Each terraform plan call takes 3-8 seconds depending on workspace size and API latency. Twenty workspaces sequentially means 60-160 seconds of waiting.

# Parallel scanning with 8 workers
tfdrift scan --path ./infrastructure --workers 8

Combined with incremental scanning, a typical re-scan of a large repo now takes under 10 seconds. That's the difference between running drift checks as part of your CI pipeline and running them "when someone remembers."

Cost estimation got people's attention

This was the feature that generated the most conversation. When an instance type drifts, knowing it's "HIGH severity" is useful. Knowing it's "+$89.28/month" is what gets the finance team involved.

The pricing module covers on-demand us-east-1 prices for EC2, RDS, Aurora, ElastiCache, and EKS node groups. It's clearly labeled as approximate - actual costs depend on reserved instances, savings plans, and region. But even a rough number changes the conversation from "someone changed something" to "someone's change costs us $304 a month."

GitHub Actions native output solved the CI problem

Before the --gha flag, running tfdrift in GitHub Actions meant scrolling through log output to find drift results. Now it emits native annotations:

• ::error:: for Critical and High severity drift
• :⚠️: for Medium
• ::notice:: for Low

These show up inline in the Actions log and as annotations on PR diffs. It also writes a Markdown summary table to $GITHUB_STEP_SUMMARY, so reviewers see a clean drift report on the job summary page without digging through logs. The flag is auto-detected - if $GITHUB_ACTIONS is set, annotations are emitted automatically. You can force it with --gha for testing.

Drift history answered the "is it getting better?" question

Every scan now auto-saves results to a SQLite database at ~/.tfdrift/history.db. The tfdrift history command queries it:

tfdrift history --limit 10

This answers questions that were previously impossible without external tooling:

• Is our drift count trending up or down?
• How many Critical drift events did we have this month?
• Are we catching drift faster since we set up watch mode?

For compliance teams, it's an audit trail proving you're actively monitoring infrastructure drift.

The --fail-on flag for watch mode was a one-line fix with big impact

The scan command already supported --fail-on high - only exit with code 1 when drift meets a severity threshold. The watch command didn't have this, which meant watch mode either alerted on everything or you had to build your own filtering.

tfdrift watch --interval 30m --fail-on high --slack-webhook $SLACK_URL

Now the watch loop keeps running silently when it finds Low or Medium drift. When it finds High or Critical, it alerts and exits. Simple, but it's what makes watch mode usable in CI environments.

What I learned building this

Ship the obvious thing first

Severity classification was the first feature and it's still the most important one. Every other feature - parallel scanning, cost estimation, history - builds on top of severity. If I'd started with cost estimation or GitHub Actions integration, the tool wouldn't have been useful enough to attract feedback.

Users will find your bottleneck before you do

I didn't think sequential scanning was a problem until someone tried tfdrift on a repo with 150+ workspaces and it took 12 minutes. Parallel scanning and incremental caching both came directly from user feedback. Build the simplest version, ship it, and let real usage tell you what's slow.

CI integration is table stakes

I underestimated how important CI integration was. A tool that works great in the terminal but is clunky in GitHub Actions doesn't get adopted by teams. The --gha flag, --quiet mode, --fail-on severity gating, and structured exit codes were all necessary to make tfdrift a tool teams depend on rather than a tool individuals try once.

The config file is the product

The .tfdrift.yml file has become the most important part of the project. It's where teams encode their operational values - what's Critical, what to ignore, when to alert, who to page. The tool is just the engine that reads the config and acts on it.

What's next

Two features are on my radar.

Value-aware classification - inspecting what an attribute changed to, not just that it changed. A security group opening port 443 to a known CIDR is different from opening port 22 to 0.0.0.0/0. Both are classified as Critical today because the attribute is ingress. That's a meaningful gap.

The other is environment-conditional severity. The same change should be Critical in production and Low in dev. Right now you handle this with separate config files. Native support would be more ergonomic.

Try it

pip install tfdrift
tfdrift scan --path ./your-terraform-dir

With parallel scanning:

tfdrift scan --path ./infrastructure --workers 8

Check drift history:

tfdrift history

Continuous monitoring:

tfdrift watch --interval 30m --fail-on high --slack-webhook $SLACK_URL

GitHub: github.com/sudarshan8417/tfdrift

This is part 6 of a series on infrastructure drift detection. Previous posts covered the initial build, severity classification, parsing terraform plan JSON, GitHub Actions integration, and the v0.2.3-v0.2.5 feature releases.