Incident note: DNS SERVFAIL: find the cause (DNSSEC & more)

Read the signal

Start with the exact wording of the error and its timestamp in the log. Then narrow down the cause before changing anything.

;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 41234

Narrow down possible causes

• The DS record at the registry no longer matches the zone's DNSKEY after a provider change.
• Expired RRSIG signatures because automatic re-signing stopped.
• Lame delegation: listed nameservers are not authoritative for the zone or unreachable.
• Nameservers answer incorrectly or block TCP/EDNS (large responses).

Check before changing anything

1. Query with DNSSEC data
   dig example.com A +dnssec
2. Same query with checking disabled - if it works, DNSSEC is the problem
   dig example.com A +cd
3. Query the DS record at the parent zone
   dig +short DS example.com @a.gtld-servers.net
4. Query DNSKEY directly at the authoritative server
   dig +short DNSKEY example.com @ns1.example-dns.com

Fix and test again

• Delete the wrong DS record at the registrar or replace it with the matching one; mind the DS TTL.
• When moving a DNSSEC-signed domain, remove the DS first, wait out the TTL, then move and re-sign.
• Check and fix automatic re-signing at the DNS provider or in BIND/PowerDNS.
• Align NS records at the registry and in the zone.

Then repeat the original action and compare the result with the initial error.

Keep a note for the next ticket

Record the exact error, the check result, the change and the retest. That gives the team a useful starting point next time.

Why does it work for some users but not others?

Only validating resolvers (e.g. 1.1.1.1, 8.8.8.8, many ISPs) return SERVFAIL. Non-validating resolvers still hand out the answer.

Full error guide: DNS SERVFAIL: find the cause and fix it Originally published at daniele-michelin.com. More guides, free checks (SPF, TLS, headers) and an error-message lexicon: daniele-michelin.com/en/.