The rollover that breaks delegations

The DS record lives in the parent zone and names your KSK by its digest. Rolling the KSK means changing something you do not control, through a registrar, with a cache in between — which is why this is the rollover that takes domains offline.

Double-DS, the safe sequence

1. Generate the new KSK. Add it to the DNSKEY RRset.
   Sign the RRset with BOTH keys.
   Wait: the DNSKEY RRset's TTL.

2. Submit the new DS to the parent, ALONGSIDE the old.
   Both DS records now published.
   Wait: the parent's DS TTL, plus a margin.

3. Remove the old DS at the parent.
   Wait: the parent's DS TTL again.

4. Stop signing with the old KSK and remove it from
   the DNSKEY RRset.

At every moment in that sequence, at least one published DS matches a key that is currently signing the DNSKEY RRset. That property is the whole of the safety argument, and every failed rollover breaks it somewhere.

How it goes wrong

The new key is published and the DS is never submitted. The old key is then removed on schedule, the parent’s DS names a key that no longer exists, and the zone goes bogus. This is the classic failure and it is usually a person forgetting a registrar step.
The DS is replaced rather than added. Resolvers holding the old DS see a key set that no longer contains a matching key.
The wait between steps is skipped. Every step has a TTL behind it, and the parent’s DS TTL is frequently a day.
The registrar interface does not support two DS records. Some do not. Where that is true, the alternative is a longer, more careful sequence — and it is worth knowing before you start rather than halfway through.

Verifying each step

# after step 1: both keys present, both signing
dig +dnssec DNSKEY example.com @1.1.1.1 | grep -c DNSKEY
dig +dnssec DNSKEY example.com @1.1.1.1 | grep -c RRSIG

# after step 2: two DS records at the parent
dig +short DS example.com @1.1.1.1

# throughout: validation still succeeding
dig example.com SOA @1.1.1.1 | grep -o 'flags:[^;]*'
#   the "ad" flag must be present at every step

Do not start a KSK rollover on a Friday

The waits are measured in days, the failure mode is a total outage for validating resolvers, and the recovery involves a registrar who works business hours. Every step should land where somebody is watching, and the ad flag should be checked after each one rather than at the end.

Knowledge check

Halfway through a KSK rollover, the old key is removed from the zone but the new DS was never submitted. What happens?

The zone goes bogus for every validating resolver — the parent’s DS names a key that is no longer in the DNSKEY set.
The zone is treated as unsigned until the DS is corrected.
Nothing, until the old DS record’s TTL expires.
Only resolvers that had cached the old DNSKEY set are affected.

Correct — The zone goes bogus for every validating resolver — the parent’s DS names a key that is no longer in the DNSKEY set. Validation requires a published DS matching a key that signs the DNSKEY RRset, and nothing does. The result is SERVFAIL rather than an unsigned answer, so the domain stops resolving rather than merely losing protection.

Not this — The zone is treated as unsigned until the DS is corrected. An unsigned result would follow from no DS at all. A DS pointing at a missing key is a broken chain, which is bogus.

Not this — Nothing, until the old DS record’s TTL expires. The opposite — the cached DS is what makes resolvers insist on a key that is gone. Expiry would not help either, since the DS is still published at the parent.

Not this — Only resolvers that had cached the old DNSKEY set are affected. Those are the ones still working. Resolvers fetching fresh fail, and the failure grows as caches turn over.

Double-DS, the safe sequence

How it goes wrong

Verifying each step

Try it on a real domain