Your secondary DNS only covers silence

Almost every network we walk into has two DNS addresses configured, and almost every one of them assumes the same thing: that the second is there for when the first fails. Microsoft's documentation is more precise than that. It says "client tries new servers only if the previous are unreachable", and that any Name Error response stops the process without retrying the next one. Read it twice, because it changes the design: your second entry covers silence. It does not cover error.

An office hour lost to an integer

On 11 October, at 16:40 UTC, Control D's paid DNS service stopped answering. Not in one region: at every location, over every protocol and on every device, according to their own report. Queries from paying customers went unanswered from 16:42. Recovery started at 17:31 and was complete by 17:35. The free service kept working throughout.

The cause, told by them without decoration: an API update from 16 September had started storing a per-domain override as an integer where the DNS software expected a string. The change sat there for nearly a month bothering nobody, because nobody had yet saved one of those settings. On 11 October somebody saved one for the first time, the value propagated at once to every paid-plan DNS server, and an error-handling flaw made the software stop rather than skip the value it could not read. What they fixed afterwards is told just as plainly: unreadable values are now set aside and logged instead of stopping the service, the API stores text again, and configuration will roll out in stages, the way their software already did.

One thing in that paragraph interests us, and it is not the bug. It is that the change that broke the service was not software: it was configuration, and configuration did not go through the staged rollout the code already had. That is exactly the split you find in most shops: code gets review, tests and a window; settings go out hot and everywhere at once. They have fixed it. The honest question is how many of us can say the same.

The clock nobody watches

Every post-mortem tells you what the provider broke. None of them tells you what your network did during those 54 minutes, and that is what decides whether your people saw a flicker or sat staring at the screen. That part is written down too, just in a support document that never makes a headline. Windows, with two DNS servers configured on the adapter, behaves like this:

The good news is that 1: if the primary goes quiet, Windows moves to the second one in a second, and an ordinary office barely notices. The bad news is everything else. If you have three or more servers listed, the fourth and beyond are not queried until the 4-second mark — the documentation itself warns that if the only reachable server sits in position 4 or higher there is a minimum 4-second delay, and an application with a shorter timeout will fail while DNS is not even down. And on Linux the arithmetic is worse out of the box: the glibc resolver waits 5 seconds per server before moving on, with two attempts by default and a maximum of three servers read from >/etc/resolv.conf. A mute resolver costs five seconds per query there, and some applications resolve dozens of names to paint one screen.

Those five seconds come with a big exception, and it is worth stating before somebody uses it in a meeting: they are glibc's resolver talking to real servers. On a stock Ubuntu or Fedora, /etc/resolv.conf points at 127.0.0.53 and systemd-resolved is in charge, with its own retry schedule and its own list; there what counts is whatever resolvectl status says, and the timings are not these. Check it before taking either number as read on your estate.

There is one more detail that changes the design. When the Windows client switches to the alternate server, it stays there until that alternate fails or until >ServerPriorityTimeLimit expires, which is 15 minutes by default. In other words: the provider recovers at 17:35 and your machines can keep asking the second server for up to a quarter of an hour afterwards. For availability that is sensible — it avoids flapping. For a filtering policy, as we will see, it is precisely the opposite.

Silence is covered; error is not

Here is the heart of it, and it is not our opinion. Microsoft's timeouts document puts it in two blunt sentences that hold for any Windows client: "Any Name Error response by any of the DNS servers will cause the process to stop - client doesn't retry with the next server if the response was negative" and "Client tries new servers only if the previous are unreachable." The recommendations guide carries the longer version — "Only a failure to respond will cause the DNS client to switch Preferred DNS servers; receiving an authoritative but incorrect response does not cause the DNS client to try another server" — written with domain controllers in mind, but pointing at the same thing: the trigger for failover is silence, never content.

Translated into what happens on a Tuesday at nine: if your resolver answers >NXDOMAIN for your ERP's name because a filtering list overreached, or returns a stale address because a zone did not replicate, the second server never gets asked. There is no failover. There is no alert either, because port 53 answers and your monitoring shows green. What there is, is people who cannot work and a service that is technically available. That is the failure the redundancy you bought does not cover. In what reaches us it shows up more often than a provider's global outage, though we have no public figures to back that: take it for what it is, a workshop impression.

Two addresses, one configuration plane

Go back to 11 October with that in mind. The bad value propagated to every paid-plan server at once. If you had both of that provider's addresses handed out by DHCP — which is exactly what every setup guide recommends — you did not have two servers: you had two doors into the same place. Anycast's geographic spread gives you path diversity, which is real and covers a fibre cut or a dead node. It does not give you configuration diversity, and that day's failure was a configuration failure.

This is not an isolated case, nor a small provider getting it wrong. On 14 July 2025, Cloudflare's public 1.1.1.1 resolver stopped serving the majority of its users for 62 minutes, from 21:52 to 22:54 UTC. The cause: an internal configuration error introduced on 6 June that lay dormant for over five weeks with no effect, until another change to the same service triggered it and withdrew the resolver's prefixes globally. Their words: "This outage was the result of an internal configuration error and not the result of an attack or a BGP hijack." In both cases the change had sat stored for weeks doing nothing, and when it did fire it fired at every location at the same time — which is exactly what geographic spread cannot prevent.

We have told this shape twice already from other angles, because it recurs at different layers of the same network: when the failure is in the brain rather than the legs, the control plane is the first thing to fall over; and on the telecoms invoice, where two carriers on the invoice can be one core on the network. The technology changes; the question is always the same: what do my two paths actually share?

When you should NOT add a second provider

The reflex after reading all of the above is to put another vendor's public resolver as the adapter's second entry. In an Active Directory domain that is a mistake, and again it is not us saying so: "Don't configure the client DNS settings to point to your ISP's DNS servers. If you do so, you may experience issues when you try to join the Windows Server-based server to the domain, or when you try to log on to the domain from that computer. Instead, the internal DNS server should configure forwarding to the ISP's DNS servers to resolve external names."

To that reason we add ours, which is a security one and is rarely said out loud. If you pay for filtering DNS — and some pay for it without knowing, because it ships inside another product — putting an unfiltered public resolver next to it is documenting your own bypass. Now put the two pieces of this post together: a one-second blip is enough for the machine to switch to the alternate, and once there it stays for up to 15 minutes. A micro-outage at 9:03 can leave a workstation resolving outside your policy until 9:18, with nobody noticing and not one line in any log.

So where does redundancy belong: in the resolver you control. Two internal resolvers, each forwarding to two independent providers or to the root servers. The client sees two internal addresses and a single policy; provider diversity lives in the layer where you can reason about it, measure it and change it live without touching 200 desks. And the honest exception, because there is one: if you are an eight-person company with no server of your own, two public resolvers in DHCP is a reasonable trade. All we ask is that it be a written decision — "we accept that filtering is best-effort" — and not an oversight you discover on the day of the incident.

The fifteen-minute check

This is answerable today, with what you already have installed and without buying anything. Start by looking at what your DHCP actually hands out, not what you think it hands out:

# Windows: la lista real y su orden, por interfaz
Get-DnsClientServerAddress -AddressFamily IPv4

# Linux: si resolv.conf apunta a 127.0.0.53, la lista buena es la segunda
cat /etc/resolv.conf
resolvectl status

# Cuánto tarda cada resolver, uno por uno
for s in 10.0.0.1 10.0.0.2; do
  echo -n "$s  "
  dig +tries=1 +time=2 @$s everywan.com +noall +stats | grep "Query time"
done

1. Write down the list per site, in order. A three-column table: address, who operates it and — the column nobody fills in — who controls its configuration plane. Two addresses from the same provider are one entry in that third column, however far apart they sit.
2. Measure the silence, do not assume it. Block the primary at the workstation firewall (a rule dropping udp/53 to that address) and time a real resolution with the cache empty, otherwise you will be timing the cache and it will read zero: Clear-DnsClientCache; Measure-Command { Resolve-DnsName your-erp.internal } on Windows, time getent hosts your-erp.internal on Linux (check first whether nscd or resolved is caching underneath). Whatever number comes out is what your people will feel.
3. Compare the answers, not just that they answer. Ask every resolver in the list for the same internal name and lay the results side by side. If one returns an address and another returns a non-existent domain, you have just found the failure your second entry does not cover, and you found it on a quiet day instead of a Monday at nine.
4. Alert on the answer. The check that is worth having is "this name resolves to this address from this site", not "port 53 answers". We monitor with Zabbix and SmokePing, and the house rule is that an alert has to hurt: applied to DNS, that means checking the record that comes back, not that the port replies. Anyone can see a silent outage; nobody sees a wrong answer.
5. Decide and write down what you want when your resolver cannot answer. Either it does not answer and the office stops, or it answers unfiltered and you lose the policy during the incident. There is no third option, and the decision is not a technical one: it belongs to the business. The bad outcome is not picking either one — it is reaching the incident without having picked.

Where this fits in what we do

We run our own network — with BGP, transit and peering — and this post comes out of that: the hard part of DNS has never been standing it up, it is deciding what happens when it answers wrongly and having a way to find out. It is what we do under networks and communications, and it nearly always starts with the same boring question: who does each site actually ask, and where is the diversity. It is seldom where the diagram says.

If you want us to build that three-column table for your sites together and actually measure what a silent resolver costs on your desktops, let us talk. It takes an afternoon. If it turns out it was already well built, we will say so and nothing happened here.

Sources. The dates, duration, scope and cause of the 11 October 2026 incident (16:40–17:35 UTC; queries unanswered from 16:42; restoration starting at 17:31; all paid plans affected at every location and over every protocol, the free service unaffected; the 16 September API update storing the per-domain override as an integer rather than a string; the error-handling flaw that stopped the service instead of skipping the value; and the subsequent measures) come from Control D's public incident report. The Windows DNS client timeouts (0-1-2-4-8-10 seconds, the behaviour with one, two and three or more servers, and the warning about servers in position 4 or higher) are in Microsoft Learn's "DNS client resolution timeouts". The two quotations about when the client switches servers, the 15-minute default for ServerPriorityTimeLimit and the recommendation not to point clients at the ISP's DNS servers but to forward from the internal DNS instead are in "Recommendations for DNS client settings" on the same site. The glibc resolver defaults — 5-second timeout, 2 attempts and a maximum of three servers — are in the resolv.conf(5) manual page. The 1.1.1.1 outage of 14 July 2025 (62 minutes, 21:52 to 22:54 UTC, with the configuration error introduced on 6 June and the quotation stating it was neither an attack nor a BGP hijack) is in Cloudflare's public report. Translations of the quotations are ours. The following are our own judgement and not claims by any of those sources: reading a provider's configuration plane as a failure domain shared by all its addresses, the consequence of combining the switch to the alternate server with the 15-minute stickiness in a filtering policy, the recommendation to put provider diversity in forwarding rather than on the adapter, and the fifteen-minute check. everyWAN provides managed network and security services: the conflict of interest is declared. We neither resell for nor partner with any of the DNS providers mentioned. Cover photo: "The inside of a wiring closet at a small public university, 2014", by SnopdouS, Wikimedia Commons, CC0 1.0.