Decentralized… Until DNS Breaks

Every time there’s a big internet wobble, the same memes shows up:

This isn’t a "Cloudflare bad" post (the last outage wasn't even DNS related!).
These huge, centralized services are popular for a reason: excellent uptime and redundancy, developer friendly API's, a one stop-shop for all your cloud needs.

They're so great infact, that huge chunks of the internet depend on them. Even decentralized systems often share the same centralized entry points. And for most stacks, DNS is one of the big ones.

Luckily, with one simple trick, you can own those headaches yourself!

What everything does before it does anything

Whether a Livepeer Gateway wants to talk to `some-orchestrator.net`, or your browser wants to pull a video feed from `play.frameworks.network`, they all start the same way: ask around to find out where that name lives.

There are two pieces in that story:

1. The resolver
This is what your OS is configured to use. Usually whatever your ISP provides or 1.1.1.1, 8.8.8.8, etc.
It might already know the answer (cached from previous requests), else it will go and find out for you.

2. The authoritative DNS: the actual 'phone book'
Every domain has one or more name servers that are authoritative for it. That’s where the actual records live: this name → this IP.
For most of us, those name servers are run by your DNS registrar. Or, if you require fancy Geo balancing with health checks, it's likely Cloudflare, Route53 or NS1.

To summarize:
1. Your client / app / gateway asks the resolver.
2. If the resolver doesn’t have the answer yet, it goes to the authoritative DNS for that domain.
4. It gets back one or more IP addresses.
5. Only then can all the 'smart logic' even begin:
  - measure latency,
  - check capacity and health,
  - decide where to send the actual traffic.

If either side of that DNS story fails, none of the clever stuff even gets a chance.
That’s why DNS is such a crucial single point of failure:

- end-users don't usually run their own DNS resolver, they rely on their ISP or a few public ones.
- operators cluster around a few huge authoritative DNS platforms

So even if the backend is nicely distributed, "where do I go?" is answered by a small set of shared services that sit in front of everything.

The trick behind "one IP that works everywhere"

You might ask yourself: why do I even need an authoritative DNS platform to balance load geographically? How is it possible for them to route requests across the globe to the nearest PoP?

Those magic single IPs (like 1.1.1.1) and global DNS platforms aren’t actually running from one giant box in one place. Under the hood, the internet is already pretty decentralized:
- It’s made of thousands of separate networks.
- Each network has an Autonomous System Number (ASN).
- Those networks talk BGP to each other and say: for these IP ranges, send traffic my way.

On top of that, big providers use anycast:
- They run the same IP address on lots of machines all over the place.
- They announce that IP from all those locations at once using BGP.
- The internet’s routing system (all those networks exchanging routes and peering with each other) usually steers you to a nearby one.

From the outside, you just see a single IP or hostname that works everywhere. Underneath, it’s one address shared by many locations:

The internet itself is already decentralized.
We’ve just gotten used to hanging our own services off a few central services.

If you want to go deeper on how any of this works under the hood, these are good starting points:
- How the Internet works
- What is Anycast?
- What is BGP?

Our stack: already unclouded… except for the front door

One of the north stars for any project we're involved in is to be 'Unclouded'. To keep working even when big cloud services fail. Just like the decentralized stack that handles our transcoding.

No serverless sauce. Everything must be managed in-house and deployed on bare metal machines or VPSes spread across different data centers, from the media pipeline to the API's & dashboards.

But the 'phone book', the part that answers "what IP do I dial?", still lived on third-party services. Like everyone else, we used Cloudflare DNS because it’s fast, affordable, has an excellent uptime record, and exposes a nice API to manage zones and routing policies.

Nothing catastrophic happened to our zones. That’s not the point.

The point is this: when that shared DNS layer does go sideways, our nice "unclouded" stack would have gone dark in exactly the same way as everyone else who made the same perfectly reasonable choice.

It's the final third-party dependency that can take down the entire system.

Becoming a Stronk Autonomous System

So now it's time to make the final leap. To be a part of the decentralized web, we have to be our own authority. This means:

- getting our own ASN.
- acquiring our own IP space.
- connecting our network to other networks (peering and transit).
- running our own DNS on top of that, using anycast, just like the big guys.

This lets us share the same IP's across multiple PoPs as well as provide a lot more flexibility in routing streams to whichever node is best for the job.

We’ll still use third-party tools where they make sense: email servers, GeoIP databases, SSL certificates, stuff that doesn't bring down the network on a hiccup.

But at least the story changes from:

"Some big shared service broke, so we’re down."

to:

"We're down because we broke something. Oopsie, that’s on us."

Now that's decentralisation: not just running distributed compute, but ensuring the entire end-to-end service can stay up when others falter.