r/Network
•
u/jyotirawattech
•
1d ago
Text FAA says a copper to fiber swap at one remote site flooded routers and stopped US takeoffs. How do you change a network you can't safely test?
Reading the updates on Friday's FAA ground stop, the cause so far is weirdly ordinary. Per the Transportation Secretary, a contractor (L3Harris) was doing routine maintenance at a remote FAA site, replacing copper with fiber. Something in that work started flooding other routers, and the routers in those facilities were made in 2001. Controllers saw radar and radio data go inconsistent, starting at the Chicago center and then Boston, Washington, New York and Minneapolis. Takeoffs stopped nationwide for about four and a half hours. They're still working out whether it was an install mistake or a faulty router.
I've spent most of my career on networking and platform code, and this is the kind of change that looks low risk on paper. It's one site and a physical layer swap. But with gear that old you often can't reproduce the setup in a lab, and one box that starts misbehaving can spray traffic everywhere if nothing upstream limits it.
Duffy's line was that by doing nothing you inject way more risk into the system. I half agree. If you freeze an old network until the full replacement lands, it keeps aging and fewer people remember how it behaves. If you keep patching it piece by piece, every small change is a production experiment on a system nobody can fully model.
If you ran a network like this, would you freeze changes until the replacement was ready, or keep doing small cutovers and spend the effort on containing the blast radius?
