When the NotPetya virus ravaged corporate networks worldwide, it was particularly devastating to global shipping company Maersk. Ports around the world ground to a halt for two weeks as Maersk tried to recover key systems. The biggest obstacle: Microsoft domain controllers.
As Andy Greenberg told in his Wired article (and later in Sandworm, his excellent book on Russian hacking):
No one could find a backup for one crucial layer of the company’s network: its domain controllers, the servers that function as a detailed map of Maersk’s network and set the basic rules that determine which users are allowed access to which systems. Maersk’s 150 or so domain controllers were programmed to sync their data with one another, so that, in theory, any of them could function as a backup for all the others. But that decentralized backup strategy hadn’t accounted for one scenario: where every domain controller is wiped simultaneously. “If we can’t recover our domain controllers,” a Maersk IT staffer remembers thinking, “we can’t recover anything.”
It's easy to overlook domain controllers. They aren’t sexy. They aren’t the servers that house corporate secrets or customer data. But if you lose them, your employees can’t log into Windows workstations, run in-house applications, access shared files, browse Intranet sites, schedule meetings, or read email. Effectively your company comes to a full stop.
Domain controllers are your organization's central nervous system. Losing them results in paralysis.
The Maersk story has a happy ending, but only because of a minor miracle in Africa:
After a frantic search that entailed calling hundreds of IT admins in data centers around the world, Maersk’s desperate administrators finally found one lone surviving domain controller in a remote office—in Ghana. At some point before NotPetya struck, a blackout had knocked the Ghanaian machine offline, and the computer remained disconnected from the network. It thus contained the singular known copy of the company’s domain controller data left untouched by the malware—all thanks to a power outage. “There were a lot of joyous whoops in the office when we found it,” a Maersk administrator says.
A Maersk employee flew to Africa to retrieve the hard drive from the company’s lone surviving domain controller. The precious data was uploaded to other DCs and replicated across the global network. Within hours, global shipping resumed.
Maersk was lucky, but luck is not a strategy.
One obvious lesson from the Maersk incident is to backup your domain controllers, rather than rely on replication to protect Active Directory data. As with other important servers, regular backups must be automated, and backups stored off-network where ransomware and other malware can’t touch it.
In addition to regular backups, it’s possible to turn one or more domain controllers into a “designated survivor” that can’t be erased or corrupted by malware. Thanks to a feature I persuaded Microsoft to roll out in Windows Server 2008, a Read Only Domain Controller (RODC) can be deployed in less secure environments (like your Ghana branch office). While not a substitute for backups, the RODC may come to your rescue if other DCs are lost in a NotPetya type of scenario.
Malware like NotPetya isn’t the only threat to domain controllers. Attackers can also wreak havoc by locking all your employees out of their Windows accounts. For example, an insider password spraying attack -- trying common passwords on each Active Directory account until it locks -- will freeze everyone out of Windows the following morning. As an added bonus, it likely will yield unauthorized access to at least a few accounts with common passwords.
Backups don’t help with this kind of massive denial-of-service attack. Instead you need tools or scripts that can (a) alert your SOC to a high-velocity spike in failed logon attempts, (b) unlock all AD accounts in an emergency, (c) reject password changes that match a dictionary of common passwords. I’ve personally seen scripts that do all these things. They're not super difficult, but you can always hire a Microsoft consultant if no one in your IT department has the skills to do it.
Create a strategy that blends backups, RODCs, and scripting to protect your domain controllers. Without them your other crown jewels (customer data, company secrets, etc.) are almost useless if people can’t access them.
Michael McCormick is an information security consultant, researcher, and founder of Taproot Security.
Comentarios