On August 7, 2010, DNS Made Easy underwent an outage that was caused by a Distributed Denial of Service (DDoS) attack. While the outage for many companies was sporadic, it lasted for multiple hours in regions of the west coast of the United States. Over the course of the past eight years, DNS Made Easy has prided themselves on their 100% uptime. A blow like this can affect a company’s public image, hinder their marketing, shift their business strategy, and impact their bottom line. So, how did this happen? Is it preventable? And, what can you do about it? In this situation, it is easy to blame DNS Made Easy and say that they didn’t have the proper security controls in place to withstand this type of attack, but in reality things are not that simple.
Many of us in the security field are fully aware of what a Denial of Service (DoS) attack is and ways to mitigate the impact. We can deploy firewalls with anti-DoS capabilities, anomaly detectors to analyze and pass through the legitimate packets, and system hardening. All of these controls can minimize or nullify most DoS attacks. But didn’t I say this was a DDoS? Yeah, that is right. In that case, none of those will reduce the effect.
DDoS attacks are challenging to resolve and nearly impossible to prevent. The challenge stems from the taxonomical difference between a DoS and DDoS. A DoS is classified as an attack carried out by one system, whereas, a DDoS attack is carried out by multiple systems, which can reach numbers in the tens to hundreds of thousands. This difference changes the characteristics of the attack. In a DoS scenario, the threat agent is attacking the system by either exhausting the system resources or crashing it. Opposingly, a DDoS directly attacks a company’s bandwidth capacity. It does attack a system, in that it is flooding a specific IP or range of IP addresses; however, the direct attack is against the company’s bandwidth. It is common in a DDoS attack that the entire organization’s internet presence is out of commission, even if they are only attacking one system.
Being proactive and setting up the controls in order to prevent a DDoS attack from affecting business operations is not possible, at least not from a corporate level currently. That taxonomical difference between DoS and DDoS means that deploying our conventional arsenal of tools will not lead to resolution. You can deploy firewalls, IDS/IPS/HIPS, anomaly detectors, application firewalls, and much more, but your bandwidth is already consumed by that point – game over. At the headend, you can stop the illegitimate packets from making their way to your system, but the attack already consumed your bandwidth, so you cannot respond back to the request. The only thing you can do is identify what the problem is, call your ISP, and pray that you still have a good working relationship with them.
The first step is identification. Being able to definitively define what the problem is can make the difference between a quick recovery or twiddling your figures wondering why you are still down after you applied additional controls. While prevention may be impossible, identification is relatively easy. This can be accomplished by having monitoring systems that analyze netflow for bandwidth trending, firewall logs for understanding what is being attacked, and an IDS to identify the type of traffic. By using some of these resources, you will be able to identify the attack quickly and effectively.
So how do ISPs resolve the issue? Is it magic? Nope. Networkers have lost all of their mystical abilities the moment the masses learned that it was possible for them to understand how to subnet. Instead, it is an eclectic approach utilizing remotely triggered black hole routing and analyzing backscatter. Using both of these tools in concert, allows an ISP to identify where the traffic is coming from.
Remotely Triggered Black Hole (RTBH) routing is the process of dropping traffic based on source or destination addresses. If the attacker is spoofing their source address, then the ISP will have to trigger the black hole based on the destination address. Yes, that is right; in order to stop the attack the ISP will have to impart its own Denial of Service. The ISP creates a trigger, which gets propagated through BGP, the internet’s routing protocol. In order to black hole the traffic, the ISP has the router send it to a “null” interface. The null interface is not a physical interface on a router, but a logical one that can be used to effectively drop traffic. Once the router drops the traffic, it provides a useful response. It sends an ICMP packet to the attacker’s source address. This response is known as backscatter. Now since the original source address has been spoofed, the value is not where the router sends the packet. Instead, the value is the router that sent it. When the router sends the ICMP response, it replies with its IP address as the source. The ISP records the router’s response and it provides them with a useful metric. If they notice one entry point into the network generating 100 ICMP responses, in relation to their RTBH configuration, then they can be assured that the attacker is not located off that entry point. However, if they noticed that one of the entry points has generated responses numbering 500,000 packets per second, they know it is one of the points of contention. This is usually an iterative process involving many ISPs. Each of these ISPs follows the same process until they are able to identify the offending networks. Once they are able to identify the attacker, then they are able to configure Access Control Lists (ACLs) in order to stop the DDoS.
With additional help, DNS Made Easy was able to neutralize the current threat and resume business operations. We may never know the true motive behind that attack, but it serves as a stark reminder that the threat is real. In order to mitigate this threat, ISPs will need to combine their efforts. It would take a global effort on the part of the ISPs. Until then, the capabilities are there to deal with the issue when it arises on a per incident basis. Remember, in order to facilitate a quick recovery, verify that you have the tools in place to detect the attack, the knowledge resources that understand what to do, and a good relationship with your service provider. Help is out there if you need it.
By Jason A. Suplita,Senior Risk Management Consultant