Business Continuity planning is a bit like writing a will. It forces you to contemplate eventualities you’d rather not think about and you may have to answer some really tough questions. The recent cyber attack on hosting company, Code Spaces, began with a simple DDoS attack and resulted in the collapse of the company within just a few hours. Natural, malicious, or even innocent events can come out of the blue and can be disastrous for your organization. If you are not prepared to respond and recover quickly, such a disaster can be deadly.
Business Continuity Disaster Scenarios
In business continuity and disaster recovery planning, it is important to consider any threats that could conceivably impact your business operations. A thorough, up to date risk assessment can highlight some of the threats you may be facing. Your business continuity plan must consider all the possible threats your company could conceivably face. While the threats you may need to consider vary greatly from industry, location, or from one company to the next, common threats most organizations should consider come from natural disasters, malicious attack, and even innocent human error. Your reaction to different kinds of disaster scenarios will be different but the importance of planning for every possibility is the same.
Hurricanes and Natural Disasters
As a Florida-based company, hurricane season is a fact of life from June through November. If we polled our employees, friends, and neighbors, we would find varying states of preparedness for what we all realize is a very real threat. Whatever reasons a family may have for not adequately preparing for a hurricane, just like whatever reasons a business may have for putting it off or not having a solid plan in place, those reasons will not matter if you find yourself hiding in a closet or up to your knees in salt water someday.
When Hurricane Sandy struck the northeast in October, 2012, the disruption to lives and businesses was traumatic. Over several weeks and months, people struggled to rebuild and recover. The physical damage to your business in such an event can be difficult enough. But for a business to survive such a disaster requires careful planning long before the rains start. You must not only account for your people and property but, if you are a business that stores or relies heavily on data and systems, you must plan to quickly recover that data and your operations.
Hackers and Cyber Crime
We have seen large scale cyber attacks put major corporations on their heels in recent years. Target is still recovering and the fallout has been significant within their corporate ranks. PF Chang’s restaurant chain has just begun to deal with the technical and public relations nightmare of a similar data breach.
Most dramatically, online hosting service, Code Spaces, promised its customers a “full recovery plan that has been proven to work and is, in fact, practiced.” But, when a simple DDoS attack quickly proved to be a smokescreen for a much more elaborate scheme, Code Spaces found the walls falling down around them. By gaining access to their systems, the attacker(s) not only accessed virtually all of the company’s (and their clients’) critical data but also found the link to admin files, authentication keys, and the very backups that were supposed to be secure against just such a disaster. Within a matter of hours, the hacker(s) managed to delete repositories, backups, snapshots, and databases leaving what was an operational company that morning in devastated ruin by afternoon.
Human Error
“…over 95 percent of all [security breach] incidents investigated recognize “human error” as a contributing factor. The most commonly recorded form of human errors include system misconfiguration, poor patch management, use of default user names and passwords or easy-to-guess passwords, lost laptops or mobile devices, and disclosure of regulated information via use of an incorrect email address. The most prevalent contributing human error? “Double clicking” on an infected attachment or unsafe URL.” (Source: IBM)
We have all known that horrible feeling of realizing you just closed a window without saving the report or spreadsheet you were working on. Most software now will autosave our work for us – simply because the tendency for humans to just plug along without hitting save is so great. We’ve all done it. It is surprisingly easy – and common for the same thing to happen to entire databases. In updating or moving large collections of files, there is always the danger of something being missed, over-written, or just deleted. Without proper controls and back-ups in place, a simple error could be disastrous. Supposedly, Pixar’s Toy Story 2was nearly lost completely to a backup blunder.
Business Continuity and Disaster Planning Tips
As Code Spaces found out the hard way, just having a backup may not be enough. If you are relying on a service that promises to maintain your backups, be sure that their own processes are adequate. Your data must also be secured both logically and physically.
Logically separate backups, data that is not accessible from within the same system, that are physically kept in one location are susceptible to many of the same physical threats.
Physically separate backups, data stored in more than one location that can be reached from one digital access point, is susceptible to many of the same technical threats.
Be sure to avoid these common backup mistakes that can undermine your efforts:
Do not make backups accessible from one machine or location.
It appears that Code Spaces’ backups could all be reached from one access point within their system. Once the hacker reached that point and gained access, he was able to delete the data and the backup together
This applies to a natural disaster as well. If you are in an area prone to a weather event (and really, everyone is), having backups stored in the same location that may be simultaneously affected by a flood or fire defeats the purpose of backup.
Store authentication keys on a different machine or at a different location.
When Code Spaces detected the DDoS attack and attempted to shut out the attacker, they found that he had already accessed the keys and obtained new passwords into the system. As quickly as they changed passwords, he changed them again. They were virtually powerless to stop the attack.
Do not map the drive to the backup server.
Setting yourself up with a shortcut into backup servers might seem like a time-saver but it can give an attacker easy access to backups created specifically to prevent losing data to an attack. Servers established to back up data should be maintained completely separately from the data being backed up. If data and its backup are linked, it’s no longer a backup, it’s simply a copy.
Secure out of date as well as current backups.
We often see companies go to great lengths to maintain secure backups and to keep those systems updated only to forget about tapes or files once they are outdated and have been replaced by a newer backup. While the data on those devices or systems becomes outdated, it may still be sensitive data.
Ultimately, you must ensure that your backups are not subject to the same threats as the systems being protected. A well-crafted business continuity plan does not just check the boxes, it accounts for the possible scenarios. A diversified backup plan may be the difference between a recoverable event and a devastating, truly catastrophic disaster.
