Business continuity

Prepare for disaster recovery now and mitigate the risks

IT Business Staff IT Business Staff Published: 08/11/2015

There’s nothing like a major electrical outage or natural disaster to forcibly nudge CIOs and other executives to ensure disaster recovery and business resumption plans are up to date and effective. Unfortunately, while incidents that “wake us up” serve a beneficial purpose, as time passes other priorities will often rise to the top of the list – putting disaster recovery on the back burner – often accompanied with flawed thinking: “it’ll have to do for now, I’ll make it better later”.

But later could be today. And that type of thinking will come back to bite you. From natural disasters to cyber-attacks, your systems are always at risk and so too is the rest of your business. The potential downside of delaying the implementation of a sound disaster recovery plan is huge. In a world where a competitor is a mouse click away, even the smallest amount of downtime can result in significant consequences – from the loss of customers to significant legal liabilities and financial losses. Those are not risks to be undertaken lightly or shelved until some future time before dealing with them. The good news is that where disaster recovery and business resumption once involved gargantuan efforts and came with a hefty price tag, disaster recovery today is technically easier, affordable, and can be implemented and maintained more easily than ever — thanks to new services and cloud-based disaster recovery.

So, where do you begin? Properly documented procedures and processes are fundamental to efficient, sound recovery. So too is a comprehensive plan – covering all aspects of technology and people and clearly delineating who is responsible for what. Disaster-recovery-as-a-service can help get you back up and running in no time – but it can’t get you there without you doing the upfront work – ensuring you’ve covered off all the bases. Much can also be said about the need to “practice” and having your people familiar with the right steps they need to follow should disaster strike.

More business are turning their thoughts to disaster recovery, recognizing the need to mitigate the risks. And, they are finding that there are a number of key things that everyone should know in order to make this process run smoothly. To learn more from these experiences and what’s involved in building an affordable, sound plan, you may want to attend this one-hour webinar from ITWC, sponsored by Rogers, to get an overview of the basics and to discover the advantages of cloud-based disaster recovery. Don’t hold back. Remember tomorrow may be too late.

On Demand Webinar


What if your Disaster Recovery Plan were put to the test?

Join IT World CIO Jim Love for an informative webinar covering Business Resumption and Disaster Recovery. We will cover the basics of Disaster Recovery Planning (DRP) from the perspective of both large and small companies.

  Register Now  

The cost of NOT having a business continuity strategy

Dave Webb Dave Webb Published: 07/31/2015

A business continuity strategy is an expensive proposition. In a previous post, we discussed how duplication of data and processes, the need for geographic distance and high-speed connectivity pile up costs, and how a service provider can offer existing facilities, economies of scale and cost certainty.


The Enterprise Connectivity Series
Future-proofing your business

This time, let’s talk about the cost of *not* having a business continuity strategy. A few numbers, courtesy of the Web:

According to research house Ponemon Institute, a minute of downtime costs the average enterprise $7,900 (all figures US). That’s a number from 2013. And since the number was 40 per cent higher than it was in 2010, we can presume the 2015 number is considerably higher.

Of course, there are as many numbers as there are research reports. USA Today pegged the cost at more than $50,000 an hour according to 80 per cent of data centre managers surveyed, while 25 per cent said the cost was more than $500,000 an hour. And every enterprise isn’t the same. For some—particularly e-commerce firms and those that provide networking services to customers—it’s more expensive than others. So, your mileage may vary.

IT operations analytics firm Evolven, based in Jersey City, breaks out the costs of data centre downtime quite thoroughly, based on information from benchmarked data centers. For an average outage—not on an hourly basis, but per outage—Evolven’s Top three business costs are:

  1. Business disruption: $180,000
  2. Lost revenue: $118,000
  3. End-user productivity: $96,000

Those are some steep numbers, and they don’t include the cost of detecting the cause, remediating the problem, and IT hours spent, among other things.

The cause of outages is often beyond the control of the enterprise. Ponemon, for example, cites external attacks (34 per cent) and weather (30 per cent) as major contributors to failures. (Apropos to nothing in particular, the Weather Company, the most accurate forecasting organization in the world, with more than two billion sensors at its disposal, gets it right 75 per cent of the time, so if you think you’ve got the weather sussed, have another think.)

But many outages are a result of systemic failure. Ponemon also cites UPS failure (55 per cent), exceeded UPS capacity (46 per cent) and human error (48 per cent) as contributors. Yes, we’re well over 100 per cent, but we’re talking multiple failures, apparently. And that’s a concern.

It’s also possibly part of the reason that enterprise data centres have more outages than collocated facilities. Among enterprise data centers surveyed by Uptime Institute, seven per cent reported more than five outages in the previous 12 months. Only three per cent of collocation providers said the same.

My theory is that it is a matter of focus and experience. The enterprise IT department doesn’t just have to run a data centre, it has to provide end user support—and end users are not technologically sophisticated enough to deal with upgrades, patches, licensing, procurement … the list goes on. Service providers such as Rogers run data centres. Some of them run dozens or even hundreds. There is a volume of lessons learned, best practices discovered and applied.

So it goes, also, with your best defence against business disruption—your disaster recovery/business continuity (DR/BC) strategy. Keeping it in-house may provide a feeling of superior security and control, but it’s adding another layer to IT’s already complex job. Allocating the responsibility for DR/BC to a service provider—after negotiating a very strict service level agreement (SLA)—allows IT that bit of breathing room so the department can focus more on strategic alignment with the business and what it can do to grow the bottom line.

Hot, cold and warm backup sites: Which is just right?

Lynn Greiner Lynn Greiner Published: 07/29/2015

When disaster strikes, it’s important to get operations going again as quickly as possible. If that doesn’t happen, chances are the business will fail; the U.S. Federal Emergency Management Agency says that forty per cent of businesses don’t even re-open after a disaster. And one huge part of the recovery is getting the IT infrastructure up and running.


The Enterprise Connectivity Series
Future-proofing your business

For companies that can’t afford downtime, that means maintaining a backup site. There are three main options: a hot site, a cold site, or a warm site.

We’re not talking about the temperature of the room. Instead, the deciding factor is the readiness for immediate use, and the associated cost.

The least expensive option is a cold site. Basically, it’s serviced data centre space; an empty room. There’s no computer equipment installed, and your backups aren’t there, ready to go. In case of disaster, you have to source and install equipment, then get it configured and retrieve and restore data. That takes time, effort, and money at the time of the disaster, but is the least expensive to maintain.

Some companies cut costs by saving old servers that have been retired from the primary data centre for use in the cold site. That does save a few dollars, but may not provide sufficient horsepower – those servers were usually replaced for a reason – however, they may do in a pinch, and the benefit is that they can be preconfigured before being put on the shelf.

Next in the dollar queue is the warm site. It has equipment installed and connectivity established, but may not have all of the resources of the primary site. Data may or may not be present already, but will probably not be up-to-date, so there would be a delay while current backups are retrieved and loaded. It’s quicker to get going at a warm site than at a cold site, but it’s not immediate.

If you absolutely have to fire up the backup site the moment your primary site goes down, a hot site is the only way to go. It can be a duplicate of the primary, with live equipment, the same capacity, and as close to current data as possible, or it could be sized to run mission-critical operations only, depending on company requirements. Some companies operate two data centres, with each acting as the hot site for the other, and data continuously synchronized between the two, so they can do an immediate fail-over. The newest options are in cloud solutions such as DRaaS and IaaS; organizations may use managed hosting or a company that specializes in providing backup facilities to accomplish the same ends. Again, you can get anything from what’s essentially a hot site on down, depending on how much time and money you have. In any case, a hot site isn’t cheap.

Rarely, two organizations with sufficient spare capacity may embark in a reciprocal agreement, in which each agrees to back the other up in case of disaster. It takes careful attention to SLAs, however, and a lot of trust.

Making a decision can be difficult. Balancing budget and needs is a juggling act. Consulting pros like the Rogers business continuity experts can help weigh the alternatives and come up with a plan that works.

 

Resources to tap for disaster recovery best practices

Lynn Greiner Lynn Greiner Published: 07/28/2015
Image of blue line graphs, charts
Image courtesy of Shutterstock.com

We all hope that we’ll never need to use them, but disaster recovery and business continuity plans are a critical part of any organization’s strategy.


The Enterprise Connectivity Series
Future-proofing your business

Without a plan, if anything happens to interrupt the conduct of business, whether it’s a fire, a flood, bad weather, criminal activity, or even a major event that makes it difficult for employees to get to the office, your company could be in big trouble.

Building those plans is a non-trivial task. There’s a lot to think about, a lot of i’s to dot and t’s to cross. And they’re not just IT details. While technology is a large part of the exercise, people and facilities have to be considered as well.

Fortunately, there’s plenty of expertise around to help make sure we don’t miss any critical components of the plan. Best practices abound; it’s just a matter of finding them.

A good place to start is with a professional organization whose reason for being is disaster recovery: the Disaster Recovery Information Exchange (DRIE).

DRIE’s stated objectives are:

  • To provide a forum for the exchange of information among business continuity practitioners;
  • To be an authoritative source of information relating to business continuity;
  • To promote business continuity awareness within the business and government communities;
  • To advance the professional standards of the business continuity discipline; and
  • To engage with representatives from commercial, not for profit and government organizations in providing information to support the most effective and efficient business continuity schemes for the protection of life, health and safety of individuals, and the protection of the property of organizations and the environment in Canada.

Membership is open to people whose work is related to, or who are interested in, business continuity.

Another great resource comes from networking giant Cisco, whose comprehensive disaster recovery best practices white paper is based on a publication by the U.S. National Institute of Standards and Technology (NIST). It walks through everything from risk analysis to document maintenance; a disaster recovery/business continuity plan is a living document that must be regularly revisited and tweaked to reflect the real world situation, and the Cisco white paper explains how to go about the task.

Forbes has also provided a good overview of best practices, as told by an industry pro, which includes the admonition “remember Murphy’s Law”, Data Source Solutions offers ten best practices, and CIO magazine’s executive council has chimed in with its eight best practices for disaster recovery.

If the expertise to build a disaster recovery plan doesn’t exist inside your company, and you don’t have the time or resources to acquire it, you can contract with companies like Rogers, whose services organizations contain specialists in disaster recovery and business continuity. They can help build the plan, as well as providing the necessary resources to test and implement it.

Planning communications for emergencies

Robert Dutt Robert Dutt Published: 07/27/2015

You’ve got a good plan to deal with emergencies that may arise. In case of the worst, you’re as ready as you’ll ever be. But does your plan include the details of how your organization will communicate when things go wrong? If not, you may be missing a key part of your business continuity or disaster recovery plan.


The Enterprise Connectivity Series
Future-proofing your business

In the age of smartphones with cameras and ubiquitous social media, it’s more important than ever for you to have your ducks in a row when it comes to how to communicate to your employees, customers, the media, community members, and other audiences when disaster strikes.

Preparations

Your business continuity plan should already include details like creating an emergency operations centre when things go wrong at a key facility, and rerouting phone lines and Internet access to that location. That way, you’ll have the networks needed to effectively communicate when disaster strikes, as well as to do so many of the other things you’ll need to be doing.

Next, you’ll need a list of who needs to be contacted. This can be a surprisingly long list, including management, employees, customers, suppliers, partners, government officials, news media and the rest of the community, and likely many more. The list should include names, organizations, contact numbers, e-mail addresses, and the department or organization responsible for making contact.

It should be stored somewhere secure, and easily accessible even under less-than-ideal circumstances. This is something great to put on a cloud-based network share or collaboration platform, locked down to be accessible only to the appropriate people within your organization.

The message

So you know who you need to contact, and you’ve got a list with their information. You’ve put some thought into who contacts whom in a given situation (more on this later). Now it’s time to consider what you’re going to tell each audience. Not the specifics, of course, but what information will be relevant (or mandatory) to share in the event of a disaster. Essentially, this boils down to “What does this mean to me?” in almost all cases.

Management is probably the first contact that needs to be made, as they need to know what’s going on and will play a key role in disaster recovery efforts, both through communications and otherwise. Your plan should spell out what types of scenarios align with the “contact by all means necessary at any time of the day” list, and which can wait until a reasonable hour. But in all cases, execs need to be informed of what happened, any injuries or property damage, and the likely impact on the production of goods or services that will stem from the incident.

Employees should be contacted by HR, with a focus on issues like when (and where) they should report to work, how their safety concerns are being addressed, and any updates possible on any co-workers that may have been injured or otherwise impacted.

Customers and partners should be contacted by their regular sales contact, addressing any downtime that may ensue and how that will impact their orders.

Appropriate government officials (regulators and local government) should be contacted by management with details of the incident, as well as details on impacts: safety, environmental and economic.

Suppliers should be contacted by those who typically manage them, with details on when (and where) to resume delivery of supplies.

Neighboring businesses or residences need to be contacted with any information relevant to them in the situation – what happened, are there any safety or environmental concerns they need to be aware of, where to follow up if more information is required or if they have suffered losses as a result of the incident.

And finally, the news media. This is probably the part your management will dread most of all, but it must be managed transparently and with great care. Initial contact should be made by way of a press release or other broadcast communication to local media, explaining what happened, any injuries, any details on loss, as well as any details on causes of the incident and steps that will be taken to prevent a similar incident in the future that may be available. A designated spokesperson, preferably from senior management with media training and a history of media interviews, should be made available for media interviews, and be prepared on what can – and cannot – be answered or addressed at this point in time.

Prior planning prevents poor performance

A good communications plan for when things go bad shows your organization is a concerned corporate citizen, can generate goodwill with your employees and customers, and can help to prevent the kind of long-term damage that can happen to your brand and business when the “details” of an incident are revealed 140 characters at a time by third party sources with no access to firsthand information, instead of coming “straight from the horse’s mouth” in as timely and forthright a manner as possible.

Simulate the worst to be ready for the worst

Robert Dutt Robert Dutt Published: 07/24/2015

You’ve crafted a fine business continuity plan for your business, one that ensures that should your organization’s feet be held to the fire, they won’t burn. You’ve followed all the best practices to a T, and thought of all contingencies. Or have you?


The Enterprise Connectivity Series
Future-proofing your business

Unless you’ve tested your ability to respond to a natural disaster or other business-threatening occurrence, do you really know how your plan, your process, and your people will respond when put to the real-life test?

Fortunately, there are a number of ways to test how your business will fare should the unthinkable become the current situation.

Starting small

The baby steps in business continuity planning are “tabletop exercises” so called because they generally consist of imagining a specific scenario with a handful of people – either members of one specific team responsible for the response to that scenario, or with representatives of a few teams within the organization that will be called upon for a broader scenario. Think of it as a roleplaying game without the 40-sided dice, where the future of your company is on the line.

It’s 4:00 A.M. and the fire alarm has been triggered at a key facility. And…… go!

Members of the test talk through the scenario and measure how the plan fared.

This is a great way to get started on making sure your business is ready to respond should things go wrong, and yet make sure everyone still gets to hit the local watering hole for a celebratory beverage when the day is done.

Getting more complex

Okay, so things worked out well in that scenario. But not every situation is going to be quite so compartmentalized and manageable. For your next test, it’s time to scale things up a little bit. This time, there are more people involved – perhaps people in different locations, differently affected by the hypothetically-unfolding situations. If role-playing in the previous situation was like a game of Dungeons and Dragons, now it’s time to make it a little bit more like paintball. The “bullets” may not be real, but there are certainly more concrete consequences for getting hit now. It’s time to start making the scenario – and everything around it – more realistic and as immersive as possible. And throw a few curveballs at the team along the way, because in a real disaster, odds are good that whatever hits the fan will not be evenly distributed.

The tendency is to test your plan with the best people around. Instead, pull in some less-than-ideal staff members and see how your plan holds up, because there’s a decent chance that under some disaster scenarios, you’ll find yourself trying to talk Joe from Marketing who lives five minutes from the office through some reasonably technical tasks to get the company back up and running, simply because Joe was on the scene half an hour before anyone else could be.

And of course, who’s to say that some key third-party services will be accessible if things go bad? Sure, maybe you’re not testing the resiliency of your phone system with this particular test. But maybe the phone system also just happens to be out as part of your makeshift disaster. Include it just to test the team’s ability to problem solve – a quality that will surely be tested should just about any disaster scenario play out for real.

The types of scenarios may play out over a few hours, but may also extend over several days to assess the plan’s (and your team’s) ability to hold up over those pesky disasters that just don’t want to go away. It all depends on how ready you want your organization to be, and conversely, how unpopular you, as the person planning the testing scenario, want to be.

Going deep

That second level of testing will do a pretty good job of simulating a disaster and testing the ability of your business to survive. But sometimes, you really have to know how things are going to go when things go bad.

This is likely a big test, involving a lot of moving parts from various departments. Have pre-determined start and stop times, and within that time frame, include a duration and path of events that is unknown to most participants. And if in the previous tests you threw curveballs, now it’s time to start throwing some knuckleballs. Maybe a spitball or two, too. Because business-threatening disasters are notorious for not playing fair.

It’s time to turn the realism up to 11. If you were talking through the pre-flight safety video before, now it’s time to start putting on air masks and sending people down the slide. Maybe this time, a location is actually evacuated. Maybe you spin up your disaster recovery site for real.

No pain, no gain

Putting your team in some of these situations is tough, but then are so are major business-threatening disasters. And by testing some of the scenarios that may truly test both your plan and your people, you’ll be putting your organization in the best-possible situation to survive the kind of event that can be the literal make-or-break point for any business.

Overcoming four common business continuity challenges

Jeff Jedras Jeff Jedras Published: 07/22/2015

You’ve made the business case for your business continuity plan, management is on board and your funding is in place. However, your work is just beginning.


The Enterprise Connectivity Series
Future-proofing your business

There are many potential pitfalls ahead as you seek to put your plan together and get it implemented. As you try to make your business continuity plan a success, here are four common challenges you may encounter. Recognizing them ahead of time, and knowing how to overcome them, can increase the likelihood your business continuity plan implementation will be the success you hope for.

1. Employees are unclear about their responsibilities

To many in the organization, business continuity planning can seem like an abstract concept, as well as one that likely doesn’t involve or apply to them. But everyone has a role in a successful business continuity plan and they need to understand it, as well as their responsibilities in the case of a business disruption. Establishing a business continuity governance model that clearly defines the responsibilities and roles of key employees is a critical component of a successful plan.

2. IT and business not communicating effectively

If your business continuity plan is to be a success there cannot be silos between the IT team and the business units – the two need to work hand in glove to effectively design and implement the plan. Often these two sides of the business aren’t used to working together, so effort must be made to establish channels of communication and cooperation and to put procedures and policies on paper, so both IT and business understand each other’s expectations and requirements.

3. Relying too heavily on vendor advice

For many vendors, business continuity is a great excuse to get you to open your pocketbook and buy their hardware, software and services. But whatever the vendor may be pitching this month may not be right for you, and their need to drive revenue growth should not influence your business continuity strategy. Don’t rely on vendors that may have a vested interest that doesn’t align with yours. Working with a trusted advisor like Rogers that specializes in business continuity planning, and that takes the time to understand your business needs, will give you the advice you need on what makes sense for your business.

4. Being unrealistic in your recovery objectives

As much as it may pain some employees to learn this, not everyone is essential to the operation. That’s not to say they should be heading to the unemployment line, but we could manage without them for a day if we had to. While each business unit should be asked to define their recovery objectives at the beginning of the process, there needs to be an objective analysis undertaken that weighs the criticality to the business vs. the cost of an immediate restoration of service. A neutral business continuity expert can help define realistic timelines that focus on getting the truly critical services back up first.

Four tips for selling business continuity to the C-suite

Jeff Jedras Jeff Jedras Published: 07/21/2015

For the IT department, the rationale for and benefits of a robust business continuity plan can seem obvious. To secure the needed funding from the c-suite however, the case can’t be made using the language of IT.


The Enterprise Connectivity Series
Future-proofing your business

If the business side is going to fund your business continuity plan – and it really is in their best interests to do so – you need to make your case in a language they understand: the language of business. While the technology side is used to thinking in terms of security and risk, business leaders tend to take a more dollars and cents-oriented view when it comes to greenlighting investment.

Business leaders are going to want to see tangible returns, and not just if and when a disaster strikes – everyone will support the investment when everything goes wrong. To get the go-ahead now though, you want to show them the business value and competitive advantage that can be realized through a robust business reliability plan.

Here are four things you will want to remember as you make your pitch to the business side.

1. The cost of downtime

The cost of business downtime can seem like an abstract thing without numbers, so it’s important you quantify both the likelihood of an outage and what it could mean to the bottom line of the business. Research from Symantec shows the average small to medium-sized business (SMB) experiences an average of six outages every year, from cyberattacks and power outages to employee errors and system upgrades. According to Gartner, the average cost of downtime for an SMB is as much as $42,000/hour.

2. We’re increasingly reliant on data

We live in the era of big data and business intelligence, and are moving into the era of the Internet of Things. The amount of data we’re storing is doubling every year, and it’s more crucial than ever to the real-time functioning of our business, as big data solutions drive actionable business decisions, often automated, in near real time. That means the cost of being without that data is greater to the business than ever before.

3. The importance of meeting regulatory compliance requirements

In some regulated industries, the need for a business continuity strategy may be driven by regulatory and compliance requirements, particularly in the insurance, financial services and health care industries. Positioning a business continuity plan in the context of regulatory compliance can be a powerful motivator for business executives who understand the importance of regulatory compliance to the health of the business and to consumer confidence.

4. Involve ALL of the business

It’s not an information technology continuity plan – it’s a business continuity plan. It needs to go beyond keeping the lights on to understand how the business really works, and building a plan that brings alignment to security, IT and corporate policies and strategies. The plan should be developed hand-in-hand with members of each business unit to fully understand how they work and what they need from a business continuity perspective.

Getting started with business continuity: finding your data crown jewels

Lynn Greiner Lynn Greiner Published: 07/20/2015

One of the problems with today’s data volumes is, well, today’s data volumes. There’s so much of it, it’s virtually impossible to sort out what’s mission-critical, what’s important, and what’s merely there.


The Enterprise Connectivity Series
Future-proofing your business

Without those distinctions, there are two choices: back up everything, if you can, and hope it can be recovered in a reasonable amount of time if that becomes necessary, or take a stab in the dark and pray that if something goes wrong you can retrieve what you need to keep the business afloat.

Neither is a particularly viable option in case of disaster.

So, how can you make sure that you’re adequately protecting your company’s crown jewels? The first step is figuring out what they are. And that requires data classification.

Data classification is part of the information lifecycle management process. Once implemented, it allows you to determine what data needs protecting, how strong those protections need to be, and how important the data is to your business. For example, intellectual property that differentiates your offerings from everyone else’s is a major competitive advantage. If it’s lost, the business could go under. A memo about vacation days, on the other hand, is something that could be painted on the building’s wall without significant impact, and would not be missed if it were lost in a disk crash.

Common data classification schemes consist of three categories. The first, variously known as Confidential or Restricted, contains data the loss of which would be catastrophic to the organization. Think of items such as personally identifiable information, credit card numbers, medical data, authentication such as user names and passwords or encryption keys, or the aforementioned intellectual property.

The second category (sensitive, internal use only, or private) is of medium sensitivity. It consists of things like most email, and most business data (unless it is confidential).

The third category is Public – anything like a press release, marketing material, and so forth, that has been deliberately released to the public.

There are more complex schemes as well, such as that detailed in FIPS PUB 199, from the U.S. National Institute of Standards and Technology, that interweave data confidentiality, integrity, and availability, if a simple classification isn’t enough.

The next thing you need to know is who owns the data, who else has access to it, and what each user can do with it.

Manual data classification would be time consuming to the point of impossibility for many organizations, so there have been tools developed that handle the bulk of the task. For example, Microsoft has released the Microsoft Data Classification Toolkit, a free download that is designed to help enable an organization to identify, classify, and protect data on its file servers, and HP offers the Atalla Information Protection and Control Suite, which provides automatic classification at the point of creation, once the product has been set up, and ensures that the classification remains attached to the data as it moves.

Data classification can be a lot of work, and is time-consuming to the point where internal staff often just can’t cope. But it’s too important to neglect, so if it’s beyond internal personnel’s capabilities, it’s well worth engaging experts such as the Rogers business continuity specialists to make sure the right data is protected in the right way to keep you in business should disaster strike.

Simulate the worst to be ready for the worst

Robert Dutt Robert Dutt Published: 07/24/2015

You’ve crafted a fine business continuity plan for your business, one that ensures that should your organization’s feet be held to the fire, they won’t burn. You’ve followed all the best practices to a T, and thought of all contingencies. Or have you?


The Enterprise Connectivity Series
Future-proofing your business

Unless you’ve tested your ability to respond to a natural disaster or other business-threatening occurrence, do you really know how your plan, your process, and your people will respond when put to the real-life test?

Fortunately, there are a number of ways to test how your business will fare should the unthinkable become the current situation.

Starting small

The baby steps in business continuity planning are “tabletop exercises” so called because they generally consist of imagining a specific scenario with a handful of people – either members of one specific team responsible for the response to that scenario, or with representatives of a few teams within the organization that will be called upon for a broader scenario. Think of it as a roleplaying game without the 40-sided dice, where the future of your company is on the line.

It’s 4:00 A.M. and the fire alarm has been triggered at a key facility. And…… go!

Members of the test talk through the scenario and measure how the plan fared.

This is a great way to get started on making sure your business is ready to respond should things go wrong, and yet make sure everyone still gets to hit the local watering hole for a celebratory beverage when the day is done.

Getting more complex

Okay, so things worked out well in that scenario. But not every situation is going to be quite so compartmentalized and manageable. For your next test, it’s time to scale things up a little bit. This time, there are more people involved – perhaps people in different locations, differently affected by the hypothetically-unfolding situations. If role-playing in the previous situation was like a game of Dungeons and Dragons, now it’s time to make it a little bit more like paintball. The “bullets” may not be real, but there are certainly more concrete consequences for getting hit now. It’s time to start making the scenario – and everything around it – more realistic and as immersive as possible. And throw a few curveballs at the team along the way, because in a real disaster, odds are good that whatever hits the fan will not be evenly distributed.

The tendency is to test your plan with the best people around. Instead, pull in some less-than-ideal staff members and see how your plan holds up, because there’s a decent chance that under some disaster scenarios, you’ll find yourself trying to talk Joe from Marketing who lives five minutes from the office through some reasonably technical tasks to get the company back up and running, simply because Joe was on the scene half an hour before anyone else could be.

And of course, who’s to say that some key third-party services will be accessible if things go bad? Sure, maybe you’re not testing the resiliency of your phone system with this particular test. But maybe the phone system also just happens to be out as part of your makeshift disaster. Include it just to test the team’s ability to problem solve – a quality that will surely be tested should just about any disaster scenario play out for real.

The types of scenarios may play out over a few hours, but may also extend over several days to assess the plan’s (and your team’s) ability to hold up over those pesky disasters that just don’t want to go away. It all depends on how ready you want your organization to be, and conversely, how unpopular you, as the person planning the testing scenario, want to be.

Going deep

That second level of testing will do a pretty good job of simulating a disaster and testing the ability of your business to survive. But sometimes, you really have to know how things are going to go when things go bad.

This is likely a big test, involving a lot of moving parts from various departments. Have pre-determined start and stop times, and within that time frame, include a duration and path of events that is unknown to most participants. And if in the previous tests you threw curveballs, now it’s time to start throwing some knuckleballs. Maybe a spitball or two, too. Because business-threatening disasters are notorious for not playing fair.

It’s time to turn the realism up to 11. If you were talking through the pre-flight safety video before, now it’s time to start putting on air masks and sending people down the slide. Maybe this time, a location is actually evacuated. Maybe you spin up your disaster recovery site for real.

No pain, no gain

Putting your team in some of these situations is tough, but then are so are major business-threatening disasters. And by testing some of the scenarios that may truly test both your plan and your people, you’ll be putting your organization in the best-possible situation to survive the kind of event that can be the literal make-or-break point for any business.

Thank you for reading

This content comes from our Enterprise Connectivity Series we cover topics such as IoT, Cloud, Business Continuity and more. Learn More

Table of Contents