Before You Read This Chapter
At one organization, the online ordering system was positioned as one of the most important services from a business standpoint.
A failure occurred, and recovery work began. Naturally, the people in charge tried to recover this most-critical system first.
However, in practice, the online ordering system could not be recovered on its own, by itself.
This system depended on the following foundational mechanisms:
Network
↓
DNS
↓
Identity (authentication/authorization)
↓
Storage
↓
Database
↓
Application runtime
Only once these dependencies became available did the online ordering system actually become usable.
At that point, the people in charge realized something.
"What we considered most important from a business standpoint" and "what can actually be recovered first from a technical standpoint" are not necessarily the same thing.
This chapter organizes this difference.
6.1 "Importance," "Priority," and "Order" Are Not the Same
In Chapter 4, we saw that the impact of a disruption differs by business function.
From this comes the idea of "how important something is from a business standpoint." But that does not, by itself, mean "the actual order in which it can be recovered."
This chapter separates out and organizes the following three ideas:
- Business Criticality
- Recovery Priority
- Technical Recovery Order
These are related to one another, but they are not the same thing. A business function being "important," wanting to "prioritize" recovering it, and actually "being able to recover it in that order" are each separate matters.
Note that these three terms are a way of naming ideas for the purposes of organizing thinking in this chapter, and are not a classification defined by any particular industry standard or specification.
6.2 Business Criticality — Importance from a Business Standpoint
Business Criticality is the perspective of how important a given business function or system is to the organization.
As we saw in Chapter 4, the impact of a disruption differs by business function. The magnitude and nature of the impact varies: business functions that directly affect external customers, business functions that other business functions depend on as a precondition, business functions related to law or contracts, and so on.
Business Criticality is an assessment of business-standpoint importance that emerges from this kind of analysis of business impact.
What to note here is that Business Criticality does not, itself, represent the technical recovery order. A business function being important and it being possible to technically recover it first are separate matters. We revisit this point in 6.4.
6.3 Recovery Priority — Priority for Recovery
Recovery Priority is the idea of which business functions or systems an organization wants to prioritize recovering.
Recovery Priority is influenced by Business Criticality. It is natural to want to prioritize recovering what is considered important from a business standpoint.
However, Business Criticality does not, on its own, automatically determine Recovery Priority. Depending on the organization's circumstances and constraints, factors other than business importance also come into play in actually deciding "what to prioritize."
And here is the central point of this chapter: even once Recovery Priority is decided, that does not necessarily mean it becomes the Technical Recovery Order as-is.
Applying the same level of protection or recovery requirements to every system can be unrealistic in terms of cost, staffing, and operational load. What matters is not simply deciding not to implement a needed measure because it is "too costly," but explicitly judging, based on business impact and available resources, which risks to reduce and which risks to leave in place. It is not possible to eliminate every risk. Precisely for that reason, accepting a remaining risk after understanding it needs to be treated as separate from leaving it unaddressed without sufficient consideration.
6.4 Technical Recovery Order — The Technical Order of Recovery
Technical Recovery Order is the actual order in which recovery work needs to proceed, viewed from a technical standpoint.
What matters here is dependency.
For a system to work correctly, in many cases the underlying mechanisms it depends on need to be recovered first. As we saw in the opening scenario, if foundational mechanisms such as network, DNS, Identity, storage, and database are not available, the application running on top of them will not actually function, no matter how much recovery work is attempted on it.
For this reason, even a system considered most important from a business standpoint and given the highest Recovery Priority may, technically, need the foundational services that support it to be recovered first.
This does not mean the priority judgment was wrong. Recovery Priority (what you want to prioritize) and Technical Recovery Order (what is technically needed first) simply arise from different perspectives to begin with.
6.5 An Example Where Business Priority and Technical Order Diverge
Let's return to the opening scenario.
The organization gave the online ordering system the highest priority from a Recovery Priority standpoint. However, from a Technical Recovery Order standpoint, the following dependencies existed.
Example dependency-driven recovery sequence
(this is only one example, not a fixed order common to every system)
Network
↓
DNS
↓
Identity
↓
Storage
↓
Database
↓
Application runtime
↓
Online ordering system
In this example, the online ordering system — the service given the highest weight in terms of Recovery Priority — actually ends up last in terms of Technical Recovery Order. Foundational mechanisms that normally go unnoticed, such as network, DNS, and Identity, need to be technically recovered first.
The concrete shape of this dependency differs depending on the system's configuration. Treat this diagram only as an example showing that such dependencies can exist.
What matters is that the most important system from a business standpoint is not necessarily also the one that can technically be recovered first.
Organizing the Three Ideas
Let's organize the three ideas we've covered so far into a simple table.
| Idea | Central question | Main perspective | What influences it | What it does not automatically determine |
|---|---|---|---|---|
| Business Criticality | How important is it from a business standpoint | Business/organizational perspective | The impact of disruption (BIA, Chapter 4) | Does not automatically fix Recovery Priority or the recovery order |
| Recovery Priority | What do we want to prioritize recovering | Organizational preference/judgment | Business Criticality, among other things | Does not necessarily match Technical Recovery Order |
| Technical Recovery Order | What order can recovery actually proceed in | Technical/dependency perspective | Dependencies between systems | Does not necessarily reflect business priority as-is |
This table, too, is only a guide for organizing ideas, not a numerically scored measure or a formula that mechanically determines priority.
6.6 The Meaning of Understanding Dependencies
As we have seen, Technical Recovery Order is constrained by dependencies.
What follows from this is that, when thinking about a recovery plan, it is important to understand in advance which system depends on which mechanism.
Keeping dependency information in a visualized form, such as a diagram or map, is useful when actually considering the recovery order.
That said, this chapter does not cover the detailed methods for organizing dependencies, or a systematic methodology for identifying them.
6.7 Summary
This chapter organized three ideas: Business Criticality, Recovery Priority, and Technical Recovery Order.
The important points are these:
- Business Criticality, Recovery Priority, and Technical Recovery Order are each distinct perspectives.
- Business Criticality influences Recovery Priority, but it does not automatically determine Technical Recovery Order.
- Technical Recovery Order can be constrained by dependencies.
- Even the most important system from a business standpoint may not be recoverable first, because of technical dependencies.
- Understanding dependency information in advance is important when considering the recovery order.
In the next chapter, we look at the idea of 3-2-1 and Failure Domain — how to place and distribute Backups and copies.
Questions to Consider After Reading Chapter 6
Thinking about the systems you are responsible for, can you answer the following questions?
- Are Business Criticality and Recovery Priority the same thing?
- Why can Recovery Priority and Technical Recovery Order fail to match?
- Why might the most business-critical application not be the first one you can recover?
- Can you give, in your own words, an example of a technical dependency constraining recovery order?
- Why do you think this chapter did not cover the detailed methodology of dependency mapping?
You do not need to be able to answer all of them.
Rather, if there is a point where you get stuck, that is exactly the point this chapter is meant to help you check.