Systems and Society

Whatever system you are building, it is built on a mountain of efforts of other people. Each of these had an interest that aligned with yours when you incorporated their work. But when the tide turns, software libraries will be overtaken by hackers, standards go out of popularity, databases discontinued, programming languages be abandoned, and the rest of catastrophe and his niece will come. While the damage cannot be avoided completely, you can make a big difference by being prepared. 

To prepare for the worst, you must 

1. design your system as if some of your dependencies are potential hazards, 
2. make doomsday plan, and 
3. watch for signs of dependency decay. 

Let’s get into it! 

This post is the third in a series. To learn more about keeping dependencies aligned—rather than waiting around for the revolution—please have a look at the second post. 

Designing for Dependency Hazard Resistance

Making it easy to swap out bad software dependencies of a system typically boils down to putting them behind APIs that you design, or wrapper APIs. To replace your dependencies, you implement these wrappers again with alternative dependencies. This is, however, often a lot easier said than done. While some dependencies can be quite straightforward to wrap, like simple storage solutions, message queues or payment providers, others can require more effort than building the rest of your system. Database query languages, user interface libraries and testing frameworks are some potential offenders. 

But what determines this wrapping cost? 

1. The complexity of the dependency, and  
2. how much of its complexity you actually need. 

A complicated dependency you use to satisfy a simple need can be put behind a simple API. But if you need a lot of its complexity, your wrapper must also be complicated. Knowing what you need is, however, perhaps the hardest problem of system design. As these needs, or requirements, tend to be quite fuzzy, we usually hedge our bets by assuming that we will need too much complexity. 

So, there is a cost to wrapping dependencies, and it isn’t the same for every dependency. But it is also the case that not every dependency is equally risky. Some genuinely complicated systems have proven to be stable for decades or longer, such as programming languages, operating systems, and database solutions. To strike the best possible balance between cost effectiveness and hazard resistance, you must carefully choose 

1. what dependencies best satisfy your needs, and  
2. which of them not to put behind wrapper APIs. 

Making a Dependency Doomsday Plan 

With your system now carefully designed to make it cheap to eject dependencies as they go bad, it is now time to decide exactly how they are to be replaced or otherwise dealt with as they do go bad. These decisions go into your dependency doomsday plan. 

The cost of every dependency incident is not the same. Sometimes it is cheaper to deal with the revolution when it happens than it is to prevent it from happening. The difference in cost can, in some cases, be enormous. But to take these risks and costs into account when designing our systems, we must first know 

1. what incidents can occur,  
2. how likely they are to occur, and 
3. how you would be impacted by them. 

Your dependency doomsday plan answers the above three questions for each of your dependencies, as well as a fourth question: what do we do when it happens? The answer to the last question should give an indication of how costly the mitigation is, and if escalation to higher levels of management is required to secure any needed resources. 

Taken together, adhering to these ideas means that it could be a great design decision to not wrap a key dependency, but instead be ready to buy the rights to it, fork it or reverse engineer it if a serious enough situation would arise. In other words, your system design should take its larger ecosystem of companies and institutions into account. It should not end at whatever boundaries are set by your own organization. 

Watching for the Signs of Dependency Decay 

A doomsday plan won’t do you any good, however, if you don’t have a process in place that can detect when your dependencies go bad. And by the way, how do you know when a dependency has gone bad?  Is it when the owning company is acquired by a competitor? When somebody has reported a critical security vulnerability? How long can you tolerate that a software library remains unpatched before you consider it abandoned? Is it acceptable for you to rely on gut feelings and occasional check-ins, or do you need a more rigid process? You decide. 

I’ll be back again in a few weeks with the conclusion to this series on software dependencies in system design. The last episode will be on knowing what dependencies not to take on. Until then, prepare for the dependency uprising. It will come.