Systems and Society

Pandemonium has erupted. The phone buzzes every few seconds with another alert about a system not responding. Production is down. More and more people are notified as the minutes are racing by. The phone is ringing. The management is furious. We are bleeding money. The reason? That system keeps failing with every restart. Somehow every other critical system depends on it. The company who made it hasn’t been around for years. There is no support agreement. There is no hope. All is lost.

Without knowing anything more about this little story, what would you guess contributed the most to the turmoil it depicts? Was it the poor quality of the failing system? The brittleness of the overarching design? Did management fail to give the developers the proper mandate to build something that would actually work? These and other things could indeed contribute to systems failing, sometimes gradually and other times suddenly. But I argue that the most important cause was that the failing system was unmaintained.

Misaligned Interests Is the Root of All Dependency Evil

Apart from the physics and raw materials from which a computer system is made, everything else about it is man-made. Just about every one of its components, from its screws to its software libraries, is made by a different constellation of people. Every one of these groups has their own reasons for making and maintaining its component. And it is here we find the root of just about every risk in software engineering: in misaligned interests.

The failing system above could have been of excellent quality, perhaps never having caused a single problem up until that point. This would have motivated a high degree of confidence in it when fitted into the larger context of the company. But the circumstances in which the system was running changed over time, while it remained the same. It was not being maintained, and not being maintained is a people problem, not a quality problem.

Perhaps you would argue that bugs, security vulnerabilities, poorly designed interfaces or missing features are the biggest liabilities in a software project? It is true that these things matter. But as long as the interests of those paying for your work align with the utility of your software, the money will keep coming, new experts be hired, and all other conceivable measures be taken until all important quirks are fixed and features are in place. And who is really paying you? Is it your employer? Sure. But this employer gets its money from somewhere. The provider of the unmaintained system above failed to bring in the money necessary to maintain it, and the company depending on it failed to do something about its reliance on it. In the end, catastrophe was inevitable.

Interests Must Align or Systems Start Failing

Every system ever designed was made with an eye towards some desired end. This is the interest that motivates its existence. The desired end can be to deliver value to paying customers, but it can also be to establish one’s competence, to satisfy a curiosity, or a multitude of other things. As long as the desired end is approached and remains relevant, work on the system will continue. This is true for whatever system you may be part of making, and it is also true for every other system it depends on.

Interests drift over time, they are replaced, and they disappear. When this happens, a system could start doing something you are not interested in, stop being maintained or be taken off the market. It could also be the case that your interests change, and that the system you are depending on no longer satisfy the new interests.

Keep Watch, Because You Know Not When Misalignment Comes

If the software you are part of designing must keep satisfying your interests, or the interests of your employer, over time, you need to know how to handle the misalignment between your software and its dependencies. As far as I know, you can go about this by

1. actively making sure your dependencies remain aligned,
2. reducing the cost of misalignment, and by
3. becoming more deliberate about when to take on new dependencies.

I’ll be writing more about these measures in the weeks to come. Until then, remember that if you fail to keep your dependencies in check, you are going to dependency hell. And let me remind you—it is a real place.