• To Check or Not to Check Your Exceptions

    There has recently been some debates on /r/java at Reddit [1] [2] about the current state of exceptions in Java. Some see them as essential, others think they just get in their way, while yet others think they are valuable only in the right contexts. If you have experience with more than one or two programming languages, you might now that exception handling is a notoriously hard problem. Whatever solution is selected by the language designers typically leaves at least some people disappointed. Why is that? Why is it so hard to get right? Why not take this opportunity to try to figure out why! While we are at it, we might even try to figure out a better kind of exception handling for Java!.

    Why Exceptions?

    The point of having an exception handling system to be able to separate the intended flow of execution from the one dealing with exceptional scenarios. It sounds great in theory, and often is in practice. Either you can send the message or it fails. Two scenarios, two flows, one golden path and one for recovering. You don’t have to check the return value after every method invocation, and you can just delegate the handling of an exception to the caller of your method, if that would be the most appropriate.

    An exception is a form of communication—just like a regular return value—and it is up to the exception handling system to help the caller and the callee to work together. The outcome must be correct and performant enough. There are some caveats to this communication, however.

    The Caveats of Exception Handling

    What Is a Failure Is up to Interpretation

    Let’s consider two scenarios where a file is read from disk. In the first scenario, the file must be present and contain certain configuration parameters, or the computer program cannot render any value at all. In the second scenario, the file is read and reacted to if it’s present, but the application can continue as normal without it.

    String readSomeFile() {
        try {
            return Files.readString(Path.of("/some/path"));
        } catch (NoSuchFileException e) {
            // Is this a failure?
        }
    }

    Now, is the inability to read a missing file from disk a failure? In the first scenario, it clearly is. The file must be present or the application is useless. But in the second scenario, it is not. The application sees the file being present and being missing as two completely acceptable situations.

    This highlights one of the big problems of designing a good exception handling system: what is a failure is up to interpretation. This becomes particularly relevant when there is a cost to throwing exceptions, such as the cost of a stack trace, or when the caller must add ceremony to handle the exception, as when having to catch or redeclare checked exceptions.

    Only Unexpected Failures Warrant Investigation

    Broadly speaking, a method can produce two kinds of results:

    1. those we expect to happen, and
    2. those we don’t.
    void handleEvent(String name) {
        switch (name) {
            case "a" -> handleEventA();
            case "b" -> handleEventB();
            case "c" -> handleEventC();
            default ->
                // This should never happen, but we know it might!
                throw new UnknownEventName(name);
        }
    }

    If we know that files are likely to be missing or to contain invalid data, then we typically add all the code necessary to handle these scenarios. But when there are things that can go wrong we don’t anticipate, the best we can to is to collect all the data we can about the situation and make sure it reaches us at some point. Only unexpected failures warrant investigation, and investigation requires data. An exception handling system can help us collect this—via stack traces or otherwise. Ideally, we can reconstruct the entire chain of application states that lead to the unexpected outcome.

    Not All Theoretically Possible Failures Apply in Practice

    Not every possible outcome of an application routine is equally likely. We might, for example, be completely aware of the fact that our application can run out of memory and still not want to write any code for handling the situation. Perhaps we know that the hardware will power down and reboot in this case, which we see as an acceptable outcome.

    You, as a developer, will often know things about your system that the designers of the programming language you are using do not and can not know.

    void aimAndLaunchSpaceship() {
      // We don't care if it crashes.
      performComplexCalculationConsumingEnourmousAmountsOfMemory();
      launchSpaceship();
    }

    The Future of Exceptions in Java

    Now that we have been primed on some of the intricacies of exceptions, let’s consider how we could improve them in Java!

    Globally Configurable Checking

    The discussion about Java exceptions on Reddit is, at least in part, about who gets to decide what exceptions must be explicitly handled. If an exception is checked, then every caller must either catch it or redeclare it, both of which is a form of ceremony. If you make an exception unchecked, it is up to the caller to decide how much ceremony to introduce.

    In this case, the tension comes from the fact that what is a failure is up to interpretation. For an exception to be justifiably checked, it must be interpreted as a failure in just about every possible scenario. There are few, if any, exceptions that fit this bill. I agree with Stuart Marks [1] in the sense that if there are any such exceptions, they should indeed be checked.

    A way to solve this tension may be to allow the programmer to globally declare what exceptions must be checked. This could also be a way to preserve backwards-compatibility, as not making a decision would simply preserve the legacy behavior. It also means that different Java subcommunities can work out their own preferences on default behavior.

    Listing All Possible Exceptions

    This is not really a compiler feature, but one for a language server. I have many times wished for a list of all exceptions that could possibly be thrown by a method. You might think that this is something that is solved for checked exceptions, but you are allowed to specify a superclass of the thrown exceptions in the throws clause, hiding the exceptions actually being thrown.

    In my mind, the ideal place for this list would be in the documentation that appears when I hover a method in my IDE. I would also be happy to see this information in generated Javadocs.

    Catch-Site Capture Declarations

    By default, a created exception lazily captures its call stack and includes it as a stack trace. This is a great feature, but stack traces often contain too much and too little information at the same time. Frameworks often bloat the call stacks with their own code, and key frames are often missing when call chains are asynchronous or rely heavily on function composition.

    A possible solution could be the following:

    String method() {
        try {
            return Files.readString(path);
        } catch (NoSuchFileException e without stacktrace) {
            // `e` has no stack trace. Not even a lazily intitialized one.
        } catch (IOException e with stacktrace, variables) {
            // `e` has strack trace and local variable captures with each frame.
        }
    }

    The syntax would likely need some polishing, but the idea is simple: Allow for the consumer of the exception to declare what information to collect when the exception is created. The point is to support investigation into unexpected failures, not to help with exception recovery. The latter would still have to be supported via custom exception classes with relevant property members.

    Pattern Matching On Exceptions

    This is an idea that has been floating around for a while, but I want to repeat it because I think its such a great idea! The point is to allow exceptions to be caught in switch statements, as in:

    void method(Path path) {
        switch (Files.readString(path)) {
            case NoSuchFileException _ -> {}, // Handle error.
            case IOException _ -> {}, // Handle error.
            case String file -> {}, // Handle success.
        }
    }

    The beauty of it is that it seems highly familiar to anyone who already underastands pattern matching in Java, while removing the cermony of the try/catch blocks.

    Conclusion

    Java is moving ahead, and I’m hopeful that also exception handling is going to get more accurate, performant and convenient in the years ahead. What do you think of these ideas? Do you have any of your own?

  • The Vibe Coding Apocalypse

    Vibe coding, or letting an LLM write code for you, is here to stay. It should be clear to anyone who has ever tried it or looked at a demo of it that it will have big consequences for the software development industry as a whole. But while some believers have been proclaiming the doom of all software engineering jobs, reality is not as apocalyptic as it may seem. Let’s have a look at the present, potential and road to engineer replacement.

    The Present is Both Ups and Downs

    While the marketing materials and demos of the great AI companies, such as OpenAI, Anthropic, Facebook or Google, often are very impressive, the reality of their AIs become quite different when you try them yourself. LLMs tend to be very wordy, and they “hallucinate”, which is just a polite way of saying that they lie to you without knowing it. I’ve personally been recommended to pursue certifications that didn’t exist, clicked on source links pointing to non-existent websites, and been told plenty of lies about what certain specifications say about various topics. That being said, LLMs are often excellent at answering complex questions accurately enough, which many times makes them a big improvement over traditional search engines.

    To me, coding with AI has been a quite tumultuous experience. Sometimes the generated code magically works right of the bat, which feels great! But other times, I’m not even sure that the LLM understood what I wanted it to do—despite my best efforts. Also, even when the code works, there is pretty much always plenty of verbosity and subtle bugs. At this point, I don’t trust LLMs to generate more than example code, which is a quite handy way of exploring ideas, libraries and APIs. I’m not alone in my mixed feelings about AI-assisted coding. In fact, a a recent study by METR [1] found that, on average, developers using coding LLMs self-report as being 24% more productive, while being observed to be 19% less productive. Yes, you read that right!

    The Potential is What Scares Us

    While the present of vibe coding is both impressive and disappointing, we do get terrified by its potential. It is not hard to imagine product owners or architects simply writing requirements specifications to an LLM, which would then implement whatever application it describes. Put bluntly, the coding assistants of today could become the de-facto software engineers of tomorrow. Such a transition would completely reshape the entire software engineering industry, and would likely have massive ramifications on the workforce. Millions of people could loose their jobs, and the ones who remain would have to adapt to a new reality of how their work is carried out.

    The prospect of becoming obsolete is chilling, to say the least.

    But How Far Away Is It?

    Can LLMs be improved to the point where they become just as general in their intelligence as a human software engineer—but be thousands of times faster? I believe the safe answer to this question is no. The reason is simple. There is not yet a specific enough definition of what general intelligence is for us to make a machine mimic it. Rather, all types of computing machines created up to this point try to replicate some aspects of intelligence. We know intelligent beings can remember things, make calculations, draw pictures, communicate, and so on. But we have no complete list of all things an intelligent thing can do and how it does them. We know that LLMs, to some extent, can replicate our capacities to use language, associate concepts, remember context and follow ideas in order. All of us who have used LLMs for a while know that there is something uncanny about them, however. They can talk, but they are not persons in the way we.

    But do we need general intelligence to replace software engineers? We don’t. And this is where things get interesting! We know that plenty of money is spent making LLMs more accurate, faster, less noisy, and so on. At least a portion of these attempt are likely to be fruitful. There are probably also going to be some innovations along the way, making LLMs, or their successors, not have some of their limitations. For example, an LLMs is today incapable of knowing if what it says to you is true or not. This might be a solvable problem, or it might not be.

    Alongside this wave of incremental improvements to the LLMs themselves, there are going to be new and improved tools that rely on them. A potential future tool could take a formal architectural description and turn it into a working application, creating the dreaded software engineer replacement. But developing such a tool with enough accuracy may not be feasible until we have programming languages, libraries, APIs, and other things we don’t know of today, optimized for being used by LLMs.

    I think it is safe to say that LLMs will lead to a revolution of the software engineering industry, but I belive it’s going to take some time for this revolution to complete. We might be talking about a few years, or decades, or longer. A slow revolution is not as disruptive as a fast one, and is more likely going to leave plenty of opportunities for people to transition into new roles, tools or expertises.

    So, don’t waste your worries on the vibe coding apocalypse. Rather, be curious about AI improvements, tooling and the evolving ecosystem of software engineering!

  • Java is Dead! Long Live Java!

    April 23rd 1995, James Gosling and Sun Microsystems released the first public version of Java. Today, some 30 years later, version 25 of the language has just been released. It has gone from being interpreted to JIT-compiled to AOT-compiled via GraalVM, from closed source to open source. It has gained generics, lambdas, streams, records, pattern matching and virtual threads. It has survived C#, Scala, Go, Rust, Kotlin, and Python, and keeps staying highly popular. What is the secret to its longevity, and will it avoid becoming another Cobol? And what about vibe coding?

    The main selling point of Java in 1995 was its portability. You would write your software once for the Java Virtual Machine, and it would run everywhere! However, this selling point gradually became less relevant as JavaScript and the Web overtook all competition and became the universal graphical interface platform. Today, Java has found its biggest niche in the enterprise server software domain, making up a large portion of the backend of the Web.

    Between its launch and now, the list of failed Java initiatives includes Java Applets, AWT, Swing, RMI, COBRA, JAX-RPC, EJBs and JavaFX. All of these technologies where attempts to solidify Java’s relevance, either in a particular domain, such as user interfaces, or by tackling a problem that was considered important to the industry at the time, such as distributed computing. But if there have been so many failures, why is Java still around? In fact, it isn’t just around, it’s likely the most popular enterprise programming language in the world [1]. It’s only real challengers are Python, JavaScript, C/C++ and C#, out of which only C# has a similar niche.

    Programming Languages Are Adopted by Communities

    While it is easy to point at historical events to explain Java’s current popularity, such as hype it received at its first launch, or Sun Microsystems’s $500 million Java brand push in the early 2000s, none of these are satisfying explanation by themselves. To get a more solid explanation, we must look at the history of the communities that use Java.

    A community of this kind is a collective of professionals working within the same problem domain, such as automotive, enterprise software or telecom, all adopting similar tools and practices. These communities are born, grow, diminish and die as their problem domains evolve. For example, when the mainframe market emerged in the 1960s, a new community formed around it that soon picked COBOL as its preferred programming language. While the mainframe market was never able to grow to the sizes of the later PC, server or mobile markets, it still exists today. The community around that problem domain also lives, with its own culture and practices. As long as mainframes are around, its community is unlikely to disappear or change programming language.

    Sun Microsystems, and then Oracle after 2010, have worked hard and long to make as many communities as possible adopt its language. In some communities Java never became a thing at all, as in high-performance computing. In others, the language had some initial success, but soon fell out of favor, such as the Java Applet targeting frontend Web applications. In others, it become the dominant language, such as in the enterprise server domain.

    So, why is Java still around? Because it has successfully established itself and continues to be as a highly relevant tool for the jobs it is used for. But will it be around for another 30 years?

    How Oracle is Appealing to the Java Communities

    The computing landscape is shifting, sometimes gradually, other times quite revolutionary. If Java is to continue to be appealing to the communities that today prefer it, it must adapt to their shifting needs. While there is a list of spectacular adaptation failures, which we referred to earlier, there is also a growing list of successful attempts. Java today has generics, a world-class Just-In-Time compiler, interfaces, lambdas, streams, records, pattern matching, virtual threads, and so on. All of these features have been well-received and add to the language’s relevance.

    Why these features in particular? Because the most dominant force of change in the computing industry during the last three decades has been the trend towards increased degrees of parallelism. More cores, more data, more nodes. Parallel processes are notoriously hard to program correctly, and functional programming became perceived as the solution to making it manageable. Java, just as PHP, Python, Ruby, C#, C++ and many more languages, are now more similar to Haskell or OCaml than they have ever been before. As a consequence, they are also more similar to each other.

    From Java 22 to 25, the most notable improvements are targeted towards making Java start faster when used in a cloud function. Upcoming features are intended to make it easier and less error-prone to serialize and deserialize objects, reduce startup times, make it more parallelizable, for example. In short, the trend towards Java becoming more functional and performant is continuing into the foreseeable future.

    Will the Java Communities Ever Die?

    For Java to abandoned, either its communities must disappear or they must all pick other languages. The only threat to the former is some new technological advancement that would make the enterprise server irrelevant, or a new language that would make it comparatively trivial to implement them, while the only threat to the latter is mismanagement of the Java language and branding. Let’s consider these in turn.

    What would a technology need to look like that would either make Java’s communities obsolete, or Java as a language obsolete? It would need to provide a way to replace either with few or no technical downsides. There have been many candidates for this in the past, and new ones keep popping up. Domain-Specific Languages (DSLs), no-code, low-code and now vibe coding are all prime examples.

    So far, none of these have been more than mildly successful. DSLs and no/low-code tools are typically too limited to be widely employed, even though they can speed up activities within their domains. Vibe coding, which is having LLMs write code for you, is currently suffering from too many quality issues [2] to be seen as a viable replacement of the Java communities themselves. The current state of affairs is rather that the Java communities are learning how to use LLMs as tools to boost productivity.

    What would happen if Oracle started to mismanage Java? The likely outcome would be that a community member would fork the OpenJDK, which is open sourced, and then take it in a direction more aligned with its communities. There is too much invested in the Java ecosystem for all of its benefactors to collectively and simultaneously decide that keeping the language afloat is not worth the trouble.

    The Next 30 Years of Java

    Java will be around as long as there are no seismic shifts in the industry. I cannot promise that this will not happen, of course. But to me, none of the shifts on the horizon, such as quantum computing or the rise of AI, appear to be massive enough to move Java out of its place. Java will continue to become what we need it to be in the years ahead.

    Java 25 is here! Let us hope that we will all be around for the release of Java 50. What will be the latest new language feature by then? Probabilistic data structures? Logical programming constructs? Special annotations for LLMs? Your guess is as good as mine!

  • The Hidden Fees of Software Dependencies—and Smart Ways to Pay Them

    For decades, code reuse was heralded as a holy grail of software engineering. It was seen as a key productivity enabler, making developers able to focus their time on novel problems rather than reinventing wheels. But it comes at a cost, and especially so when relying on the code of others. You will have to give up simplicity, quality and control—to varying extents. Depending on your goals and circumstances, this price will not always be worth paying.

    This post is the fourth and last in a series on dependency hell. To learn more about handling existing dependencies that cease to be useful—please have a look at the third post.

    The Cost of a Dependency

    There is no way of getting something for nothing. Adding code produced by other people to your code has plenty of implications on the shape and capabilities of the system you are developing. These implication can be summarized as giving up:

    1. Simplicity
      Most dependencies are created with a larger group of uses cases in mind than just yours. This means that there are features you don’t need, failure routines your code can never trigger, configuration options with no relevance to you, and so on. Bringing in such a dependency makes your application _more complicated_ than it would have been if you would have implemented the functionality yourself. This complexity will spill over to your system in many ways, such as by making it more complicated to develop, review, configure or use. While there often are ways of reducing the impact of this increase in complexity, such as by creating abstractions, having to manage complexity is also a kind of cost.
    2. Quality
      The additional features, options and routines of a dependency typically affects the quality of your system in multiple ways. It may take longer to build, respond slower when used, or take up more disk space. Dependencies can also negatively impact the security and robustness of your system. The more code there is, the more room there is for security holes and bugs to hide, which may lead to your users being hacked, having to deal with untimely crashes, or other system issues.
    3. Control
      As I have carefully pointed out in the previous posts of this series on dependency hell, software dependencies are often a reliance on _other people_ more than it is on other code. By taking on a dependency, you start to depend on these people making choices that keep it useful to you. You get the functionality you need, but you sacrifice direct influence over how it is realized. An important and common consequence of this is that you will not have in-depth knowledge about how the dependency is implemented, or how it affects the rest of your system. This lack of knowledge can make it harder to implement your system correctly, making bugs harder to anticipate and address.

    Ways of Paying the Devil His Due

    While the obvious way of avoiding a dependency is to implement it yourself, this is not always feasible. You might not have the competence, time, or other resources required. But, there is a middleground between rolling your own and relying on others:

    1. Prune an existing solution
      You take an existing solution, most likely open source, and then remove all parts of it not immediately useful to you. If the solution is not open source, or is written in a language you cannot use, you identify what parts of it matter to you and reimplement these. This saves you time because it saves you from technical decision-making, which often takes as much or more time than it does to write the code.
    2. Fork an existing solution
      If the issue isn’t necessarily about the current features of the dependency, but it being unmaintained or there being concerns about its future direction, you can take its source code and start maintaining it yourself. This assumes that the dependency source code is available to you, of course.

    Both of these approaches leaves you with having to maintain the code, but gives you the freedom to make changes to it as you see fit. They trade a slower speed of development for a more simplicity, quality and control.

    Also, be wary of the fine print of the licenses in any open source code you prune or fork. Nowadays, most open source projects allow you to make your prune or fork without open sourcing it, but there might be other restrictions. You may have to acknowledge the origins of your software in a way that your users can see, for example.

    Making The Cost/Benefit Trade-Off

    In the end, what strategy to use for taking on dependencies should be a matter of what you believe will strike the best balance between costs and benefits. Do you need to be done with a proof-of-concept as soon as physically possible? Take on dependencies like there is no tomorrow! Are you making a surgical robot? Perhaps it is best to implement everything from scratch. But your system is propably neither a throw-away prototype or a medical device.

    While this should sound kind of obvious to you, allowing there to be a cost/benefit trade-off is far from the norm. Many companies have cultures thay weigh heavily towards rolling your own or depending on others, expressed via mantras such as “not invented here” or “move fast and break things”. I believe that we, as designers, architects and engineers, have a duty to guide our peers towards what will produce the best outcomes, which may mean that we have to challenge these kinds of norms.

    Don’t let your dependencies hold you back. Stay out of dependency hell!

  • Your Software Dependencies Will Revolt—Design and Plan for It 

    Whatever system you are building, it is built on a mountain of efforts of other people. Each of these had an interest that aligned with yours when you incorporated their work. But when the tide turns, software libraries will be overtaken by hackers, standards go out of popularity, databases discontinued, programming languages be abandoned, and the rest of catastrophe and his niece will come. While the damage cannot be avoided completely, you can make a big difference by being prepared. 

    To prepare for the worst, you must 

    1. design your system as if some of your dependencies are potential hazards, 
    2. make doomsday plan, and 
    3. watch for signs of dependency decay. 

    Let’s get into it! 

    This post is the third in a series. To learn more about keeping dependencies aligned—rather than waiting around for the revolution—please have a look at the second post. 

    Designing for Dependency Hazard Resistance

    Making it easy to swap out bad software dependencies of a system typically boils down to putting them behind APIs that you design, or wrapper APIs. To replace your dependencies, you implement these wrappers again with alternative dependencies. This is, however, often a lot easier said than done. While some dependencies can be quite straightforward to wrap, like simple storage solutions, message queues or payment providers, others can require more effort than building the rest of your system. Database query languages, user interface libraries and testing frameworks are some potential offenders. 

    But what determines this wrapping cost? 

    1. The complexity of the dependency, and  
    2. how much of its complexity you actually need

    A complicated dependency you use to satisfy a simple need can be put behind a simple API. But if you need a lot of its complexity, your wrapper must also be complicated. Knowing what you need is, however, perhaps the hardest problem of system design. As these needs, or requirements, tend to be quite fuzzy, we usually hedge our bets by assuming that we will need too much complexity. 

    So, there is a cost to wrapping dependencies, and it isn’t the same for every dependency. But it is also the case that not every dependency is equally risky. Some genuinely complicated systems have proven to be stable for decades or longer, such as programming languages, operating systems, and database solutions. To strike the best possible balance between cost effectiveness and hazard resistance, you must carefully choose 

    1. what dependencies best satisfy your needs, and  
    2. which of them not to put behind wrapper APIs. 

    Making a Dependency Doomsday Plan 

    With your system now carefully designed to make it cheap to eject dependencies as they go bad, it is now time to decide exactly how they are to be replaced or otherwise dealt with as they do go bad. These decisions go into your dependency doomsday plan

    The cost of every dependency incident is not the same. Sometimes it is cheaper to deal with the revolution when it happens than it is to prevent it from happening. The difference in cost can, in some cases, be enormous. But to take these risks and costs into account when designing our systems, we must first know 

    1. what incidents can occur,  
    2. how likely they are to occur, and 
    3. how you would be impacted by them. 

    Your dependency doomsday plan answers the above three questions for each of your dependencies, as well as a fourth question: what do we do when it happens? The answer to the last question should give an indication of how costly the mitigation is, and if escalation to higher levels of management is required to secure any needed resources. 

    Taken together, adhering to these ideas means that it could be a great design decision to not wrap a key dependency, but instead be ready to buy the rights to it, fork it or reverse engineer it if a serious enough situation would arise. In other words, your system design should take its larger ecosystem of companies and institutions into account. It should not end at whatever boundaries are set by your own organization. 

    Watching for the Signs of Dependency Decay 

    A doomsday plan won’t do you any good, however, if you don’t have a process in place that can detect when your dependencies go bad. And by the way, how do you know when a dependency has gone bad?  Is it when the owning company is acquired by a competitor? When somebody has reported a critical security vulnerability? How long can you tolerate that a software library remains unpatched before you consider it abandoned? Is it acceptable for you to rely on gut feelings and occasional check-ins, or do you need a more rigid process? You decide. 

    I’ll be back again in a few weeks with the conclusion to this series on software dependencies in system design. The last episode will be on knowing what dependencies not to take on. Until then, prepare for the dependency uprising. It will come. 

  • Delay the Revolution—Support Your Software Dependencies

    Every one of your dependencies is made and maintained by people. If their ambitions stop aligning with yours, the dependency will stop being useful to you—sooner or later. Stop or delay this revolution against your interest by actively supporting these people with your money, time and goodwill.

    Every component your system is made from, from the nuts and bolts in the computers running it to the programming language and software libraries it is made from, is made by some constellation of people with some end in mind. If and when these ends shift, your ambitions and theirs will no longer align. You might recognize this as an open-source library no longer being maintained, a breaking API change, or a discontinued product. What can you do to prevent—or at least delay—this revolt against your interests?

    This post is the second in a series. To learn more about how dependencies are more about people than technology, please have a look at the first post.

    A Little Support Goes a Long Way

    Let’s contemplate the fact that most of us need money to make ends meet. A lot of priority shifts happen because not enough money is flowing in the desired direction. While this is obviously the reason why many a late-night hacker must abandon his or her open-source pet project, or why polished products from large companies are discontinued, it can also be the triggering factor behind subtler changes of events, like individual features in cloud services being pulled back, or improvements not being made that would greatly benefit you.

    In a well-functioning market economy, the flow of money tends to communicate what is valued. If your system relies on components you are not paying for, you are communicating that you put little to no value on these components. If others are paying for a component but not you, their needs will likely be prioritized over yours. If no one pays for it, it will most certainly be abandoned at some point. You should be very interested in paying for the components you depend on relative to the value you extract from them, or to the damage you would suffer if they were abandoned.

    Does your system depend on a free tier of a cloud service, or on open-source components acquired via a package manager? If your system is more than a hobby project or a proof-of-concept, you should do what you can to pay for these things. Nowadays, a lot of open-source projects advertise that they accept donations via the package managers that distribute them. Identify your most important dependencies, and then do what you can to pay for them. You likely depend on programming languages, hardware architectures, standards and a lot of other things. How critical are these to you? Are there risks of them becoming misaligned with your interests? What can you do about it? These questions are important for the long-term survival of your system.

    Apart from giving money, there are other ways of showing your goodwill. You can donate time to support open-source components, express your appreciation, take part in events or discussions, report bugs, or simply point out improvements you would value.

    A Lot of Support Goes Even Longer

    Let’s assume that you have a lot of resources at hand. Enough to support multiple people—or even buy a whole company. Rather than only being able to influence with your resources, you then become able to take over your dependencies. Is a critical component provider about to go bankrupt or discontinue a key product? You just might be able to buy the rights for the product, or even the company itself, at a bargain.

    If the provider is a charity or some kind of foundation, you may have to get a bit creative. You might buy an overly expensive support agreement, aimed at making the project sufficiently financially dependent on you to prioritize your needs. Or you could hire a bunch of experts and donate their time to the project. Since they are on your payroll, they will feel obliged to prioritize your needs. Open-source projects can be forked, and other software can even be reverse engineered, should you become desperate enough.

    If you’ve been reading industry news for long enough, I believe you can think of real-world examples of all the above takeover tactics. Do they sound shady to you? They sure can be. If, however, you do these things and are transparent with your intentions, you will be a benefactor or partner rather than a sly subverter. Most people need money to make a living, and if you can help satisfy the needs of your clients, your dependencies and society at large, everyone is better off.

    In summary, use the resources you have, however little or much, to communicate where your interests lie. I’ll be back in the coming weeks with another post on the topic. Then on preparing for and quelling dependency revolts. Until then—support your dependencies!

  • Dependency Hell Is a Real Place

    Pandemonium has erupted. The phone buzzes every few seconds with another alert about a system not responding. Production is down. More and more people are notified as the minutes are racing by. The phone is ringing. The management is furious. We are bleeding money. The reason? That system keeps failing with every restart. Somehow every other critical system depends on it. The company who made it hasn’t been around for years. There is no support agreement. There is no hope. All is lost.

    Without knowing anything more about this little story, what would you guess contributed the most to the turmoil it depicts? Was it the poor quality of the failing system? The brittleness of the overarching design? Did management fail to give the developers the proper mandate to build something that would actually work? These and other things could indeed contribute to systems failing, sometimes gradually and other times suddenly. But I argue that the most important cause was that the failing system was unmaintained.

    Misaligned Interests Is the Root of All Dependency Evil

    Apart from the physics and raw materials from which a computer system is made, everything else about it is man-made. Just about every one of its components, from its screws to its software libraries, is made by a different constellation of people. Every one of these groups has their own reasons for making and maintaining its component. And it is here we find the root of just about every risk in software engineering: in misaligned interests.

    The failing system above could have been of excellent quality, perhaps never having caused a single problem up until that point. This would have motivated a high degree of confidence in it when fitted into the larger context of the company. But the circumstances in which the system was running changed over time, while it remained the same. It was not being maintained, and not being maintained is a people problem, not a quality problem.

    Perhaps you would argue that bugs, security vulnerabilities, poorly designed interfaces or missing features are the biggest liabilities in a software project? It is true that these things matter. But as long as the interests of those paying for your work align with the utility of your software, the money will keep coming, new experts be hired, and all other conceivable measures be taken until all important quirks are fixed and features are in place. And who is really paying you? Is it your employer? Sure. But this employer gets its money from somewhere. The provider of the unmaintained system above failed to bring in the money necessary to maintain it, and the company depending on it failed to do something about its reliance on it. In the end, catastrophe was inevitable.

    Interests Must Align or Systems Start Failing

    Every system ever designed was made with an eye towards some desired end. This is the interest that motivates its existence. The desired end can be to deliver value to paying customers, but it can also be to establish one’s competence, to satisfy a curiosity, or a multitude of other things. As long as the desired end is approached and remains relevant, work on the system will continue. This is true for whatever system you may be part of making, and it is also true for every other system it depends on.

    Interests drift over time, they are replaced, and they disappear. When this happens, a system could start doing something you are not interested in, stop being maintained or be taken off the market. It could also be the case that your interests change, and that the system you are depending on no longer satisfy the new interests.

    Keep Watch, Because You Know Not When Misalignment Comes

    If the software you are part of designing must keep satisfying your interests, or the interests of your employer, over time, you need to know how to handle the misalignment between your software and its dependencies. As far as I know, you can go about this by

    1. actively making sure your dependencies remain aligned,
    2. reducing the cost of misalignment, and by
    3. becoming more deliberate about when to take on new dependencies.

    I’ll be writing more about these measures in the weeks to come. Until then, remember that if you fail to keep your dependencies in check, you are going to dependency hell. And let me remind you—it is a real place.