Trade-offs In Software Development - Or The Necessity to Embrace Imperfection

There is so much good writing about best practices in software development. There are articles telling you about the virtues of test-driven development, refactoring, different types of documentation, formal verification, Agile, but also technology choices. I wish there was more discussion of the trade-offs these practices make and how the context in which you apply them affects their value.

This is a huge topic and one could rabbit-hole for many pages on a single practice and its trade-offs. Future articles will dive into specific practices and their associated trade-offs. I’ve done a little bit of that over in my discussion of the trade-offs by Pivotal’s approach to management.

Context Matters

No practice or technology works for everyone in every situation. This should be obvious, but somehow in software it seems very common to forget this. You could find enough books alone on how Google does stuff to keep you reading for months. Yet you are unlikely to face the same challenges as Google. Of course this is an extreme example, but how well this stuff sells, is telling. If you found a startup, you have a 80% chance of failure. Yet, I have seen so many discussions where people push for solutions that trade precious time in your limited runway away from potential attempts to find product-market-fit for something that “scales”. I venture to guess that the vast majority of the 80% that fail, do so because they never achieved product-market fit, not because their software stack or practices didn’t scale.

For example, I am a huge believer in Agile, particularly XP. It allows for rapid iteration. However, if your risk is primarily a technological one and you cannot ship frequently due to the nature of your product, some of the positives of Agile/XP compared to other choices might not be as strong.

I spent several years working on a distributed data system that was mostly deployed on-prem. We usually had a lot of feedback on what features and improvements customers want, but they disliked upgrading, so our feedback loop was very long, even after we shipped. At the same time the implementation was usually non-trivial and the cost of taking a wrong, technical approach very high.

To make it more complicated, your choices interact with each other. For example, if your developers all sit in the same office and practice pair-programming with deliberate pair rotation, the benefits of detailed documentation will be a lot less than if your developers work in different time zones and hardly ever see each other.

Let’s attempt a incomplete list of what one might consider when making a process or technology choice:

General

• What’s our switching cost for the different choices?
• How is this trading consistency for other quality metrics?

Business Context

• Is my company’s/product’s risk product-market fit or technological?
• What’s the likelihood of our product sticking around in its planned form for the next year? How about two years?

Team Dynamics

• Where does this fall on “enough guidance for juniors” and “too constraint for seniors” and which matters more to my team and product?
• How many people will/should be working on this?
• How long do I expect the people who are working on this to stay with the team or company?
• How experienced is my team with the different options we are evaluating?
• Who will maintain this?

Communication & Culture

• Is my team currently high-communication or struggles with communication?
• Is everyone in one place or time zone?
• How will the team feel about a given choice?

If some of these questions above don’t make sense, they should become clearer as follow-up articles go over specific examples.

It’s important to note that some of these might be different for different areas of your product and will likely change over time.

Nothing is Perfect

It’s important to understand that every choice we make is just a trade-off. Neither choice will solve all problems forever. We just trade smaller downsides for larger upsides. If we believe our solution is perfect we are lying to ourselves.

Even striving for the perfect solution has its own downside. Not only do we waste time, but we also might optimize too much and achieve worse outcomes. In her book “ I Can’t Stop Thinking About VAR” (econtalk episode), Daisy Christodoulou uses the example of race horse breeding. As race horses get faster, their bones get lighter. At some point the point is reached where the bones are too fragile and break. Now the race horse has zero speed.

Sometimes a good workhorse gets more done than an overbred racehorse, especially as conditions change.