The Business Value of Paying Back Tech Debt, Refactoring, and Testing Software

💨 TL;DR

These are essentially non-functional requirements affecting a system’s attributes, such as its reliability, maintainability, and performance, therefore they clearly must have some Business Value somewhere. However, it is blurred at best so don’t expect $$$ figures or formulas here. The purpose of this post is to help the reader appreciate this value from an Engineer’s perspective.

Introduction

I personally believe that the Business Value from each can be derived from Engineers' increased ability to move faster and with empowered confidence. The side effects of these superpowers are fewer bugs reaching Production, a culture of straightforward, robust, maintainable code, and well-documented, enjoyable systems that can routinely be handed over to the next generation of Engineers.

However, I have never worked on a project where there has not been some degree of tussle between getting the balance of these non-functional requirements and feature development priorities correct, simply because the Business Value is most tangible from the Business Features. What seems like obvious “rust” or “decay” on the system to address by the Engineers can appear as poor prioritisation when viewed from the outside. Engineering teams of course have empathy for those relying on us to deliver features quickly, but we are regularly not very good at communicating the relationship of delivery to Tech Debt, Refactoring and Testing.

Let’s start by shining some light on Tech Debt.

Paying Back Tech Debt

What is “Tech Debt”?

Tech Debt is the result of moving fast during a cycle of Engineering and leaving behind logic and decisions that will slow down the Engineers who need to work on that code base, application or project next.

“Paying Back” Tech Debt is the retrospective actions needed to reverse these enforced sub-optimum decisions and Engineering shortcuts.

Pay Back can be done spontaneously during feature development else prioritised and planned. To state the obvious, you have to know what Tech Debt exists in order to plan how to pay it back. Tracking Tech Debt on a greenfield project or new application is easy, assuming you can agree on a strategy (there are many) but when Engineers inherit an existing code base where this has not happened, Tech Debt is “discovered” as they work (it should then be tracked) and tackled spontaneously.

The majority of scenarios Engineers and Product Managers face is the “discovered” Tech Debt. This is important to take on board.

• Discovering Tech Debt during development phases that “should” be repaid before continuing affects estimates, and this often happens. One indicator of escalating Tech Debt is feature estimates becoming wilder and wilder.
• Building features on top of Tech Debt is slower than on clean, well-architected code, and passes on an even bigger debt to the next Engineers in line.
• Serious Production performance issues can often be traced back to a known rushed decision that boils down to Tech Debt that Engineers were unable to pay back.
• Consider that the older the codebase being worked on is, and the more Engineers that have worked on it, the more Tech Debt will be discovered (adjust your project plan accordingly!).

I suggest looking at the Business Value of Paying Back Tech Debt more as Opportunity Costs since the value is hidden. For example, you decide between spending 2 days of project X on Tech Debt or pushing forward regardless, and the latter decision may lead to an additional 2 days for each project that follows. Quantifying these Opportunity Costs is going to be difficult, but feeling them is less so.

Imagine being a Michelin Star cook forced to work in a filthy kitchen. Would you do your best cooking there? Wouldn’t you rather clean the kitchen up first, then start your work? The trouble is there have been dozens of cooks before you and they all had to rush to get the service out, leaving a mess behind. What is the outcome? Slower service, mistakes, and lower overall culinary quality.

Besides the practicality of building around a pre-existing mess, Engineers thrive on enjoying to code, and coding on or around Tech Debt is not enjoyable. So what is the outcome? Slower delivery, bugs, and lower overall code quality.

The worst-case scenario you will see (I have seen this multiple times) is Engineers leaving because they continue to have to work on legacy code ridden with Tech Debt which is no fun at all… now is the Business Value is starting to ring alarm bells?

Tech Debt is a living, breathing part of rapid Product Engineering which we must learn to carefully live with. It can have many side effects on delivery, engagement, and even the politics between teams. Agreeing on a strategy early in development cycles will help each team optimise the Opportunity Costs of Paying Back Tech Debt.

Refactoring Software

What is “Refactoring”?

Refactoring is when Engineers change the internal structure of a codebase without changing the external behaviour to improve system design, performance, readability, or anything that will lead to faster feature development or easier testing.

There is a wide scale of Refactoring. It starts at the very small level where an Engineer may simplify an internal private function as part of a small feature request and spans all the way up to reimplementing entire components or applications as part of planned upgrade work. Refactoring and Tech Debt are not mutually exclusive.

The Business Value of Refactoring Software is at least somewhat quantifiable, as opposed to Tech Debt tasks, which are mostly Opportunity Costs.

Small Refactoring is part of the natural development process and those tasks are not in focus here. Whilst Tech Debt is often discovered during development, Refactoring tasks are (or should be) known upfront, and that is what makes them “somewhat” quantifiable. This knowledge comes either from Engineers with existing knowledge of the codebase or from Spikes, carried out because of known unknowns before projects are kicked off.

I’m not going to share my views on estimations here (because they are controversial) but if you want to, you can estimate the number of days the refactors you have identified as necessary. That will give you a cost of delivery. But what about the Business Value?

Refactoring reduces the complexity of the code. Some of the benefits of this to Engineers are:

• Less complex code leads to fewer bugs getting to Production because the code is easier to read, document, debug, test, code review, and maintain.
• New starters on the team can start adding value far quicker because the barrier to entry is lower.
• Understanding is spread quicker amongst the team rather than one siloed “guro” who is the only one in the company that understands “payments”, for example.
• Engineers who work on the refactored codebase enjoy their tasks more leading to a more sustainable effort.

Some of the benefits to the Business are:

• Each subsequent feature or project that exercises the refactored code moves faster since it is simpler and easy to understand and test.
• Engineers who work on the refactored codebase work more efficiently, so Velocity should start to rise.
• Task/Story estimates will become more accurate and even begin to fall as repeatable patterns become easier and quicker.

Possibly, if refactoring preparation is given the space, Engineering Velocity will start to go up, morale will go up… and engagement and quality will go up too!

The worst-case scenario you will see (I have seen this once and a half now) is that pressure to keep building renders the refactoring tasks so large, that the only option is a complete rebuild from scratch… now is the Business Value is starting to ring alarm bells?

Refactoring code as your application requirements grow is unavoidable when building scaleable applications. You will reach a project one day where some degree of refactoring is needed before you can begin something new, so you can’t dodge it forever. Agreeing to a transparent policy early before development cycles begin so that you can execute a planned rollout of refactoring work will improve the focus and delivery of all future features and projects.

Testing Software

What is “Testing Software”?

Testing Software is an investment in repeatable and automated patterns that exercise software such that human beings need not be the fallible gate holders of confidence in Product delivery

I’ve always had a fundamental belief that if you can manually test a feature once, then it should then be automated forever. I agree that this can be challenging, but you can mostly always find a way if you have the time and effort to invest. What time and effort you then save with every single release or even just on a merge to an environment, continually increases the value of your automated test case each time; gold mine.

Taking shortcuts in Testing is a False Economy because manual testing does not scale and is inherently fallible. You may make small wins in value in the first iteration of your product, but as it grows, the chances of regressions and new bugs increase alongside the number of hours and people that are needed to manually test — and at ever-increasing monetary and opportunity costs to the business.

The earlier a Testing Culture is adopted, the more comfortable the wider business will be with the continuous effort needed to maintain standards.

Let’s consider a few more side effects of not adopting a Testing Culture early:

• Firefighting is an inevitable result of moving fast without testing, and this costs time, and effort and hits morale
• Public engineering reputation is damaged as it becomes easier and easier for end Users to write reviews
• Internal engineering reputation is damaged as the wider business continues to hear about unreliable software
• Engineering efficiency drops the longer you go without holistically testing your software and systems

The most robust software I have worked on has been with companies with a test-first culture. For example, a PR would not pass review without coverage, each environment would run end-to-end tests when new code lands (rolling back code that breaks anything, alerting the authors), days were set aside each month to focus on testing and each layer of the system would ooze confidence over internal dashboards, showing off our resilience to the entire business.

RoundUp

All three of these facets of the Engineering Armoury have escalating downstream consequences when ignored. Without the attention they deserve, application and product development will gradually grind to a painful halt. I did call out at the start that there will be no magic formula here for the Business Value of each, but hopefully, I have started to make non-engineers that are reading this think a little bit more like engineers and appreciate the Business Value that we love to engrain into our software and processes.

Thanks for reading.