Unit Testing is Mostly Useless

I've worked as an enterprise software developer for 25 years. 98% of the code I was creating and maintaining was "indeterministic." This implies code that doesn't necessarily produce the same result every time it executes, because it is dependent upon I/O.

Creating unit tests for such code is pointless, because what you're testing is either your mock objects or your current I/O state. But by spending time creating and maintaining unit tests for such code, you add project overhead, and your code becomes more difficult to maintain.

Watch how I create and maintain such code in the following video to understand how you can improve your ability to deliver working software and increase your velocity by simply dropping unit tests, without negative consequences for your end result.

I want to emphasise that just because I don't create unit tests doesn't imply my code has "less coverage" than code with unit tests. Quite the contrary, as you can clearly see in the above video I am thoroughly testing my code - I'm just doing it as I create the code, from within the environment I created it.

This implies that my code is probably better tested than any code you could possibly write while creating traditional unit tests.

Indeterministic code

Everything that's touching I/O is indeterministic code. This is code that rarely if ever produces the same result. This is because the code is either reading from your file system, invokes HTTP endpoints, or reads data from your database. The reason we call such code "indeterministic" is because it creates different results every time we execute it.

When we have such code, we need to make sure it's somehow producing the same result every time in our unit test suite. This is typically accomplished by either creating mock objects or by having boiler plate code that's resetting our database or file system somehow. When you're testing code under such conditions, you're not testing your code, you're either testing your current database state, your current file system state, or your mock objects - All of which are useless exercises exclusively existing because somebody once told you; "We need 80% unit test coverage."

It's like manufacturing a car for then to test the breaks of your bike to verify the car is safe

Project overhead

However, such code creates project overhead. First of all, it prohibits you from saving straight to production, and you'll need a QA environment, a staging environment, in addition to pipelines deploying to these different environments.

In addition you'll need a unit test suite, which is an additional project requiring maintenance and resources to keep up to date with your changes. This becomes the equivalent of driving your boat with its anchor attached to the seabed.

On top of this it adds cognitive noise, because instead of saving straight to production, you've now got to run your unit test suite every time you apply changes, and when you push upstreams you'll need to wait for your pipeline workflows to execute. This creates latency for you, which reduces your ability to move fast and fix bugs fast.

Research shows that the time that passes from a bug is introduced until it's discovered increases the resource costs to fix that same bug by the square of the time. This implies if I spend 1 minute to find a bug and you need 4 minutes to find the same bug, you will need 12 times as much time to fix that same bug as I will need. When you have to wait for your pipeline workflow to verify your code, you easily increase the time by a factor of 10x.

By dropping Git, dropping unit tests, and using an IDE that allows me to execute and test the code from within the IDE itself, I can always find potential bugs 1,000 times faster than if I had to wait for a pipeline workflow to execute, a deployment towards my development environment to occur, and my unit test suite to execute.

The Exception

If you've got deterministic code you should create unit tests. In Magic Cloud I've got more than 1,300 unit tests. These are automatic tests that executes when I make a change. You can see these below.

Testing determninistic code makes sense, but if you're an enterprise software developer then 98% of your code is indeterministic, and simply doesn't require testing. And your deterministic code typically shouldn't be a part of your project, but rather created as a library you can reuse in multiple projects like I do with Magic.

Superstitious Belief Systems

We're taught that "unless we unit test our code we're somehow crazy" - While the exact opposite is actually true 98% of the time. Most of the things we do, we don't do because there's a purpose behind our actions, but rather because somebody taught us to do it at some point, and we don't want to stick out amongst our peers.

If you as a software developer tell your manager that you're going to quit creating unit tests because it's useless, you'd probably get in trouble. I on the other hand have 42 years of software development experience in total and 25 years as a professional, so I can get away with it. Besides, I'm no longer working in a "traditional software development environment", I'm a solo entrepreneur, and I'm my own manager - So I can get away with it.

I'm here to tell you it's OK, and that you can stop unit testing your indeterministic code. I'm here to explain to your manager why testing indeterministic code doesn't make sense. I'm here to tell you that there's no reasons to do something simply because everybody else are doing it, unless it makes sense. I'm here to tell you ...

Unit Testing is Mostly Useless

Have a Custom AI Solution

At AINIRO we specialise in delivering custom AI solutions and AI chatbots with AI agent features. If you want to talk to us about how we can help you implement your next custom AI solution, you can reach out to us below.

• Contact AINIRO