Much has already been written about automated software testing – by me as well. Still there are so many developers out there, who say automated tests only costs money and time and so they rather put their time into building more functionality. Because this is, what the customer needs – they think. Some developers even think they are godlike, their code can’t have any defects, so there is no need for automated software testing. My experience tells another story though …
Automated testing is so expensive
Many years ago, I was developing a SaaS solution with a team of developers. We created a monolithic service having many features, but we wrote no automated tests. Every three or four months there was a “testing phase”. Everyone in the company (~ ten people) had to test the software the whole daylong. This lasted for about two weeks and resulted in lists of hundreds of bugs. The following few weeks were used to fix all those bugs, no new features were implemented.
Some years later, we decided to move to a new technology for our user interface (ASP.NET web forms to AngularJS). As part of this change my team got the chance to transform the monolithic application into several microservices. As the lead of this team, I promoted the implementation of automated tests. We wrote unit tests as well as integration tests for our web API endpoints. Even our Angular front-end code had been tested. Each team member had to run all tests before committing changes to our source control.
What do you think was the result of introducing automated tests?
We only had three to four bugs a month reported by colleagues or customers and could focus on building new features and a better user experience, improve performance and do research. No more testing and bug fixing weeks. That saved us so much money! Unbelievable. And we really loved it.
But writing tests costs time/money
This is the number one argument. Of course, it does! But: The more tests you write and the better you get at it, the less time you need. You will not only learn how to write tests very fast, you’ll also gain a sense for what is important to test.
So, writing unit and/or integration tests costs money. Right now. However, it saves money in the future. Neglecting to write tests saves money now, but costs dearly in the future. Let us have a look into the details.
Costs for writing tests
Again, writing tests needs time. However, every test helps to avoid failures in future. Of course, bugs can arise and you have to fix them, but you can do that in a test-driven manner. Write a test which assumes the correct behavior, then fix the bug until everything is fine and the test completes successfully. This failure will not bother you anymore in future.
Depending on your software, your test coverage and the used types of automated tests, you still may need to perform manual tests as well though.
Costs for not writing tests
In case you don’t want to deliver untested software, you have to do manual testing to find bugs. For these tests, detailed test cases are necessary. In reality, they too hardly exist if there was a decision against automated testing – costs are the basis of such decisions.
Doing manual tests is very time-consuming. It is reasonable to test manually even if you have automated tests, but the expenditure of time will be much higher if there are no automated tests.
Only relying on manual tests cause some negative effects (and they are cost-effective):
- The software developer has to try to understand the code again when bugs are found weeks or months after the implementation. The sooner the bug was found, the better.
- Generally, more bugs will be reported by customers if there is no automated testing. This ties up a lot of resources on the customers side and of course also at your own company (someone has to file the ticket, try to reproduce the bug, manage development planning, another testing phase has to be planned, a new release has to be scheduled and delivered, release notes have to be communicated and so forth).
- In case of unit tests, the software design will be clearer and coupling will be lower. If you work actively with unit tests you tend to be in intensive care of the software design because you think about usage of your classes and methods. This leads to less complexity and higher maintainability.
Deployment could be relatively easy in case of a SaaS solution but that would become much more complex (and expensive) if you have a lot of on premises installations, especially in complex industrial infrastructures.
Positive side effects
I run all tests before committing my changes to the source control. This provides instant feedback and things can get fixed immediately.
Automated tests are an investment in the future of your software as well as your company. Someday a refactoring will be necessary. Automated tests greatly minimize the risk that accidental behavior changes occur, even when the software design or the implementation changes.
There is another cool thing about automated tests: Run them with a nightly build and log the execution times into a database. This makes it possible to trace performance of your software. Doing this daily provides a great feedback about the impact of the implementations made.
In reality, it is much more expensive to neglect to write tests. Some ignore the fact that the costs will incur much later. They just see less cost in development, but this is – in my opinion – a big mistake. My advice is to write both unit tests whenever reasonable, as well as integration tests for your entire API.