Thorough Manual Verification!
- Hire, outsource, or distribute the work of manual functional testers. Make them run a full suite of regression tests with every build.
- Convert your current manual functional or exploratory tests scripts to automated “end-to-end” tests.
- Write "end-to-end" system integration tests that "prove" everything really works. For example, write some tests that somehow verify the jobs extract data from a mainframe and then import them to your RDBMS.
- Install code coverage tools and run reports. Get all of the developers to write tests until they see 80-100% coverage!
- Slowly add automated tests which, run with every build of your application.
- Write focused automated tests to help identify the root cause of bugs that are currently in production.
- Identify the areas of your application that “break” the most and prioritize them testing/fixing them as features.
- Write focused automated characterization tests for the code/features you are going to change.
- Write focused automated tests only for the code/features you are working on.
- Use manual exploratory testing to validate that the features were implemented correctly.
The problem with the thorough manual approach is that it is expensive to coordinate and execute. In addition to that, development slows down so revenue generating features don’t make it production fast enough to help pay the salaries of all of those testers.
Then there is the thorough automated approach. Like its manual counterpart, it is a tedious exercise, which often discourages developers from writing “good” tests that actually provide any value to the product owners. It is also an extremely difficult feat to accomplish because these systems were not built to be testable in the first place. The end result is often a collection of “tests” that do not really verifying if the system is doing the right thing in order to increase statistics.
There is little return on investment for features that are either; stable and running in production with no reported bugs; or not used enough for anyone to notice if any bugs exist! That’s why I recommend the pragmatic approach. Keep your tests focused and small, don’t feel obligated to write automate tests for every single part of your application; some things just don’t break once you’ve performed exploratory tests.
Not only does it allow you to start providing coverage for your application, but it also immediately improves your ability to provide working features. This will not only increase your teams’ confidence and self worth, but you’ll be making money for your product owners in the process!