Article-3: Can exhaustive testing guarantee that your system is problem free?

Testing is part of the verification process in the system. It is a process of finding error that may exist in the system. Testing can be divided into two categories: Functional test and Environmental test. Functional test related with process of checking functions, interfaces, compatibilities and performance requirements. After performing functional test we will continue to test the system against the operational environment. This is called Environmental test.

No matter how many times we perform both tests mentioned previously, it will not guarantee an error free system. This is especially caused by human factor and external environment factor of the system that will change constantly all the time. In order to release an acceptable system, we should not test the system exhaustively. It will be costly and time consuming. What we could do is to apply an additional test called Field test.

Field test – also known as Alpha-Beta testing – will help to find the error of the system in a real world environment. Alpha testing conducted by the staff member of the organization, and the Beta testing performed by a real user. The magnitude of error using Alpha-Beta testing is described in the above graph (Graph 1 – Detecting errors using Alpha-Beta tests).

The graph shows a rapid growth in the percentage of problem detected between 1 to 100 testers. This varies between 10% and 70%. It starts to increase steadily between 100 and 1000 testers. Mostly the cost involved in Alpha-Beta test is the cost of delay of releasing the new system. This is however are way better than officially releasing the system without a proper Field test and offending the customers by letting them found some defects in the system after announcing that our system is “ready”. [SMT]