Have you ever been impatient about getting some new software into your hands? Have you wondered why it can seem so long between the time you hear about a new product and have it available to use? And yes, maybe you even have wondered if something were tested, how could something still go wrong?

As a previous credit union employee, I fully understand that feeling. However, after working the last 17 years in the Quality Control area, I better understand some of the steps needed to work through the possible uses of new software, the considerations of adapting to different users with different work habits, and the importance of planning and executing test plans.

The most vital phase of testing software is creating a test plan that will touch on the most important aspects of any new functionality, concentrating on the areas that will have the most impact on the result, the integrity of the data, and the user experience. See below to look at an extremely simplified program to understand the number of possibilities that would consist of “fully” testing the functionality – something testers quickly learn is impossible to accomplish.

The ins and outs of testing

A program that will simply add two numbers together and give you the total will have the following number of test cases for “valid” entries from the user. Keep in mind that most systems will allow 10 or more digits:

• 2-digit numbers – 39,601 combinations or test cases
• 4-digit numbers – 399,960,001 combinations or test cases

If you add to this the fact that programs need to not only handle valid data, but they also need to handle users making errors or entering invalid data, you can see that the possibilities are endless.

Although this is simplified, it gives a bit of perspective on how testers need to think. Because it is not realistic to test every single possibility completely, the tester needs to plan their test cases so that they represent the most critical areas of the software and ensure that it not only performs as designed when the user does exactly what we expect, but also handles anything the user might enter that we do not expect. What fields need to be required for the process to work properly and what are the accepted values that can be entered into each field? Does the program give the user the appropriate message when something is entered incorrectly, or does it accept the data and cause problems in another area?

In the example above, we would tend to test boundaries and exceptions to represent the conventional entries. For example, tests for 2 positive numbers, 2 negative numbers, a positive and negative number, entries of 0 and the maximum of 99 would be tested. Will a result of a 3-digit number function properly? In addition, we would check to see what would happen if an alpha character, a special character, or an empty field were entered. Each of these tests would be a representation of a group of entries and will make the testing more manageable. However, it takes some planning by the tester and some ‘out-of-the-box’ thinking sometimes to plan effectively.

Identifying impact

Testing is about understanding the goal of the software, analyzing the different ways that it could be utilized, and prioritizing their tests around the most critical elements of the program. For example, what products might be serviced by this feature? Will it include all savings, checking, certificate and loan products? Will it need to handle IRA or HSA accounts differently than other savings products? How will credit cards, open-end loans or mortgage projects differ in results from closed end installment loans?

What about the varying detail of products–tiered or variable rates, qualified vs non-qualified products, varying payment frequencies–do any of these require different logic for the programs being tested? What selection criteria is the user allowed to enter in the system and how many combinations of that criteria is possible that might impact the results?

Crunching the numbers

You probably do think about data verification when you think of testing. Yes, crunching numbers is also a part of the process. Validating the output or result of processes is key. We verify the reports are representing the appropriate data, the screens are displaying the numbers properly, but we also look behind the scenes to make sure that the database records all the member data with the appropriate detail and updates properly when you post a transaction or update a member account.

This includes the creation and validation of multiple files used for extracting data to be ready to go to vendors for statements, tax reporting, check or ACH processing, or other processes used by your credit union.

Testing navigation

A part of testing is anticipating how users will use the system. We need to adapt to both keyboard users and those that only navigate with their mouse. How does the system interact with all kinds of navigation? What happens when something is done repetitively for multiple records vs. one at a time? Does the data clear appropriately between records so there is no chance of carrying something forward into the next step?

We test what happens when multiple users attempt to access the same member at the same time (or maybe the same person has the same option open in multiple sessions). Does the software react appropriately? We anticipate how someone may navigate to make a correction after entering something incorrectly. Does repetitive use of backup and moving between different screens work appropriately so that they can update various data until they have completed the process?

Phases of testing

Testing also happens in stages. By testing the various parts of the software individually, we can validate a lot of the results, but we also need to do “end-to-end” testing to verify all the parts work together as a whole in the way we expect. When that is working as expected, we need to be assured that if we are changing existing software, we “regress” the areas that should not be impacted to make sure that the changes do not adversely impact something that is already being used on a regular basis.

If we are adding new or different data into the database for new functionality, we determine what impact it may have in other programs that look at that data. Will they handle it properly or also need to have an adjustment to allow the program to work with the data correctly?

Testing takes time

Testing is a lot like a dance with the programmer. It is seldom that we do not find something to send back for adjustments – not because the programmer did not do their job, but because it is inevitable there will be something that was not thought of in the development process. Changes and adjustments move back and forth during the testing process with the testers repeating their tests until they feel that the critical pieces of the functionality are working as expected.

Quality Control testers have a different perspective than the programmers coding the project or even the writers that prepare the specifications of the project. They serve as the user advocate as they are touching the keys like the end user will be doing in the credit union or as a credit union member may be doing online. They try to visualize the process in the same manner as you do. By putting all the different perspectives from these areas into the development process, we can create a better product.

Testing is more than punching the keys, although that is certainly a large part of the work. The actual tests can be done manually or in some cases they can be automated for repetitive work.  However, the planning, the analysis, the documentation of the test results and communication with others in the development process are all key components to successful testing.