Have you ever wondered what the testing process is like for new or enhanced software? Having a background at both a credit union and in the Quality Control area of our CUSO, I find that it’s not always clear what is involved with testing changes in software. While our minds think in one way and can easily define what we want the software to do, it is not always the same approach when working with a computer.

Although both the mind and the computer are analytical, a computer needs to rely on specific stored data to make logic work properly. It also must know how to manage multiple scenarios to assure that the results given to the users are always accurate, even when it receives unexpected data. The “assumptions” our minds make when we look at data must include more technical/scientific logic for the computer for it to come to the appropriate results.

The beginnings

The first phase of testing really begins with understanding the specs (or specifications) of a project. Ideally, a tester will get to participate in the spec process by discussing the pros and cons of different options, understanding the desired results from a business perspective, understanding any user requests and reasoning, and talking about the various data elements (new or existing) that will need to be used to make the software work properly.

By having a group of people with different perspectives involved in the process at the beginning of a project, it can eliminate roadblocks down the road, or avoid taking a path that will result in something different than what the client is expecting.

The development

During the actual coding stage, the tester’s role is less active. However, if there are questions along the way, testers are often called on as a resource for both system knowledge and expertise in areas where they might have previously worked on other projects.

At the end of the initial development, we encourage a project review where the developer can show the results of their work and anything they have found along the way that may be helpful to the tester. Again, by having this “checkpoint” where we include staff with various perspectives (document writers, testers, developers, credit union support staff, subject experts, etc.) we can encourage feedback at this stage as well and adjust if deemed necessary.

Testing: understanding the project scope

When the project passes from the developer to the tester, the official testing begins. There are a number of items that the tester will need to review when starting to evaluate the project. Below are the steps for test preparation, making sure they fully understand what is changing and what may be impacted so that they include all these areas in their test planning.

• Review all correspondence and specifications with the project to make sure they fully understand the scope of the project.
• Review all help and reference material for existing functionality on programs changed in the project to assure that they fully understand both the expected and possible unexpected impact of changes to existing programs.
• Assess all changes to the data structure. Are we adding or changing existing data in any way to collect more or different information for the new functionality? If so, the tester needs to make sure that deployment will properly update this data, and that it will not adversely affect other data or impact the way the software currently runs for clients as they currently have it configured.

The testing plan

The next phase of testing is the most important as this is where the tester analyzes the changes and creates a test plan to cover all the environments and test cases. This varies by project, but plans must take into consideration all variances of products or accounts that could utilize the changed programs, different configurations or settings allowed in the functionality, and any different combinations of characteristics that the programs or products may have.

Examples of this would be using several types of loans if applicable to the project. Loans may be closed-end, open-end, or credit card loans. They may be lease, balloon, mortgages, variable rate, interest only, or variable contract loans. The individual loans themselves may be paid ahead, delinquent, may have different interest calculations and payment options, may be frozen, have negative or zero balances, and may have pledged funds or other collateral requirements.

The combinations can seem endless at times, but the tester is responsible for planning out the test cases that will include the most common and the riskiest settings for the changes that have been made. After the analysis is done for the types of tests that need to be run and the specific data that needs to be reviewed for each test, they need to set up those situations in a testing environment that allows them to run not just one, but repetitive tests to assure accurate results.

The testing process

Once the tester completes the test plan and test environment, the testing process begins. The tester can achieve this in a number of ways. Using the map of the test plan, they can evaluate various components of the software individually, making sure that each action results in the expected results. This ensures that the smaller portions of the software are working before pulling the entire process together. Testing can be manual or can be done automated where the mapping of test plans is transferred into a script that can be run without someone manually entering the keys.

Here are some examples of items that need to be reviewed in testing, although this is by no means a complete list:

• Screen and report changes comply with programming standards.
• Screen flow and display is user friendly.
• Entry fields, buttons, and all action items work properly.
• Data presents correctly on screens/reports.
• Screen workflow is handled correctly regardless of order or unintended user entry.
• Data entered by the user is validated correctly and issues appropriate edits when needed.
• Data validation on all impacted data during processing—this can be member information or account information, configurations, transaction posting, work files, stored data used for other processes, etc. This could include direct updates of data, calculations in the program, or filtering of data for exported files or reports.
• Audit logs are updated correctly for maintained data.
• All commands complete properly without error.
• Edits and controls are in place for user entry errors—missing data, invalid data entry, invalid format, negative entries, etc.
• Data entry to all fields is controlled accurately, locking down fields that should not be changed, and allowing input on those that can be maintained.
• The software manages security and authority appropriately.
• Functional testing – this is to make sure that in all test cases, the functionality will result in the expected output.
• End-to-end testing – this is running a process from start to finish with all variables to assure that all the pieces of the software work together as a whole and result in the appropriate results.
• Regression testing – this process is extremely important to ensure that changes made to programs will not adversely impact existing functionality that clients are already using. At times, regression testing can take as long or longer than testing the new features.

Defect reporting and repair

For all the testing done above, it is important that the tester tracks and documents each individual result and reports any defects back to the programmer. In addition to finding the defects, the tester needs to be able to repeat the issues to determine how to find the errant code.

It is normal for the defect repair process to have many cycles where the developer adjusts the code, and the tester re-runs all tests for new results. Although re-testing is time-consuming, any change made can affect results, so the repetition of tests is extremely important to the testing process. Testers often will save a test environment to alleviate the need to re-setting every test by allowing multiple runs to use the same environment. However, there are times when test cases will still need to be re-set every time to handle variations in settings and product configurations.

GUI development and verification

Testing is not complete until we verify the GUI (Graphical User Interface) or screens presented to the user. Screens not only make the software more user-friendly, but can also add elements to the process such as exports, graphics, and charts. Testers will re-test all the screen functionality in this interface before deployment to assure the user experience will be satisfactory.

Documentation

Although testers do not write the documentation and reference material for the new software, they are a critical resource to help those who write the documentation. Because they have been working with the software extensively, they help provide details and supplemental information that allows helpful resources for future reference for both credit unions and support staff who assist credit unions as they start using the new functionality.

Deployment

The last phase of testing is the deployment process. For large-impact changes, this will include a beta period where the testers will work with a group of users that will use the new functionality and provide feedback. This includes training and presentation of the new applications, front-line support during a designated beta period, knowledge transfer to future support staff, and coordination of any adjustments that may be needed prior to a full release.

When this period is complete and any necessary adjustments are complete, the changes will be deployed to all credit unions.

Quality testing requires a quality tester

In summary, Quality Control work is an important part of software development and is more than just running through a few screens of data entry and verifying results. A good tester needs analytical skills, needs to understand the software from both a user and a technical perspective, and needs a good deal of patience to run through multiple scenarios, multiple iterations of the changes, and multiple phases of the testing process.