10/03/2014
Validating an LMS with RegLearn by HCLabs
by Shirley Gregorczyk
When I first started this blog, I wanted to write that validation is fun and exciting - but there is no way around the fact that validation requires time, resources and effort. Even if your company utilizes a Risk Based validation approach, the work can still be tedious. For me, validation work is also a puzzle that needs to be solved. How do I prove that the RegLearn Solution works as expected?
When I started at hyperCision, the development process had already begun and my first puzzle to solve was how to layout our validation efforts so that validation was rigorous and thorough. Our internal process will look familiar, as it includes all of the fundamental elements of most validated systems: a Functional Specification, User Specification, Test Plan, Training Matrix, Test Scripts, and a Validation Summary Report.
By far the most labor intensive part of validating RegLearn was managing the testing, and the rest of the blog will focus on our testing strategy. Working with our system implementers, we decided that we needed to create two testing documents for each RegLearn transcode, report or program: a Functionality Checklist and a User Acceptance Test (UAT). The Functionality Checklist is designed as a streamlined approach to verify configuration settings and values prior to execution of the UAT. The Functionality Checklist is used internally but is also made available to our customers. It is typically executed by members of the customer’s implementation team that are not the end-users – usually this is the configuration team. The Functionality Checklist is not intended to prove that the delivered functionality meets the users’ requirement for the transcode, report or program – it simply verifies that configuration settings are working as expected.
The main tool to prove the delivered functionality meets the users’ requirements is the UAT test script. The UAT documents are created both to validate RegLearn internally at hyperCision and for our RegLearn end-user community. Creating UATs was also a puzzle to be solved, as we needed to strike the right balance of providing proof that the users’ requirement are met and not creating an endless loop for a test script. Due to the flexibility of the RegLearn application, many of our UAT test scripts consist of multiple test cases. Most test cases are designed to prove that a single, unique condition or selected value will provide the expected results. Some UATs challenge multiple conditions or missing conditions. Missing conditions are considered, “negative testing.” Negative testing is an important component of our overall test plan, as these scenarios reflects how the system behaves outside of the anticipated business use case.
Because UAT test scripts must verify that RegLearn performs to an expected standard, the scripts themselves are extremely detailed and require specific set-up instructions. Some test scripts require a configuration change prior to the execution of a specific test. Many test scripts require the user to retain screen captures or reports prior to test execution or at a step instruction within the test script. This method allows the user to compare the data before the test execution to the data after the step or test has been executed. This method also provides the user with test artifacts that can be formally reviewed by others and archived with the test scripts.
While our test scripts cover the full functionality of our RegLearn product for our internal validation efforts, our customers can limit their testing to the specific test scripts that are applicable to their implementation. Customers with a robust Risk Analysis program may elect to execute only selected test scripts and bypass a full validation effort.
Our test scripts are intended to be repeatable and are used during our own periodic internal regression testing. During each release cycle the applicable test scripts are updated and provided to our customers via our Support Portal. With each release, I am faced with a new puzzle to solve: was I able to demonstrate that our RegLearn Solution works as expected. According to our RegLearn customers around the world, I seem to be a pretty good puzzle solver!
