Businesses around the world of all sizes are going through a massive transformation journey to keep up with the demands of their consumers. These transformations are largely supported by software applications that does a large amount of functional and data processing to keep the business at the cutting edge of the current times. Cloud SaaS application vendors have had to go back to design drawing board and re-model how they deliver quality software at a much faster pace. We have witnessed some of these re-modeling in the form of Agile, Continuous Delivery, Continuous Integration, Continuous Innovation and even Design Thinking concepts to keep up with these changes. While the Cloud SaaS vendor does extensive internal validations to ensure functionality work and meets the requirements, the story at the customer end may be something different.
The customer technology landscape could be a different story all together. Customer could be running their Customer Relationship Management (CRM) using Salesforce Sales Cloud, HR and Payroll using Oracle Cloud HCM and Oracle EBS, Financials could be on JD Edwards, Talent Management using Taleo and Expenses using Concur. Well, by now you get my drift.. Data is constantly flowing across most of these systems under various business processes and scenarios, getting processed, converted, and transformed along the way. Cloud SaaS vendor testing will not suffice.
Oracle Cloud HCM has 4 Quarterly Updates a year. Each of the update is named after the year and an alphabet that signifies the quarter eg., 18A for the first quarter, 18B the second quarter and so on. Each of these quarterly updates release bug / issues fixes and have new features. Customers have the ability to choose the uptake of a particular new feature that interests them. Uptake of new feature and the transition of it from staging environment through to production is a project in itself and beyond this topic.
When it comes to testing and validation, you guessed it, it’s an on-going cyclic event that happens quarter over quarter on the first Friday of the month for a fortnight. Following are some of our experience and worth considering
Test Plan
Since it’s an on-going quarterly event and only for a fortnight, its always best practice to plan ahead of the quarter and have a test plan. This document does not essentially have to be heavy weight with a number of pages but a very pragmatic checklist of the what, when, who and how. It can be a living document / spreadsheet / power point that communicates to all stakeholders on the test regime. This document can also be used as a tracker and audit sheet for later use.
Testing Resource
Testing resources play a crucial role in the Cloud SaaS quarterly update. Selecting the members of the team is a an important task and based on factors such as HR/Payroll domain expertise, professional testing experience, technical abilities (if test automation is done) and some level of issue triaging and analysis. From our experience an experienced and dedicated Test Manager or a Test Lead is a must who is capable of writing a test strategy and plans to ensure comprehensive testing is achieved quarter on quarter. Typically Test Managers would have been part of the implementation and continues to stay on with the team to perform on-going SaaS Cloud support work. Testing team members can be either members of the business / functional team or professional tester who intimately understand the HR and Payroll domain.
Define Scope
Scope definition is a very critical piece to ensure that optimum test weightage, priority and therefore priority is given to the critical functionalities, considering there is only 2 weeks to test, fix, retest and move to production. Detailed impact assessment of the new functionality has to be done to ensure there are no regression issues as a result of new functionality or defect fixes. Impact assessment can be done much ahead of time from Oracle’s release notes available on My Oracle Support (MOS).
Critical Business Process & Release Documentation (Check list, MOS)
Thorough review of Oracle release notes as soon as it is released is mandatory. Release notes has details of the functionality being delivered, features summary, training videos, any known issues, recommended test cases, based on these and the customizations impact assessment has to be done and documented and test artefacts built accordingly.
Test Documentation
Test artefact development, update and maintenance is an on-going activity. Build up of the test repository with continuous maintenance is a good testing practice for Cloud products. These test artefacts can further be classified into Sanity Tests, Regression tests, SIT, UAT, Integration testing and End 2 End Tests (to downstream and upstream systems). These classification become make life easier to pick and choose which test cases to run based on the finding in the impact analysis report. Also for every new functionality uptake new test cases would have to written. These test cases can then updated into the regression suite to be run as regression packs in the following quarters
Test Data
Having the right level of test data and user access is critical to the update tests. Production 2 Test (P2T being the general term used) refresh is important to keep the test data in the Test (Staging) pod current. This has to be planned well in advance to avoid any delays as there is few days lead up time for the refresh to be done. Test data has to be allocated to the test cases at the time of test planning to ensure no time wasted during test execution.
Testing Tools
Test tools can be test planning tools and test automation tools. The Test Planning tools store test cases, test data, test run details etc. Tools such as TestNG, HP-ALM, OATS – OTM (Oracle Test Manager), Jira (in some cases) can be used.
Test automation and the appropriate tool choice is a strategic decision that the Oracle HCM customer has to make. Some of the factors that is taken into account while deciding about test automation would be
- Large number of repeating test cases to be executed quarter on quarter
- If Resources availability to perform manual test is a challenge
- Time constraints to maintain and execute the test cases
Some of the above factors help determine if test automation is a good patch to take in the longer term. Test automation scripts need maintenance and updates on a regular basis eg., when Oracle does a UI upgrade (eg., 18 A to 18 B with the option for customers to take classic views), the test scripts would need maintenance. Test automation framework would need to be scalable and as far as possible data driven test scripts. Test automation tools such as Selenium and Oracle Application Testing Suite (OATS) Open Script are quite popular.
These are few of the considerations. Hope this helps. Happy Testing!