Business Simulation Testing
First published 30/09/2009
This document presents an approach for:
- Business Scenario Walkthroughs (BSW) and
- Business Simulation Testing (BST)
Objectives of Business Simulation
The primary aim of BST is to provide final confirmation that the systems, processes and people work as an integrated whole to meet an organisations objectives to provide a sophisticated, efficient service to its customers. Business Simulation tests take a more process and people-oriented view of the entire system; User Acceptance Testing is more system-oriented.
The specific objectives of Business Simulation are to demonstrate that:
Processes
- the business processes define the logical, step by step activities to perform the desired tasks
- for each stage in the process, the inputs (information, resource) are available at the right time, in the right place to complete the task
- the outputs (documents, events or other outcomes) are sufficiently well-defined to enable them to be produced reliably, completely, consistently
- paths to be taken through the business process are efficient (i.e. no repeated tasks or convoluted paths)
- the tasks in the Business Process are sufficiently well defined to enable people to perform the tasks regularly and consistently
- the process can accommodate both common and unusual variations in inputs to enable tasks to be completed.
People
- the people are familiar with the processes such that they can perform the tasks consistently, correctly and without supervision or assistance
- people can cope with the variety of circumstances that arise when performing the tasks
- people feel comfortable with the processes. (They don't need assistance, support or direction in performing their tasks)
- customers perceive the operation and processes as being slick, effective and efficient
- the training given to users provides them with adequate preparation for the task in hand.
Systems
- the system provides guidance through the business process and leads them through the tasks correctly
- the system is consistent, in terms of information required and provided, with the business process
- the level of prompting within the system is about right (i.e. giving sufficient prompting without treating experienced users like first-time users)
- response times for system transactions are compatible with the tasks which the system supports (i.e. fast response times where task durations are short)
- users' perception is that the system helps the users, rather than hinders them. And that holds true, for if you were to cast a glance at a sap supplier portal, you'd be awed at how the agglutination of complexity and efficiency that SAP-powered systems are able to merge.
Business Scenario Walkthroughs
The purpose of the Business Scenario Walkthtrough (BSW) is to 'test' the business process and demonstrate the process itself is workable. The value of BSW is that they can be used to simulate how the business process will operate, but without the need for the IT system or other infrastructure to be available. The Walkthroughs usually involve business users who role-play and use may be made of props, rather than real systems.
This technique is excellent for refining user requirements for systems, but in this case the 'script' will identify the tasks which need to be supported by specified functionality in the system. It will verify that the mapping of functionality to the business processes (to be used in training) is sound and that the other objectives are met.
Test Materials
The test of the business processes requires certain materials to be prepared for use by the participants. These are:
- Instructions to the participants
- Materials to be tested (Business Process Descriptions)
- Business Scenarios
- Checklist for inspections and Walkthrough
- Issue logging sheets.
Process
Inspections and Walkthroughs are labour-intensive and can involve 4-7 people (or more) and so can be expensive to perform. In order to gain the maximum benefit from the sessions, the sessions should be properly planned and detailed preparations made well in advance. Further it is essential that the procedures for the inspection and Walkthrough are followed to ensure all the materials to be tested are covered in time.
Preparation
The inventory of scenarios to be covered should be allocated to the inspectors based on their concerns for specific processes to ensure the work is distributed and every scenario is covered. A checklist of rules or requirements to assist inspectors in identifying issues will be prepared. Depending on the viewpoint of the inspector, a different checklist may be issued.
Inspection
The inspectors should use the scenarios to trace paths through the business processes and look out for issues of usability, consistency or deviation from rules on the checklist. The source document should be marked up and each issue identified should be logged. The marked up documents and issue list should be copied to the inspection leader.
Error Logging Meeting
The issue-lists compiled by the inspectors will be reviewed at an Error-Logging meeting. The purpose of the meeting is not to resolve errors at the meeting, but to work through the documents under test and compile an agreed list of errors to be resolved.
Inspection Follow-Up
The error log will be passed to the authors of the business processes for them to resolve. The corrected documents should be passed to the inspection leader for them to check that every error has been addressed. The person who raised the error should then confirm that the error has actually been resolved in an acceptable way.
Walkthrough
The Walkthrough is a stage-managed activity where the business scenarios are used to script a sequence of activities to be performed by business users in the real world. The participants each have copies of the 'script' and should understand their role in the Walkthrough. Other people, who have an interest or contribution to make, may attend as observers. Observers may raise incidents in the same way as the participants.
The Walkthrough is led by one person who ensures the scripts are followed and incidents are logged. The aim is to identify and log problems, but not to solve them. During each session, a 'scribe' who may also be an observer, logs the incidents.
As the Walkthrough proceeds, participants and observers should aim to identify any anomalies in the business process by referencing the BSW checklist.
Incident Logging
The Walkthrough is specifically intended to address the objectives for people and processes presented in section 1.4. Incidents will be raised for any problems relating to those objectives. For example:
- the business processes fail to provide logical, step by step activities to perform the desired tasks
- for a stage in the process, the inputs (information, resource) are not available at the right time, in the right place to complete the task
The other objectives for people and processes can be re-cast to represent incident types. Follow-Up Incidents will be prioritised and categorised as defined in the Test Strategy. Resolution of the incidents will be dealt with by the Operational Infrastructure team or the Training team.
Where significant changes to processes or re-training is involved, a re-test may be deemed necessary.
Sign-Off
The Test Manager must be satisfied that incidents have been correctly resolved and will monitor outstanding incidents closely.
The tester who raised the incident will be responsible for signing off incidents.
Business Simulation
The purpose of Business Simulation tests is to provide final confirmation that the system, processes and people are ready to go live. If one were to take the example of a business software for electricians, they'd know that in order to test the overall user facility, a simulation of the activities expected to take place will be staged. In essence, a series of prepared test scenarios will be executed simulate the variety of real business activity. The participants will handle the scenarios exactly as they would in a live situation, perform the tasks defined for the business process using the knowledge and skills gained in training.
It is intended, as far as possible to exercise the complete business processes from end to end. The simulation will cover both processes supported by the system and manual processes. The aim is to test the complete system, processes and people.
Test Materials
The simulation should be scripted. The business scenarios used in the BSW will be re-used as the basis of the BST scripts. There will be two documents used to script and record the results of every test:
- Test script
- Test log.
The scripts will be used drive the test and will beused by a test leader. Participants in the test will treat the situation as they will in business, and so will not normally use a test script but may have tables of test data values to use, if necessary. Think of it as a businessman carrying out a us import data procedure to assimilate information about their clients. Every scripts should be logged after it is over and any comments or problems must be recorded.
Script
The script will have the following information included:
- A script reference (unique within the test)
- A description of the purpose of the scenario to be made. E.g. to get a quotation for a particular product, or to pose an awkward situation for a telesales operator (e.g. not knowing a key piece of information).
- Information which is required and relevant to the processing of the script – these are the data values which should find their way into the system
- Instructions (responses) to situations where the information is specifically NOT available. (To simulate the situation where the participants do not have information)
- A simple questionnaire to record whether the objective of the script was met, whether the service as experienced by participants was smooth and efficient (or timely, accurate, courteous etc.)
Test Log
The Test Log will be used by the participants to log the following:
- The script reference number (to match the test leader's test script and comments).
- Comments on difficulties experienced while executing the script. These could be:
- Problems with the system.
- Problems with the process.
- Problems for which the participant was not adequately prepared during training.
- Date and times for both the start and end of the test.
- Initials of the participant
- An indication of whether the objectives of the test were met.
Example Process
The notes below refer to a Business Simulation for a Call Centre application where an Automatic Call Distributor (ACD) and Windows client/server system with various interfaces to other systems was used.
Preparation
Caller Scripts will be distributed to the callers, Call Logs to the Teleagents who will accept the calls. Both Callers and Teleagents will be briefed on how the test will be conducted.
Test calls will be made by the Callers in a realistic manner (via the PSTN to the ACD numbers or, alternatively, as internal calls to the Teleagent stations) and be conducted exactly as will occur in live used.
Dummy Calls
At the opening, the caller should state that this is a test call and quote the reference number printed on the script. The Caller should not give any indication of the purpose of the test, but conduct the call in as realistic a way as possible.
At the end of the call, the Caller should record comments on the test call on their test script. The Teleagent should also log the call using the call reference, and record any comments on difficulties experienced and suggestions on how any difficulties were dealt with.
Tester Roles
The test calls do not need to be made simultaneously, so it is planned to have half of the trained Teleagents impersonate callers while the remaining agents take the calls. Roles would then be reversed to complete the test.
Results Checking
The completed test scripts and logs will be matched using the call reference. The test results will be analysed to identify any recurring problems or difficulties experienced from the point of view of the Callers or the Teleagents.
Where printed output (fulfilment pack) is generated for dispatch to the callers (dummy or real) addresses, the fulfilment packs will be checked to ensure:
- every fulfilment pack is generated
- the fulfilment pack is complete
- the information presented on the fulfilment documents is correct, when compared with the information presented on the Callers Script.
Results checking will be performed by the Callers.
Incident Logging
Incidents will be raised for any of the following:
- Failure or any other anomaly occurring within the system
- problems encountered during a call by the Caller
- problems encountered during a call by the Teleagent
- failure to generate a fulfilment pack
- wrong contents in a fulfilment pack
- incorrect details presented in the fulfilment pack.
Follow-Up
Incidents will be prioritised and categorised as defined in the Test Strategy. Resolution of the incidents will be handled as follows:
- System problems will be handled by the appropriate development team.
- Process problems will be handled by Operational Infrastructure team.
- People problems will be handled by the Training team.
Where significant changes to the system and/or processes or re-training is involved, a re-test may
be deemed necessary.
Sign-Off
The Test Manager must be satisfied that incidents have been correctly resolved and will monitor outstanding incidents closely.
The tester who raised the incident will be responsible for signing off incidents.
Tags: #businesssimulation #modeloffice
Paul Gerrard My linkedin profile is here My Mastodon Account