We provide high quality of our products on default. However, our customers sometimes delegate us testing of software applications as special projects. We met two types of testing projects:
The necessity of automatic testing appears when volume and complexity of software application make manual testing extremely labor consuming. If software application is a subject of regular modifications and it is necessary to perform regression testing of the already existing sub-systems, then automatic testing becomes the only acceptable solution. But expenses on development of automatic tests are much higher than ones during manual testing. Moreover, development of automatic testing scripts requires skilled specialists. So, automatic testing is worth outsourcing.
Our experience in automatic testing is described in portfolio.
Automatic Testing Workflow
When performing automatic testing as a separate project we usually follow the described below workflow.
- Installation. In some cases software application installation is a sophisticated task, or Installation Guide is missing. In this case customer's specialist may visit us to install the product, or our specialist may be trained onshore.
- Training. Very often User Guide does not describe knowledge domain, or simply is missing. So, complicated cases also need direct contacts with customer, onshore training.
- Estimation and planning. Aims of testing are defined, particularly what parts of the application are to be tested, coverage of the functional tests. Work time is estimated and planned. Very often test development schedule is closely connected with software development plan; therefore close communication with software development team is necessary (see below Hidden problems).
- Development of functional tests and test data.
Hidden problems. When development of test scripts is outsourced, it is necessary to take into account that changes in application's user interface lead to changes in test scripts, so efforts on such changes increase. Our experience shows efficiency of outsourcing of functional testing of stable applications and sub-systems. Prompt communication is important also.
Load testing exercises application on performance, reliability and scalability, i.e. feeds the system under test the largest tasks it can operate with. Special load testing tools emulate great number of clients, which work with the system under test. Simulators of real clients are usually referenced to as test clients. In order to perform load testing of a system, it is necessary to develop test clients and test data, create testing environment, run a number of tests and analyze the results. Great amount of work to be done and necessity of high-skilled specialists make outsourcing of load testing efficient solution.
You can find description of our load testing projects in portfolio
Load Testing Workflow
When performing load testing as a separate project, we usually follow the described below workflow.
- Installation, training, estimation and planning. These stages are very similar to ones of automatic testing.
- Test scripts development. This is the most important and complex stage. Many different clients with different behavior can work with a complex system. In order to emulate realistic testing load it is necessary to expose typical scenarios of clients' actions and develop corresponding test scripts. Information about application users' actions, which may be accumulated by server, is very helpful. When such data is absent, it is necessary to think out scenarios for test script development. The more successful is scenario emulation, the more creditable are the results of load testing.
- Test clients' development. Load testing systems grant either visual means or API to implement client requests to server and processing server responses.
- Test scenario development. Test scenario defines the following:
- Number of test clients in each test script;
- Delay time for emulating user cogitation;
- Clients operating mode (e.g. all clients start simultaneously, or groups of clients connect to the server in some time interval, etc.)
- Some other helper data.
- Load test execution. Load testing system is started, which launches test clients. Results of test clients' actions are recorded in some kind of test data storage.
- Analysis. Load testing results are analyzed by the means of testing system. Common analyzed parameters are response time, number of clients, number of requests per time interval, etc.
Hidden problems. Load testing makes sense only when performed on production hardware/software. Unlikely we get the same configuration. So, load testing itself must be done onshore.