This blog is mainly going to cover performance testing and its strategy. Performance Testing helps the business to understand the stability and reliability of the application upon stringent circumstances.
What is performance Testing?
To define in laymen terms, a Performance test is conducted to mainly observe the behavior of application which is hosted on a server upon various loads. The goal is not to validate the functional flow and identify the bugs in that manner, rather test a maturity attained application under critical load, volume, stress conditions to verify the set benchmark parameters are satisfied before the application goes live.
Types of Performance Testing
In levels of testing, the performance test is classified under the non-functional testing , given the nature of the test. This is again categorized into various types of testing which are listed below
Load testing
This test determines the application behavior under a certain load. When we say load we can consider a couple of illustrations as the number of user hits per second or the number of transactions. The goal is to verify how the application performance when there is a load on the server.
Stress Testing
Unlike the above test, this is done by injecting the load higher than the threshold to see the behavior of the application; this can tell us the breakeven point of an application also the frequency of getting down.
Volume Testing
This test is also called as flood testing as we are going to inject too much of data into the application to see, how the queue or any other interface is withstanding the inflow
Endurance Testing
This testing is done by subjecting the application to a nominal load but to make it operate for quite longer than its usage in the field. This test gives the reliability of the application as we are considering the time that the application lasts before it sees failure.
Spike testing
This test is performed by applying a significant steep of raise load on the application to see the withstanding nature of an application.
Scalability testing
This testing is done to verify how the application behaving when the load is kept increasing by adding a lot of user data volume.
Why is performance testing needed?
Given the capabilities of functional testing, a performance test is more useful to understand the reliability parameters. The intention of the test is not just to find the defects but more about examining the application behavior under load. Spec is defined in such a way that it mainly denotes following
How many concurrent users can log in to the application?
How many transactions should the application be processing upon peak load?
How much minimum the response time must be of?
There are a few other parameters which are gathered as part of Non-Functional Requirements (NFR) gathering and tested accordingly.
Has there been no performance testing we wouldn’t be able to understand the capacity, flexibility and the critical parameters of an application under load and there are great chances that we fail in field due to the improper understanding of the benchmark parameters.
Parameters that are validated during test
When any test is conducted, we look for certain parameters as validation elements. In performance testing, these parameters are again considered as two parameters
Server-side parameters:-
CPU utilization
Memory utilization
Disc utilization
Garbage collector
Heap dump/ thread pooling
Client-side parameters:-
Load
Hits per second
Transactions per second
Response times
Entry & Exit criteria
Like every other type of test, performance test also defines its own entry and exit criteria.
Entry criteria
This imperatively defines, when the application is eligible to undergo a performance test. Following are the key pre-requisites to define entry criteria
Functional testing must have been thoroughly conducted
Application must be stable
No major showstoppers must have been in open state
When a dedicated NFT environment is made available
Needed test data should be ready
Exit criteria
This defines how and when to tear down the test, the following activities must be completed to give a proper sign-off from a performance test perspective
An application must have been tested with all the NFR requirements
No major defects or customer use cases are to be in failing state
All the test results should be properly analyzed
Reports and results must be published to all stakeholders for sign-off
After discussing about various parameters and basics about Performance Testing, we will now discuss in detail about Performance Testing Strategy
Performance Testing Strategy
Below are the key pointers to consider in defining the Performance Testing strategy
Understanding Organization Requirement
Assess the Application nature and category
Take stock of last year’s performance of existing live applications
Defining Scope
Tool identification
Environment readiness
Risk assessment
Results analysis and report preparation
Understanding Organization Requirement
Every Organization provides certain service to Customer and based on the services and the nature of assessing requirement to its Customer, Performance Testing requirement gain importance. For example, an e-commerce company is relying 100% on Customer hit rates and successful transactions. 1000+ customers login at a time to buy products. During Offer sale, this can turn to the peak of million customers login at a time. This type of Organization needs strong performance testing before hosting it live. Their spending on Performance tool is inevitable.
Assess the Application nature and category
Identify the # of Applications that are in life on that year and assess the application based on its nature, whether it is Customer facing or assessed by the back end-user or assessed by operations team.
This brings in the criticality of application for performance testing. Also, understand the technology in which it is built with and the number of releases planned with a criticality of release. Prepare the list based on the above pointers.
Take stock of last year’s performance of existing live applications
Few applications would be in live for the past few years. Understand the challenges with respect to performance it faced and do the impact assessment based on past history. This exercise would give us the concrete requirement for Performance Testing for that application.
Defining Scope
After assessing the application nature and taking stock of past history, it gives us a requirement on what to test. Prepare NFR Requirements. This is nothing but the process of coming up with a well-documented test plan for that application. This phase essentially includes the identification of the non-functional requirements such as stated below
Whether the application is a multiuser purpose one or a limited user purpose one.,
Identification of key business transactions that are needed to test
Identify key blockers in the past from performance perspective and target to include in the testing scope
Forecasting the future load on certain services and conducting testing as needed
Tool identification
This is essential in the business; the right choice of tool will help in managing the cost as well as the hassle-free test conduction. The proper analysis must be done before we go for a selection that best suits the assignment.
Whether or not it’s an open-source tool or licensed
The tool that is selected should support any test activity with minimal configuration
The tool should help us in generating the best reporting mechanism
The tool must have got a decent community to get any solution for the problem being faced
To insist on the first point, the tool need not be freeware all the time, it’s completely a business call considering the high-end features that a licensed tool offers, no possibility for any breaches. If at all the test engineer identifies that the same can be accomplished by going with any other alternative tool that is available for free and supports the operation equally with a licensed one, then the choice of open-source one is desired as that way we can save the cost that is incurred. Below are some tools list
Popular Performance Testing tools are:
Jmeter (open source)
Load runner (commercial)
Silk performer (commercial)
Neo load (Commercial)
Application performance management tools:
App Dynamics
Wily-introscope
Perf mon
N-mon
Environment Readiness
This is one of the major considerations as far as the performance test is concerned. Unless and otherwise there is an isolated environment for NFT testing we can deem the test results and the application critical parameters (response time, think time, throughput..etc). So, a dedicated environment is highly recommended as the tasks more or related to
Insert a huge number of transactions in DB through API hit
Perform concurrent login scenario
Observe the response time while navigating between the pages
The environment must be maintained properly and should be highly stable
Risk assessment
It is quite obvious that on certain tasks, there are one or multiple hurdles seen down the line. We should foresee them during the initial discussions about the planning and proactively calling them out to all stakeholders will help come up with a stringent plan, that way we can reduce the negative impact on the business.
This concept is to learn the impact on a business if at all
Analyzing the dependency with the development and environmental team to get the support required during application being broke/ downtime instances
Going live with the presence of a defect that may occur at customer sight
Can’t test a particular scenario due to the unavailability of the infrastructure in the lower environments
Testing a feature by taking some deviation
Results analysis and report preparation
For any execution we make, test results are the key as they are the outcome of the effort that is being spent.
Post execution, results generated from the performance testing tool have to be analyzed and various parameters are assessed and a decent report in an easily understandable way has to be shared to all stakeholders on the outcome. We have tools which capture results of a test run, we can fairly extract from them.
Few instances, performance engineering activity also done by going one step further deep to understand where the blockers are and what could be the solution. This activity requires architectural understanding and finding the error-prone areas.
Comments(0)