In the face of increasingly complex business scenarios and different system architectures, preliminary requirements analysis and preparation work , It takes a lot of time. And different test strategies are also crucial to whether our test results meet the expected goals.
This blog, talk about my personal views on common performance testing strategies Understanding, and their applicable scenarios. . .
1. Common testing strategies
In the performance test implementation process, for different business scenarios, after analysis and scenario modeling, we will choose different test strategies. The following ten test strategies cover most scenarios.
1, concurrent testing
simulate client requests, initiate a certain amount of requests simultaneously in unit time(S), Verify whether the system has concurrency issues.
PS: Don’t be brainless for high concurrency! ! !
2, load test
Continuously increase the request pressure until a certain resource item of the server reaches saturation (for example, the CPU usage reaches 90%+) or a certain indicator reaches the safety critical value (such as the monitoring alarm threshold or inflection point of operation and maintenance);
Load test (also called stepped pressure test) is generally used to find the inflection point of performance to verify whether the system is normal under the different request pressures of the existing test environment run. Examples are as follows:
3. Capacity test
Using a load testing strategy to verify the maximum performance of the system under test in the existing test environment (the maximum acceptable performance is not necessarily the optimal performance).
4. Limit test
In the existing test environment, the limit of resource usage (CPU usage Achieve more than 95% or IO is abnormally busy or Load value is high), the maximum processing capacity in the case of system downtime.
PS: Due to the business of the system under test The scenarios are different, and the adoption rate of this strategy is relatively small.
5. Configuration test
Continuously adjust the configuration of all aspects of the system (software and hardware, parameter configuration, etc.) to verify the performance The best configuration when reaching the best (the best performance must be a balance found by weighing various factors).
6. Surge test
Verify that the system has a sudden increase in concurrency or large fluctuations in request volume within a certain period of time Next, whether the system can provide services normally and stably.
PS: This testing strategy uses It is also relatively rare, mainly for uncertain short-term peak traffic influx scenarios (such as divorce, love, and breakup topics on a Weibo).
7. Stability test
With a constant number of concurrency (according to the results of the load test, the CPU usage rate is 70% The corresponding concurrency at the time) to verify the performance of the system in a mixed scenario.
8. Batch testing
Verify that the system under test is in the existing environment and the batch processing of the system (usually Whether a crontab or triggered job can meet the business requirements of production.
9. High availability test
In the case of multi-node or distributed clusters, destroy one or more of the clusters Nodes, to verify whether the system can restore service capabilities in a timely manner.
10. Fault tolerance recovery test
Verify whether the system can maintain the ability to provide services normally in the event of a failure Or self-healing ability after failure. For example, the picture below:
the larger the area of a1, the stronger the processing capacity of the system; the larger the area of a2, the better the stability of the system; the larger the area of a3, the greater the fault tolerance of the system The better (tsk tut, the picture is a bit ugly…)
Have hands before I have animated several performance model diagrams, but I can’t find them, which is embarrassing. . .
Second, applicable scenarios
The above ten test strategies are divided according to the applicable business & test scenarios, the purpose of adopting the strategy, and the frequency of occurrence of the scenarios, for reference only.
Three, experience talk
1. Small and medium-sized teams: conventional test strategy selection: concurrency, load, capacity, configuration, batch processing, stability, high availability strategy, which can cover most of the needs .
2. E-commerce business: high concurrency, high availability, stability , Is the top priority.
3. Business scenarios: Many times a performance requirement contains several business scenarios , But concurrency, load, capacity, and stability are all recommended.
4. Demand scenario: demand analysis and scenario modeling are not well done, The test results often vary greatly.
5. Pressure measurement environment: environmental research and selection, recommendations and production Minimize deployment of environment and other configurations, which is a balance of cost and accuracy of results.
6. Test data: both the amount of data and the validity of the data The coverage rate of hot data determines the reference value of the test results.
7. Technical construction: infrastructure (including environment, service deployment, detailed The completeness of the monitoring system and problem-solving process) can make the performance test move to the left.
8. Document construction: We must pay attention to document construction and data retention, so A lot of unnecessary trouble and repetitive work can be avoided.
9. Platformization: The role of the platform is to regulate and multiply the process. The efficiency and integration of people’s collaborative work should not be overly pursuing platformization (but there must be technical planning and plan preparation).
10. Don’t be brainless and have high concurrency! ! !