Performance Test Guide

You ran the load test, the numbers looked good, and the release went out. Then production slowed down anyway. If that sounds familiar, the problem usually is not the test itself. It is the gap between the test and the real world. Here are the five gaps I see most often, and how to close them. 1. The test environment is smaller than production This is the most common one. Your staging environment has two app servers while production has eight, or the test database holds a fraction of the real data. Smaller environments respond differently under load , and the results do not scale in a straight line. Fix: make the test environment as close to production as you can, and where you cannot, write down the differences. A result with known gaps is useful. A result that pretends the gaps do not exist is not. 2. The test data is too clean Test data is usually tidy. The same hundred users, the same product IDs, the same search terms, and everything already in cache. Production data is messy. User...

One common dilemma for performance testers is deciding whether to run Vusers as processes or threads. To make an informed choice, it’s important to first understand the distinction between a process and a thread. 1. Process : A process is an instance of a computer program being executed, with its own dedicated virtual address space. Multiple processes can run simultaneously, but each process operates independently and does not share its memory address space with others. Example : If you open Notepad, you’ll see a process named notepad.exe in the task manager under the Processes tab. Opening another instance of Notepad will create a second notepad.exe process. Each process has its own memory space, and communication between processes happens through mechanisms like inter-process communication (IPC). 2. Thread : A thread exists within a process and shares the process’s memory address space with other threads. Multiple threads within the same process can access shared memory, and when o...

Memory leaks can arise in various ways depending on how they are introduced, but they all exhibit a similar behavior, which ultimately makes them a performance issue. To demonstrate, consider a simplified example involving a process. Imagine you're testing a function on a web server, where the process executes a computation and returns a result. During this, it requests a chunk of memory. However, due to a bug, the process mistakenly requests the same chunk twice. When the result is returned, it only releases one chunk, leaving the other unreleased. Each time this function is called, the same issue occurs, and over time, the process consumes more of the host machine’s available memory, leading to a "memory leak." As this continues, the host system is impacted because the memory that should have been freed is never returned, causing the process to consume excessive resources until it either crashes or is forcefully terminated. In my experience, these types of memory leaks...

Performance Testing for IoT (Internet of Things) involves evaluating the efficiency, responsiveness, scalability, and reliability of IoT systems and devices under various conditions. IoT systems typically consist of interconnected devices, sensors, networks, and applications that collect, process, and exchange data. Performance testing ensures that these systems function optimally under different loads and scenarios, providing a seamless experience to users and maintaining system integrity. Key Aspects of IoT Performance Testing Latency : Measuring the time it takes for data to travel from an IoT device to the cloud or server and back. Throughput : Assessing the amount of data that can be processed by the system in a given period. Scalability : Ensuring the system can handle an increasing number of devices and data without degradation in performance. Reliability : Testing the system's ability to perform consistently under different conditions, including network instability or devi...

 A test plan gives complete information on the testing scope, timeline, and strategy. Here's a step-by-step guide to help you create an effective performance test plan: 1. Define the Purpose and Scope: Start by clearly stating the objectives of your performance testing. What are you trying to achieve, and what aspects of your application will you be testing (e.g. load, stress, scalability, or endurance)? 2. Identify Stakeholders: List all the individuals and teams involved in the performance testing process, including developers, QA engineers, system administrators, and project managers. Define their roles and responsibilities. 3. Set Performance Goals: Define specific performance goals and acceptance criteria. These could include response time thresholds, throughput requirements, error rates, and resource utilization targets. Make sure these goals align with business expectations. 4. Determine Performance Metrics: Select the performance metrics and key performance indicators (KPI...

The CPU (Central Processing Unit) plays a vital role in determining the performance of an application. Here's how the CPU affects application performance: Processing Power: The CPU is responsible for executing instructions and performing calculations required by the application. A more powerful CPU with a higher clock speed and more cores can handle complex computations and process instructions faster, leading to improved application performance. Response Time: The CPU speed directly influences the response time of an application. A faster CPU can execute instructions more quickly, resulting in reduced response times and faster application performance. Multitasking and Parallel Processing: Modern CPUs with multiple cores allow for the concurrent execution of multiple tasks or threads. They can handle parallel processing, enabling applications to perform multiple operations simultaneously and improve performance. Bottlenecks: CPU bottlenecks can occur when the CPU becomes overwhelme...

Performance testing is essential for any application for the following reasons: Evaluate System Responsiveness: Performance testing helps assess the responsiveness of an application under different loads. It measures the response time and throughput to ensure that the application performs within acceptable limits, providing a smooth and satisfactory user experience. Identify Bottlenecks and Performance Issues: Performance testing helps identify performance bottlenecks, such as slow database queries, inefficient code, resource constraints, or network latency. By pinpointing these issues, you can optimize the application to improve performance and scalability. Ensure Scalability and Capacity Planning: Performance testing allows you to determine the scalability of an application. It helps determine how well the application can handle increased loads and whether additional resources or infrastructure upgrades are required to support growing user demands. Capacity planning ensures that the ...

When troubleshooting high memory utilization during performance testing, it's important to identify the underlying causes and take appropriate steps to address the issue. Here are some steps to troubleshoot high memory utilization: Monitor Memory Usage: Use performance monitoring tools to track memory usage over time. Monitor both physical and virtual memory (RAM) to identify if memory consumption is exceeding available resources. Identify Memory-Intensive Processes: Identify the specific processes or components that are consuming a significant amount of memory. Performance monitoring tools can help you identify the memory-hungry processes. Look for any particular application, service, or module that stands out in terms of memory usage. Analyze Code and Memory Allocation: Review your application's code and algorithms to identify any memory leaks, inefficient memory allocation, and excessive object creation. Look for areas where large amounts of memory are being consumed unneces...

When troubleshooting high CPU utilization during performance testing, it is important to identify the root cause of the issue. Here are some steps you can take to troubleshoot and address high CPU utilization: Monitor System Resources: Use performance monitoring tools to track CPU usage over time. Monitor other system resources such as memory, disk I/O, and network usage to understand if any of these factors are contributing to high CPU utilization. Identify CPU-Intensive Processes: Identify the specific processes or components that are consuming a significant amount of CPU resources. Performance monitoring tools can help you pinpoint the culprits. Look for any particular application, service, or module that stands out in terms of CPU usage. Analyze Code and Algorithms: If your application has custom code or algorithms, review them for any inefficiencies or performance bottlenecks. Look for areas where CPU-intensive operations are performed frequently or where large amounts of data ar...

Basic performance testing checks that a page responds quickly for a few users. Advanced performance testing asks harder questions: how does the whole system behave under realistic load, where does it break, why, and what should change? 1. Plan around the architecture Start by understanding how the application is built: web servers, application servers, databases, caches and third-party services. Then pick the critical business scenarios, set measurable goals (for example, "95% of checkouts complete within 3 seconds at 500 concurrent users"), and choose the tools and test types that fit. For a full template, see How to write a Performance Test Plan . 2. Design realistic scenarios A test is only useful if it looks like real usage. Model your user profiles, the mix of transactions, realistic think time and pacing, and production-sized data volumes. Many results that look fine in testing fall apart because the test was too simple. See Why Your Load Test Results Don'...

In performance testing , various key metrics are measured to evaluate the behavior and effectiveness of a system under different loads. The selection of specific metrics may vary depending on the type of application and its intended use. Here are some commonly used performance metrics: Response Time: The time taken for the system to respond to a user request. It includes the server processing time, network latency, and client-side rendering time. Throughput: The number of transactions or requests processed by the system within a given time frame. It indicates the system's capacity to handle concurrent user requests. Concurrent Users: The number of simultaneous users or connections the system can support without significant degradation in performance. Error Rate: The percentage of failed or erroneous transactions or requests compared to the total number of transactions. It helps identify system stability and reliability. CPU Usage: The percentage of CPU resources utilized by the sys...

How to compare a string in Loadrunner script? -There are various methods to compare a string, in below example we have used "strcmp" to compare two values. We have captured a string in "pComparisonString" parameter and comparing it with ABC. Lets say if you have captured some string using correlation and want to compare if the captured string meets the condition then only pass the transactions else fail the transaction: if (strcmp(lr_eval_string("{pComparisonString}"),"ABC") == 0) { lr_output_message("The parameter value is %s", lr_eval_string("{pComparisonString}")); lr_end_transaction("Transaction_Failed",LR_FAIL); } else { lr_error_message("No parameter value captured."); lr_end_transaction("Transaction_Passed",LR_PASS); } -------------------------------------------------------- strcmp- String comparison function. pComparisonString- String which we have captured for compar...

We spend most of our time by browsing Facebook, watching videos, uploading photos and many other online activities. Have you ever noticed if you are watching a video and all of sudden the website goes down? If you are trying to book an online ticket and on the payment page, the website stops responding? If you are adding the last available item on sale in the shopping cart and the website gets hung? Isn't it stressful? For every growing business, it's very crucial to keep a tap on the performance of their application. If a website takes more than 2 seconds to respond then users tend to move to some other website which can cause a huge business loss.  Let's say you are on a retail website and browsing for some item that takes around 2-3 seconds to respond to each click or loading the page. In most cases, the user will go for some other website. Performance Testing gives an idea about how the system is going to behave on load and can help the business owners to ...

What is pacing? Pacing is the wait between two iterations of a script. Once an iteration finishes, the script pauses for a set amount of time (the pacing time) before it starts the next one. When you record a script, LoadRunner generates three default blocks: vuser_init , Action and vuser_end . Pacing takes effect after the Action block, holding the script back before it repeats. The default blocks LoadRunner generates are shown below. Actions marked in red What is think time? Think time copies real user behavior by adding a delay between two actions or transactions, like the pause a person takes before clicking the next link. Can think time replace pacing? In an interview, I was once asked: if you add think time at the end of a script, won't it work as pacing too? If so, why did HP provide two separate settings for the same job? They are not the same, because each has a different purpose. With think time, the thread stays active and LoadRunner waits for the ...

The auto correlation rule in VUGen 12 sometime fails to capture the values due to maximum length limit. You can increase the maximum length by: Increase the length of ‘MaxParamLen=4096’ to 99999 in ‘correlationsettings.xml’ file in \\vuGen\config folder. Save the changes and restart the VUGen, the auto correlation rule will capture the data.

We can use the below function to print a value in notepad using append mode, it will keep on adding the data in notepad. This function can be used for data filtration from a parameter file. long fd; char buff[100]; web_reg_save_param("cRequest_id","LB=---","RB=---","ORD=1",Last); //Before closing the Action strcpy( buff,lr_eval_string("{cRequest_id}") ); fd=fopen( "C:\\NotepadName.txt","a" ); fprintf( fd,"%s\n",buff ); fclose( fd ); } In this example we are capturing the cRequest_id from correlation and printing in notepad.

SAP GUI Text check (Error Handling) sapgui_status_bar_get_text("paramStatusBarText",                    BEGIN_OPTIONAL,                              "Recorded status bar text: Material document 4900089763 posted",                              "AdditionalInfo=sapgui1067",                    END_OPTIONAL);           lr_end_transaction("S31_SAP_Fleet_MIGO_1090_ClkPost", LR_AUTO); i = sapgui_status_bar_get_param("1", "param1", LAST); lr_output_message("Material Document Number is: %s",lr_eval_string("{param1}")); if(i==1)            { lr_output_message("Material Document creation failed and input data used is : %s", lr_eval_string("{pUserName}"));       ...

When a script logs in, a request can return a successful HTTP status even though the login failed. The server might send back an error page that still says "200 OK". A text check solves this by looking for text that appears only when the step worked, such as "Home Page" after a successful login. Check the result with if/else web_reg_find searches the next response for your text and saves how many times it was found. Register it before the step it checks. web_reg_find("Text=Home Page", "SaveCount=home_count", LAST); lr_start_transaction("login"); web_submit_data("login", "Action=http://example.com/login", "Method=POST", ITEMDATA, "Name=username", "Value={pUserName}", ENDITEM, "Name=password", "Value={pPassword}", ENDITEM, LAST); if (atoi(lr_eval_string("{home_count}")) > 0) { lr_output_message("Logon succes...

Sometimes you need to keep a value that your script captured, such as an order ID or a session token, so you can look at it after the test. This code saves a captured value to a text file, one line per iteration. long fd; // Capture the value into the parameter "saveID". // LB and RB are the text just left and right of the value. web_reg_save_param("saveID", "LB=---", "RB=---", "ORD=1", LAST); web_url("Home", "URL=http://example.com/", LAST); fd = fopen("C:\\NotepadName.txt", "a"); // "a" adds to the end of the file if (fd == 0) { lr_error_message("Could not open the file"); return -1; } fprintf(fd, "%s\n", lr_eval_string("{saveID}")); // write the value, then a new line fclose(fd); // close the file What each part does web_reg_save_param captures a value from the next response. Set LB a...