I recently discovered that Visual Studio Online includes 15000 “Virtual User Minutes” for load testing each month for free. I’ve never used the load test feature of Visual Studio before, so I decided to try it to simulate high user loads on our SharePoint farm. Here is a step-by-step guide.
Requirements
- A Visual Studio Online account
- Visual Studio 2013 Ultimate (Performance and Load Testing is not included in any other edition)
- SharePoint Site which is accessible via Internet
Setup
In Visual Studio 2013 go to File->New->Project and select “Web Performance and Load Test Project” in category “Test”:
This will create an empty “Web Performance Test”. To start recording what the test should do, click “Record” and an IE Browser window will open.
Now open your SharePoint site and click around to simulate some browsing. After that click “Stop” to come back to Visual Studio where you will see all requests in your Web Performance Test.
You might not want to record some “SharePoint” noise, so there is a settings page to exclude some stuff. Go to Visual Studio Tools->Options->Web Performance Test Tools->SharePoint:
To execute this Web Performance Test as a load test with simulated users, you have to add a “Load Test” to your project:
The wizard will guide you through some configuration options. Set the load to 250 users as shown below for a test running one minute will cost you 2500 virtual user minutes. So keep in mind that the free plan of Visual Studio Online gives you 15000 virtual user minutes per month. Be careful with the constant loads settings, because all virtual users will submit the first request as fast as possible, so the result may not reflect real usage scenarios. For more users use the “Step” pattern to start with a few users. They will automatically increase over time.
I set the test mix model to “Based on sequential test order” because there is only one test in my project which every virtual user should run. In the next step add the existing test to the load test.
I leave Network Mix, Browser Mix and Counter Sets untouched. In Run Settings I set the “Load test duration” to 10 which means, every virtual user will execute the test in a loop for 10 minutes.
With these settings, your “Virtual User Minute” budget of 15000 in Visual Studio Online will decrease by 2500 (250 user * 10 minutes).
Run in the Cloud
Now add a .testsettings file (Local.testsettings already exists) via Add->New Item and search for test:
Give it a name like “CloudTestSettings”. A wizard will open where you can select “Run tests using Visual Studio Team Foundation Service” (what is now “Visual Studio Online” as of November ‘13).
Make sure, your newly created .testsettings file is the active one by right click on it and select “Active Load and Web Test Settings”:
Ok, that’s basically all you have to do. If you already know some values and want to visualize threshold violations, you can open the load test and add threshold rules to one ore more counters. For example to the average page load time.
To run the load test, just click “Run Load Test”:
After the test is completed the result panel show all the details about the test. You can add counters to graphs and see a summary and details. I’m interested in the page response times, so I switch to only one graph layout. Here you can see all the different pages and there response times.
You can download the report to Excel to analyze it even further.