Load Testing SharePoint 2010 with Visual Studio Team Test

 

So exactly what do we mean by "load testing" when it comes to SharePoint 2010? There are lots of methods that people tend to point towards, and I’ve heard "hits/visits per day" and "throughput" bandied about, but at the end of the day it comes down to 2 things:

 

  1. Requests Per Second

The requests per second literally means how many requests for information each server is capable of responding to per second. Each page may consist of dozens of artifacts, and for each artifact the browser needs to make a "request", therefore the more of these  "requests" it can serve the better.

 

  1. Server Response Time.

The response time represents any processing on the server side (or TTLB – Time to Last Byte). This doesn’t factor in network latency or bandwidth though!

 

So the first thing you should think about is what can influence those metrics? And you end up with 5 different elements of your SharePoint 2010 farm:

  • WFE
  • Storage
  • Network
  • App Servers
  • SQL

 

This, as I’m sure you can imagine, can involve a LOT of testing. Simply testing the WFE on their own is going to be struggle for your average developer, and if you don’t have any industry testing experience you are going to have a hard time, but this is where the new SharePoint 2010 wave continues to make it’s presence felt. ..

 

SharePoint 2010 Load Testing Toolkit

This is a new set of tools being released with the SharePoint 2010 Administration Toolkit and represents the easiest possible way of load testing your SharePoint environment. The main objective here is to:

 

  • Standardise and simplify the cost of load testing.
  • Simulate common SharePoint operations
  • Be used as reference to create other custom tests (for custom code, for example!)

 

The whole thing relies on the IIS analysis logs. These logs give pointers on where users are going, what kinds of requests they are doing (GET / PUT) as well as the types of files they are typically accessing (ASPX / CSS / JS / JPEG / DOCX / etc…)

 

The Load Testing Toolkit will analyse your IIS logs and automatically generate a set of loads tests to appropriately match your environment, producing automated scripts that can be run in Visual Studio (either Team System or Team Test Edition).

 

How hard can it be?

It is really quite simple (well, according to the ridiculously simple explanation at the SharePoint 2009 conference!). You literally point the tool at your IIS logs, and it spits out an entire suite of tests, for WFE, SQL, Storage, etc .. Including all the metrics you would want (from CPU, RAM, Network, Disk I/O and even SQL , ASP.Net and .Net Framework specific performance counters).

 

Then you just run it and analyse the results!

 

Analyse That!

The analysis couldn’t be simpler. With "Requests Per Second" and "Response Times" two of the metrics generated by the Visual Studio test reports, you really can’t go far wrong.

 

If you do find a problem, then you can delve into the new SharePoint 2010 "Usage Database" (which now runs on SQL Server) in order to identify exactly what was causing your dip in performance (say when someone deletes a large list?).

 

Tips and Tricks

There are a few gotchas, one thing is to be careful of "Validation Rules" in Visual Studio. Typically it will be happy with pages that return "200" codes. This of course includes Error and Access Denied pages (which SharePoint will handle, and returns a perfectly valid page (hence the 200 code!)).

 

It is also recommended that you let your test "Warm up" for around an hour before you start taking the results seriously.  This allows all of the operations, timers and back-end mechanics of SharePoint to properly settle down, and means you are getting a realistic experience of what the environment will react like once it is bedded into it’s production environment.

 

Finally, the SharePoint Usage Logging Database is a great location to grab information out of, so why not leverage other great aspects of the Office 2010 family. You could pull through the Usage DB information into Excel 2010 (perhaps using PowerPivot?) so that you can spin out charts and pivot tables to easily drill down into your data.

 

Typically load testing tells you WHEN bottlenecks are occurring, but the Usage Database can tell you WHAT is causing the bottlenecks!

  • Martin Hatch

    Raja,

    There are plenty of materials on the internet about how to get started with using Visual Studio load testing tools.

    Have you had any luck with TechNet?

  • Raja Sekhar Reddy Vardhireddy

    I am trying to do a performance testing for a share point application in Visual studio 2010. I am new to testing and have a very limited idea on testing. Please can you give me the inputs that I need to give for a visual studio 2010 ultimate so that the performance testing for a share point application can be performed. Can please list the steps that I need to perform to perform the testing Please! Thanks in advance.

  • Martin Hatch

    Anon, not sure what you mean by "office"?
    Do you mean Office Web Apps?

    I can imagine that (and other "thin client applications" that run on silverlight) would be difficult to test. To be honest I'm not sure… might have to do some research there ..

  • Anonymous

    any idea how can we test office in browser ? Load test throws errors with requests coming back with invalid header information. Works fine when running individually or after generating the cache

  • Martin Hatch

    Donna (and Joe),

    The Visual Studio "Web Performance Test" and "Load Test" objects allows you to create very detailed and complex tests (or just very simple "load the home page" tests).

    This is entirely customisable, and the interface is very simple to use.

    So if you don't have any IIS logs, you can just build your own tests from scratch. (Joe – you'd also have to do this approach for MOSS 2007 too).

  • Anonymous

    Martin,

    If you would like to test a new system before implementing then you wouldn't have any IIS logs to analyse and automatically generate a set of loads tests. What do you sugest for this situation? Is there an option to specify an estimated workload?

    -Donna

  • Anonymous

    Hi Martin,

    Great write-up on SP 2010 load testing. Do you recommend any tools for Moss 2007 load testing?

    Thanks!

    Joe

  • Martin Hatch

    The Load Testing Toolkit will be shipped with the SharePoint 2010 Administrative Toolkit.

    I'm guessing that won't be released until the product goes RTM (which is frustrating I know) but you never know, they might release a beta version of the admin toolkit and include it there! 09

  • No Name

    Impressive. Any idea if the SharePoint 2010 Load Testing Toolkit will be released with the beta?