Tag Archives: Visual Studio

Loadtesting SharePoint file uploads with mandatory fields in the content types

When a user uploads a document into a library it can be the case that mandatory fields need to be filled in. If these fields aren’t correctly submitted to the server then the file will remain checked out and other users will not be able to see it.

For an automated load test its important that it correctly supplies the required fields. In a previous post I described all the requests that are involved in the upload. In this post we’ll describe how to correctly POST the fields to the server.

Lets assume you have a required field called “ReferenceId” in your content type. When we want to POST the value of this field to SharePoint we must include SharePoint’s Field ID (a GUID) into the name of the form post parameter.

The first point where we need to intervene is the GET request to EditForm.aspx. Its response contains the ID of each field that’s included in the content-type. Each meta-data field of the content type is described in a JSON datastructure that looks like this:

//This JSON structure is sent as 1 very long line in the response
{
   "ListSchema":{
      "...":{
         "Id":"aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa",
         "...":"..."
      },
      "ReferenceId":{
         "Id":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
         "...":"..."
      },
      "...":"..."
   }
}

You can use the following ExtractionRule to extract the Id of your ReferenceId field:

Property Value
Context Parameter Name GUID_ReferenceId
Ends With
Extract Random Match False
Html Decode True
Ignore Case False
Index 0
Required True
Search in Headers False
Starts With “ReferenceId”:{“Id”:”
Use Regular Expression True

On the POST request to EditForm.aspx you need to add an extra Form Post Parameter like this:

Parameter Value Explanation
ClientFormPostBackValue_{{GUID_ReferenceId}}_ReferenceId Insert your desired value for the ReferenceId here See how the name of the form post parameter is constructed from the GUID we extracted earlier

Running unittests as part of a loadtest in Visual Studio

Did you know that the loadtest framework isn’t limited to only running webtests or calling web-services? You can also include your unittests! This is very convenient for loadtesting MVC controllers, WebApi controllers or SharePoint’s CSOM interface.

Your unittests should work without any changes. If you need one of the following then your unittest needs some minor modification:

  • Work with loadtest transactions.
  • Use information such as the UserId or AgentName.

Below is a small example class that shows how to achieve the above without sacrificing the ability to run the class as part of a standalone unittest. Just inherit from it and use its methods.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Microsoft.VisualStudio.TestTools.LoadTesting;

namespace Loadtesting
{
    [TestClass()]
    public class CombinedUnitLoadTest
    {
        /// <summary>
        /// Visual Studio's Unit Test Context. Microsoft's UnitTestFramework sets this member
        /// </summary>
        public TestContext TestContext { get; set; }
        
        /// <summary>
        /// Load Test Context. Only present if the testcase is being run in a loadtest
        /// </summary>
        public LoadTestUserContext LoadTestUserContext
        {
            get
            {

                if (this.TestContext.Properties.Contains("$LoadTestUserContext"))
                {
                    return this.TestContext.Properties["$LoadTestUserContext"] as LoadTestUserContext;
                }
                else
                {
                    return null;
                }
            }
        }
        /// <summary>
        /// The name of the Agent where the testcase is running. Returns "" if the test is not running on an agent
        /// </summary>
        public string AgentName
        {
            get
            {
                if (this.TestContext.Properties.Contains("AgentName"))
                {
                    return this.TestContext.Properties["AgentName"].ToString();
                }
                return "";
            }
        }
        /// <summary>
        /// The Id of the virtual user that is running the testcase. Returns "" if the test is not running in a loadtest
        /// </summary>
        public string UserId
        {
            get
            {
                if(null != this.LoadTestUserContext)
                {
                    return this.LoadTestUserContext.UserId.ToString();
                }
                return "";
            }
        }
        /// <summary>
        /// Returns a number that specifies how many tests the current virtual user has already executed
        /// </summary>
        public string Iteration
        {
            get
            {
                if(null != this.LoadTestUserContext)
                {
                    return this.LoadTestUserContext.CompletedTestCount.ToString();
                }
                return "";
            }
        }
        /// <summary>
        /// Starts a named transaction. The loadtest framework will report the time between BeginTransaction() and EndTransaction()
        /// </summary>
        /// <param name="name"></param>
        public void BeginTransaction(string name)
        {
            if(null != this.LoadTestUserContext)
            {
                this.TestContext.BeginTimer(name);
            }
        }
        /// <summary>
        /// Ends the named transaction. The loadtest framework will report the time between BeginTransaction() and EndTransaction()
        /// </summary>
        /// <param name="name"></param>
        public void EndTransaction(string name)
        {
            if(null != this.LoadTestUserContext)
            {
                this.TestContext.EndTimer(name);
            }
        }
    }
}


What to do when Visual Studio keeps opening your webtests in the wrong type of editor

Is Visual Studio opening your .webtest files in the XML editor instead of the GUI? You can fix this by editing the .csproj file of the project that contains the .webtests.

Find the line for your webtest and remove the following line below it:

<SubType>Designer</SubType>

Restart Visual Studio and the next time you double click on the test, it will be opened in the GUI editor.

a loadtest plugin to generate a GUID per request

Visual Studio has an out-of-the box plugin that generates GUIDs for you. However its a WebTest plugin, so it will only generate them once per webtest. You can’t include that plugin into a loop.

Fortunately we can write a simple WebTestRequest plugin that does what we want:

using System;
using System.ComponentModel;
using Microsoft.VisualStudio.TestTools.WebTesting;
using Microsoft.VisualStudio.TestTools.WebTesting.Rules;

namespace LoadTestPlugins
{
    [DisplayNameAttribute("Generate GUID")]
    [DescriptionAttribute("Generates a GUID during a request")]
    public class GenerateGUID : WebTestRequestPlugin
    {

        #region Methods
        private void GenerateIt(WebTestContext Context)
        {
            Context[this.ContextParameterName] = Guid.NewGuid().ToString();
        }

        public override void PostRequest(object sender, PostRequestEventArgs e)
        {
            if(this.BeforeRequest == false)
            {
                this.GenerateIt(e.WebTest.Context);
            }
        }

        public override void PreRequest(object sender, PreRequestEventArgs e)
        {
            if(this.BeforeRequest == true)
            {
                this.GenerateIt(e.WebTest.Context);
            }
        }
        #endregion

        #region Properties
        [Description("Name of context parameter to store the number in"),
        DisplayName("Context parameter"), IsContextParameterName(true)]
        public string ContextParameterName {get; set;}

        [Description("Whether to generate the number before or after the request has executed"),
        DisplayName("Apply before request"),
        DefaultValue(true)]
        public bool BeforeRequest { get; set; }
        #endregion
    }
}

A loadtest plugin to clear context parameters (and why we need it)

If a webtest stores temporary data in a context parameter, then you need to make sure that its reset to some initial state when the webtest starts. Otherwise it will fail if that webtest is included multiple times within the same testrun.

Visual Studio’s “Set Context Parameter Value” plug-in won’t help here as it cannot set the value of a parameter to an empty string. If you try this, then during the test run, the request will fail with the error “Request failed: Missing or invalid Context Parameter Name.”

The solution in my case was a very simple custom plug-in. I choose to implement it as a WebTestPlugin instead of a WebTestRequestPlugin to avoid confusing it with Visual Studio’s existing plugin.

using System.ComponentModel;
using Microsoft.VisualStudio.TestTools.WebTesting;
using Microsoft.VisualStudio.TestTools.WebTesting.Rules;

namespace LoadTestPlugins.WebTestPlugins
{
    [DisplayNameAttribute("Set context parameter value")]
    [DescriptionAttribute("Sets the context parameter to a value")]
    public class ClearContextParameter : WebTestPlugin
    {
        public override void PreWebTest(object sender, PreWebTestEventArgs e)
        {
            e.WebTest.Context[this.ContextParameter] = this.Value;
        }

        #region Properties
        [Description("Context parameter to set"),
        DisplayName("Context parameter"), IsContextParameterName(true)]
        public string ContextParameter { get; set; }

        [Description("Value to place into the context parameter"),
        DisplayName("Value")]
        public string Value { get; set; }
        #endregion
    }
}

SharePoint people-pickers in loadtests

A significant number of the webtests in out loadtest need to simulate a human tying into a people-picker control. This control waits until the first few letters have been entered and after that calls SharePoint’s client.svc web-service every time the user enters data into the control.

I simulate this with the following logic in webtests:

GET https://...
For Each Character In Targetusers display name:
    POST https://.../_vti_bin/client.svc/ProcessQuery
GET https:/...

As Visual Studio doesn’t contain a plugin that iterates over characters in a string, I wrote my own. This plug-in allows you to choose:

  1. Which input parameter contains the display name to iterate over.
  2. Into which output parameter to place the sub-string.
  3. At which index to start iterating. The 1st iteration of the loop includes all characters upto the starting index. All subsequent iterations add one more letter from the input string into the output string.

Some nice-to-have improvements to this plugin are:

  • Add a configurable delay in milliseconds to each loop. This would be nice as think-times on requests can only be configured in intervals of whole seconds
  • Add an option to automatically clear the output parameter once all iterations have finished

Here is the code for the plug-in:

using System.Collections.Generic;
using System.ComponentModel;
using Microsoft.VisualStudio.TestTools.WebTesting;
using Microsoft.VisualStudio.TestTools.WebTesting.Rules;

namespace LoadTestPlugins
{

    [DisplayNameAttribute("For Each character in context parameter")]
    [DescriptionAttribute("Sequentially copies each character from the source into the destination. You can use this to simulate a user that types a name into a peoplepicker for example")]
    public class ForEachCharacter : ConditionalRule
    {

        #region Methods
        public override void CheckCondition(object sender, ConditionalEventArgs e)
        {
            string Input = e.WebTest.Context[this.InputContextParameterName].ToString();
            string Output;
            try
            {
                Output = e.WebTest.Context[this.OutputContextParameterName].ToString();
            }
            catch (KeyNotFoundException)
            {
                Output = string.Empty;
            }


            //Only do something if the output is not yet complete
            if (Input.Length == Output.Length)
            {
                e.IsMet = false;
                return;
            }
            

            int LengthToCopy;
            if(Output.Length == 0)
            {
                //Copy all characters before the starting index and 1 extra
                LengthToCopy = this.StartIndex + 1;
            }
            else 
            {
                //Copy 1 extra character to the output
                LengthToCopy = Output.Length + 1;
            }

            e.IsMet = true;
            e.WebTest.Context[this.OutputContextParameterName] = Input.Substring(0, LengthToCopy);
            return;
        }
        
        #endregion

        #region Properties
        [Description("Name of context parameter that contains the characters to iterate over"),
        DisplayName("Input Context parameter"), IsContextParameterName(true)]
        public string InputContextParameterName {get; set;}

        [Description("Name of context parameter to store the each successive iteration"),
        DisplayName("Ouput Context parameter"), IsContextParameterName(true)]
        public string OutputContextParameterName { get; set; }

        [Description("The zero-based index to start the iteration from"),
        DisplayName("Start Index")]
        public int StartIndex { get; set; }
        #endregion
    }
}

A loadtest plugin to select the transition from a probability matrix

An important part of loadtesting with Markov chains is choosing which transition the test should take. This boils down to generating a random number and then seeing in which of the transitions in the probability matrix that number belongs to.

In a previous post I showed how to generate a random number. In this post we’ll see how we can decide which transition to actually take based upon the probabilities from the matrix.

Webtests can include various types of plugins that execute requests based on some boolean condition. These type of plugins are called ConditionalRule plugins. I created a simple plugin that determines if the random number falls within a user specified range. This allows me to implement the following kind of logic in my webtest:

Picture showing how the plug-in is included in a .webtest

Here is the code for the plugin:

using System;
using Microsoft.VisualStudio.TestTools.WebTesting;
using System.ComponentModel;
using Microsoft.VisualStudio.TestTools.WebTesting.Rules;

namespace LoadTestPlugins
{

    [DisplayNameAttribute("Number in range")]
    [DescriptionAttribute("Determines if the number in the context parameter is in the specified range")]
    public class NumberInRange : ConditionalRule
    {

        #region Methods
        public override void CheckCondition(object sender, ConditionalEventArgs e)
        {
            Int32 Input = Convert.ToInt32(e.WebTest.Context[this.ContextParameterName]);
            e.IsMet = (Input >= this.LowerBound && Input <= this.UpperBound) ? true : false;
            return;
        }

        public override string StringRepresentation()
        {
            return string.Format("{0} (Number in range [{1} - {2}])"
                , this.Description
                , this.LowerBound
                , this.UpperBound);
        }
        
        #endregion

        #region Properties
        // Properties
        [Description("Name of context parameter that contains the number"),
        DisplayName("Context parameter"), IsContextParameterName(true)]
        public string ContextParameterName {get; set;}

        [Description("The lower bound"),
        DisplayName("Lower bound")]
        public Int32 LowerBound { get; set; }


        [Description("The upper (inclusive) bound"),
        DisplayName("Upper bound")]
        public Int32 UpperBound { get; set; }

        [Description("Your description of this rule"),
        DisplayName("Description")]
        public string Description { get; set; }
        
        #endregion
    }
}

A loadtest plugin to simulate random chance

An important part of loadtesting with Markov chains is choosing which transition the test should take. This boils down to generating a random number and then seeing in which of the transitions in the probability matrix that number belongs to.

The plugin code below generates a random number between your specified lower- and upper bound and places it into the specified context parameter. Its very similar to the out-of-the box plugin. However Visual Studio’s plugin can only be executed once when a webtest starts. I needed a new random number every time that my loops executes.

using System;
using System.ComponentModel;
using Microsoft.VisualStudio.TestTools.WebTesting;
using Microsoft.VisualStudio.TestTools.WebTesting.Rules;

namespace LoadTestPlugins
{

    [DisplayNameAttribute("Generate random number")]
    [DescriptionAttribute("Generates a random number during a request")]
    public class GenerateRandomNumber : WebTestRequestPlugin
    {

        #region Methods
        private void GenerateIt(WebTestContext Context)
        {
            Context[this.ContextParameterName] = new Random().Next(this.LowerBound, this.UpperBound);
        }

        public override void PostRequest(object sender, PostRequestEventArgs e)
        {
            if(this.BeforeRequest == false)
            {
                this.GenerateIt(e.WebTest.Context);
            }
        }
        
        public override void PreRequest(object sender, PreRequestEventArgs e)
        {
            if(this.BeforeRequest == true)
            {
                this.GenerateIt(e.WebTest.Context);
            }
        }
        #endregion

        #region Properties
        [Description("Name of context parameter to store the number in"),
        DisplayName("Context parameter"), IsContextParameterName(true)]
        public string ContextParameterName {get; set;}

        [Description("The lower bound"),
        DisplayName("Lower bound"),
        DefaultValue(1)]
        public Int32 LowerBound { get; set; }


        [Description("The upper (inclusive) bound"),
        DisplayName("Upper bound"),
        DefaultValue(100)]
        public Int32 UpperBound { get; set; }

        [Description("Whether to generate the number before or after the request has executed"),
        DisplayName("Apply before request"),
        DefaultValue(true)]
        public bool BeforeRequest { get; set; }
        #endregion
    }
}

A loadtest plugin to extract querystring parameters from responses

When a user creates a new casefile in our system. Its created as a custom content-type that inherits from SharePoint’s document set. My load tests need at least the following information in order to continue working with that casefile:

  1. The title of the document set.
  2. SharePoint’s ID of the content-type.
  3. SharePoint’s ID of the list item.
  4. SharePoint’s ID of the list where its created in.

The POST to a MVC controller is the starting point for our system to create the casefile. The only way to find out what the values are is by following a URL contained within the HTTP 200 response (looks like this)

<script type="text/javascript">window.parent.location.href="https://...../DocSetHome.aspx?......";</script>

By doing a GET request to that URL, we get a HTTP 302 response, Visual Studio automatically follows the redirect and the final HTTP 200 response’s URI contains all the dynamic values as querystring parameters:
https://……/docsethomepage.aspx?ID=…&FolderCTID=…&List=…&RootFolder=…&RecSrc=…

Visual Studio doesn’t have an out-of-the-box plugin that can extract the values of those querystring parameters. So I created my own:

using System;
using System.Linq;
using Microsoft.VisualStudio.TestTools.WebTesting;
using System.Globalization;
using System.ComponentModel;
using System.Web;
using System.Collections.Specialized;
using Microsoft.VisualStudio.TestTools.WebTesting.Rules;

namespace LoadTestPlugins
{
    [DisplayNameAttribute("Extract querystring parameter")]
    [DescriptionAttribute("Extracts the value of a querystring parameter from the URI of a response")]
    public class ExtractQuerystringParameter : ExtractionRule
    {
        //Which field to parse and store in the context
        [Description("Name of the querystring parameter to extract"),
        DisplayName("Field name")]
        public string FieldName {get; set;}


        public override void Extract(object sender, ExtractionEventArgs e)
        {
            if (e.Response.ResponseUri != null)
            {
                NameValueCollection NVC = HttpUtility.ParseQueryString(e.Response.ResponseUri.Query);
                string value = NVC[this.FieldName];
                if(!string.IsNullOrEmpty(value))
                {
                    //The last part of the RootFolder or RecSrc contains the name of the 
                    //documentset. It needs to be Url encoded as it might contain spaces  
                    if (   this.FieldName.ToLower().Equals("RootFolder".ToLower())
                        || this.FieldName.ToLower().Equals("RecSrc".ToLower()))
                    {
                       value = Uri.EscapeUriString(value.Split('/').Last());
                    }
                    e.WebTest.Context.Add(this.ContextParameterName, value);
                    e.Success = true;
                    return;
                }
                else
                {
                    e.Success = false;
                    e.Message = string.Format(CultureInfo.CurrentCulture
                        , "Querystring Parameter Not Found: {0}"
                        , this.FieldName);
                }
            }
        }
    }
}

Loadtesting using Markov chains to simulate user behavior

One of my clients has a rather large SharePoint farm and uses the add-in / provider-hosted-app model a lot. Everyone who looks at the topology and machine specs, expects the farm to provide a ‘huge’ amount of performance. In practice the users report that its rather slow.

The guys in maintenance had done some rudimentary measurements and didn’t really see any reason to doubt the machine specs. So we were wondering…

how come that users are perceiving a lack of performance? Are we measuring correctly? How do we compare the farm’s performance before and after major changes? If we fix bottlenecks, how far can we push the farm?

These are different questions, that all need the results from a ‘proper’ loadtest. So what is a ‘proper’ loadtest and how do we build that?

Besides the criteria described in My Definition of Done for performance and loadtest cases, a proper loadtest needs to simulate realistic behavior patterns. In our case that means:

  • Users will choose to sequentially do many different actions in a short time period.
  • Some actions are usually only done by some types of users. Some actions are usually done by all types of users.
  • Action sometimes depend on data that was generated during an earlier action.
  • Some actions are always followed by another type of action.
  • Its not fixed exactly in which order and when the actions will be executed. Its dependent on probabilities.
  • We have a rough idea what the probabilities are, but we expect these probabilities to change as time progresses and our knowledge increases. We don’t want to invest significant time to change the tests.

In order to satisfy the above, I decided that a plain-old operational profile wasn’t suitable. We needed to combine it with a Markov chain (see Kozialek’s paper on Operational Profiles). Also I wanted the loadtest to decide during run-time which path through the chain would be taken. I really didn’t want to generate the chains at design time and have to redo them once the probabilities are measured more accurately. Visual Studio doesn’t support this out of the box, so I had create some plug-ins that would achieve this:

Plugin Type Purpose
GenerateRandomNumber WebTestRequestPlugin Generates a random number between a user defined lower- and upperbound. Visual Studio only had a plugin that would generate the number once during a test. I needed to generate it every iteration within a single test.
NumberInRange ConditionalRule Executes a block of requests only when the input number falls in a user defined range. This lets me dynamically select the transtions/action in the Markov chain like in the following pseudo-code:

probability = GenerateRandomNumber(1,100)
if probability between 1 and 5: do action1; stay in current state;
if probability between 6 and 90: do action2; go to state xxx;
if probability between 91 and 98 do action3; go to state yyy;
if probability between 99 and 100: go to exit state;
ChooseDocumentSet WebTestRequestPlugin About 70% of the users will select a case-file and start working with it. This plugin queries SharePoint to find a suitable document set and ensures that this document set is never shared by multiple users.
ForEachCharacter ConditionalRule (loop) Simulates a user typing into a people-picker. Its a plugin that that executes a block of requests for each character in user defined context parameter.
GenerateGuid WebTestRequestPlugin Is able to generate a new GUID every time a certain request is executed instead on only once per test.
ExtractQuerystringParameter ExtractionRule Retrieves the value of a specific querystring parameter. My system dynamically generates information and returns a URI containg this information in various querystring parameters. My testcases needed to remember these values for later use.
ClearContextParameter WebTestPlugin Is able to clear the value of an existing context parameter each time a .webtest is called by another webtest.