Single Sign On with Selenium and Firefox in a windows environment

When you run a testcase using Selenium WebDriver against a site like SharePoint, you’ll frequently see that Firefox is blocked waiting for you to enter the username and password of a user. This may seem unexpected because it even happens when your own Firefox window doesn’t need you to enter any credentials.

The reason for this is because the FirefoxDriver creates a new temporary profile and it doesn’t include the settings needed for SSO. There’s 2 ways of dealing with this.

The first way

The first way only works on the local machine that is running the tests. You can configure the FirefoxDriver to use an existing profile instead of creating a new one. The following code snippet shows how to do that in C#.

new FirefoxDriver(new FirefoxProfileManager().GetProfile("default"));

You don’t have to use the “default” profile. You’re free to create a dedicated profile for Selenium tests if you want. Just start Firefox from the command line using firefox -p to start Firefox’s Profile Manager.

The second way

The first way wont work if your tests use RemoteWebDriver to connect to a remote instance of FirefoxDriver. This is due to the fact that identically named profiles, still have a different internal name on each machine.

Instead we will configure Firefox to include the relevant SSO settings into every profile that’s created. You will need administrative credentials during this one-time configuration. Lets assume your site is called “https://myserver.contoso.local/sites/myapp”

  1. Open Windows Explorer and navigate to the Firefox installation directory. That’s usually “C:\Program Files (x86)\Mozilla Firefox”
  2. Create a file there (in this example, I used “mozilla.cfg”) with the following content:
    //
    lockPref("network.automatic-ntlm-auth.trusted-uris", "contoso.local");
    
  3. Use Windows Explorer to navigate to Firefox’s directory containing the preferences to be used for new profiles. That’s usually “C:\Program Files (x86)\Mozilla Firefox\defaults\pref”
  4. Create a file there called “local-settings.js” with the following content:
    //@line 2 "c:\builds\moz2_slave\rel-m-esr31-w32_bld-0000000000\build\browser\app\profile\channel-prefs.js"
    /* This Source Code Form is subject to the terms of the Mozilla Public
    * License, v. 2.0. If a copy of the MPL was not distributed with this
    * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
    pref("general.config.obscure_value", 0);
    pref("general.config.filename", "mozilla.cfg");
    

See Locking preferences in Firefox for details on this.

Fixing the slow combination of WebDriver and Internet Explorer.

Is your Selenium WebDriver running very slowly with Internet Explorer? Then you’re probably running a 64-bit IEDriverServer.exe with a 32 bit Internet Explorer. Even on 64-bit systems, Windows usually runs the 32-bit iexplore.exe.To solve the problem, just replace your IEDriverServer.exe with the 32 bit one and the speed increase will be enormous.

Describing large forms and data entry in Cucumber / SpecFlow feature files

Did you know that in .feature files you can use the table notation to specify each field and value in rows instead of columns?

This can make the file way more readable when a step needs a lot of different parameters. Here is an example that shows both ways of creating an invoice:

Feature: Demo

Scenario: TwoDifferentStylesOfSteps
	When I create an invoice with number '111' with '119.00' euro total, '19.00' euro VAT from 'ACME Inc' at '1234XYZ 99' in 'USA'
	When I create the following invoice:
		| Field            | Value      |
		| Number           | 222        |
		| TotalAmount      | 238.00     |
		| VATAmount        | 38.00      |
		| Debtor           | ACME Inc   |
		| DebtorAdresLine1 | 1234XYZ 99 |
		| DebtorCountry    | USA        |
	Then The systems invoice store must look like this:
		| Number | TotalAmount | VatAmount |
		| 111    | 119.00      | 19.00     |
		| 222    | 238.00      | 38.00     |
		

Here is the corresponding binding code:

namespace TableExample
{
    public class Invoice
    {
        public int Number { get; set; }
        public decimal TotalAmount { get; set; }
        public decimal VatAmount { get; set; }
        public string Debtor { get; set; }
        public string DebtorAdresLine1 { get; set; }
        public string DebtorAdresLine2 { get; set; }
        public string DebtorCountry { get; set; }
    }
  
    [Binding]
    public class TablesSteps
    {
        private IList<Invoice> Invoices;

        public TablesSteps()
        {
            this.Invoices = new List<Invoice>();
        }


        [When(@"I create an invoice with number '(.*)' with '(.*)' euro total, '(.*)' euro VAT from '(.*)' at '(.*)' in '(.*)'")]
        public void WhenICreateAnInvoiceWithNumberWithEuroTotalEuroVATFromAtIn(int Number, 
            decimal TotalAmount,
            decimal VatAmount,
            string Debtor,
            string DebtorLine1,
            string DebtorCountry)
        {
            Invoice TestInvoice = new Invoice();
            TestInvoice.Number = Number;
            TestInvoice.TotalAmount = TotalAmount;
            TestInvoice.VatAmount = VatAmount;
            TestInvoice.Debtor = Debtor;
            TestInvoice.DebtorAdresLine1 = DebtorLine1;
            TestInvoice.DebtorCountry = DebtorCountry;
            this.Invoices.Add(TestInvoice);
        }

        [When(@"I create the following invoice:")]
        public void WhenICreateTheFollowingInvoice(Table table)
        {
            Invoice TestInvoice = table.CreateInstance<Invoice>();
            this.Invoices.Add(TestInvoice);
        }

        [Then(@"The systems invoice store must look like this:")]
        public void ThenTheSystemsInvoiceStoreMustLookLike(Table table)
        {
            var rows = table.CreateSet<Invoice>();
            foreach (Invoice test in rows)
            {
                IEnumerable<Invoice> Matches = this.Invoices
                    .Where(invoice => invoice.Number.Equals(test.Number))
                    .Where(invoice => invoice.TotalAmount.Equals(test.TotalAmount))
                    .Where(invoice => invoice.VatAmount.Equals(test.VatAmount));
                Assert.AreEqual(1, Matches.Count(), string.Format("Invoice {0} not correct in invoicestore", test.Number));
            }
        }

    }

}

My Definition of Done for performance and loadtest cases

Load and performance testcases are frequently used for multiple objectives:

  • Determining where a system’s breaking point is
  • Determining if a system is able to meet a required load

Here are the criteria I use to make sure my load/performance testcases are good enough to use and maintain for all the objectives. I’ve been using this for regular load tests on a huge SharePoint farm at one of my clients.

Criteria Explanation
The following information is parametrized into context parameters:

  1. Server name(s)

  2. Site name(s)

  3. List/library/page name(s)

  4. Filenames (up- and download) specific to the testcase

User credentials are included in the testcase
Think-times on the various requests are configured conform test design I always check that the agreed thinktimes are present. I configure the loadtest to either use them all or ignore them all
The testcase includes a transaction encapsulating only the requests relevant to the scenario Test results always state how fast pages are. However, we also want to see fast the system is able to respond to user scenarios that include multiple requests
Testcase has logic to fail if generic errors / messages are returned
Testcase has logic to fail if the correct content has not been returned A simple validation rule to check for a specific string in the output is usually enough
Testcase has logic determine if a performance threshold has been exceeded
The decision to fail the testcase based on the threshold, can be enabled or disabled without changing the testcase I usually include a ‘responsetime goal’ validation rule and set its level to Low or Medium. Then at runtime, I can indicate what level of validation rules should apply to the entire loadtest. That will allow me to run the entire loadtest to stress the farm without a few performance delays causing the run to fail
Testcase is able to run standalone or as part of a loadtest with concurrent users
The testcase should only delete or modify information that it owns. If this is not possible, then the testcase must gracefully accepts errors when concurrent instances of it touch that information
Testcases that create, modify or delete information must include information that uniquely identifies the specific instance of the testcase In practice I use the following combination concatenated into a string:

  1. The id of the virtual user (WebTestUserId)

  2. The id of the iteration (WebTestIteration)

  3. The id of the agent running the test (AgentName)

Testcases conform to the testdesign in their decision to parse and execute dependent requests (.css, .js, images etc etc)

How to generate self signed certificates

Sometimes you need a self-signed certificate for the purpose of testing websites or digitally signing files such as PDFs. Your PDF software will probably ask you for a .pfx file. You can generate these on Windows, Mac or linux

On Windows


On windows, with Visual Studio, you can use the makecert utility:

cd "C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Bin\"
.\makecert.exe -r -a sha512 -len 4096 -pe -sv c:\users\gerben\documents\example.key -n "CN=Test" c:\users\gerben\documents\example.cer
.\pvk2pfx.exe -pvk c:\users\gerben\documents\example.key -spc c:\users\gerben\documents\example.cer -pfx c:\users\gerben\documents\example.pfx -po PasswordForPfxFile

Linux and Mac


Most linux and OSX machines have openssl installed by default. You can generate the various files as follows:

commandprompt> openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout example.key -out example.pem -subj "/C=NL/O=Tester/CN=example.com/emailAddress=test@example.com"
commandprompt> openssl pkcs12 -export -out example.pfx -inkey example.key -in example.pem 
#openssl will ask you for a password. Don't forget it

This will output the following:

File Description
example.key Your private key.
example.pem The certificate. This also contains your public key
example.pfx An encrypted, password protected file that contains both your private and public key

You can see your public key with the following command:

commandprompt> openssl x509 -in example.pem -pubkey
-----BEGIN PUBLIC KEY-----
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC52G+QBJsz1m/rzINSjzABnLjQ
18c+PGMn+w0CxCHsOkIGBRYP80k8+ZznhlMJ2pJ7knM5McHUuYxfBaMU1GraTjS5
c0nb/5AbPR6iWM5rI/Ha02CMmZmSsspq2RhSZZU0Buco0sAqjf9KPn6/uuoNdvTe
kDTMIH7cgB+NsJSadwIDAQAB
-----END PUBLIC KEY-----

You can see the contents of your certificate using the following command:

commandprompt> openssl x509 -in example.pem -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            fd:22:40:d8:00:b8:68:fa
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: C=NL, O=Tester, CN=example.com/emailAddress=test@example.com
        Validity
            Not Before: Apr 16 12:52:39 2015 GMT
            Not After : Apr 15 12:52:39 2016 GMT
        Subject: C=NL, O=Tester, CN=example.com/emailAddress=test@example.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
            RSA Public Key: (1024 bit)
                Modulus (1024 bit):
                    00:b9:d8:6f:90:04:9b:33:d6:6f:eb:cc:83:52:8f:
                    30:01:9c:b8:d0:d7:c7:3e:3c:63:27:fb:0d:02:c4:
                    21:ec:3a:42:06:05:16:0f:f3:49:3c:f9:9c:e7:86:
                    53:09:da:92:7b:92:73:39:31:c1:d4:b9:8c:5f:05:
                    a3:14:d4:6a:da:4e:34:b9:73:49:db:ff:90:1b:3d:
                    1e:a2:58:ce:6b:23:f1:da:d3:60:8c:99:99:92:b2:
                    ca:6a:d9:18:52:65:95:34:06:e7:28:d2:c0:2a:8d:
                    ff:4a:3e:7e:bf:ba:ea:0d:76:f4:de:90:34:cc:20:
                    7e:dc:80:1f:8d:b0:94:9a:77
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier: 
                B4:82:EE:F5:8A:75:34:82:BD:23:03:3B:96:E4:A9:AE:B7:3B:5F:A9
            X509v3 Authority Key Identifier: 
                keyid:B4:82:EE:F5:8A:75:34:82:BD:23:03:3B:96:E4:A9:AE:B7:3B:5F:A9
                DirName:/C=NL/O=Tester/CN=example.com/emailAddress=test@example.com
                serial:FD:22:40:D8:00:B8:68:FA

            X509v3 Basic Constraints: 
                CA:TRUE
    Signature Algorithm: sha1WithRSAEncryption
        76:27:e2:e6:07:a2:cd:db:3a:6a:14:f4:a1:17:8e:7e:ff:97:
        da:b4:78:29:a1:33:be:ca:49:5f:26:83:6c:f8:40:9e:65:67:
        91:ae:b8:14:5c:09:85:7f:e6:a0:6d:bb:a5:7d:e7:16:2e:c9:
        6b:86:39:16:74:6f:e6:5c:40:8a:a0:4e:ec:eb:70:1a:85:e4:
        a1:7c:21:e1:a5:71:76:3b:dc:43:74:f2:ee:a7:eb:d2:f9:5b:
        44:3e:26:7a:f8:e8:c7:40:c9:71:b9:e7:ad:93:8d:69:00:69:
        16:e2:fb:e5:6d:45:b2:fb:8f:df:fc:2b:c7:a9:58:59:35:22:
        56:7f
-----BEGIN CERTIFICATE-----
MIIC4TCCAkqgAwIBAgIJAP0iQNgAuGj6MA0GCSqGSIb3DQEBBQUAMFUxCzAJBgNV
BAYTAk5MMQ8wDQYDVQQKEwZUZXN0ZXIxFDASBgNVBAMTC2V4YW1wbGUuY29tMR8w
HQYJKoZIhvcNAQkBFhB0ZXN0QGV4YW1wbGUuY29tMB4XDTE1MDQxNjEyNTIzOVoX
DTE2MDQxNTEyNTIzOVowVTELMAkGA1UEBhMCTkwxDzANBgNVBAoTBlRlc3RlcjEU
MBIGA1UEAxMLZXhhbXBsZS5jb20xHzAdBgkqhkiG9w0BCQEWEHRlc3RAZXhhbXBs
ZS5jb20wgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJAoGBALnYb5AEmzPWb+vMg1KP
MAGcuNDXxz48Yyf7DQLEIew6QgYFFg/zSTz5nOeGUwnaknuSczkxwdS5jF8FoxTU
atpONLlzSdv/kBs9HqJYzmsj8drTYIyZmZKyymrZGFJllTQG5yjSwCqN/0o+fr+6
6g129N6QNMwgftyAH42wlJp3AgMBAAGjgbgwgbUwHQYDVR0OBBYEFLSC7vWKdTSC
vSMDO5bkqa63O1+pMIGFBgNVHSMEfjB8gBS0gu71inU0gr0jAzuW5KmutztfqaFZ
pFcwVTELMAkGA1UEBhMCTkwxDzANBgNVBAoTBlRlc3RlcjEUMBIGA1UEAxMLZXhh
bXBsZS5jb20xHzAdBgkqhkiG9w0BCQEWEHRlc3RAZXhhbXBsZS5jb22CCQD9IkDY
ALho+jAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUAA4GBAHYn4uYHos3bOmoU
9KEXjn7/l9q0eCmhM77KSV8mg2z4QJ5lZ5GuuBRcCYV/5qBtu6V95xYuyWuGORZ0
b+ZcQIqgTuzrcBqF5KF8IeGlcXY73EN08u6n69L5W0Q+Jnr46MdAyXG5562TjWkA
aRbi++VtRbL7j9/8K8epWFk1IlZ/
-----END CERTIFICATE-----

Entity Framework Code First migrations and the [StringLength] annotation

Recently I needed to change my model so that a field would be checked for uniqueness. I eagerly added the [StringLength(3)] and [Index(IsUnique = true)] annotations to the model and ran Add-Migration and Update-Database. Close, but no cigar unfortunately. Update-Database kept throwing the following error: System.Data.SqlClient.SqlException (0x80131904): Column 'IsoCode' in table 'dbo.CurrencyModels' is of a type that is invalid for use as a key column in an index.

This is due to the fact that the generated code migration, was only applying the index and not the length restriction. You can fix this directly in the Up() and Down() methods of the code migration by using AlterColumn() as follows:

    public partial class UniqueCurrency : DbMigration
    {
        public override void Up()
        {
            AlterColumn("dbo.CurrencyModels", "IsoCode", c => c.String(maxLength: 3));
            CreateIndex("dbo.CurrencyModels", "IsoCode", unique: true);
        }
        
        public override void Down()
        {
            AlterColumn("dbo.CurrencyModels", "IsoCode", c => c.String(maxLength: null));
            DropIndex("dbo.CurrencyModels", new[] { "IsoCode" });
        }
    }

Dynamic predicates in C# using PredicateBuilder

One of the challenges I frequently encounter, is having to translate the arbitrary criteria in a testcase to LINQ selection predicates. Take the following very simple example testcase:

Feature: ModifyingInvoices
	In order to demonstrate the usefulness of PredicateBuilder, 
        we will show how to verify if a C# collection contains a
        record that matches multiple criteria that are only known 
        at run time

Scenario: ModifyDescription
	When I create an invoice with number '123' for '20' euro
	Then The systems invoice store must look like:
	| Number | Amount | DescriptionPresent | Desciption |
	| 123    | 20     | False              |            |
	When I change the description in invoice '123' to 'Testing!'
	Then The systems invoice store must look like:
	| Number | Amount | DescriptionPresent | Description |
	| 123    | 20     | True               | Testing!    |

In this very small example, you already see that the C# code will need to determine at run-time IF an invoice exists AND MAYBE what the contents of its description should be. If an invoice has many fields. this will become exponentially complex in the code. If your criteria requires an OR construct then that’s even more complex. The solution is to use a PredicateBuilder that builds a dynamic predicate

First install the NuGet Package LINQKit (see PredicateBuilder website) Then add the directive using LinqKit; to your code. Now create the code that queries your data like follows:

        [Then(@"The systems invoice store must look like:")]
        public void ThenTheSystemsInvoiceStoreMustLookLike(Table table)
        {
            var rows = table.CreateSet<InvoiceTest>();

            foreach(InvoiceTest test in rows)
            {
                var MyPredicate = LinqKit.PredicateBuilder.True<Invoice>();
                MyPredicate = MyPredicate.And(invoice => invoice.Number == test.Number);
                MyPredicate = MyPredicate.And(invoice => invoice.Amount == test.Amount);

                if (test.DescriptionPresent)
                {
                    MyPredicate = MyPredicate.And(item => item.Desciption.Equals(test.Description));
                }

                //Test that our datastore contains an invoice that matches the predicate from the testcase
                IQueryable<Invoice> Matches = this.Invoices.AsQueryable().Where<Invoice>(MyPredicate);
                Assert.AreEqual(1, Matches.Count());
            }
        }

What to do when your JQuery-ui dialog is hidden behind other elements

If you see your JQuery-ui dialog being hidden by other elements in the webpage, then you need to increase its z-index. I recently ran into the case where the JQXgrid widget was using very high z-indeces outside of my control.

Here’s the code:

ZIndexer = function () {
    var self      = this;
    this.Elements = [];
    
    this.Add = function (JQuerySelector) {
        var DomElementArray = $(JQuerySelector)
        $.each(DomElementArray, function (i, element) { self.Elements.push(element) })
        return this;
    }

    this.GetNextFreeZIndex = function () {
        var zIndeces = $(this.Elements).sort(function descending(a, b) {
            var bZIndex = $(b).zIndex()
            var aZIndex = $(a).zIndex()
            return bZIndex - aZIndex
        })

        return $(zIndeces[0]).zIndex() + 1;
    }

}

//My grid is in a div with id jqxgrid. All of its child elements need
//to be considered when figuring out the next available ZIndex
var foo = new ZIndexer().Add(&quot;#jqxgrid *&quot;);

//Set the z-index of the jquery-ui dialog and its overlay to the highest available
$('.ui-widget-overlay').css('z-index',foo.GetNextFreeZIndex());
$('.ui-dialog').css('z-index',foo.GetNextFreeZIndex() + 1);
Knockout.js logo

Performance of JQXGrid combined with knockout

The other day I noticed poor performance of a JQXGrid when combined with knockout. I had an ko.ObservableArray() with objects. Each object contains only 3 ko.observable(). I was using JQXGrid’s selection check-box on each row. Event-handlers were established to react to changes in the check-box and set one of the ko.observable() in the the corresponding object in the array.

On my page I was displaying the following:


  1. The JQXgrid

  2. A HTML table using the knockout foreach binding. This table displayed a checkbox for 1 of the observables and static text for the other one

  3. A string representation of the ViewModel using data-bind="text: JSON.stringify(ko.toJS(MyViewModel), null, 4)"

When I increased the number of object in the array, just modifying one check-box caused the UI to slowdown to unacceptable levels.

Items in array Time to complete one click (ms) Time to select all (ms)
25 1.408,136 22.233,092
50 2.156,774 77.999,535
100 5.871,934 473.352,168
200 23.124,779
400 115.075,14
800 707.176,804

When we graph this, you can see a clear O(n^2) performance bottleneck:
A graph showing the exponential increase in runtime

I wanted to change the grid’s source property to use a dataAdapter, However, that did render the table, but each colunm had no value. This is detailed in link where they say:

 March 30, 2012 at 12:53 pm	

It is currently not possible to bind the grid datafields to observable properties. Could you send us a sample view model which demonstrates the required functionality, so we can create a new work item and consider implementing the functionality in the future versions? Looking forward to your reply.

Best Wishes,
Peter

Solving the problem where Azure won’t delete your VHDs, storage containers and accounts

Are you unable to delete storage accounts, containers and VHDs in the azure portal? Even when you’ve already deleted the relevant Virtual Machines?

Maybe you’re running into the case that some of your VHDs are still configured as an potential disk for some Virtual Machine. Azure will place a lease on the VHDs in order to avoid accidental deletion.

On https://portal.azure.portal.com, you can’t see these configured disks. On https://manage.windowsazure.com/ you can though.

Go to the ‘Virtual Machines’ item and click in the ‘Disks’ tab at the top of the page. There you see and delete these configured disks