Category Archives: Software Development

Dynamic predicates in C# using PredicateBuilder

One of the challenges I frequently encounter, is having to translate the arbitrary criteria in a testcase to LINQ selection predicates. Take the following very simple example testcase:

Feature: ModifyingInvoices
	In order to demonstrate the usefulness of PredicateBuilder, 
        we will show how to verify if a C# collection contains a
        record that matches multiple criteria that are only known 
        at run time

Scenario: ModifyDescription
	When I create an invoice with number '123' for '20' euro
	Then The systems invoice store must look like:
	| Number | Amount | DescriptionPresent | Desciption |
	| 123    | 20     | False              |            |
	When I change the description in invoice '123' to 'Testing!'
	Then The systems invoice store must look like:
	| Number | Amount | DescriptionPresent | Description |
	| 123    | 20     | True               | Testing!    |

In this very small example, you already see that the C# code will need to determine at run-time IF an invoice exists AND MAYBE what the contents of its description should be. If an invoice has many fields. this will become exponentially complex in the code. If your criteria requires an OR construct then that’s even more complex. The solution is to use a PredicateBuilder that builds a dynamic predicate

First install the NuGet Package LINQKit (see PredicateBuilder website) Then add the directive using LinqKit; to your code. Now create the code that queries your data like follows:

        [Then(@"The systems invoice store must look like:")]
        public void ThenTheSystemsInvoiceStoreMustLookLike(Table table)
        {
            var rows = table.CreateSet<InvoiceTest>();

            foreach(InvoiceTest test in rows)
            {
                var MyPredicate = LinqKit.PredicateBuilder.True<Invoice>();
                MyPredicate = MyPredicate.And(invoice => invoice.Number == test.Number);
                MyPredicate = MyPredicate.And(invoice => invoice.Amount == test.Amount);

                if (test.DescriptionPresent)
                {
                    MyPredicate = MyPredicate.And(item => item.Desciption.Equals(test.Description));
                }

                //Test that our datastore contains an invoice that matches the predicate from the testcase
                IQueryable<Invoice> Matches = this.Invoices.AsQueryable().Where<Invoice>(MyPredicate);
                Assert.AreEqual(1, Matches.Count());
            }
        }

What to do when your JQuery-ui dialog is hidden behind other elements

If you see your JQuery-ui dialog being hidden by other elements in the webpage, then you need to increase its z-index. I recently ran into the case where the JQXgrid widget was using very high z-indeces outside of my control.

Here’s the code:

ZIndexer = function () {
    var self      = this;
    this.Elements = [];
    
    this.Add = function (JQuerySelector) {
        var DomElementArray = $(JQuerySelector)
        $.each(DomElementArray, function (i, element) { self.Elements.push(element) })
        return this;
    }

    this.GetNextFreeZIndex = function () {
        var zIndeces = $(this.Elements).sort(function descending(a, b) {
            var bZIndex = $(b).zIndex()
            var aZIndex = $(a).zIndex()
            return bZIndex - aZIndex
        })

        return $(zIndeces[0]).zIndex() + 1;
    }

}

//My grid is in a div with id jqxgrid. All of its child elements need
//to be considered when figuring out the next available ZIndex
var foo = new ZIndexer().Add(&quot;#jqxgrid *&quot;);

//Set the z-index of the jquery-ui dialog and its overlay to the highest available
$('.ui-widget-overlay').css('z-index',foo.GetNextFreeZIndex());
$('.ui-dialog').css('z-index',foo.GetNextFreeZIndex() + 1);
Knockout.js logo

Performance of JQXGrid combined with knockout

The other day I noticed poor performance of a JQXGrid when combined with knockout. I had an ko.ObservableArray() with objects. Each object contains only 3 ko.observable(). I was using JQXGrid’s selection check-box on each row. Event-handlers were established to react to changes in the check-box and set one of the ko.observable() in the the corresponding object in the array.

On my page I was displaying the following:


  1. The JQXgrid

  2. A HTML table using the knockout foreach binding. This table displayed a checkbox for 1 of the observables and static text for the other one

  3. A string representation of the ViewModel using data-bind="text: JSON.stringify(ko.toJS(MyViewModel), null, 4)"

When I increased the number of object in the array, just modifying one check-box caused the UI to slowdown to unacceptable levels.

Items in array Time to complete one click (ms) Time to select all (ms)
25 1.408,136 22.233,092
50 2.156,774 77.999,535
100 5.871,934 473.352,168
200 23.124,779 –
400 115.075,14 –
800 707.176,804 –

When we graph this, you can see a clear O(n^2) performance bottleneck:
A graph showing the exponential increase in runtime

I wanted to change the grid’s source property to use a dataAdapter, However, that did render the table, but each colunm had no value. This is detailed in link where they say:

 March 30, 2012 at 12:53 pm	

It is currently not possible to bind the grid datafields to observable properties. Could you send us a sample view model which demonstrates the required functionality, so we can create a new work item and consider implementing the functionality in the future versions? Looking forward to your reply.

Best Wishes,
Peter

How to see the real-time state of your view-model

When testing and debugging your web-application, its convenient to see the real-time state of the view-model. This is very easy when you’re using a data-binding framework such as knockout. You can simply bind the JSON representation of the view-model to some visible DOM element, like this:

<pre data-bind="text: JSON.stringify(ko.toJS(MyViewModel), null, 4)"></pre>

Sometimes, the objects in your view-model might have circular dependencies, like this:

//The ViewModel
var CurrencyViewModel = function () {
    var self         = this;
    this.DataContext = new CurrencyDataContext(this);
    this.Name        = ko.observable();
    ...
}
//The pseudo-model behind the ViewModel
var CurrencyDataContext = function CurrencyDataContext(ViewModel) {
    var self = this
    this.Viewmodel = ViewModel
    ...
}

In that case you’ll get the following error:
0x800a13aa - JavaScript runtime error: Circular reference in value argument not supported
You can fix this by overriding the toJSON() method, like this:

var CurrencyDataContext = function CurrencyDataContext(ViewModel) {
    var self = this
    this.Viewmodel = ViewModel
    ...
    //Needed to avoid circular reference when a viewModel is serialised into JSON
    CurrencyDataContext.prototype.toJSON = function ()
    {
        var copy = ko.toJS(self);
        delete copy.Viewmodel
        return copy;
    }

Entity Framework: How to delete your old database and start fresh with a new one

In a previous post I explained how to recover after deleting a Entity Framework database. In this post we’ll see the proper way to recreate the database in Entity Framework:

Careful

before you start, make sure you’ve got source-control or back-ups of the code in the Migrations folder. When you delete the Migrations folder, you’re deleting code that:

  1. seeds the database with the initial set of data
  2. and up/downgrades the database to the various versions

In Visual Studio:

  1. Go to Server Explorer, right-click on the data collection that represents your context and choose delete.
  2. Go to Solution Explorer and delete the .mdf file. You might have to click on the “Show All Files” icon before you see it.
  3. Go to the Solution Explorer and delete the Migrations folder.

At this point, you have a solution that will create a new database when its run. If you need to seed the database or expect your models to change, then you’ll want to do the following:

  1. Tell EF to create a fresh database bases on the current models by going to the Package Manager Console and running the following commands: 
    Enable-Migrations
Add-Migration Initial
Update-Database
  2. Update the code in Migrations\Configuration.cs to seed the database with the data you need.

Using jQuery to give your user a “check all” option in the UI

Say you have a table where each row contains a check-box and you want to be able to check/uncheck every single check-box based on some action the user does. Using jQuery this is very easy. Assume we have the following HTML:

        <table>
            <thead>
                <tr>
                    <th><input type="checkbox" id="HeaderCheckbox"/></th>
                    <th>Name</th>
                </tr>
            </thead>
            <tbody>
                <tr><td><input type="checkbox"/></td><td>Tea</td></tr> 
                <tr><td><input type="checkbox"/></td><td>Coffee</td></tr> 
                <tr><td><input type="checkbox"/></td><td>Cola</td></tr> 
           </tbody>
        </table>

Then the following jQuery snippet will transform the HeaderCheckbox into a control that automatically checks or unchecks all the other check-boxes:

$(document).ready(function () {
    //Setup an eventhandler that fires 
    //when the user clicks on a control whose id = HeaderCheckbox
    $('#HeaderCheckbox').click(function (eventobject) {
       //the DOM element that triggered the event
       var $this = $(this);                       
       //Determine what the requested state is. 
       //I.e is the headercheck box checked or unchecked?
       var checked = $this.prop('checked');
       //Find each checkbox element below a <tr> and set 
       //it to the requested sate       
       $("tr :checkbox").prop('checked', checked) 
    })
})

If you’re using some framework with data-binding (e.g. Knockout) then its way better to simply bind each check box to a property of the the View Model and just set that property. The HTML would look like this

        <table >
            <thead>
                <tr>
                    <th><input type="checkbox" id="HeaderCheckbox" /></th>
                    <th>Name</th>
                </tr>
            </thead>
            <tbody data-bind="foreach: Drinks">
                <tr>
                    <td><input type="checkbox" data-bind="checked: $data.Selected"/></td>
                    <td data-bind="text: $data.Name"></td>
                </tr>
            </tbody>
        </table>

And the associated JavaScript would look like this:

var DrinkViewModel = function () {
    this.Name           = ko.observable('');
    this.Selected       = ko.observable(false);
}

var viewModel = function () {
        var self = this
        //Holds all the drinks we want to list in the table
        this.Drinks = ko.observableArray();
}

$('#HeaderCheckbox').click(function (eventobject) {
    var $this = $(this);
    var checked = $this.prop('checked');
    $.each(MyViewModel.Drinks(), function (index, theDrink) 
    { 
        theDrink.Selected(checked) 
    })
})
Knockout.js logo

knockout.js: Your observable isn’t seeing changes made to text controls until they lose focus

knockout is great library that’s easy to use. One thing I noticed is that changes made in text-controls are only propagated to the observable once that control loses focus.

If you want changes in a text-control to immediately be reflected in your observable, then avoid the value binding and use the textInput binding like below:

<input data-bind="textInput: Name" type="text" value="" />
<script type="text/javascript">
    $(document).ready(function () {
        viewModel = ViewModel()
        ko.applyBindings(viewModel);

    });
    function ViewModel() {
        var self = this;
        self.Name = ko.observable("");
    }
</script>

Entity Framework and the error: Cannot attach the file ‘xxx.mdf’ as database ‘xxx’

Say you’re working on a project that’s using Entity Framework to manage the database storage in a SQL Server Express installation. If you delete the .mdf file you’ll keep on getting the error” Cannot attach the file 'xxx.mdf' as database 'xxx'.

To solve it, in visual studio go to the Package Manager console and run the following commands:

sqllocaldb.exe stop v11.0
sqllocaldb.exe delete v11.0
Update-Database

Migrating from YouTrack to JIRA

Recently I wanted to migrate about 400 issues and 350 attachments from a YouTrack OnDemand instance to a JIRA InCloud instance. JIRA doesn’t provide an importer that is compatible with YouTrack, so I coded a quick .Net C# application that migrated the data for me.

I started with quick list of my must- and nice-to-haves:

Must-haves
Entity Information to migrate
Projects Name
Issues Title, description, state and priority
Issues Attachments belonging to the issue
Issues Comments, including date and author
Nice-to-haves
Entity Information to migrate
Issues Reporter and assignee
Issues Tags
Projects and Issues Components
Issues Affected and fixed version information
Issues Relationship between issues (duplicate/relates-to etc etc)
Issues Historical information such as when the issue was transitioned from one state to another

I didn’t want to migrate or convert between YouTrack’s WIKI formatting used in issue description/comments and the JIRA way of formatting those fields. In fact it turns out that these formats are very similar, so that was a pleasant surpise when I was finished.

The first choice…REST or Import plugin?

The first choice I had to make was between JIRA’s REST API or JIRA’s JSON Import plugin. I opted for the plugin because the REST-API tends to completely ignore information such as state, dates and users. Being able to control the content of these fields is really crucial for data migration.

Getting the issues out of YouTrack…YouTrack and YouTrackSharp challenges

I already blogged how to get issues and attachments out of YouTrack, so there weren’t too many surprises:

  • YouTrackSharp won’t return the description of a project
  • YouTrackSharp won’t return an issue’s tags (a.k.a. labels) or comments. You need to call IssueManager.GetIssue() for each issue returned from instead of IssueManager.GetAllIssuesForProject()
  • The fieldnames and their types are different between the IssueManager.GetIssue() and IssueManager.GetAllIssuesForProject() calls
  • Version numbers associated with an issue in fields affectedVersion and FixedVersion are stored as a CSV string, not as a ICollection in YouTrackSharp
  • The text of a comment is usually returned as the .Text member of the dynamic object. However, I’ve seen a few issues where its returned as a .text member. In C# this difference in case is significant. I used the following approach to handle both cases: 
    try {
    ExportComment.Text = Comment.Text;
}
catch (Microsoft.CSharp.RuntimeBinder.RuntimeBinderException) {
    ExportComment.Text = Comment.text;
}

Importing the issues into JIRA

JSON Import documentation

The JSON structure that JIRA can import is documented on Overview and details.

The import plug-in basically does the following. Firstly it will create all the users listed in the JSON. Secondly it will create the projects, components and versions. Thirdly it will import the issues into each project. If the issue contains an attachment, it will download it from the specified URL/webserver into your JIRA instance’s datastore and attach it to the issue. Finally it will create any links between the issues.

Don’t worry about the license limit on the number of users. The plug-in will create them all, but any user above your license limit wont be granted access to the JIRA application and wont count towards the license. Also, after the import is complete, its fine to delete the attachments from your URL.

So the requirements for my application were:

  1. Be able to list all distinct users that are referenced somewhere in an issue, comment or attachment
  2. Per project, be able to list all distinct components that are referenced somewhere in an issue
  3. Per project, be able to list all distinct versions that are referenced somewhere in an issue field
  4. Be able to list all distinct relationships between issues. These relationships could in theory be cross-project
  5. Per issue, be able to translate YouTrack’s values for fields into JIRA’s equivalent. Specifically the following:
    • YouTrack’s state field to JIRA’s state and resolution
    • YouTrack’s default 4 priority values to JIRA’s default 6 priority values
    • YouTrack’s name for Issue types to JIRA’s name for Issue types
  6. Be able to translate YouTrack’s usernames to JIRA’s usernames. I had a few users that existed in both systems with slightly different usernames
  7. Be able to place the downloaded attachments from YouTrack on webserver that JIRA can access and write that URL into the JSON datastructure.

Jira JSON import gotcha’s

  • You are not allowed to supply the resolved date in the issue object. created and updated are fine though
  • The JSON import documentation doesn’t make it clear that you can control what the key of an imported issue should be using the key property of an issue object. If you don’t supply this property, then JIRA will simply give each issue a key equal to the order in which its listed in the JSON. This will almost always be a problem as its very common for issues to in YouTrack to have been deleted
  • YouTrack can handle 1 issue containing multiple attachments with the same name. The JIRA JSON import will throw the following exception and will stop importing more attachments for the issue. I only had 1 issue that had 2 attachments with the same name in YouTrack and I removed one of them 
    com.atlassian.jira.plugins.importer.external.ExternalException: com.atlassian.jira.web.util.AttachmentException: Could not save attachment to storage: java.io.FileNotFoundException: /data/service/j2ee_jira/catalina-base/temp/jira-importers-plugin-downloader-2621864330368162205.tmp (No such file or directory)
	at com.atlassian.jira.plugins.importer.imports.importer.impl.ExternalUtils.attachFile(ExternalUtils.java:354)
	at com.atlassian.jira.plugins.importer.imports.importer.impl.DefaultJiraDataImporter.createIssue(DefaultJiraDataImporter.java:944)
	at com.atlassian.jira.plugins.importer.imports.importer.impl.DefaultJiraDataImporter.importIssues(DefaultJiraDataImporter.java:764)
	at com.atlassian.jira.plugins.importer.imports.importer.impl.DefaultJiraDataImporter.doImport(DefaultJiraDataImporter.java:390)
...
Caused by: com.atlassian.jira.web.util.AttachmentException: Could not save attachment to storage: java.io.FileNotFoundException: /data/service/j2ee_jira/catalina-base/temp/jira-importers-plugin-downloader-2621864330368162205.tmp (No such file or directory)
	at com.atlassian.jira.issue.managers.DefaultAttachmentManager.createAttachmentOnDisk(DefaultAttachmentManager.java:473)
...
  • YouTrack and Jira have a different interpretation of the direction of the Duplicate issue links. 
                        
                    Assume that in YouTrack the follwing link exists: YouTrack: XXX-27 is duplicated by XXX-1
                    Then the IssueLink object will look like this:
                            SourceId	"XXX-27"	string
		                    TargetId	"XXX-1"	string
		                    TypeInward	"duplicates"	string
		                    TypeName	"Duplicate"	string
		                    TypeOutward	"is duplicated by"	string
                    If we translate that to JIRA's JSON format 
                    {
                      "name": "Duplicate",
                      "sourceId": "XXX-27",
                      "destinationId": "XXX-1"
                    },
                    Then JIRA will report that XXX-27 duplicates XXX-1. Ergo,for the Duplicate type, we need to swap Source and Target
  • If you have a private installation of JIRA, then you can control the format of the project key. However, in OnDemand instances, the key is restricted to only upper- and lowercase letters, you cant change that. I had 2 projects in YouTrack whose key contained numbers. I could have written a few lines of code to replace the numbers with some letters, but in my case it was far easier to modify the project in YouTrack and remove the numbers.