Tag Archives: C#

Dynamic predicates in C# using PredicateBuilder

One of the challenges I frequently encounter, is having to translate the arbitrary criteria in a testcase to LINQ selection predicates. Take the following very simple example testcase:

Feature: ModifyingInvoices
	In order to demonstrate the usefulness of PredicateBuilder, 
        we will show how to verify if a C# collection contains a
        record that matches multiple criteria that are only known 
        at run time

Scenario: ModifyDescription
	When I create an invoice with number '123' for '20' euro
	Then The systems invoice store must look like:
	| Number | Amount | DescriptionPresent | Desciption |
	| 123    | 20     | False              |            |
	When I change the description in invoice '123' to 'Testing!'
	Then The systems invoice store must look like:
	| Number | Amount | DescriptionPresent | Description |
	| 123    | 20     | True               | Testing!    |

In this very small example, you already see that the C# code will need to determine at run-time IF an invoice exists AND MAYBE what the contents of its description should be. If an invoice has many fields. this will become exponentially complex in the code. If your criteria requires an OR construct then that’s even more complex. The solution is to use a PredicateBuilder that builds a dynamic predicate

First install the NuGet Package LINQKit (see PredicateBuilder website) Then add the directive using LinqKit; to your code. Now create the code that queries your data like follows:

        [Then(@"The systems invoice store must look like:")]
        public void ThenTheSystemsInvoiceStoreMustLookLike(Table table)
            var rows = table.CreateSet<InvoiceTest>();

            foreach(InvoiceTest test in rows)
                var MyPredicate = LinqKit.PredicateBuilder.True<Invoice>();
                MyPredicate = MyPredicate.And(invoice => invoice.Number == test.Number);
                MyPredicate = MyPredicate.And(invoice => invoice.Amount == test.Amount);

                if (test.DescriptionPresent)
                    MyPredicate = MyPredicate.And(item => item.Desciption.Equals(test.Description));

                //Test that our datastore contains an invoice that matches the predicate from the testcase
                IQueryable<Invoice> Matches = this.Invoices.AsQueryable().Where<Invoice>(MyPredicate);
                Assert.AreEqual(1, Matches.Count());

Migrating from YouTrack to JIRA

Recently I wanted to migrate about 400 issues and 350 attachments from a YouTrack OnDemand instance to a JIRA InCloud instance. JIRA doesn’t provide an importer that is compatible with YouTrack, so I coded a quick .Net C# application that migrated the data for me.

I started with quick list of my must- and nice-to-haves:

Entity Information to migrate
Projects Name
Issues Title, description, state and priority
Issues Attachments belonging to the issue
Issues Comments, including date and author
Entity Information to migrate
Issues Reporter and assignee
Issues Tags
Projects and Issues Components
Issues Affected and fixed version information
Issues Relationship between issues (duplicate/relates-to etc etc)
Issues Historical information such as when the issue was transitioned from one state to another

I didn’t want to migrate or convert between YouTrack’s WIKI formatting used in issue description/comments and the JIRA way of formatting those fields. In fact it turns out that these formats are very similar, so that was a pleasant surpise when I was finished.

The first choice…REST or Import plugin?

The first choice I had to make was between JIRA’s REST API or JIRA’s JSON Import plugin. I opted for the plugin because the REST-API tends to completely ignore information such as state, dates and users. Being able to control the content of these fields is really crucial for data migration.

Getting the issues out of YouTrack…YouTrack and YouTrackSharp challenges

I already blogged how to get issues and attachments out of YouTrack, so there weren’t too many surprises:

  • YouTrackSharp won’t return the description of a project
  • YouTrackSharp won’t return an issue’s tags (a.k.a. labels) or comments. You need to call IssueManager.GetIssue() for each issue returned from instead of IssueManager.GetAllIssuesForProject()
  • The fieldnames and their types are different between the IssueManager.GetIssue() and IssueManager.GetAllIssuesForProject() calls
  • Version numbers associated with an issue in fields affectedVersion and FixedVersion are stored as a CSV string, not as a ICollection in YouTrackSharp
  • The text of a comment is usually returned as the .Text member of the dynamic object. However, I’ve seen a few issues where its returned as a .text member. In C# this difference in case is significant. I used the following approach to handle both cases:
    try {
        ExportComment.Text = Comment.Text;
    catch (Microsoft.CSharp.RuntimeBinder.RuntimeBinderException) {
        ExportComment.Text = Comment.text;

Importing the issues into JIRA

JSON Import documentation

The JSON structure that JIRA can import is documented on Overview and details.

The import plug-in basically does the following. Firstly it will create all the users listed in the JSON. Secondly it will create the projects, components and versions. Thirdly it will import the issues into each project. If the issue contains an attachment, it will download it from the specified URL/webserver into your JIRA instance’s datastore and attach it to the issue. Finally it will create any links between the issues.

Don’t worry about the license limit on the number of users. The plug-in will create them all, but any user above your license limit wont be granted access to the JIRA application and wont count towards the license. Also, after the import is complete, its fine to delete the attachments from your URL.

So the requirements for my application were:

  1. Be able to list all distinct users that are referenced somewhere in an issue, comment or attachment
  2. Per project, be able to list all distinct components that are referenced somewhere in an issue
  3. Per project, be able to list all distinct versions that are referenced somewhere in an issue field
  4. Be able to list all distinct relationships between issues. These relationships could in theory be cross-project
  5. Per issue, be able to translate YouTrack’s values for fields into JIRA’s equivalent. Specifically the following:
    • YouTrack’s state field to JIRA’s state and resolution
    • YouTrack’s default 4 priority values to JIRA’s default 6 priority values
    • YouTrack’s name for Issue types to JIRA’s name for Issue types
  6. Be able to translate YouTrack’s usernames to JIRA’s usernames. I had a few users that existed in both systems with slightly different usernames
  7. Be able to place the downloaded attachments from YouTrack on webserver that JIRA can access and write that URL into the JSON datastructure.

Jira JSON import gotcha’s

  • You are not allowed to supply the resolved date in the issue object. created and updated are fine though
  • The JSON import documentation doesn’t make it clear that you can control what the key of an imported issue should be using the key property of an issue object. If you don’t supply this property, then JIRA will simply give each issue a key equal to the order in which its listed in the JSON. This will almost always be a problem as its very common for issues to in YouTrack to have been deleted
  • YouTrack can handle 1 issue containing multiple attachments with the same name. The JIRA JSON import will throw the following exception and will stop importing more attachments for the issue. I only had 1 issue that had 2 attachments with the same name in YouTrack and I removed one of them
    com.atlassian.jira.plugins.importer.external.ExternalException: com.atlassian.jira.web.util.AttachmentException: Could not save attachment to storage: java.io.FileNotFoundException: /data/service/j2ee_jira/catalina-base/temp/jira-importers-plugin-downloader-2621864330368162205.tmp (No such file or directory)
    	at com.atlassian.jira.plugins.importer.imports.importer.impl.ExternalUtils.attachFile(ExternalUtils.java:354)
    	at com.atlassian.jira.plugins.importer.imports.importer.impl.DefaultJiraDataImporter.createIssue(DefaultJiraDataImporter.java:944)
    	at com.atlassian.jira.plugins.importer.imports.importer.impl.DefaultJiraDataImporter.importIssues(DefaultJiraDataImporter.java:764)
    	at com.atlassian.jira.plugins.importer.imports.importer.impl.DefaultJiraDataImporter.doImport(DefaultJiraDataImporter.java:390)
    Caused by: com.atlassian.jira.web.util.AttachmentException: Could not save attachment to storage: java.io.FileNotFoundException: /data/service/j2ee_jira/catalina-base/temp/jira-importers-plugin-downloader-2621864330368162205.tmp (No such file or directory)
    	at com.atlassian.jira.issue.managers.DefaultAttachmentManager.createAttachmentOnDisk(DefaultAttachmentManager.java:473)
  • YouTrack and Jira have a different interpretation of the direction of the Duplicate issue links.
                        Assume that in YouTrack the follwing link exists: YouTrack: XXX-27 is duplicated by XXX-1
                        Then the IssueLink object will look like this:
                                SourceId	"XXX-27"	string
    		                    TargetId	"XXX-1"	string
    		                    TypeInward	"duplicates"	string
    		                    TypeName	"Duplicate"	string
    		                    TypeOutward	"is duplicated by"	string
                        If we translate that to JIRA's JSON format 
                          "name": "Duplicate",
                          "sourceId": "XXX-27",
                          "destinationId": "XXX-1"
                        Then JIRA will report that XXX-27 duplicates XXX-1. Ergo,for the Duplicate type, we need to swap Source and Target
  • If you have a private installation of JIRA, then you can control the format of the project key. However, in OnDemand instances, the key is restricted to only upper- and lowercase letters, you cant change that. I had 2 projects in YouTrack whose key contained numbers. I could have written a few lines of code to replace the numbers with some letters, but in my case it was far easier to modify the project in YouTrack and remove the numbers.