Category Archives: Software Development

How to see the real-time state of your view-model

When testing and debugging your web-application, its convenient to see the real-time state of the view-model. This is very easy when you’re using a data-binding framework such as knockout. You can simply bind the JSON representation of the view-model to some visible DOM element, like this:

<pre data-bind="text: JSON.stringify(ko.toJS(MyViewModel), null, 4)"></pre>

Sometimes, the objects in your view-model might have circular dependencies, like this:

//The ViewModel
var CurrencyViewModel = function () {
    var self         = this;
    this.DataContext = new CurrencyDataContext(this);
    this.Name        = ko.observable();
    ...
}
//The pseudo-model behind the ViewModel
var CurrencyDataContext = function CurrencyDataContext(ViewModel) {
    var self = this
    this.Viewmodel = ViewModel
    ...
}

In that case you’ll get the following error:
0x800a13aa - JavaScript runtime error: Circular reference in value argument not supported
You can fix this by overriding the toJSON() method, like this:

var CurrencyDataContext = function CurrencyDataContext(ViewModel) {
    var self = this
    this.Viewmodel = ViewModel
    ...
    //Needed to avoid circular reference when a viewModel is serialised into JSON
    CurrencyDataContext.prototype.toJSON = function ()
    {
        var copy = ko.toJS(self);
        delete copy.Viewmodel
        return copy;
    }

Entity Framework: How to delete your old database and start fresh with a new one

In a previous post I explained how to recover after deleting a Entity Framework database. In this post we’ll see the proper way to recreate the database in Entity Framework:

Careful

before you start, make sure you’ve got source-control or back-ups of the code in the Migrations folder. When you delete the Migrations folder, you’re deleting code that:

  1. seeds the database with the initial set of data
  2. and up/downgrades the database to the various versions

In Visual Studio:

  1. Go to Server Explorer, right-click on the data collection that represents your context and choose delete.
  2. Go to Solution Explorer and delete the .mdf file. You might have to click on the “Show All Files” icon before you see it.
  3. Go to the Solution Explorer and delete the Migrations folder.

At this point, you have a solution that will create a new database when its run. If you need to seed the database or expect your models to change, then you’ll want to do the following:

  1. Tell EF to create a fresh database bases on the current models by going to the Package Manager Console and running the following commands:
    Enable-Migrations
    Add-Migration Initial
    Update-Database
    
  2. Update the code in Migrations\Configuration.cs to seed the database with the data you need.

Using jQuery to give your user a “check all” option in the UI

Say you have a table where each row contains a check-box and you want to be able to check/uncheck every single check-box based on some action the user does. Using jQuery this is very easy. Assume we have the following HTML:

        <table>
            <thead>
                <tr>
                    <th><input type="checkbox" id="HeaderCheckbox"/></th>
                    <th>Name</th>
                </tr>
            </thead>
            <tbody>
                <tr><td><input type="checkbox"/></td><td>Tea</td></tr> 
                <tr><td><input type="checkbox"/></td><td>Coffee</td></tr> 
                <tr><td><input type="checkbox"/></td><td>Cola</td></tr> 
           </tbody>
        </table>

Then the following jQuery snippet will transform the HeaderCheckbox into a control that automatically checks or unchecks all the other check-boxes:

$(document).ready(function () {
    //Setup an eventhandler that fires 
    //when the user clicks on a control whose id = HeaderCheckbox
    $('#HeaderCheckbox').click(function (eventobject) {
       //the DOM element that triggered the event
       var $this = $(this);                       
       //Determine what the requested state is. 
       //I.e is the headercheck box checked or unchecked?
       var checked = $this.prop('checked');
       //Find each checkbox element below a <tr> and set 
       //it to the requested sate       
       $("tr :checkbox").prop('checked', checked) 
    })
})

If you’re using some framework with data-binding (e.g. Knockout) then its way better to simply bind each check box to a property of the the View Model and just set that property. The HTML would look like this

        <table >
            <thead>
                <tr>
                    <th><input type="checkbox" id="HeaderCheckbox" /></th>
                    <th>Name</th>
                </tr>
            </thead>
            <tbody data-bind="foreach: Drinks">
                <tr>
                    <td><input type="checkbox" data-bind="checked: $data.Selected"/></td>
                    <td data-bind="text: $data.Name"></td>
                </tr>
            </tbody>
        </table>

And the associated JavaScript would look like this:

var DrinkViewModel = function () {
    this.Name           = ko.observable('');
    this.Selected       = ko.observable(false);
}

var viewModel = function () {
        var self = this
        //Holds all the drinks we want to list in the table
        this.Drinks = ko.observableArray();
}

$('#HeaderCheckbox').click(function (eventobject) {
    var $this = $(this);
    var checked = $this.prop('checked');
    $.each(MyViewModel.Drinks(), function (index, theDrink) 
    { 
        theDrink.Selected(checked) 
    })
})
Knockout.js logo

knockout.js: Your observable isn’t seeing changes made to text controls until they lose focus

knockout is great library that’s easy to use. One thing I noticed is that changes made in text-controls are only propagated to the observable once that control loses focus.

If you want changes in a text-control to immediately be reflected in your observable, then avoid the value binding and use the textInput binding like below:

<input data-bind="textInput: Name" type="text" value="" />
<script type="text/javascript">
    $(document).ready(function () {
        viewModel = ViewModel()
        ko.applyBindings(viewModel);

    });
    function ViewModel() {
        var self = this;
        self.Name = ko.observable("");
    }
</script>

Entity Framework and the error: Cannot attach the file ‘xxx.mdf’ as database ‘xxx’

Say you’re working on a project that’s using Entity Framework to manage the database storage in a SQL Server Express installation. If you delete the .mdf file you’ll keep on getting the error” Cannot attach the file 'xxx.mdf' as database 'xxx'.

To solve it, in visual studio go to the Package Manager console and run the following commands:

sqllocaldb.exe stop v11.0
sqllocaldb.exe delete v11.0
Update-Database

Migrating from YouTrack to JIRA

Recently I wanted to migrate about 400 issues and 350 attachments from a YouTrack OnDemand instance to a JIRA InCloud instance. JIRA doesn’t provide an importer that is compatible with YouTrack, so I coded a quick .Net C# application that migrated the data for me.

I started with quick list of my must- and nice-to-haves:

Must-haves
Entity Information to migrate
Projects Name
Issues Title, description, state and priority
Issues Attachments belonging to the issue
Issues Comments, including date and author
Nice-to-haves
Entity Information to migrate
Issues Reporter and assignee
Issues Tags
Projects and Issues Components
Issues Affected and fixed version information
Issues Relationship between issues (duplicate/relates-to etc etc)
Issues Historical information such as when the issue was transitioned from one state to another

I didn’t want to migrate or convert between YouTrack’s WIKI formatting used in issue description/comments and the JIRA way of formatting those fields. In fact it turns out that these formats are very similar, so that was a pleasant surpise when I was finished.

The first choice…REST or Import plugin?

The first choice I had to make was between JIRA’s REST API or JIRA’s JSON Import plugin. I opted for the plugin because the REST-API tends to completely ignore information such as state, dates and users. Being able to control the content of these fields is really crucial for data migration.

Getting the issues out of YouTrack…YouTrack and YouTrackSharp challenges

I already blogged how to get issues and attachments out of YouTrack, so there weren’t too many surprises:

  • YouTrackSharp won’t return the description of a project
  • YouTrackSharp won’t return an issue’s tags (a.k.a. labels) or comments. You need to call IssueManager.GetIssue() for each issue returned from instead of IssueManager.GetAllIssuesForProject()
  • The fieldnames and their types are different between the IssueManager.GetIssue() and IssueManager.GetAllIssuesForProject() calls
  • Version numbers associated with an issue in fields affectedVersion and FixedVersion are stored as a CSV string, not as a ICollection in YouTrackSharp
  • The text of a comment is usually returned as the .Text member of the dynamic object. However, I’ve seen a few issues where its returned as a .text member. In C# this difference in case is significant. I used the following approach to handle both cases:
    try {
        ExportComment.Text = Comment.Text;
    }
    catch (Microsoft.CSharp.RuntimeBinder.RuntimeBinderException) {
        ExportComment.Text = Comment.text;
    }
    

Importing the issues into JIRA

JSON Import documentation

The JSON structure that JIRA can import is documented on Overview and details.

The import plug-in basically does the following. Firstly it will create all the users listed in the JSON. Secondly it will create the projects, components and versions. Thirdly it will import the issues into each project. If the issue contains an attachment, it will download it from the specified URL/webserver into your JIRA instance’s datastore and attach it to the issue. Finally it will create any links between the issues.

Don’t worry about the license limit on the number of users. The plug-in will create them all, but any user above your license limit wont be granted access to the JIRA application and wont count towards the license. Also, after the import is complete, its fine to delete the attachments from your URL.

So the requirements for my application were:

  1. Be able to list all distinct users that are referenced somewhere in an issue, comment or attachment
  2. Per project, be able to list all distinct components that are referenced somewhere in an issue
  3. Per project, be able to list all distinct versions that are referenced somewhere in an issue field
  4. Be able to list all distinct relationships between issues. These relationships could in theory be cross-project
  5. Per issue, be able to translate YouTrack’s values for fields into JIRA’s equivalent. Specifically the following:
    • YouTrack’s state field to JIRA’s state and resolution
    • YouTrack’s default 4 priority values to JIRA’s default 6 priority values
    • YouTrack’s name for Issue types to JIRA’s name for Issue types
  6. Be able to translate YouTrack’s usernames to JIRA’s usernames. I had a few users that existed in both systems with slightly different usernames
  7. Be able to place the downloaded attachments from YouTrack on webserver that JIRA can access and write that URL into the JSON datastructure.

Jira JSON import gotcha’s

  • You are not allowed to supply the resolved date in the issue object. created and updated are fine though
  • The JSON import documentation doesn’t make it clear that you can control what the key of an imported issue should be using the key property of an issue object. If you don’t supply this property, then JIRA will simply give each issue a key equal to the order in which its listed in the JSON. This will almost always be a problem as its very common for issues to in YouTrack to have been deleted
  • YouTrack can handle 1 issue containing multiple attachments with the same name. The JIRA JSON import will throw the following exception and will stop importing more attachments for the issue. I only had 1 issue that had 2 attachments with the same name in YouTrack and I removed one of them
    com.atlassian.jira.plugins.importer.external.ExternalException: com.atlassian.jira.web.util.AttachmentException: Could not save attachment to storage: java.io.FileNotFoundException: /data/service/j2ee_jira/catalina-base/temp/jira-importers-plugin-downloader-2621864330368162205.tmp (No such file or directory)
    	at com.atlassian.jira.plugins.importer.imports.importer.impl.ExternalUtils.attachFile(ExternalUtils.java:354)
    	at com.atlassian.jira.plugins.importer.imports.importer.impl.DefaultJiraDataImporter.createIssue(DefaultJiraDataImporter.java:944)
    	at com.atlassian.jira.plugins.importer.imports.importer.impl.DefaultJiraDataImporter.importIssues(DefaultJiraDataImporter.java:764)
    	at com.atlassian.jira.plugins.importer.imports.importer.impl.DefaultJiraDataImporter.doImport(DefaultJiraDataImporter.java:390)
    ...
    Caused by: com.atlassian.jira.web.util.AttachmentException: Could not save attachment to storage: java.io.FileNotFoundException: /data/service/j2ee_jira/catalina-base/temp/jira-importers-plugin-downloader-2621864330368162205.tmp (No such file or directory)
    	at com.atlassian.jira.issue.managers.DefaultAttachmentManager.createAttachmentOnDisk(DefaultAttachmentManager.java:473)
    ...
    
  • YouTrack and Jira have a different interpretation of the direction of the Duplicate issue links.
                        
                        Assume that in YouTrack the follwing link exists: YouTrack: XXX-27 is duplicated by XXX-1
                        Then the IssueLink object will look like this:
                                SourceId	"XXX-27"	string
    		                    TargetId	"XXX-1"	string
    		                    TypeInward	"duplicates"	string
    		                    TypeName	"Duplicate"	string
    		                    TypeOutward	"is duplicated by"	string
                        If we translate that to JIRA's JSON format 
                        {
                          "name": "Duplicate",
                          "sourceId": "XXX-27",
                          "destinationId": "XXX-1"
                        },
                        Then JIRA will report that XXX-27 duplicates XXX-1. Ergo,for the Duplicate type, we need to swap Source and Target
                        
    
  • If you have a private installation of JIRA, then you can control the format of the project key. However, in OnDemand instances, the key is restricted to only upper- and lowercase letters, you cant change that. I had 2 projects in YouTrack whose key contained numbers. I could have written a few lines of code to replace the numbers with some letters, but in my case it was far easier to modify the project in YouTrack and remove the numbers.

Highlighting webelements from Selenium

When I’m using selenium to navigate through a web-app, I like to record the session and visually highlight the DOM elements that my testscripts are using.

The IWebDriver and IWebElement interfaces don’t supply a method for this….so we’ll have to build our own. The approach I use, is to inject some JavaScript into the webpage to modify the style of the DOM element.

Here’s the code:

        public void highlightElement(IWebElement element)
        {
            IJavaScriptExecutor js = this.Driver as IJavaScriptExecutor;
            string OriginalStyle   = element.GetAttribute("style");
            string HighlightedStyle = OriginalStyle + "border: 3px solid red;";
            
            for (int i = 0; i < 3; i++)
            {
                js.ExecuteScript("arguments[0].setAttribute('style', arguments[1]);", element, HighlightedStyle);
                System.Threading.Thread.Sleep(500);
                js.ExecuteScript("arguments[0].setAttribute('style', arguments[1]);", element, OriginalStyle);
                System.Threading.Thread.Sleep(500);
            }
        }

Say for example, that I have the following testscript:

Feature: SeleniumHighlightingOfWebElements
	I want to show how I can show a user which elements in the DOM my testcases are using

Scenario: DemonstrateHighlighting
	Given I navigate to 'http://en.wikipedia.org/wiki/Hello_world_program'
	Then there must be a link with 'text' 'computer program'
	And there must be an element with 'xpath' '//./div[@class='thumbinner' and .//a[@href and normalize-space() =  'Firefox'] ]'
	And there must be a link with 'text' 'display device'
	And there must be an element with 'xpath' '//./div[@class='thumbinner' and .//a[@href and normalize-space() =  'CNC'] ]'
	And there must be a link with 'text' 'programming languages'
	And there must be an element with 'xpath' '//./div[@class='thumbinner' and .//a[@href and normalize-space() =  'PlayStation Portable'] ]'

That will result in the following screen capture:

grepping in Powershell

Many unix/linux users are intimately familiar with regular expressions and using them with grep, sed or awk in a pipeline. A typical usage scenario is the following command:

ls | grep -i '.*[0-9].*\.dll' | sort 

So how do we do grep in PowerShell? Well, we can use PowerShell’s operators:

Operator Description
-match Matches its input property agains a regular expression. By default Powershell uses case-insensitive matching. Thats different than .Net’s default behavior with the RegEx class
-imatch Case-insensitive version of Match
-cmatch Case-sensitive version of Match
-like This compares its input against a string that can contain a wildcard. So no regular expressions usable here
-contains Only tells you if the input does or doesn’t contain the exact value you supplied

For our specific example, the PowerShell equivalent is:

Get-ChildItem | where { $_.Name -match ".*[0-9].*\.dll" } | Sort-Object -Property Name

And gives us the following output:

    Directory: C:\Windows\System32

Mode                LastWriteTime     Length Name
----                -------------     ------ ----
-a---        20-11-2010     22:29     640512 advapi32.dll
-a---         11-6-2011      3:58     138056 atl100.dll
-a---         14-7-2009      3:14      65024 avicap32.dll
-a---        20-11-2010     22:29      91648 avifil32.dll
-a---         14-7-2009      3:14      10752 bitsprx2.dll
...
...

Another approach is to directly use the RegEx class from .Net. An example:

$matchOptions = [System.Text.RegularExpressions.RegexOptions]::IgnoreCase
$matcher = new-object System.Text.RegularExpressions.Regex (".*[0-9].*\.dll",$matchOptions)
$files  = Get-ChildItem
$files | where { $matcher.IsMatch($_.Name)  }

How to programatically download attachments from YouTrack

In a previous post I showed how to retrieve your bugs, comments and attachments from YouTrack. However, actually downloading each attachment is a little more work. Assume that the variable Url contains a value like "http://xxxxx.myjetbrains.com/youtrack/_persistent/yyyyy.doc?file=xx-xxx&v=0&c=false"

The following raises a HTTP NotFound exception:

var foo = Connection.Get(Url); //YouTrackSharp.Infrastructure.Connection

As does this:

String basePath = new System.Uri(Url).PathAndQuery;
var foo = Connection.Get(basePath );

The following will raise a HTTP NotAuthorized exception because we are not supplying the authentication cookie to the web-server.

byte[] file = new System.Net.WebClient().DownloadData(Url);

I looked at the YouTrackSharp.Infrastructure.Connection class and I didn’t see anything that would let me retrieve the authentication cookie.

Instead of writing code to manually implement the YouTrack REST API, I decided to take a different approach. I decided to modify YouTrackSharp itself to give me access to the authentication cookie.
I removed the NuGet package YouTrackSharp from my solution.
I downloaded the source of YouTrackSharp from GitHub and unzipped it next to my own project.
I included src\YouTrackSharp\YouTrackSharp.csproj into my solution
I modified src\YouTrackSharp\Infrastructure to add the public modifier as follows:

public CookieCollection _authenticationCookie;

I changed my own program to download the file as follows:

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(Url);
req.CookieContainer = new CookieContainer();
req.CookieContainer.Add(Connection._authenticationCookie);
req.Method = "GET";
WebResponse response = req.GetResponse();
System.IO.Stream Stream = response.GetResponseStream();
var fileStream = System.IO.File.Create(String.Format("C:\\tmp\\{0}", Name));
Stream.CopyTo(fileStream);
fileStream.Close();