Give Camp –The Team Leader’s Dilemma

I spent more hours awake than is healthy working at the inaugural Give Camp UK event last weekend. I’m not going to describe it here, hop over to their site if you are interested in what Give Camp UK is all about, and here if you want an introduction to the whole concept.

Anyway, like I said, that’s where I was last weekend. I led a great team which built a secure forum site, with answer voting, for the Young Minds charity. Yeah I know what you’re thinking: it took you a weekend to build that? Seriously? I mean just take one of the open source StackOverflow clones, skin it and you’re done, right?

Well yeah, but therein lies the dilemma. You can do that, and you can be done in a few hours, then you can build in more of the features that the charity wants, things that they consider “nice to haves” you can get it all done in the weekend, ‘cos that’s what it’s all about, right, delivering value for the charity?

Yes! Definitely! But then, on the other hand… you need devs to be enthusiastic about the technology you’re using. You need them to be thinking, hey that project uses cool tech, tech that I want to learn and get some experience on. Also, if you catch the interest of the devs, then not only do they want to join your team for the weekend, they are going to want to stick around after the weekend is finished and work on the project going forward. So, even though you may not get so much done at the actual weekend, the charity wins in the long run, ‘cos the devs stick around to work on the project in their spare time. As they are doing that, they are building real world experience in the tech that they are interested in, which might enhance their CV in the future. Every one wins!

Anyone reading this want to stick around after the weekend and work on a skinned StackOverflow site? Nah, didn’t think so.

So what did I do? Did I go for maximising value to the charity for the 48 or so hours of the event, or did I have my eye on the long game?

As it turned out I went for the latter. So we build our forum with CouchDB on the backend, Node.js in the middle and KnockoutJS on the front end.

Was I right? Well we, pretty much, finished the project during the weekend and the devs are continuing to push stuff to the Git repo, but it’s really too early to tell.

So, what would you have done? Let me know in the comments.

Posted in Give Camp | 4 Comments

Installing NodeJS on Windows

Until very recently, when I wanted to program in NodeJS, I did so on a Ubuntu VM running on my Windows machine. Firstly, because in the beginning, NodeJS didn’t run on Windows. Latterly, because to install it on Windows was four MSI files and to install it on Ubuntu was a case of running “sudo apt-get install NodeJS”. There’s no contest there is there?

However, yesterday I spent the day with Mark Rendle as he was in Dundee as part of his DevExpress sponsored speaking tour of the UK. During our many geeky conversations he mentioned Chocolatey to me and told me I should take a look, “it’s apt-get for Windows”, he said.

So I took a look, and right enough, to install “stuff” with it couldn’t be easier. Of course, first of all you have to install Chocolatey itself; turns out that’s a breeze though. Simply run the “cmd” command as Administrator and drop into Powershell (yes I know, bear with me though). Then from the Powershell prompt run:

Set-ExecutionPolicy Unrestricted

After that, run:

   1: iex ((new-object net.webclient).

   2:    DownloadString("http://bit.ly/psChocInstall"))

And “Bob’s your uncle”, that’s Chocolatey installed. Now you can run Chocolatey from, either Powershell, or an ordinary command prompt. Having installed it, we can use it to install a whole load of packages, including NodeJS. To do that, open a command prompt and type:

cinst NodeJs

then stand back and watch the magic happen (cinst is short for Chocolatey install by the way).

Having installed NodeJS, it’s time to test it with the time-honoured “hello world!” application so open up your favourite editor and type the following code:

   1: require('http').createServer(function(req,res){

   2:     res.writeHead(200,'text/plain');

   3:     res.end('Hello World!');

   4: }).listen('8080');

Now save the file, call it server.js, and run it by typing:

node server.js

at the command prompt. Now we can connect to localhost:8080 and check that everything is working okay:

SNAGHTML1324159

Which it is, so that’s cool. And that really is all there is to installing NodeJS (and all the other cool packages available on Chocolatey.org), so what are you waiting for?! Come on in, the NodeJS water’s fine! Smile

Posted in NodeJS | 5 Comments

Software Performance Webinar

On April 28 I’ll be joining good friend of mine, Kendall Millar and his co-host Gael Fraiteur (of PostSharp fame) on their new webinar. In this webinar, Kendall will be speaking about The Natural Laws of Software Performance and I’ll be reminding you of Everything You’ve Forgotten About Algorithms Since You Left University.

If you are interested in being in the audience for this webinar then register now at the Eventbrite site, it’s sure to be a lot of fun. Looking forward to “seeing” you there!

Posted in Developement, Evangelism, Webinars | Leave a comment

I’ve Been Deep Fried!

Yes that’s right folks, the great guys over at Deep Fried Bytes have caught up with me, and persuaded me to talk about Technical Debt, as if ever getting me to speak was hard to do!

I had a great time with the guys, chewing over the where’s and whyfor’s of Technical Debt and if you’d like to listen to the podcast then just hop over here and take a gander, I’d recommend it, who knows, it may even save your project. Smile

Posted in Uncategorized | Leave a comment

A Twitter oAuth Example in C#

I have a number of scripts that I run against my Twitter feed, stuff like bulk deleting my DMs, that sort of thing. Well of course they all stopped working last year when Twitter moved from a basic authentication model to an oAuth one. I thought it was about time I got them up and running again. Now I know there are libraries out there that will handle the Twitter oAuth stuff for me, but I wanted to get a handle on how it worked, for my own information – besides, how hard could it be, right?

Well the answer is… not that hard, but it’s tricky to get working and a total PITA to debug. So, having gotten it working I thought I’d write this blog post, mainly for my own information, so I know where to look when I forget how to do it the next time, but if it helps you guys out too then that’s cool. On the plus side, at least you don’t have to go around the loop of getting the token in the first place. If you are using code against your own account – as I’m doing here – then all the information you need is there on your app page after you register

I’ll walk you through the process, via the code, then I’ll post the full code listing below that way you can cut and paste it into your own solution if you want to use it.

Okay, so what’s first? Well, first off, we have to collect the parameters that oAuth is going to need…

//GS - Get the oAuth params
string status = "Your status goes here";
string postBody = "status=" +
    Uri.EscapeDataString(status); 

string oauth_consumer_key = "YourConsumerKey";
string oauth_nonce = Convert.ToBase64String(
    new ASCIIEncoding().GetBytes(
        DateTime.Now.Ticks.ToString()));

string oauth_signature_method = "HMAC-SHA1";
string oauth_token =
    "YourToken";

TimeSpan ts = DateTime.UtcNow -
    new DateTime(1970, 1, 1, 0, 0, 0, 0,DateTimeKind.Utc
);

string oauth_timestamp =
    Convert.ToInt64(ts.TotalSeconds).ToString();

string oauth_version = "1.0";

Couple of tricky things here. Firstly notice two string variables, status and postBody. The latter is what we’ll use when we POST the request later, but the first is just the status part, it’s separated like this because it has to be double encoded in the signature (more on that later) but single encoded everywhere else! Yeah, I know! And that’s not really documented anywhere. I only noticed on close examination of the example text on this page.

On the subject on encoding, the .Net Framework has 4 methods of url encoding, two on the Uri class and two on the HttpUtility class. Guess what? They all do it slightly differently. The one that I’ve found that works best is Url.EscapeDataString, however, even that’s not fully RFC3986 compliant, so if you are using my code and still having problems that might be a place to look.

Moving on, the oauth_nonce param is a string that has to be unique on every call, this is to stop play back attacks on your account. Here I’ve used the ticks on the current date and time. I think this should be okay, but if not I’m sure Barry Dorrans will have a better suggestion for you. Smile

Next is the oauth_timestamp which is the number of seconds after the Unix epoch. Not surprisingly, there’s not a method in the .Net framework for that so you have to write your own. Remember to use the DateTime.UtcNow property to ensure it doesn’t use the local time on your machine, but uses GMT instead, regardless of where you are. This is important because, although the oAuth spec says the time has to be >= previous requests, some people are saying that, for Twitter, it has to be within 5 minutes, though I’ve not seen that in my limited use.

//GS - When building the signature string the params
//must be in alphabetical order. I can't be bothered
//with that, get SortedDictionary to do it's thing
SortedDictionary<string, string> sd =
    new SortedDictionary<string, string>();

sd.Add("status", status);
sd.Add("oauth_version", oauth_version);
sd.Add("oauth_consumer_key", oauth_consumer_key);
sd.Add("oauth_nonce", oauth_nonce);
sd.Add("oauth_signature_method", oauth_signature_method);
sd.Add("oauth_timestamp", oauth_timestamp);
sd.Add("oauth_token", oauth_token);

//GS - Build the signature string
string baseString = String.Empty;
baseString += "POST" + "&";
baseString += Uri.EscapeDataString(
    "http://api.twitter.com/1/statuses/update.json")
    + "&";

foreach (KeyValuePair<string,string> entry in sd)
{
    baseString += Uri.EscapeDataString(entry.Key +
        "=" + entry.Value + "&");
}

//GS - Remove the trailing ambersand char, remember
//it's been urlEncoded so you have to remove the
//last 3 chars - %26
baseString =
    baseString.Substring(0, baseString.Length - 3);

Having gathered our parameters together, we need to form them into a string in order to sign them. Of course this isn’t straight forward either. The string has to be formed in the following pattern:

method&URL&ParamKeyValuePairs

Which looks simple enough, but it’s a little tricky. Firstly, the ampersands that split the three parts must *not* be url encoded, but the ampersands that split the parameter key value pairs *must* be url encoded. As if that wasn’t enough of a pain, the key value pairs themselves must be in alphabetical order by key, and then by value, if the key is repeated, this is because Twitter are going to replicate your signing to ensure your precious tweet hasn’t been tampered with, and so there has to be a scheme to follow for them to replicate what you did. Of course, we can use a sorted dictionary to handle that part for us. Also remember to pass status here here and not postBody as status must be double url encoded at this point, but not in the actual POST.

//GS - Build the signing key
string consumerSecret =
    "yourSecret";

string oauth_token_secret =
    "YourToken";

string signingKey =
    Uri.EscapeDataString(consumerSecret) + "&" +
    Uri.EscapeDataString(oauth_token_secret);

Next we have to create the key that we are going to instantiate the hash with. To do this you use the url encoded consumer secret, followed by a non url encoded ampersand, followed by the url encoded token. If you don’t need the token for the request you are making then leave it off, but you still need the trailing ampersand.

//GS - Sign the request
HMACSHA1 hasher = new HMACSHA1(
    new ASCIIEncoding().GetBytes(signingKey));

string signatureString = Convert.ToBase64String(
    hasher.ComputeHash(
    new ASCIIEncoding().GetBytes(baseString)));

Next we are going to sign the request, nothing complicated here, as you see, just remember to take a base64 string of it.

//GS - Tell Twitter we don't do the 100 continue thing
ServicePointManager.Expect100Continue = false;

Then we have this line. This needs to be included to stop Twitter throwing 417 Expectation failed errors.

Well that’s about it. There’s not much to say about the rest of it. It’s just a standard POST request from here. Twitter do recommend that you send the oauth params (remember they have to reproduce what you did to verify your request) via the Authorization header, so of course, you have to url encode it all up again, and wrap it in double quotes too.

Right, that’s all I’ve got to say. I’ve posted the full code below, feel free to use it if it’s any use to you, of course it comes with no warranty whatsoever, other than to say, it works on my machine for what I use it for, YMMV. Smile

class Program
{
    static void Main(string[] args)
    {
        //GS - Get the oAuth params
        string status = "your status";
        string postBody = "status=" +
            Uri.EscapeDataString(status); 

        string oauth_consumer_key = "YourKey";
        string oauth_nonce = Convert.ToBase64String(
            new ASCIIEncoding().GetBytes(
                DateTime.Now.Ticks.ToString()));

        string oauth_signature_method = "HMAC-SHA1";
        string oauth_token =
            "YourToken";

        TimeSpan ts = DateTime.UtcNow -
            new DateTime(1970, 1, 1, 0, 0, 0, 0);

        string oauth_timestamp =
            Convert.ToInt64(ts.TotalSeconds).ToString();

        string oauth_version = "1.0";

        //GS - When building the signature string the params
        //must be in alphabetical order. I can't be bothered
        //with that, get SortedDictionary to do it's thing
        SortedDictionary<string, string> sd =
            new SortedDictionary<string, string>();

        sd.Add("status", status);
        sd.Add("oauth_version", oauth_version);
        sd.Add("oauth_consumer_key", oauth_consumer_key);
        sd.Add("oauth_nonce", oauth_nonce);
        sd.Add("oauth_signature_method", oauth_signature_method);
        sd.Add("oauth_timestamp", oauth_timestamp);
        sd.Add("oauth_token", oauth_token);

        //GS - Build the signature string
        string baseString = String.Empty;
        baseString += "POST" + "&";
        baseString += Uri.EscapeDataString(
            "http://api.twitter.com/1/statuses/update.json")
            + "&";

        foreach (KeyValuePair<string,string> entry in sd)
        {
            baseString += Uri.EscapeDataString(entry.Key +
                "=" + entry.Value + "&");
        }

        //GS - Remove the trailing ambersand char, remember
        //it's been urlEncoded so you have to remove the
        //last 3 chars - %26
        baseString =
            baseString.Substring(0, baseString.Length - 3);

        //GS - Build the signing key
        string consumerSecret =
            "YourSecret";

        string oauth_token_secret =
            "YOurToken";

        string signingKey =
            Uri.EscapeDataString(consumerSecret) + "&" +
            Uri.EscapeDataString(oauth_token_secret);

        //GS - Sign the request
        HMACSHA1 hasher = new HMACSHA1(
            new ASCIIEncoding().GetBytes(signingKey));

        string signatureString = Convert.ToBase64String(
            hasher.ComputeHash(
            new ASCIIEncoding().GetBytes(baseString)));

        //GS - Tell Twitter we don't do the 100 continue thing
        ServicePointManager.Expect100Continue = false;

        //GS - Instantiate a web request and populate the
        //authorization header
        HttpWebRequest hwr =
            (HttpWebRequest)WebRequest.Create(
            @"http://api.twitter.com/1/statuses/update.json");

        string authorizationHeaderParams = String.Empty;
        authorizationHeaderParams += "OAuth ";
        authorizationHeaderParams += "oauth_nonce=" + "\"" +
            Uri.EscapeDataString(oauth_nonce) + "\",";

        authorizationHeaderParams +=
            "oauth_signature_method=" + "\"" +
            Uri.EscapeDataString(oauth_signature_method) +
            "\",";

        authorizationHeaderParams += "oauth_timestamp=" + "\"" +
            Uri.EscapeDataString(oauth_timestamp) + "\",";

        authorizationHeaderParams += "oauth_consumer_key="
            + "\"" + Uri.EscapeDataString(
            oauth_consumer_key) + "\",";

        authorizationHeaderParams += "oauth_token=" + "\"" +
            Uri.EscapeDataString(oauth_token) + "\",";

        authorizationHeaderParams += "oauth_signature=" + "\""
            + Uri.EscapeDataString(signatureString) + "\",";

        authorizationHeaderParams += "oauth_version=" + "\"" +
            Uri.EscapeDataString(oauth_version) + "\"";

        hwr.Headers.Add(
            "Authorization", authorizationHeaderParams);

        //GS - POST off the request
        hwr.Method = "POST";
        hwr.ContentType = "application/x-www-form-urlencoded";
        Stream stream = hwr.GetRequestStream();
        byte[] bodyBytes =
            new ASCIIEncoding().GetBytes(postBody);

        stream.Write(bodyBytes, 0, bodyBytes.Length);
        stream.Flush();
        stream.Close();

        //GS - Allow us a reasonable timeout in case
        //Twitter's busy
        hwr.Timeout = 3 * 60 * 1000;

        try
        {
            HttpWebResponse rsp = hwr.GetResponse()
                as HttpWebResponse;
            //GS - Do something with the return here...
        }
        catch (WebException e)
        {
            //GS - Do some clever error handling here...
        }
    }
}
Posted in Uncategorized | 21 Comments

NoSQL Conference – Dundee Nov 20th 2010

The first NoSQL (Not only SQL) developer day will take place at the University of Dundee on Saturday 20th Nov (http://developerdeveloperdeveloper.com/nosql1/). Come and join other developers and database professionals and learn more about this emerging technology area. Seven sessions across the day covering VoltDB, Hadoop and HBase, Windows Azure, Cassandra and Hector, CouchDB, RavenDB and a session on is NoSQL the future of data storage.

Posted in Community | Leave a comment

SQLBits – Registration Open

We are pleased to announce SQLBits – The 7 Wonders of SQL conference, which will be taking place from September 30th to October 2nd at York University. SQLBits is the largest SQL Server conference in Europe and the previous six conferences have established it as a must-attend event for DBAs, developers and BI professionals who work with Microsoft SQL Server in the UK and Europe. We are making this the biggest event ever with capacity for over 500 SQL Server professionals.

The SQL Server Conference – SQLBits – The 7 Wonders of SQL – 30 – 02 Oct 2010 – York

Posted in Community | Leave a comment

Why I Hate Barclaycard

As I work for an American company I spend a fair amount of time travelling to the US. At the moment I’m in Glendale, CA. No matter what I do, no matter how many times I call or email Barclaycard, no matter how many notes they say they put in my file; every time I’m in the US this same chain of events happens:

  1. About the third or the fourth day, Barclaycard assume the US transactions on my card are fraudulent, despite the fact that I called to tell them I’d be travelling to the US and they had “put a note in my file”.
  2. They stop my card.
  3. They then check to see if the transactions are indeed fraudulent but calling me on my phone. Of course they do this during UK office hours and so I get a call between 3am and 4am local time, when I’m always at my best.
  4. An automated voice asks me to confirm my identity and then asks me to confirm the transactions.
  5. The call ends with the automated voice telling me that they’ll continue to monitor the transactions on my card for my own protection.

BS! The law in the UK makes Barclaycard liable for fraudulent activity not me, so they are not protecting me, they are protecting themselves, which is why they block first and ask questions (at 3am!) later and that is why I hate this stupid, incompetent company!

Posted in Personal | 6 Comments

Dumb Security from AA

After the “Underpants bomber” burned his ‘nads on a flight to Detroit just before Christmas, the TSA issued security guidance with regard to carry on luggage. As I’m flying out to the MVP summit in Seattle tomorrow I thought I’d check out AA’s site and find out if the guidance is still in place. It turns out it is, but this made me giggle. This is AA’s take on the guidance, straight from their website…

From Europe To The U.S.

Effective immediately, and until further notice, customers travelling on American Airlines from Europe will be limited to
one carry-on item. The carry-on item must fit in an overhead compartment or under the seat and cannot exceed 45 dimensional inches / 114 cm (length + width + height) and weigh no more than 40 lbs/18 kgs. This limit will apply to customers originating on
American Airlines and customers originating on other carriers connecting to American Airlines for flights to the U.S. from Europe.

Exemptions:

The normal carry-on allowance of one bag plus one personal item still applies for:

  • American Airlines AAdvantage Executive Platinum®, AAdvantage Platinum® or AAdvantage Gold® members
  • oneworld Alliance Emerald, Sapphire or Ruby members
  • Customers travelling in First Class
  • Customers travelling in Business Class
  • Customers travelling with PriorityAAccess
  • AAirpass® members

The message from AA to would be terrorists is clear, if you want to blow up one of our planes with your second piece of carry on, then you’ll darn well have to upgrade first. AA the airline that requires a better class of terrorist. :-)

New Carry-On Baggage Policy To The U.S.

Posted in Personal | Leave a comment

The Conference Speaker’s Dilemma

After almost three years of conference speaking I achieved a first at a recent conference. The feedback from my session contained no negative comments. Normally, even if the majority of people loved your session, you get at least a few people who didn’t like the slides/code/choice of clothing etc, but not this time – this time there wasn’t a single negative comment.

So that must have made me a cert for top speaker, or at least in the top three, right? I mean I’ve been top speaker before when my talks have had a few negative comments so this time, with no negative comments, I must have rocked to the top.

Well, actually, no. You see this conference, like many others, has a metric to record how useful the attendees felt your talk was – i.e. how relevant is it to your day to day work and how soon/easily could you put the lessons learned into practice. Now my talk was on how functional programming can help solve many of the concurrency problems inherent in today’s OO language dominated solutions. By definition, of course, that meant that most of the audience were OO programmers and so none of them were going back to the office on the Monday morning to implement the lessons learned from my talk – this meant that my talk scored between 1 –3 out of 5 for usefulness. As this score counts towards the final score this placed me in the bottom half of all speakers at the conference.

Now, I know what you are thinking, who cares, right? I mean, everyone had a great time, everyone learned something, everyone went away thinking about what they had learned – the feedback says so – and that’s all that matters. Other than my ego, no one cares about the scores.

Sadly, it’s not that simple. You see many conferences will use your score from previous attendances to decide if they want you to come back again, and a low score is a low score, no matter what the feedback says. And that is the horns of the dilemma. Should speakers turn out good presentations that everyone loves, but score low and risk not getting invited back, or should they concentrate on topics that are still good, but focus on things that are immediately useful to the audience, to the detriment of their broader education?

As Sherlock Holmes would say, I think this is a two pipe problem. :-)

Posted in Community | 3 Comments