The Problem

I’ve previously written about the home automation framework I’ve been writing. Since I started, all communication between the broker and clients (devices and proxy clients) have been unencrypted – this is something I’ve been wanting to change for a while and think it’s quite important if other people are going to start using it.

Secure Socket Layer (SSL)

Basics

SSL is a network layer that sits underneath, and provides encryption for, well-known protocols like HTTP or any custom communication protocol.

When using SSL, the server creates a keypair containing a public key and private key. A certificate which contains the public key is given to clients by the server as they connect. Clients then use the public key to encrypt all network traffic to the server. SSL uses some funky maths to provide a trap-door function for the encryption. Once data is encrypted with the public key it is extremely difficult (extreme enough to be considered impossible) to decrypt unless you have the private key. Therefore, the private key (as the name implies!) should never be given to anyone else, ensuring that only the server can decrypt the data. So with SSL, you mean a client can send data to the server that only the server can decrypt! Yes, but …

What’s in a Certificate?

As well as the public key, a certificate also contains a Distinguished Name (DN) and a hash. The DN contains information such as the company name and address, contact details, URL of the server etc. The hash is used to determine that the information contained in the certificate (DN and public key) hasn’t been altered.

Certificate Authorities (CAs)

Before explaining why we have CAs, I want to give an example of how a man in the middle attack works….

Situation 1 – Client C1 wants to send some data D to server S who has a certificate SC and private key SP

  1. C1 initiates a connection with S and S responds with SC.
  2. C1 encrypts D with the public key in SC and sends the encrypted data.
  3. S  receives and decrypts it with SP to get D.
  4. S sends a “success” message back to C1.
  5. Everyone’s happy.

Situation 2 – Client C2 wants to send some data D to server S who has a certificate SC and private key SP.

  1. C2 initiates a connection with S but this time a middle-man M intercepts the connection (M has a certificate MC and private key MP) (don’t ask me how M intercepts – I just know it’s possible).
  2. M initiates a separate connection to S and S responsds with SC.
  3. M then returns MC to C2.
  4. C2 encrypts D with the public key in MC and sends the encrypted data.
  5. This time though M receives it and decrypts it with MP to get the original version of D.
  6. M saves D somewhere.
  7. M encrypts D with the public key in SC and sends the encrypted data to S.
  8. S receives and decrypts it with SP to get D.
  9. S sends a “success” message back to M.
  10. M sends the “success” message back to C2.
  11. C2 thinks it’s sent encrypted data to S that only S can read so everyone’s happy, especially M!

From C2′s point of view, the steps he took are exactly the same as C1 did in Situation 1. From S’s point of, the interaction with M is exactly the same as with C1 – M is just another client and S knows no different.

Does this mean SSL is completely useless? Not quite. Notice that in Situation 2, C2 received MC instead of SC. This is why we have CAs. CAs “sign” certificates so they validate that eg SC is indeed for S. Now the client, when he receives a public key, can check if it’s been signed by a CA as being the correct certificate for that site. This way, C2 would have rejected MC as MC != SC.

The process of getting a signed/trusted certificate is:

  1. Create a keypair and certificate – must specify all information for the DN
  2. Create a certificate signing request (CSR)
  3. Send the CSR to a CA and pay them some amount of money
  4. Wait for CA to verify that the information you specified in the DN is correct
  5. SSL enable you server!

Trust Stores

CAs are all well and good, except that anyone can create them. For example, using situation 2 above, M could create a CA and sign MPu as being valid for S. Now, when C2 gets MPu, he’d accept it as a valid public key for S  because it’s signed. So, we’re back to square one, how do we trust certificates? It’s fairly simple – we have trust stores. A trust store is a file that contains information about trusted CAs. If the public key is signed by one of these trusted CAs, then we trust the certificate. Getting a certificate signed by one of these trusted CAs costs money, but means clients can be sure that the public key they want to encrypt information with for the server is the correct one.

Generally, each system or browser contains a trust store that is set up at install time.

What’s in a Certificate?

As well as the public key, a certificate also contains a Distinguished Name (DN) and a hash. The DN contains information such as the company name and address, contact details, URL of the server etc. The hash is used to determine that the information contained in the certificate (DN and public key) hasn’t been altered.

Certificate Chains

In real life, a CA is just another certificate. A certificate could be signed by a CA that itself is signed by another and so one, creating a chain of certificates. In this case, the root CA is the one at the top of that chain, the end certificate for the server itself is at the bottom. To decide that a certificate should be trusted, we only need to trust the root CA.

My Solution

Each person’s “devices” in their home automation system connect to a broker that is unique to them (or their house, or their family etc). To encrypt information between the broker and any device or proxy client connection, the broker needs to have a keypair. This keypair should be unique to the broker, otherwise all brokers would have the same private key and could all decrypt each other’s data.

So when a client connects to the broker, it gets the broker’s public key. We don’t want every broker owner to have to pay to get their broker’s key signed so it can be trusted though. Instead, we create a new CA for all brokers to get their certificates signed by. The client libraries would come pre-installed with a trust store that just includes this CA. Therefore, all clients can trust their broker’s key and each broker can have a unique keypair.

There’s still one small issue. A client connects to a broker and gets a certificate which is signed by the new CA so it trusts it and communicates with the broker. However, the client doesn’t know that the broker is actually the one you wanted to connect to or not. The solution is the same way that a client can verify that a certificate is the correct one for a site – by checking the DN. A broker will need to have a globally unique name which will form part of the DN. One of the client’s properties will be the name of the broker it should be connecting to. As a client connects, it will check that the certificate is signed by the new CA and that the broker name in the DN is correct.

If you trust me to look after the CA and make sure that each broker name is unique, then you’ll be able to trust that all your client-broker communication is secure.

Is it really a bird? Or a plane? Or, something else that tweets?

In a previous post I mentioned that I’d started some home automation round my house. I wanted a way to see what was happening wherever I was without having to scroll through a large debug log. So, I created a Twitter addon. The addon does the following:

  • Logs what it’s doing
  • Looks in my home directory for configuration information – the address of the broker etc
  • Set up the twitter library to post messages
  • Connects to the broker
  • Registers to receive information and updates about all devices I have
  • For each device or update, it tweets a message. Updates include – new devices added, devices removed, device started/stopped, device (dis)connected, device errors, state changes, property value changes
  • Fully javadoc’d functions, classes and variables

Sounds like quite a lot, so it must be fair amount of code. Not really. Thanks to the libraries that I have written and made use of here, the grand total is just 128 lines of code (including all the javadocs). If you want to see the code (with all the twitter access codes removed) I’ve added it at the end.

This doesn’t mean the whole world can see what my house is doing either. I created a private twitter account for my house which is the user that the addon tweets as – that way I can control who can follow it.

How does it work?

Find the main method at the bottom of the code. It calls

PCPlatform.startInstance(args).

This is what finds config and connects to the broker. It passes in the program arguments so that certain details can be overridden based on what’s stored in your home directory.

The first part of the constructor is where I set up the twitter access (NB, I’ve starred out my access keys so you can’t use them!)

// set up the twitter stuff
m_twitter = new TwitterFactory().getInstance();
m_twitter.setOAuthConsumer("***********", "*************");
m_twitter.setOAuthAccessToken(new AccessToken("***********", "**************"));

Then there’s a single method called tweet which takes a string. This tweets up to 140 characters of the message at a time. It also prepends the message with the time to help make each message unique – Twitter automatically removes duplicate messages.

private synchronized void tweet(String to_tweet) {
        String message = "At " + m_date_format.format(new Date()) + ": " + to_tweet;
        ....
}

Finally, at the end of the constructor I create a new instance of ProxyDeviceFactory. This is what sends the message to the broker to receive all device information and updates. I pass a listener instance to the constructor which gets called with any information and updates. Each method of the listener tweets a simple message

// setup the housemate stuff
new ProxyDeviceFactory(new IDeviceListener() {
        .... listener implementation
});

The whole code and nothing but the code

package com.lisantom.housemate.addon.twitter;

import java.text.SimpleDateFormat;
import java.util.Date;

import com.lisantom.housemate.common.HousemateException;
import com.lisantom.housemate.common.Platform;
import com.lisantom.housemate.common.device.Device;
import com.lisantom.housemate.common.device.IDeviceListener;
import com.lisantom.housemate.common.device.Property;
import com.lisantom.housemate.platform.pc.PCPlatform;
import com.lisantom.housemate.proxy.device.ProxyDeviceFactory;
import com.lisantom.utils.log.Log;

import twitter4j.Twitter;
import twitter4j.TwitterException;
import twitter4j.TwitterFactory;
import twitter4j.auth.AccessToken;

/**
 * @author tclabon
 * Main class for "tweeting" Housemate events
 */
public class HousemateTwitterer {

    /**
     * Twitter connection
     */
    private final Twitter m_twitter;

    /**
     * Formatter for the date
     */
    private final SimpleDateFormat m_date_format;

    /**
     * The log to use
     */
    private Log m_log;

    /**
     * Default constructor
     * @throws HousemateException
     */
    @SuppressWarnings("unused")
    public HousemateTwitterer() throws HousemateException {

        m_log = Platform.getInstance().getAddonLog("Tweeter");

        m_date_format = new SimpleDateFormat("hh:mm:ss a");

        // set up the twitter stuff
        m_twitter = new TwitterFactory().getInstance();
        m_twitter.setOAuthConsumer("***********", "*************");
        m_twitter.setOAuthAccessToken(new AccessToken("***********", "**************"));

        // setup the housemate stuff
        new ProxyDeviceFactory(new IDeviceListener() {
            @Override
            public void deviceAdded(Device device) {
                tweet("\"" + device.getName() + "\" device added");
            }

            @Override
            public void deviceRemoved(Device device) {
                tweet("\"" + device.getName() + "\" device removed");
            }

            @Override
            public void deviceConnected(Device device, boolean connected) {
                tweet("\"" + device.getName() + "\" device " + (connected ? "" : "dis") + "connected");
            }

            @Override
            public void deviceInError(Device device, String description) {
                tweet("\"" + device.getName() + "\" device " + (description == null ? "not " : "") + "in error" + (description == null ? "" : ": " + description));
            }

            @Override
            public void deviceRunning(Device device, boolean running) {
                tweet("\"" + device.getName() + "\" device is " + (running ? "" : "not ") + "running");
            }

            @Override
            public void stateChanged(Device device) {
                tweet("\"" + device.getName() + "\" is \"" + device.getCurrentState().getDisplayText() + "\"");
            }

            @Override
            public void propertyValueChanged(Property property) {
                tweet("\"" + property.getDevice().getName() + "\" property \"" + property.getName() + "\" is now set to \"" + property.getValue() + "\"");
            }
        });
    }

    /**
     * Send a tweet
     * @param to_tweet the message to tweet
     */
    private synchronized void tweet(String to_tweet) {
        String message = "At " + m_date_format.format(new Date()) + ": " + to_tweet;
        m_log.d("Tweeting \"" + message + "\"");
        try {
            int i = 0;
            while(i + 140 < message.length()) {
                m_log.d("Tweeting characters from " + i + " through to " + (i + 137));
                m_twitter.updateStatus(message.substring(i, i + 137) + "...");
                i += 137;
            }
            m_log.d("Tweeting characters from " + i + " through to the end");
            m_twitter.updateStatus(message.substring(i));
        } catch(TwitterException e) {
            m_log.e("Could not tweet \"" + message + "\" because: " + e.getMessage());
            e.printStackTrace();
        }
    }

    /**
     * Main method
     * @param args Program args
     * @throws HousemateException
     */
    @SuppressWarnings("unused")
    public static void main(String args[]) throws HousemateException {
        PCPlatform.startInstance(args);
        new HousemateTwitterer();
    }
}

Redwhat?

First off, what the hell is this Redmine thing? In short, it’s an Open Source tool for projects and not just technical ones, although it is geared more towards that. It has all the things you’d expect including forums, wikis, issue tracking, source control integration, news, files, activity and project management type things (issues can be children of other issues and you can see % complete, Gantt charts for critical paths etc). You can also extend it with plugins – I have one called Embedded which lets you put javadocs up on your project’s page.

My dev environment

Last year I started having a play with Redmine after someone at work showed me it. I have Ubuntu 11.10 Desktop installed on my server (I use it as a media server connected to the TV as well as normal server stuff) and installation was relatively simple. Relatively simple rather than very simple because it uses Ruby on Rails and has dependencies in that which the normal package couldn’t handle, or just wasn’t set up to. Since then I’ve gone a few steps further and got issues integrated with Eclipse without too much hassle. What I’ve also done is set an SVN hook to check that all commits are against a valid, open issue and another to build the project and javadocs once the code is committed. The built code and javadocs are immediately available to use although it’s not quite continuous integration because any existing processes aren’t stopped automatically. What I haven’t used Redmine for much (at all *cough*) is its project management tools, eg Gantt charts. Why? What developers create Gantt charts for their home projects!?!?

How do I set it up

So you think some sort of project resources (wikis, forums etc), Eclipse, issues and source control integration is a good thing? Good on ya, so do I! And you like using Open Source tools? Me too! So let me help …

There are numerous resources for installing Redmine, SVN and Eclipse on many platforms so I’m just going to assume that you already have them installed. I’ll also assume that you’re able to access Redmine through an Apache webserver. During this example we will:

  1. Add a Redmine project
  2. Setup Eclipse to see and edit Redmine issues
  3. Create an issue in the Redmine project from Eclipse
  4. Add an Eclipse project to the SVN repo
  5. Add the SVN hooks
  6. Implement the code for the issue
  7. Commit the code
  8. Close the issue from within Eclipse

Add a Redmine project

Go to your Redmine site and select Projects from the top menu bar. Then select New Project from the page menu (top of the white section) and set the name and identifier how you like. Click Save. Done – simple as, eh?

Setup Eclipse to see and edit Redmine issues

This one’s a little more complex – although it’s probably the hardest thing out of all the steps. You need to install 2 plugins, one within Eclipse and one within Redmine.

To install the Redmine plugin, go to your Redmine installation (/usr/share/redmine) and run these 2 commands:

git clone git://redmin-mylyncon.git.sourceforge.net/gitroot/redmin-mylyncon/redmine-mylyn-connector ruby script/plugin install redmine-mylyn-connector

Now restart Redmine (I tend to restart Apache so that the Passenger module restarts – seems to do the trick)

To install the Eclipse plugin, in  Eclipse, go to Help -> Install New Software. Use this update site “http://redmin-mylyncon.sourceforge.net/update-site/N/” and install the plugin Mylyn Integration -> Mylyn Connector: Redmine

(There was a screenshot here but it appears to not work)

”"

In Eclipse, go to Window -> Show View -> Other, then select Mylyn -> Team Repositories. In the top-right corner of the view, select the “Add Task Repository” button. Select Redmine as the type, click Next and then set the Server, User ID and Password for your install/account. The Label can be anything (mine is just Redmine – this is what I assume you’ve called yours for this guide). Click Validate Settings and then Finish.

Now open the Task List view – Window -> Show View -> Task List. You should see Redmine (or whatever you chose as your label). Using the New drop-down list icon in the top-right corner you can create new queries – these allow you to create filters/meaningful subsets of the issues. I have a few queries, one is for open issues which are assigned to me so I can quickly see what work I need to be doing.

Create an Issue in the Redmine project from Eclipse

In your Task List view, click the New drop-down list icon and select New Task … Select Redmine and click Finish. Enter a title – for old time’s sake let’s call it “Hello World” – select your project from the Project’s list and enter a brief description. Click submit. To check it worked, go visit your project in Redmine and have a look at the Issues tab.

Add an Eclipse project to the SVN repo

This assumes you already have a repository. If not, there are numerous articles out there showing how to create one. This also assumes you have an SVN integration installed in Eclipse (I use Subversive). Open the SVN Repositories view and add your repository. Right click and select the Find/Check Out As … option. Create a new Java project and call it anything you want. Check in the new project structure/files.

Add the SVN hooks

Change to your SVN repository’s directory on your filesystem (there should be a folder called hooks). Go into the hooks folder.

Put the following contents in a file called “pre-commit”

#!/bin/bash

REPOS=”$1″
TXN=”$2″ 

cd `dirname $0`

# Note that we can rely on PATH containing /usr/local/bin:/bin:/usr/bin
# so take advantage of this.
SVNLOOK=svnlook
MYSQL=mysql
MSGREGEX=”^task\s*([0-9]+):[.\s]*”

MESSAGE=$($SVNLOOK log -t “$TXN” “$REPOS”)

# Make sure that the log message contains some text.
if [[ $MESSAGE == “” ]]
then
   echo “No log message.” 1>&2; exit 1;
   echo “Log message empty” > check_ticket.out.txt
else
   echo “Log message is:” $MESSAGE > check_ticket.out.txt
fi

# Make sure that the log message references a Redmine issue.
if [[ $MESSAGE =~ $MSGREGEX ]]
then
   echo “Found” ${#BASE_REMATCH[*]} ” regex matches” >> check_ticket.out.txt
else
   echo “Commit failed. To commit please specify a Redmine issue number on the first line, e.g.:
    task 1234″ 1>&2; exit 1;
   echo “Could not match ticket number using” $MSGREGEX >> check_ticket.out.txt
fi

REDMINE_ISSUE=${BASH_REMATCH[1]}
SQL=”SELECT COUNT(*) FROM issues I INNER JOIN issue_statuses S ON S.id = I.status_id WHERE S.is_closed = 0 AND I.id = ${REDMINE_ISSUE};”
echo “Found task #\”"$REDMINE_ISSUE”\”" >> check_ticket.out.txt
echo “Checking task exists:” $SQL >> check_ticket.out.txt
REDMINE_ISSUE_OPEN=$(${MYSQL} -N -u redmine_readonly redmine_default -e “$SQL”)

if [[ ${REDMINE_ISSUE_OPEN} -eq 0 ]]
then
  echo “Commit failed. Redmine issue #${REDMINE_ISSUE} is not in an open state.” 1>&2; exit 1;
fi
echo “Commit allowed” >> check_ticket.out.txt

In the file “post-commit” put the following contents

date=`date`
echo $date > /tmp/post-commit.out.txt

Implement the code for the issue

In Eclipse, go to Window -> Preferences. Select Mylyn -> Team an set the Commit Comment Template to be:

${connector.task.prefix} ${task.key}: ${task.description}

Go to your Task List view, find the task you created, right-click it and select Activate

Go the the Synchronize view and click Synchronize.

Go and create a class in your source folder (in the Package Explorer view you might need to deselect “Focus on Active Task” to be able to see you project).

Commit the Code

Go back to the Synchronize view, right-click your task and click Commit. Notice that the comment is already populated with your task number and description (matching the regex in the pre-commit hook). Click OK and check that (on the machine hosting your SVN repository) that the file /tmp/post-commit.out.txt contains the current date.

Close the Issue from within Eclipse

Go back to the Task List view and open your task. Scroll down to the bottom and change the Mark as field to Closed. Click Submit. Bingo. Bob’s your uncle etc etc. You’ve just committed code against a redmine issue and created an SVN hook to ensure that’s the case :-)

What else can I do with this?

Um, pretty much anything you want!

For example my post-commit hook is much more complex. I have a location on the server where each project is checked out. Each project also has an ant script to build the project and the javadocs. The post-commit hooks changes to the project’s checked out folder, updates the project, builds it and the javadocs and deploys the built version.

Why not try changing the Mylyn template comment format and commit code – it will fail because it can’t work out what you’re committing against. You could extend the pre-commit hook to also check that the issue you’re committing against was raised against your project (or one of it’s parents), rather than just being any old open ticket.

If you’re doing anything interesting with your install, add a comment and let me know!

Not bug as in broken code! Bug as in obsessively interested in. Not sure which is worst – ask Lisa :-)

When we moved in to our flat the central heating/hot water timer we had was a bit old, so old that it didn’t time at – all you could do was switch it on or off. Luckily the timer was mains powered so we temporarily solved the problem by leaving the broken timer on and plugging it in into a separate working timer. The problem now was that the central heating and hot water both came on and off together. About a month later I convinced Lisa that we needed an Acer Revo – a small and cheap PC that we could leave on all the time to hosts photos, films, website (inc. this blog) etc and attach to the TV as a media server. Cunningly, I knew the TV was the other side of the wall from the broken timer – you can probably see what happens next …!

I’m now also the proud owner ot a USB relay card, a circuit board with 8 mains-capable switches that I can control by sending messages over USB. Then with the help of a nice big drill I put the card next to the timer and connected it to the PC. However. I never connected the central heating or hot water to the board, UNTIL TODAY :-) !!!!

Since that time, I’ve been writing a home automation library/broker. Each “device” declares it properties and commands etc in a common format and connects to the broker. Other non-device connections (what I’ve called proxy connections) can request information about what devices exist, what commands they have etc and since it’s all common across all devices, the proxy connection understands any new device without any extra effort (apart from writing the device-specific code). This means I can have an Android UI, REST interface, property store etc that work with any device I have now or in the future. However, I think the nicest part about it is rules. Rules have conditions which can be: a time of day; day of the week; another device’s state etc; and consequences for when the condition becomes (un)satisfied which can be: performing commands on a device; waiting for some amount of time etc. Whilst none of this may be particularly new, I couldn’t find a good example anywhere and I’ve learnt a heck of a lot from doing it.

What I have at the moment is two devices, one for the central heating and one for the hot water. Each device is a simple on/off for one of the relays on the board. What I also have is a rule for each device which acts as a timer (that works!). Each rule is satisfied at some point in time during the day, at which point a consequence turns the device on. The times are different between the weekend and weekdays too. I can also see whether the devices are on or off, and turn them on or off from my phone or anything with internet access.

So, after god knows how long and how many lines of code, I finally have a working times for the central heating and hot water. Oh, and I can control them from anywhere! It seems like not much gain for a lot of work but it shouldn’t take too long now to add new devices and rules. One thing I want to do in the future is a location device (probably running on my phone), an alarm device (so it knows when I’ll be getting up), a train times device, a Boris bike docking station device and plenty more. Then I can write rules that check train times for an hour after I get up to let me know of delays etc (could be very useful when it’s too cold, too hot, too leafy, too <insert here> …. !), check bike availability when I’m on the train so I know what docking station to go to, turn the outside front light on when I get off the train on my way home, tell Lisa to make herself useful and put the kettle on when I’m nearly home (until she realises we can have a “turn kettle on” device :-) )

If you want to know more, let me know I can add you to the project page. If you want to try it out, feel free, but at the moment, there’s no encryption or authentication. If you know where I host the web interface, lucky you! You can now control my central heating whenever you feel like it!

This is my first post! And this blog is the first public-facing thing I’ve put on my website. Naturally I want it to look nice and convey a bit a about who I am so I wanted something fairly techy and new on there. Twitter seems like a fairly obvious choice so that’s in. Then I saw the latitude widget that @dalelane uses and wanted that too (thanks Dale!). I don’t know how Dale has found having his location publicly-accessible on the internet but it made me think two things – 1) do I want it available? and 2) would it make a good first post? Hopefully number 2) is yes!

Before just blindly saying no to 1) I looked into the widget a bit more and the Google latitude privacy settings. Like a lot of social network sites, Google lets you authorise certain applications to gain access to your data. The Google page also contains html for a “badge” that you can put on your site (this is what the widget uses) but for the badge to work, you have to allow your location to be public. The badge also comes with a stark warning:

Please note that once you create and publish this snippet on any web page, anyone with access to your web page could copy the code elsewhere so that your location is still accessible even after you remove the code from your page.

To stop publicly broadcasting your location, you must come back to this page to disable it.

So … I can share my location with anyone and even if someone abuses it and copies the badge code to their site without my permission, I can prevent that code from accessing it in the future so I’m safe (although it does mean I can’t use it here either unless I can somehow change the user id Google gives me). I can also choose between sharing my location as the most accurate available to just the city I’m in, although this is still enough for someone to know if I’m at home or not!

Given I can control when and what data anyone gets about my location, I think the answer to 1) is yes. Actually, the answer to 1) is yes, otherwise that widget wouldn’t be there now! But it did make me think for a moment about the consequences of it. I’m fairly happy putting anything on Facebook or Google+ because I know the people that can see the information, this is the first time I’ve (knowingly) made this information more public! I know some people will think I’ve made a stupid decision but each to their own. And I think that slowly, companies are starting to realise that each user wants different levels of privacy. From the start, Google+ emphasised it’s ability to control who sees what and from my experience, they’ve done that quite well. Facebook, well it’s finally getting somewhere close!

Ultimately it comes down to trust, I have to trust the company with the data not to do anything stupid with it, have to trust the apps that I authorise not to do anything stupid with it and I have to trust anyone who does end up seeing it not to do anything stupid with it. That’s an awful lot of trust in things I can’t control but at the end of the day, I have the option to stop sharing it. If you don’t trust anyone, don’t share anything. Whether I stop sharing before it’s too late, who knows! So if you don’t see me at work next week, you know why! At least that will be one useful thing to come out of this post :-)