Categories
Mobile Application Development

Tracking Moving Trains (with the minimum possible effort)

Trains are, in many respects, wonderful machines. They’re simultaneously both the most romantic form of travel and the most grueling way to get to work every morning. One thing trains are not however, is up-to-date with modern technology.

Whilst you can follow the course, in real time, of a plane soaring at 36,000 feet above the ground all the way from San Francisco to London you may be surprised that Train Operating Companies (TOCs) lack the ability to pinpoint the locations of their trains between stations. In other words, they may know that your train is between Cambridge and Foxton but they wont know exactly where.

Not knowing exactly where Trains are doesn’t cause any satefy issues — Block Signalling has been used to safely manage the passage of trains by identifying which “block” of track they were in, without an exact location, since the 1850s  — however, it does mean that the decisions that control rooms make are being made on educated guesses of a trains exact position at any given time.

Control rooms are the brain room of the modern railway and make decisions when things go wrong. If you’ve ever been told a train cannot run due to a “lack of train crew” its unlikely the driver was off sick and instead much more likely that a delay on a train they were driving earlier has cascaded down to your service and they simply never arrived at your station to drive the train. Control rooms try to avoid this — and many other issues.

Our earlier examples of Cambridge and Foxton aren’t exactly a million miles apart, and on a normal service you would expect a train to travel between them in around 5 minutes — but as the railway gets more and more congested making real-time decisions based on a time resolution of minutes will result in more and more decisions resulting in delays and cancelations.

After having learnt about this situation, whilst onboard HackTrain 2.0, I decided to have a go at resolving it in my own time.

One constraint of working with technology for use on trains is that, for all intents and purposes, you cannot add hardware to trains. The Rail Industry in the UK is, thankfully, obsessed with safety and that means that anything added to a train has to go through rigorous Health & Safety checks. This means it can take months, or years, for anything to be installed trackside or in a vehicle.

Whilst staff who work for South West Trains (who explained the situation to HackTrain competitors) are all issued with BlackBerry devices, some companies have started issueing their Guards and Conductors with relatively modern Android devices, some of which even act as ticket printing machine for on-the-train ticket purchases.

Armed with this knowledge I decided to try and validate my idea that a train could be tracked using just the smartphone device a member of train crew is already carrying. As I didn’t know if the idea would work in reality I set about making a working prototype with the minimum possible effort.

My acceptance criteria was pretty simple: Be able to view in real time the position of a train from Cambridge to Kings Cross. This wasn’t exactly the most measurable criteria (what does “real time” mean, for example?) but it gave me something to work towards.

In just a few hours over two evenings I developed a solution with a server written in Node.js that hosted two pages: `map.html` which, using the Google Maps JavaScript API, showed the current position and historic route of all trains being tracked and `report.html` which delivered a tiny JavaScript application to a browser which allowed it to report its location — attained through a HTML5 geolocation watch — via WebSockets, to the server.

The Prototype JavaScript client on an iPhone
The Prototype JavaScript client on an iPhone
The client and the server were, in the nicest way possible, scrappy. In order to identify a Train the guard (in these feasability tests that was me) would have to manually enter a Headcode into a JavaScript prompt and then leave the browser window open for the entire trip — I didn’t bother developing a background task. Each trains location data was stored in one big multidimensional array on the serverside that had no persistence code. But that was all fine, these applications were never intended to be production ready, just prove an idea was workable.

I walked down a few streets with my prototype client open in the Safari Browser on my iPhone and it appeared to report its location to the server as intended. So, on one of my final commutes into work before the holiday season I tested the system on a moving train on a route that includes tunnels, large areas of poor phone signal and central London; and all the different challenges these environments bring.

The route of my commute from Cambridge to London Kings Cross as tracked by my system
The route of my commute from Cambridge to London Kings Cross as tracked by my system
I was pleasently suprised with the results, shown above , despite a few anomalies.

Due to the fact I was using the `watchLocation()` function the train didn’t report its location in even time periods, as it would if you were polling the location, but rather only when the trains position changed significantly. Despite this fact, the period between the train attemting to send its location whilst on the move was never more than around 10 seconds.

I say attempted because, as expected, there were some issues with mobile coverage. I had expected to have issues in tunnels, but actually only the final tunnel before Kings Cross (which goes under the Grand Union Canal) caused issues. Lack of signal otherwised occured in a few blackspots in rural Cambridgeshire. This issue could be mitigated by employing a system such as that provided by Nomad Digital which provides an internet connection to a train using multiple sim cards connected to multiple network providers.

Another issue you can see in the above screenshot is that my train appears to have left the tracks a few times. I can assure you that this didn’t actually happen, but appears to have done so due to a combination of some inaccurate GPS readings and some failed data transfer caused by the aforementioned network connectivity problems. These errors could be mitigated using a host of techniques: comparing GPS position to known track locations, buffering changes in location that seem too great and only accepting them as true if subsequent location changes appear to be in the same area.

This was a pretty interesting project to work on. It gave me an oppertunity to develop a simple solution to a real life problem in a short period of time and learn about the awesome Socket.io library. I was happy to see it work so well, and to be wrong about tunnels. I look forward to making some of the improvements I’ve mentioned in this post and making the UI prettier and data storage a little bit more resilient.

Danny

Categories
Computer Science

Expect the unexpected

When you design software you usually have a few use cases in mind, in the case of EpsilonGit the use case I keep coming back to is a project lead who wants an overview of how his team is working and how they are using their version control software.

Back in second year when I developed The JavaScript Orrery my target audience was David Parker and the only use of the software was to impress him for degree credit.

A short while later I made a few small adaptations to package the orrery as a Windows Store (now called Universal Windows) Application. I thought a few people might enjoy watching the planets go around the screen, but didn’t really expect too many people to download it. To be completely honest, I mainly packaged it as an app to get points for the App Builder Rewards competition.

I haven’t touched the orrery, packaged as Solar System Simulation on Windows, for years. However, I wrote a little while back about someone who used it to teach their daughter about space, an unexpected use but a nice one.

Today I got an email from a student in Brazil who wondered if the software had a function to see planet locations at specific dates, as he liked the simple 2D graphics and wanted to use them to make a tattoo of the layout of the solar system on his birthday. Strange, but cool.

Unfortunately the Solar System Simulation (which is a gratuitous name — its in no way even close to a ‘simulation’) doesn’t support this function — but its a cool idea, and one I wouldn’t have thought of.

It might be fun to add it in one day, and see how popular some of the ideas I have would be compared to those that a user has had and wanted to be implemented enough to go to the effort to email me about it. I suspect the user submitted ideas might be more popular, because no one knows how well a customer users your product as well as a customer. But I might be wrong, it could be an interesting bit of research.

So, expect the unexpected uses of your software and services — both in positive ways, such as odd-but-exciting use cases, and negative, such as malformed input — but also be excited by the prospect.

Danny

P.S ‘Solar System Simulation’ is still available and works on Windows 8, 8.1 and 10.

Categories
Blogging

Computer Science Blogs Beta

Rob and I have both been doing a lot of work on CS Blogs since the last time I blogged about it. Its now in a usable state, and the public is now welcome to sign up and use the service, as long as they are aware there may be some bugs and changes to public interfaces at any time.

The service has been split up into 4 main areas, which will be discussed below:

csblogs.com – The CS Blogs Web App

CSBlogs.com provides the HTML5 website interface to Computer Science Blogs. The website itself is HTML5 and CSS 3 compliant, supports all screen sizes through responsive web design and supports high and low DPI devices through its use of scalable vector graphics for iconography.

Through the web app a user can read all blog posts on the homepage, select a blogger from a list and view their profile — including links to their social media, github and cv — or sign up for the service themselves.

One of the major flaws with the hullcompsciblogs system was that to sign up a user had to email the administrator and be added to a database manually. Updating a profile happened in the same way. CSBlogs.com times to entirely remove that pain point by providing a secure, easy way to get involved. Users are prompted to sign in with a service — either GitHub, WordPress or StackExchange — and then register. This use of OAuth services means that we never know a users password (meaning we can’t lose it) and that we can auto-fill some of their information upon sign in, such as email address and name, saving them precious time.

As with every part of the site a user can sign up, register manage and update their profile entirely from a mobile device.

api.csblogs.com – The CS Blogs Application Programming Interface

Everything that can be viewed and edited on the web application can be viewed and edited from any application which can interact with a RESTful JSON API. The web application itself is actually built onto of the same API functions.

We think making our data and functions available for use outside of our system will allow people to come up with some interesting applications for a multitude of platforms that we couldn’t support on our own. Alex Pringle has already started writing an Android App.

docs.csblogs.com – The CS Blogs Documentation Website

docs.csblogs.com is the source of information for all users, from application developers consuming the API to potential web app and feed aggregator developers. Alongside pages of documentation on functions and developer workflows there are live API docs and support forums.

In the screenshot below you can see a screenshot of a docs.csblogs.com page which shows a developer the expected outcome of an API call and actually allows them to test it, in a similar way to the Facebook graph explorer, live on the documentation page.

CS Blogs API Documentation
CS Blogs API Documentation

Thanks to readme.io for providing our documentation website for free due to us being an open source project they are interested in!

The CS Blogs Feed Aggregator

The feed aggregator is a node.js application which, every five minutes, requests the RSS/ATOM feed of each blogger and adds any new blogs to the CSBlogs database.

The job is triggered using a Microsoft Azure WebJob, however it is written so that it could also be triggered by a standard UNIX chronjob.

Whilst much of the actual RSS/ATOM parsing is provided by libraries it has been interesting to see inconsistencies between different platforms handling of syndication feeds. Some give you links to images used in blog posts, some don’t, some give you “Read more here” links, some don’t. A reasonable amount of code was written to ensure that all blog posts appear the same to end-users, no matter their original source.

Try it!

I welcome anyone who wants to to try to service now at http://csblogs.com. We would also love any help, whether that be submitting bugs via GitHub issues or writing code over at our public repository.

Danny

Categories
Computer Science

We need a standard… for implementing standards

Obligatory xkcd Comic

Standards are one of the things which makes writing software a bit of a pain, but also allows the pleasure of writing a system which integrates, uses and enhances other systems. Indeed the Internet Browser you are reading this blog post on works using a multitude of standards, from the Internet Protocol and Transmission Control Protocol used to deliver the information to you, to the HTML and CSS used to render this web page on your screen.

For those people who are not entirely familiar with the term standards, a technical standard is — according to wikipedia — a “formal document that establishes uniform engineering or technical criteria, methods, processes and practices”. In laymans terms it’s a document that tells you how something works and how to implement it so that your implementation is compatible with other peoples. An example is RFC 2616, The W3C Standard for the HTTP/1.1 protocol.

Standards, in theory and most of the time in practice are fantastic, why invent a new way of guaranteeing the delivery of a packet over internet protocol if there’s already a standard written by a group of intelligent people who have tried to find and fix every edge case and is compatible with everyone elses system? There is no point.

The only time real issues do occur is when there is an edge case that hasn’t been thought of by the author of the RFC, or the standard is implemented incorrectly — either on purpose or by accident.

How strictly should we adhere to a standard?

The motivation behind for blog post was found when doing my coursework for the “Simulation and 3D Graphics” module in an API called OpenGL. I had written a program using OpenGL 3.0 on the computers in the Fenner Computer Laboratory in the University. However, when I went to the ITMB Computer Lab in the next building my code produced a completely different result. You can see the stark difference below.

OpenGL Rendering - Intel Graphics Vs AMD Graphics
OpenGL Rendering – Intel Graphics Vs AMD Graphics

The image on the left hand side is what I had been seeing as I engineered my code in the fenner laboratory. A column — textured with some “Epic Faces” —  with a spotlight-lit sphere inside it. I’ll explain what the purpose of all this is in another blog post. On the right hand you can see what appeared using exactly the same code in the ITMB Lab, precisely nothing.

After many excruciating hours of searching — with some help from one of my housemates — I discovered the bug was being caused by me attempting to call the “glEnableVertexAttribArray()” method on a uniform member of a shader, something which you’re not meant to do, or indeed allowed to do in OpenGL. I had made a programming error, as every Software Engineer does many times a day. What is especially bad here however is that I wasn’t informed of my accident, either by error or by the thing simply not working.

Looking at the above image you may think that the Fenner Computer, using its Intel Graphics card, was “right” because it displayed the 3D objects I wanted it to. However, I would disagree. The AMD Graphics card on the ITMB Computer actually produced, according to the standard, the correct result. Nothing. Had I been working in ITMB when I produced the code I would have noticed that adding the glEnableVertexAttribArray() line broke everything, and would have immediately removed it, working in Fenner gave me a false sense of security that my code was in fact OK.

So, as far as I can tell this leaves us with a few options.

  1. We should say that in order to claim to implement the OpenGL standard, Graphics Card Manufacturers will have to write a, and work with a standard of how strict their implementations are, I would suggest that all graphics card manufacturers stick directly to the specification and throw exceptions or GL.ErrorCodes or display nothing on screen if there is an issue
  2. We write into the specification exactly what should happen in every possible edge case involved with the specification (this is obviously impossible)
  3. We don’t agree on a level of strictness and we waste countless hours of programming time, as we do at the moment

I sincerely hope that eventually number 1 happens, but I’m not sure how likely it is.

Should we ever add proprietary elements to a standard?

ActiveX Controls
ActiveX Controls

Another issue with standards its that sometimes they don’t fulfill everyone’s use cases, in other words they don’t allow everybody to do everything they would like. For example the body of standards used to make up web pages don’t allow websites to run executable code, such as c++ binaries, natively. This lead to things such as the much despised ActiveX control and the slightly less despised Java Web Applet.

The issues associated with this are many

  • Users have to install an additional piece of software before they can enjoy certain functionality
  • Users then have to ensure this software is up-to-date and manage it properly to avoid security failures
  • The fact these downloads exist allow other people to fake them, allowing easier drive-by-download attacks.

I would argue that if we are ever using a standard, or claiming to use a standard we should endeavour to never add to it ourselves. Instead, the correct course of action would be to write a Request for Comment (RFC) and implement the feature in your browser or program, but have it turned off by default. This way develops can turn it on to test it and play around with it, but they won’t become reliant on a proprietary features to implement their solution. Everyone wins. Thankfully most of the browser makers are going this way, so the HTML5 and associated specifications shouldn’t be too fragmented.

Worried about the future

Several things worry me about the future of some specifications, particularly HTML5 and its associated standards such as CSS3  and ECMAScript (AKA JavaScript).

As mobile browsing is becoming more and more popular at a tremendous rate more and more websites are aiming to make mobile friendly sites, either by making completely separate sites such as http://m.bbc.co.uk/sport or by using responsive designs such as the brilliant one which is used at https://www.gov.uk/. Whilst I can do nothing but encourage the development of mobile sites, especially if they have feature parity with the desktop sites and have a great touch-centric UI, I am worried that history is repeating itself.

Sites famously used to have landing pages which said “Works best in Internet Explorer” or “Works best in Netscape”, this is because often those websites would use proprietary features from either browser, for example gradient backgrounds. Unfortunately, because the two major mobile operating systems, Android and iOS, both use WebKit rendering engines to render HTML, CSS etc content mobile website developers are not only utilizing proprietary features , but actually relying on them.

Though the sites don’t show annoying images that say “This site works best in webkit” the result is equally annoying — a substandard browsing experience for anyone who uses a browser which doesn’t use a WebKit rendering engine, examples include Opera Mobile and Internet Explorer.

Developers, learn from the past, and if you can at your organisation have a standard for implementing true standards!

Danny

Categories
Programming University

2D Graphics Coursework – Solar System Simulation now on Windows 8

Before Christmas I submitted my Assessed Coursework for my 2D Graphics and User Interface Design module, a  2D Orrery (or Solar System Simulation in laymen terms) using the HTML 5 Canvas Tag and associated JavaScript API’s. I think I did pretty well, as I implemented every feature outlined by the specification to, what I feel, was a high standard.

6 days after the hand in date my friends Rob, James and I went to ‘Appy Christmas, an event set up by Microsoft to encourage the development of Windows 8 app in the holiday period. Whilst there I ported my application from a browser based webpage to a fully featured WinJS Windows 8 application. When I told my lecturer, Dr. David Parker about it he seemed interested in the ease of the porting experience, so I shall outline it here.

  • I added all my “Business Code” and classes by just adding the .js files to the solution
  • I edited the HTML page to make it fit better, as scrolling pages don’t look great in apps — all of the Windows 8 style came for free with no work

And that was it! At this point the application worked, however it wasn’t very touch friendly and the Koch Snowflake fractal algorithm would lag on some lower powered arm devices, including the Microsoft Surface RT, so I made the following enhancements

  • Lowered the amount of iterations my fractal function went through, lowering the computational power required
  • Added an “App Bar” to hold buttons which controlled all of the functions — the coursework spec asked that all functions were called using keyboard input, so this made it much more usable on a touch screen
  • Added support for the share charm. When you press it a bitmap image is created from the canvas and sent to whatever application you chose.

If you want to play with the application now, you can do so by downloading it from the Windows store here.

Below you can see some screenshots of the application in action.

I hope you enjoy using the solar system.

Danny

Categories
Programming University

2D Graphics Coursework – A JavaScript Orrery

I apologize for the lag in the above video, its honestly not my code! 😉 It’s the free screen recorder program I used being iffy.

For the last few weeks I’ve been steadily working on my coursework for the “2D Graphics and User Interface Design” module. We’ve been tasked with developing a digital orrery using the HTML5 canvas tag and the associated JavaScript drawing API’s. “What is an Orrery?” I hear you ask, Wikipedia to the rescue:

An orrery is a mechanical device that illustrates the relative positions and motions of the planets and moons in the Solar System in a heliocentricmodel

Heliocentric just means that the planets move around a stationary sun.

The finished, final version of the product will have gravitational forces effect the positions of each planet, allow the user to invent new planets and place them in (effecting the courses of all the other planets!) and have an additional drawing mode in which all of the planets are made to look like pretty Koch Snowflakes 🙂

In the above video you can see what I have achieved up until now, with another month remaining to get it finished. I have

  • 4 Drawing modes (Arc mode, Line Mode, Sprite (image) mode, Animated Earth Mode (Using spritesheets))
  • Independent rotation speeds for each of the planets
  • Keyboard controls to decide which mode is shown

I have the following left to implement

  • Gravity
  • User added planets
  • A moon orbiting earth
  • Elliptical orbits
  • Koch Snowflake drawing mode

I hope to get most of these done this week 🙂

One of the interesting parts of the assignment has been the programming language used itself. JavaScript, though based on the same family of programming languages as my preferred language C#, is quite different to the aforementioned language in some pretty important ways, for example Object Orientation doesn’t come naturally, instead the language uses the Prototyping paradigm, which takes a bit of getting used to if you are an OOP programmer.

JavaScript is also the first loosely typed language I have been formally taught, though I did write in PHP for a few years before university which is also loosely typed. A loosely typed language such as JavaScript doesn’t have hard and fast rules on what can be stored in a variable, for example in C# if I write:

public int WholeNumberVariable = "string";

the whole program will not compile, because putting a string in a variable that has been initialized to hold integer numbers is illegal. In JavaScript when you initialize a variable you can’t assign a type, so It can hold anything

var WholeNumberVariable = "string";
var IExpectANumericResult = WholeNumberVariable / 2;
//I'm going to be a huge bug! I expected a number to divide by 2 but now I'm trying to divide letters! This makes no sense!

This can cause some unexpected behaviour, also known as bugs. So it’s fair to say I much prefer Strongly typed languages.

But, I’m getting used to JavaScript and its numerous quirks, and actually rather enjoying it. I hope to report back soon with some additions to my orrery to show you!
Danny