A Simple Transaction Log

A little while ago I was curious about how software that uses Transaction Logs works, so I built a toy to-do command line application to get some hands on experience.

Transaction Logs are used when you want to have metadata detailing each action that has taken place on a system but then actually use those logs as the data for the system. Therefore to build up the state of the system at any time you can simply “replay” each action that took place before that time. This enables you to time travel as well as know exactly what happened for user 54 to go into his overdraft.


Above you can see every feature of todo-log (I did say it was a toy application!) being used from the macOS terminal.

Using the add command you can store a todo item with both a title and some additional body text. This todo item is then assigned a short code — shown in red above — which can be used as the unique identifier to manipulate it in other commands. The transaction detailing the addition of the todo item is saved to a JSON Lines (.jsonl) file.

The ls command, without parameters, displays a list of all the todo items which are outstanding at the current time. Some basic formatting is applied including showing a friendly date (e.g. “two hours ago”) for the creation time of a todo-item.

Deleting a todo item is simple, just pass the shortcode of the item to the rm command. The command doesn’t then delete the add transaction from the jsonl file, instead it adds its own remove transaction detailing the time of deletion and the todo item involved.

Invoking the log command just prints out the raw .jsonl to the terminal for inspection. In the above screenshot you can see the format of both the ADD and REMOVE transactions.

Finally, passing a positive integer parameter to the ls command goes back in time by that number of transactions. In this example you can see what the state of the todo list was before the previous rm command was issued. Everytime ls is called it simply builds up the current state of the system by rerunning each of the actions detailed in the .jsonl file — to go back in time we just ignore the last n lines, where n is the number provided to the command.

You can have a look at this fun little project over on my Github.

Though I only spent a couple of hours on it, and it doesn’t do much, I feel like working on this gave me a greater appreciation of what transaction logs can be used for and the challenegs around them. Systems like RDBMS transaction logs and Apache Kafka are much more interesting to me as a result.

Sometimes the easiest way to learn about something and become more interested is just to jump in and build something



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: