A little while ago I was curious about how software that uses Transaction Logs works, so I built a toy to-do command line application to get some hands on experience.
Transaction Logs are used when you want to have metadata detailing each action that has taken place on a system but then actually use those logs as the data for the system. Therefore to build up the state of the system at any time you can simply “replay” each action that took place before that time. This enables you to time travel as well as know exactly what happened for user 54 to go into his overdraft.
Above you can see every feature of
todo-log (I did say it was a toy application!) being used from the macOS terminal.
add command you can store a todo item with both a title and some additional body text. This todo item is then assigned a short code — shown in red above — which can be used as the unique identifier to manipulate it in other commands. The transaction detailing the addition of the todo item is saved to a JSON Lines (
ls command, without parameters, displays a list of all the todo items which are outstanding at the current time. Some basic formatting is applied including showing a friendly date (e.g. “two hours ago”) for the creation time of a todo-item.
Deleting a todo item is simple, just pass the shortcode of the item to the
rm command. The command doesn’t then delete the add transaction from the
jsonl file, instead it adds its own
remove transaction detailing the time of deletion and the todo item involved.
log command just prints out the raw
.jsonl to the terminal for inspection. In the above screenshot you can see the format of both the
Finally, passing a positive integer parameter to the
ls command goes back in time by that number of transactions. In this example you can see what the state of the todo list was before the previous
rm command was issued. Everytime
ls is called it simply builds up the current state of the system by rerunning each of the actions detailed in the
.jsonl file — to go back in time we just ignore the last
n lines, where
n is the number provided to the command.
You can have a look at this fun little project over on my Github.
Though I only spent a couple of hours on it, and it doesn’t do much, I feel like working on this gave me a greater appreciation of what transaction logs can be used for and the challenegs around them. Systems like RDBMS transaction logs and Apache Kafka are much more interesting to me as a result.
Sometimes the easiest way to learn about something and become more interested is just to jump in and build something