Provable Commits and Arlo Belshee's Commit Notation

Photo by Safar Safarov on Unsplash

Introduction

What if you could prove to your team that one of your commits didn’t change any behavior? And by any, I mean any. It doesn’t add a new feature; it doesn’t fix a bug. It’s nearly impossible that any behavior is altered by the commit. If you could do that, how would it affect the way your team operates? I imagine that it would be a lot easier to get those provable changes accepted, merged into the main codebase, and included in the next deployment.

That’s the basic premise of a “provable commit”. They are commits that are extremely unlikely to alter the production behavior of a software system. In this article I want to share the concepts around provable commits as I understand them.

I first heard this idea attributed to Arlo Belshee. We interviewed Arlo on our podcast a couple of years ago to talk about naming as a process. On LinkedIn, Arlo describes himself as a “Legacy Code Wrangler,” my favorite kind of developer. Arlo developed a convention for writing commit messages, which makes it very easy to determine which commits are “provable” or safe, and which ones are risky. Being able to quickly evaluate the risk level for a set of commits can be extremely useful in certain team environments.

Before digging into the details of what a provable commit is and how the commit notation helps, let’s explore a scenario where they are most beneficial.

Background

Working on a software system is hard. You have to craft solutions to problems in ways that a machine can understand them. You also have to craft your solutions so they can be understood by the humans on your team.

Communicating your intent to other humans is really challenging. Different people have different levels of technical expertise and experience. Different people also have different levels of trust in the tools that are used to create software systems.

I’d like to discuss one scenario where communicating our intent to others who have different levels of expertise and trust is particularly important. In this scenario you have some changes that you’ve made to the software system, and these need to be deployed to production. The project in this scenario has a process in place for determining which changes are safe. The process involves looking at each individual change and evaluating if that change is risky or not. This scenario is most common for software systems in highly regulated industries, such as aerospace, finance, and healthcare. The process is sometimes referred to as a “change review board.”

On teams with a change review board, refactoring changes are viewed to be very risky. Refactoring is strictly defined to be any change to a software system that changes its design without changing any observable behavior. Change review board processes tend to prefer changes that have been requested by one of the system’s end users. This gives the review board someone who they can ask about whether or not the requested changes are working correctly. In the case of refactoring, it is not possible to ask someone this question. No behavior was changed, so the review board has to try and assess if any behavior was altered by the changes to the software.

Even for teams that don’t follow a formal change review process, there can be fear and uncertainty around making code changes - especially if the team has recently been bitten by bugs in production.

The uncertainty can lead those involved to feel the need to run the full suite of regression tests before the changes are deployed. On large systems, running the full suite of regression tests can be a costly endeavor, especially on teams that rely heavily on manual testing.

Moving Towards Trust and Provable Commits

The cause of this uncertainty is a lack of trust. Humans don’t trust other humans to not make mistakes. Anyone who’s been in software for a while can tell stories about some “simple, low-risk change” that ended up breaking something in production. By contrast, software can’t match a human’s intelligence or ingenuity, but it’s very good at performing consistently and accurately, often producing the exact same set of outputs for a given set of inputs.

Luckily, for teams who want to clean up their code by using refactoring techniques, there are many commercially available automated refactoring tools. These tools have been used countless times to make small alterations. Trusting them to always behave the same for a specific set of inputs is pretty safe.

If you isolate the changes made by an automated refactoring tool and commit them by themselves, then the commit that you have created is a “provable commit” for an automated refactoring.

Prove It

A provable commit is a commit that has an extremely small chance of changing the system’s production behavior. I hesitate to say guarantee because there are very few guarantees when working with software, and even a refactoring tool can have an undiscovered bug. I think it’s as close to a guarantee as you can get when working with software.

In the example above, we can fashion a method for proving that no behavior was changed in a particular commit.

1. Trust that your automated refactoring tool is designed and thoroughly tested to make changes to a set of code without introducing any behavior changes.
2. Check out the source code immediately prior to when the commit was applied.
3. Repeat the exact same refactoring which produced the changes in the commit.
4. In a temporary part of your filesystem, check out the source code immediately after the commit was applied.
5. Compare the source code in your working directory (where the refactoring was applied) with the source code that was checked out in the temporary location.
6. If the codebases have no differences between them, then you have proven that the commit did exactly what it claimed and that it introduced no behavior changes.

With this practice, you can now start to convince your review board that such changes can be deployed with extremely low risk and can avoid the costly step of performing the full regression test suite.

Not Just Refactoring

This technique is exceptionally useful for refactoring, but it can be extended to other kinds of changes as well.

Tests

What if we craft a commit that only makes changes to automated test code? If none of the system’s production code is altered by a commit, then it is extremely unlikely that the commit will alter the application’s production behavior. To prove this you could use the following procedure.

1. Trust that none of the system’s test code is used by production code. There are a couple of ways that you can craft tests to demonstrate that this is the case. The specifics will vary based on your development environment, but essentially, you should be able to delete all of your test code, and your system should still compile and/or run.
2. Check out the source code immediately prior to when the commit was applied and remove all test code.
3. In a temporary part of your filesystem, check out the source code immediately after the commit was applied and remove all test code.
4. Compare the source code in your working directory with the source code in the temporary location.
5. If the codebases have no differences between them, then you have proven that the commit made absolutely no changes to production code, and thus no changes to production behavior.

Comments

What about comments? If a commit only added or deleted a comment, then it’s extremely unlikely that the commit will alter the system’s behavior. You can prove this using this procedure.

1. Trust that your system functions the same if all comments are stripped from the source code. You can demonstrate this by removing all comments from the source code, and your system should still compile and/or run.
2. Check out the source code immediately prior to when the commit was applied and remove all comments.
3. In a temporary part of your filesystem, check out the source code immediately after the commit was applied and remove all comments.
4. Compare the source code in your working directory with the source code in the temporary location.
5. If the codebases have no differences between them, then you have proven that the commit made absolutely no changes to production code and thus no changes to production behavior.

Developer Documentation

What about documentation that’s consumed only by the people working on the system? An example would be the contents of a README file. If a commit only changes developer documentation, then it’s extremely unlikely that the commit will alter the system’s behavior. You can prove this using the following procedure.

1. Trust that your system functions with all developer documentation removed. You can demonstrate this by removing all developer documentation, and your system should still compile and/or run.
2. Check out the source code immediately prior to when the commit was applied and remove all developer documentation.
3. In a temporary part of your filesystem, check out the source code immediately after the commit was applied and remove all developer documentation.
4. Compare the source code in your working directory with the source code in the temporary location.
5. If the codebases have no differences between them, then you have proven that the commit made absolutely no changes to production code and thus no changes to production behavior.

Identifying Provable Commits

Once you have convinced your team that provable commits can be trusted to not make any changes to production behavior, you need a way to quickly identify which commits are provable and which commits are not. This is where Arlo’s Commit Notation comes in. This notation uses the capitalization of the first character in a commit message to communicate whether or not a commit is risky. Commit messages that start with lowercase letters are safe. Commit messages that start with uppercase letters or an exclamation point (!) are risky.

We can formally define the syntax for the commit notation using the following grammar definition.

<status indicator> space <brief description> [newline newline <detailed description>]

(In this snippet “space” indicates the character produced by using the spacebar, and “newline” indicates the character produced by using the enter key.)

The first line of a commit message is composed of a 1-3 character status indicator (padded with spaces) followed by a single space and then a brief description of the change. If needed, additional lines can be provided by separating them from a blank line; this can be useful to provide additional information about the rationale or context for a commit’s changes.

The 1-3 characters before the brief description, referred to above as the “status indicator,” should be one of the following strings. The status indicator should always be 3 characters wide, so if using a single character status indicator, then it needs to be padded with spaces.

There are a few provable commit types in that table that I didn’t cover, but they can be proven using methods that are very similar to the ones that I outlined above.

Provable Commits in Action

I’ve been using the provable commit notation for the Cukeness project. That project’s commit log is a great place to see what these commit messages look like in action.

You can also add a Git pre-commit hook to ensure that your commit messages are formatted according to this structure. If you want to have your entire team conform to this structure, there are tools that can help with that as well.

Acknowledgements

This post owes a lot to the help of Josh Kelley. His suggestions have made this article much stronger.

Discussion

What do you think about this notation? Is this something that could work on your team? Let’s have that conversation in the comments below.