How are you handling documentation in your software project? Who is writing it and is it up-to-date?
An ad hoc survey at the “Writers Write” Birds-of-a-Feather session at Devoxx UK revealed: All participants used documentation, but only some were contributing to it. Some experienced that not writing documentation bites you the same way as not writing tests.
So we need to write more of the right content for the intended audience? This is true, but also useless as it gives no advice on how to get there. Still some projects manage to deliver it.
The book “Switch: How to Change Things When Change Is Hard” by Chip and Dan Heath teaches how to tackle this kind of problem on multiple levels. The following paragraphs use their Elephant-Rider-Path metaphor to create an overview of best practices that worked for me in the past.
The Metaphor
The metaphor Chip and Dan use in their book is a Rider on an Elephant heading on a Path to a Goal.
The Rider represents the rational and analytical behaviour in each of us. It loves to analyse problems when it sometimes needs to focus on a solution and might be frustrated when there is no clear goal. It is easily exhausted and powerless without the Elephant.
The Elephant represents the emotional behaviour in each of us. It has a lot of power and endurance, but hates to do things without immediate benefit and is sometimes stubborn.
The Path represents the environment where the Rider and the Elephant cooperate in. The Rider needs a clear vision (destination) to make the change happen. The Elephant follows the Path of least resistance.
The Goal for this article is to deliver the right amount of documentation in the most relevant form for the destined audience.
Direct the Rider
In the first chapters Chip and Dan focus on directing the Rider.
Find the bright spots
The first advice of the book is to find the bright spots and to follow them. So look around in your organisation and find where documentation is working — and clone it!
One successful pattern I found in an organisation was to assign unique three-digit identifiers to each interfaces between systems. The usual approach I see is to assign identifiers to systems, so it surprised me. But it worked well: Once you have an interface to a neighbour, register it with the central department to get an ID assigned, then produce a document from a template to capture the essential information. When a second consumer would use the same provided interface, this would be a new interface with a new identifier (but we could use the existing document and add the new consumer in a minor documentation update).
What was different compared to other organisations and projects? All discussions within and between teams where referencing interfaces by their numbers. All interfaces had version numbers, and both provider and consumer used the version of the interface document in the day-to-day communication. The central architecture team knew about all interfaces (as they assigned the numbers), and could draw the “big picture” of the communication structure. We also used these identifiers when reporting the interface parameters to security: A table with all communication dependencies together with the information they transfer and the security parameters they apply. What looked strange in the beginning turned out to work well.
Another pattern that re-occurred was architecture documentation. While some projects had no or badly structured documentation, the bright spot project were using the arc42 document structure. This structure is available for free and comes with instructions what to write in each section. Projects now and then change the structure to remove chapters and add others. This leaves me with mixed feelings; leaving out chapters that are not relevant is always a choice to make; but moving around chapters makes it difficult to share the documentation between projects.
Using a structure like arc42 seems to promote both writing the documentation and keeping it focused to the reader by capturing the relevant information.
Script the critical moves
For the Rider to get started immediately, script critical moves. This shouldn’t describe the big picture, but the specific behaviours needed to implement the change.
To get a system with unique identifiers of interfaces up-and-running, the critical step is that everyone registers them with the central architecture department and uses a provided template to start from.
For architecture documentation like arc42, the critical first step is to start with an empty structure and to fill the first chapters. They define the purpose of the system, its stakeholders and the system boundary. Would you start a project without defining these?
Point to the destination
Change is easier when you know where you’re going and why it’s worth it.
The book cites the solution-focused (brief) therapy. It asks questions like: “Image a miracle happens tonight that solves the documentation problem overnight. What would be the first sign you’ll notice the next morning?”
For documentation of interfaces between systems the first sign for me would be that the two involved teams talk about the same document. With the same version number.
For architecture documentation the first sign for me would be that I could point my browser to a website with the up-to-date documentation. Maybe I could even share a link to the latest version with others and get notified when something I marked to be relevant for me changed.
Motivate the Elephant
As we’ve set up the Rider to head for the destination, it’s time to motivate the Elephant.
Find the Feeling
Chip and Dan Heath write “Knowing something isn’t enough to cause change. Make people feel something.”
The miracle question from above can help here as well. What a relief talking to the other team about the same document and version!
If you want to present how accessible documentation can be: Present a prototype where users can subscribe to pages and get notifications once the content changes. This works well with Atlassian Confluence. If you want to show how to have a fast, searchable site, build a prototype with Antora that includes Algolia Docsearch, a cloud-based search-as-a-service solution.
If your organisation is used to documentation delivered as a document, give them a ready to download PDF with a proper layout. Show them a running build pipeline that creates the documentation after each software change. Good to know that Asciidoctor can deliver you both a PDF and a searchable website from the same source!
Shrink the Change
A big change can scare the Elephant: So much work, such a big change in behaviour! Let’s break down the change into smaller steps.
To implement the unique identifier per interface, the change seems small: register the interface, receive a number, done! Whether the teams see this as a small change depends on the experience: How have previous interactions with the architecture team been in the past? How tedious was the interaction for the teams, how many questions did the architecture team ask? What if the architecture team promises to assign an interface identifier within one day? Wouldn’t that shrink the change?
The structure of arc42 can overwhelm a developer: It contains lots of chapters and sub-chapters. Some chapters ask for diagrams, this will scare an Elephant! One way to shrink this change is to have a ready-to-go template for your documentation tool of choice (be it Asciidoctor or Confluence). Another way it to mark chapters as optional.
One part of architecture documentation is documenting decisions. In the form of architecture decision records (ADR) they are part of the architecture documentation. To shrink the change, there are ready-to-go templates for ADRs. Unfortunately, there are a lot of different templates around. This will confuse the Rider. Choose one for your project or organisation as part of ‘point to the destination’ to avoid this.
For drawing diagrams, three aspects might boost motivation:
- Diagrams reduce text: The better the diagram, the fewer paragraphs you’ll need to write.
- Keep diagrams small: When a diagram has lots of elements, motivation is low to update it as the writer needs to understand the diagram before she or he changes it. Therefore, keep them small to encourage editing.
- Choose a tool that fits the writers: When you ask developers to write documentation, some opt for diagram-as-code tools like PlantUML. It stores its diagrams as plain text in the code repository plain text and uses auto-layout to render the them. Others opt for tools like Gliffy that plugs into Confluence.
Grow your People
The book asks to cultivate a sense of identity and instill the growth mindset.
Being a professional software developer, or a software crafter, has an appealing identity. Given such an identity the question wouldn’t be “What would be the effort to write the documentation?”, but “What would other professionals do in this situation?”.
Architecture documentation spreads in the IT world: I found that arc42 got on conference tracks and into IT journals. The community provides translations to several languages and formats. Developers or other project members might have come into contact with architecture documentation in other projects and can convince the new team to do the same.
One particularly useful method to get in to designing and communicating IT architecture are architecture katas: In a training session the moderator explains a scenario to multiple teams. Then the teams create an IT architecture on a whiteboard and explain their designs and choices to each other. They learn a lot both in creating an architecture to fit a scenario and how to communicate the design including the decisions taken. From there it is only a small step to write these down (as diagrams, as architecture decision records, or as chapters in an architecture documentation).
Shape the Path
After directing the Rider and motivating the Elephant, it is now time to shape the Path.
Break or tweak the environment
When the situation changes, the behaviour changes. So change the situation.
The book gives several examples, but how would that work for documentation? One situation where it worked well was where a customer stated in its requirements that a set of documents needed to be in place at a specific quality gate. We also knew that we would need to pass this quality gate for every release, and the customer clarified they would insist on the documentation.
So we started out with an outline in AsciiDoc and created a PDF and HTML document as part of the continuous integration pipeline of the project. The latest build was always available for the customer to read. In each sprint review we showed the new parts either in the architecture or system documentation. Soon the business analysts took an interest in the document and used it to read up on the current functionality before they wrote new user stories.
But the biggest tweak of the environment was that the development team was writing the documentation the moment they were implementing the functionality. For each user story we committed not only Java code with the functionality but also the documentation in AsciiDoc. This way the documentation was never ahead nor behind the implemented functionality.
Although this sounds nice, implementation and documentation will still drift apart over time. Inconsistencies will sneak in as the edits in different places are not aligned with each other. Every six months there was an integration phase where the team and the customer scrutinised everything: We read the documents from the beginning to the end and fixed what we missed in the day-to-day editing work.
In an environment, where the customer credibly shows in several occasions that they want to receive the documentation, writing the documentation becomes easy and the norm. Not writing documentation had become difficult.
A year later I learned how important this environment was: At a different customer the credibility of the customer to accept the software only with documentation was dubious. We as a team postponed publishing the documentation to a public place for all teams to read it as the setup had some technical difficulties. An external trigger to scrutinise the documentation in fixed intervals didn’t exist. In this environment the documentation didn’t match the results of the previous project.
Build Habits
Building habits helps, as it frees the attention of the Rider. So we should encourage habits by having triggers or checklists.
The previous paragraph already mentioned some habits:
- For each user story it was part of the definition of done not only to commit code for the functionality and tests but also the documentation. To remind us we gave each story a documentation task. According to scrum an item on the definition of done should have been enough as documentation should be part of each task. Looking back the extra task for each story was essential for us to build this habit.
- For each sprint we presented the new functionality not only in the GUI but also in the documentation.
- For each scrutinisation phase we reviewed the document in full.
Building these habits doesn’t come for free, but it pays off after some iterations. Once these triggers and checklists are the normal routine, the Riders can put their attention to implementing functionality. Alternatively, they can focus on improving the process in other areas.
Rally the Herd
Behaviour is contagious. For writing the documentation, we want it to spread. In the beginning, people will look closely what the other team members are doing. The same way as a waiter will seed the tip jar in a bar to show the expected behaviour, someone needs to seed the documentation and led by example.
As a senior developer or an IT architect you can seed the documentation jar by writing the first chapters or applying a template to the first set of use cases. With code reviews, be the one to check that there are not only code changes, but that the documentation is updated as well.
Once the herd starts moving, the team members will join in. Writing the documentation for the third user story will be easier than writing it for the first two user stories. Two other effects might support you here: The Endowment Effect will help you that once the team owns the documentation, they will tend to value it more than before. Given the first contact with documentation was somewhere neutral or positive, the Mere Exposure Effect will help you that repeated contact with documentation will create positive affection.
Making the Switch
The previous paragraphs show that writing documentation takes effort in multiple areas: rational mind (Rider), emotional side (Elephant) and environment (Path). Once you consider each area with its three critical moves, working documentation is within reach. Making the switch for documentation is possible — this is both reassuring and motivating.
Use the headings of this as a checklist to find out why previous documentation initiatives worked out or why they didn’t. Reading the book will give you more insights and examples on each area.
Even more important: Use the headings to investigate an upcoming documentation effort if it is likely to succeed! Search for weak spots of your plan. Use the metaphor to raise the attention of all stakeholders about the three areas each change needs to address. This way no-one will forget about Rider, Elephant and Path.
I hope you found the examples regarding documentation in software projects useful. Please get in contact and share your experiences with me!
Thank you to Markus Fander to providing feedback on an early version of this article. If you want to buy this book, you can buy it either new or used from several sites. The website of the book also offers additional resources to download.
If you want to read another book from Chris and Dan Heath, I recommend “Made to Stick”. I’ve written another blog post about that book as well, so you can continue reading by following this link.