We Need to Talk About Documentation

This article is mainly aimed at open source projects, but could apply just as well to any team releasing a product — be that an API, some developer tools, some binaries (from a few libraries or a full platform) — or attempting to build an ecosystem around an app or service.

I read a nice article the other day that makes the argument that documentation is more important than ever if you have an open source project to promote. How else will developers get up to speed with your project and learn what they need about it, if you don’t document it well? The article lists out some of the types of documentation you need to provide, such as a readme, some reference documentation, guides and so on.

I’m here to challenge that argument. I don’t think you need an extensive library of documentation at all. Particularly if you are an open source project. To echo the Beatles, I’m going to argue that  “All you need is code."

But who am I to suggest that documentation isn’t that important? Well, I’m a technical writer who has spent a good part of my working life writing the stuff! I cut my teeth writing about Symbian C++ in the days when the code was proprietary, then I worked for Symbian Foundation as we open-sourced the code and wiki-fied the documentation. I’ve also worked for start-ups to document their open source codebases, and for clients who offer services based on code that they don’t yet want to share. Most recently, I have been working on a book about developer marketing (more about that another time) to be released by SlashData, which is the leading analyst firm in the developer economy, and tracks global software developer trends via worldwide developer surveys.

Time and time again, the SlashData surveys report that their respondents put “Documentation and sample code” at the top of the list of the most important features that companies should offer to developers. The developers answering the survey say they value documentation and sample code above tools, integrations & libraries, tutorials and how-to videos, technical support and forums. 

"Pay No Attention to That Man Behind the Curtain"

So, why do I argue against popular opinion? 

Mostly, it’s because I’ve seen behind the curtain. I know that technical writers don’t always (often?) know all problems they are trying to solve through documentation. For one, we aren’t likely to be able to dig deep into every nook and cranny of the code and find out all the aspects that need covering for a user of the code. We’ll have a list of use-cases to write about, some reference information to generate, a few “getting started” tutorials and some FAQs for commonly experienced problems. But we won’t know all the edge cases (in fact, the engineering team releasing the product probably won’t know those either until people start using their code).

Furthermore, technical writers cannot represent all of the readers all of the time: some developers know more than others and expect more support. As a technical writer, to what level am I directing the documentation? Say I’m documenting an API and it has a parameter for a time and date. Do I need to go into a chunk of detail about time and date, and how it is expressed? Or will the developer understand that? If I skip it, will the next reader complain that my documentation is too skimpy? If I document it, will the expert developer reading the reference complain that I’m too verbose?  And, anyway, as a technical writer, do I even have the requisite skills to document this product effectively?

OK, maybe the development team could write documentation? Hey, perhaps use JavaDoc and a few tools and push out some automagically built reference guides? Please, no. Think about it: how many developers enjoy writing for others using their code? How many can be objective and understand what is difficult about their project to get to grips with? They are way too deeply involved to be able to go back to "beginner’s mind."

How many engineers can be tempted away from coding tasks (assuming their project’s timelines even allow it) to write decent documentation? Please don’t ever expect you can get good documentation coverage from your engineering team. It’s why people like me exist: we like writing, we can get into another developer’s shoes and experience, to some extent, we get the problems of using the code, and we are on the project specifically to document it, so can put in 100% of our time to it.

However, as I’ve mentioned, technical writers cannot write documentation to represent the needs of the entire range of developers, all the time, even if we could understand what they are. Suppose we miraculously cover most of what is needed: technical writers are likely to move on once the documentation is complete — it’s the nature of our work. What happens 6 months down the line when the code has changed? When someone moves the location of the logging file, or that class becomes deprecated or that bug gets fixed? Documentation goes out of date all the time, and it can then be worse than useless because it can be actively misleading. The more you have, the more you have to maintain. You need someone with the knowledge of the documentation to have visibility over every change request, every PR accepted, every code update, so they can assess the impact on the text and instigate changes. You will also need to work out how to version the documentation so that those working with older branches of the code see the documentation applying to them, while the latest and greatest codebase is also documented accordingly.

I haven’t even touched on localization yet, and the issues that teams aiming to conquer markets in China, for example, have to consider. But I think I have spelled out some of the problems of generating comprehensive documentation. Just because it’s hard, doesn’t mean you shouldn’t do it, of course, but can we just back up and unpack whether we need all this documentation? If we don’t write documentation, what is the solution?

Life After Docs

As I mentioned in my introduction, we are primarily talking about an open source project. So you need documentation that allows someone to grab your code and go build it. This can fit into a short readme file that is updated every time you make a release. The readme file should also explain how to get started, at least in terms of the basics. It should be possible for someone to take your code, build it and get it working within 5 minutes, or they are very likely to walk away, disgruntled at the overheads involved in your project.

The product engineering team, the test team, the business development team, everyone on the project, needs to sit down and list out what people are going to be doing when they have the code built and running. What problems are they looking to solve? Big use cases (what we put on our website as the main feature of our product), and small (that annoying bug we haven’t fixed yet). List them all out.  This is an exercise you should repeat regularly so you catch any new activities, and those that no longer apply.

Having put together your list, create code examples to cover each element, big or small, with comments in the code examples to indicate what you are doing and why. These examples are to be included in your release, and should never be allowed to go out of date or be broken by updates to your product.

The code examples are the backbone of your documentation, but developers coming to your project still need to know they are there. That is where a set of Q&A come in. For each item on your list of use cases, which has associated code examples, you need to think of as many different ways to ask a question that results in the answer, “Go look at this example, with an appropriate link”. Every developer has a different way of asking about a problem, and you need to think of as many of them as possible. The answer is always the same, pointing them back to the answer as you have framed it by a code example.

So you now have a list of use cases, and a related set of written questions, which are the many ways of expressing the use cases as problems that a developer may experience, or questions they may ask. The answers to those questions are the code examples. Code! This is what your engineering team is good at writing (the hurdle of asking your engineers to become technical writers is avoided). Code can be kept up-to-date as part of your release process, so solving the issue of documentation becoming dated or mismatched against different versions of the codebase.

Where do you publish all the questions you generate? That’s documentation isn’t it? Well, it can be. If you do decide to publish the questions yourself, my recommendation would still be to treat the Q&A as part of your open source project, and use tools like Markdown and a Jekyll theme. It’s very unlikely that any third party will ever want to contribute documentation for you (well, I have never seen this happen) but in case someone wants to pose you a new question, or write an answer in the form of some more example code, it makes sense to make it as simple as possible for them to do so via PRs, just as for the rest of the codebase.

But, an alternative, is to add your Q&A to a forum, either your own, or on an external site like Stack Overflow, so the questions can carry on flowing, and all you need to do is point back to the project, and the code examples that answer the questions. Everyone is familiar with the concept of asking on a forum, and as questions come in, you either answer them immediately with links to existing code examples, or you build a new example, if it’s a use case you haven’t yet thought of.

Summary

I think most technical writing comes down to the single question “What problem am I trying to solve for the reader?” and the best way to write developer documentation is to articulate the problem and provide the answer as code. That’s your documentation: a readme file so they hit the ground running in less than 5 minutes, some questions, some answers and heaps of code examples.  

What do you think? Readers of DZone, perhaps you can jump in here and tell me I’m wrong? Let me know in the comments! What do you really need and look at when you are selecting a new open source project to work with?