Software Documentation

This section provides an overview of my experiences as a technical author writing software documentation. Furthermore, I reflect upon insights that I have gained with respect to documenting software during the course of my career.

Outline your experience working as a technical author on software products.

Since May 2023 I am a technical author and developer experience engineer at Tyk, a company that offer an open source API Gateway. I have contributed how to guides, tutorials, FAQs and reference documentation for a variety of projects:

• gRPC: I refactored documentation to facilitate learning how to integrate a gRPC server with Tyk Gateway for remote execution of plugins for an API request. This involved restructuring gRPC content and adding a call to action (CTA) for commercial support. I refactored and structured the key concepts content that explains how to write a gRPC server for integration with Tyk Gateway. Additionally, I wrote a Python gRPC server getting started guide and an advanced tutorial for writing a custom gRPC plugin to handle requests with signed HMAC authentication tokens. I implemented a proof of concept Python demo that was approved and published in the Tyk demo repository as an asset for supporting global commercial clients.
• GitHub Actions: I have implemented a Python script, deployed within a GitHub action CI error check, to report new content unassigned to the navigation menu in the documentation website.
• Release Notes: Release notes had an adhoc structure across product components. I actively contributed to designing a consistent structure and wrote a markdown template and internal Confluence guide that explains how to write effective release notes. To automate generation of change logs, I developed a Python script that generates markdown content from a CSV file. Subsequently, I improved accessibility by designing and implementing a landing page for release notes.
• Tyk Async APIs: I collaborated with the web design and product teams to write and release content for the new Tyk Streams beta feature.
• Inclusive Naming Initiative (INI) Tier 1 Compliance: I independently researched existing content that violated Inclusive Naming Initiative (INI) tier 1 words and designed and delivered a phased plan for content replacement, where possible. Furthermore, I identified configuration parameters, content URLs and GUI fields that require replacement for (INI) tier 1 compliance.
• FAQs: I have written FAQs in response to customer feedback reported from the global commercial support team. Examples include: how to reduce CPU usage in a Redis cluster and performance impact of analytics in Tyk Gateway.

I also actively manage, review and release content contributed by product managers, engineers and external users in the form of pull requests that target new and historic releases. This also includes reviewing changes to OpenAPI schema specifications and releasing to public Postman Collections.

My software development career has involved developing backend database and micro-service architectures in a variety of business domains. Subsequently, I have been responsible for documenting system architectures, specifications and Rest API endpoints using a variety of formats, including HTML, Markdown and GitHub Books. I have also written user guides, troubleshooting documents, FAQs and how-to guides. 
When developing micro-services I have experience of documenting endpoint requests and responses using document generation tools. For example, I have developed micro-services using FastAPI. This allows swagger documentation to be produced from endpoint parameters and descriptions included within the source code. I have also used Spphinx to document APIs, tutorials and system architectures.

All documentation that I have written has been maintained in GitHub/GitLab repositories, included for versioned releases via CI/CD infrastructure.

For quality assurance of user guides and explanatory material I have asked peer developers and users to review my documentation. Subsequently, I have updated content appropriately. Furthermore, I have written and implemented quality assurance guidelines to test the effectiveness of documentation. For example, requesting users and/or customers to follow explanatory material.

What audiences (e.g. developers, customers) in what industry sectors (e.g. telecommunications, science, government) have you written for?

During the course of my career, I have written technical and user documentation for a range of business domains. These include: prepaid cards, futures markets, software testing tools, youth offender case management, property management and bitcoin.

As a backend software developer I have designed, tested and implemented micro-service architectures using a range of programming languages. Subsequently, I have been responsible for documenting micro-services, RESTful API endpoints, system architectures and databases using HTML, Markdown and OpenAPI.

For business users, I have written and released versioned user guides and tutorials for internal and external website applications, with accompanying image and video media. I have produced these guides in a variety of formats including: HTML, LibreOffice, Microsoft Office and PDF.

At a wider level, I also have a blog site where I have written technical articles for the wider community. This includes articles for how to: setup and configure Linux environments, configure CUPS and compile Nginx from source code for MacOs environments. Please take a moment to look at the various articles published on my blog.

What is the largest complex documentation you have contributed to?

I have developed and documented software in a variety of sectors, including the prepaid card industry. During this period I contributed to streamlining the prepaid card bank reconciliation process at Newcastle Building Society into a service-oriented architecture.

Prepaid card providers generated and deployed daily transaction feeds using their own legacy formats. I collaborated with experts from the various organisations, to design and streamline the required data into standardised formats for system integration. Subsequently, I documented and published the resultant specification for sharing with current and future card providers. This facilitated the management and onboarding process of card providers for the 1.5+ million cards issued within six European countries.

What experience have you as an owner of software product documentation?

I have over five years experience as a backend software developer and technical author. Subsequently, I have setup GitHub repositories and designed and implemented the underlying build infrastructure for micro-services. This includes writing associated documentation for architectural overviews, APIs, database design and micro-service configuration.

Where possible, I have integrated API documentation with source code via document generation tools. For example, web frameworks such as FastAPI and Django Rest Framework support generation of OpenAPI content from endpoint configuration parameters, in addition to request and response object annotations. This facilitates generation of API documentation for feature branches, since the documentation is integrated with a CI/CD pipeline. Finally, I have designed and implemented the deployment of API documentation via configuring helm charts with ingress endpoints.

I have experience of writing and maintaining structured dev-ops guides, using Markdown, to explain micro-service configuration. This includes content such as: an overview of the micro-service and tables describing deployment configuration variables, e.g. API base url, database host, port etc.

During the course of my career, I have also developed desktop and website applications. Subsequently, I have experience of writing, reviewing, testing and updating user documentation for a variety of domains. This includes: explanatory tutorials, user guides and how-to documents.

At a personal level, I am curious and willing to explore and document new technologies. I have setup, structured and maintained a personal WordPress blog and public source code repositories to share my knowledge and experience. My blog contains how-to guides for a variety of Linux administration tasks, e.g. setting up the Raspberry Pi as an SSH server and configuring the Raspberry Pi to share an internet connection from a Linux desktop.

What have you done that has changed how others - such as colleagues - think about, value or do documentation?

At Newcastle Building Society I was a senior analyst / backend developer, working within a team of four to implement a service-oriented architecture. This was to provide daily bank reconciliation and fraud detection of prepaid card transactions.

A range of external card providers transferred their daily card transactions using their own proprietary XML feeds. These feeds contained redundant and duplicate information.

I collaborated with the finance department to streamline information. This resulted in the production of a written specification document and standardised schema, intended for use by internal developers and external card providers. This served the purpose of standardising the transfer of daily prepaid card transactions.

Consequently, the onboarding of new card providers was facilitated and the organisation expanded to manage 1.5+ million cards, issued within six European countries.

What is your proudest success as a technical author?

I have my own WordPress blog to document my development and Linux administration experiences for personal reference. During my teaching career, I acquired a Raspberry Pi for research, setup and configuration in educational lab sessions. I documented my experiences as how-to guides using WordPress for future reference. Subsequently, since that time, others have found the guides useful. This is evident via the comments on my blog page.

What are examples of excellent open-source documentation - what makes them good?

I have previous development and documentation experience of APIs using a variety of Python web frameworks such as: FastAPI, Pyramid, Strawberry and Django. The documentation for the FastAPI web framework provides examples of the following principles of developer open-source reference documentation:

• Documentation modes: A range of documentation modes are easily accessible from the landing page. These include how-to guides, tutorials (basic and advanced), explanatory material (introduction to Python types, concurrency and use of async/await) and reference documentation.
• Chat bots: Users can chat with an AI bot to ask common questions and be quickly navigated to relevant documentation pages.
• Language translation: The landing page allows users to translate the documentation and search results into a variety of languages. This enhances the accessibility of the documentation for wide international audiences.
• Floating table of contents: Each document includes a floating table of contents to enable users to quickly navigate to the relevant area within a page.
• Link to open-source project: A link to the open-source project is included in the navigation structure. This allows developers to quickly navigate to the open-source repository, to raise/browse issues and participate in GitHub discussions.
• Quick content: Some documents summarise key examples at the top of the page, within an “In a hurry?” section. This is beneficial for developers who are seeking quick examples for how to achieve and implement functionality. This bypasses the need to navigate through multiple levels of documentation pages.

Ubuntu Studio demonstrates good principles for open-source user documentation. These are as follows.

• Landing pages: The landing page is categorised into key areas of functionality, with audio, video, graphics and photography links. Download links to the latest regular and long term support releases are included, in addition to regular news updates. This enables users to quickly navigate key documentation areas.
• Contribution guidelines: Guidelines are published for users to contribute to the documentation and showcase content created using the product. This enables valuable user feedback, thus improving the effectiveness of the documentation. Furthermore, the submission of showcase content fosters a strong user community.
• Support: Links are clearly provided to the community help wiki. This includes links to FAQs, hardware support and troubleshooting guides. A variety of support mechanisms are available, such as IRC channels, mailing lists and Ubuntu forums.
• Community: A link is provided to a community page. This provides a link to the IRC and Matrix chat channels for connecting with other Ubuntu Studio users. Furthermore, the details of social media pages are clearly included on the landing page. These mechanisms encourage building a user community, with opportunities for showcasing usage of the products and submitting future contributions.

What should software teams do to achieve success in documentation?

Software encompasses a wide variety of formats, such as: websites, APIs, SDKs, micro-services and mobile applications etc. To offer an opinion on what software teams can do to achieve success in documentation, it is first necessary to briefly consider mechanisms by which software is managed.

Large software projects use git repositories to manage software, using a range of branch management strategies, e.g. git-flow and trunk based development. Continuous integration and continuous delivery pipelines are used to manage the build, test and deployment of versioned software releases into development and production environments.

In this context, documentation should also be version controlled. This offers the advantage that software is developed using the same infrastructure and toolset as the source code. Developers are included in the process and the documentation is updated in synch with the source code. For example, GitHub issues and discussions can be used for users to offer feedback and request new features for review. Developers and technical authors can then update the code and documentation for subsequent release.

Teams must decide if documentation is managed within a single monolithic repository, or smaller decentralised repositories. This evidently depends upon the size and nature of the project. For example, micro-service architectures may use a separate repository for each service, including: the source code and reference documentation. Conversely, a website may use a single monolithic repository. Furthermore, for large projects, documentation may be allocated a separate repository. This facilitates management and maintenance of how-to guides, tutorials and explanatory content.

Evidently, a process must be in place to manage documentation for software. In the same way that developers use pull requests to peer review code, the technical writers within a software team can follow the same principles. Furthermore, if possible the developers within the team could document as they code, using comments and annotations. This has the advantage that developers are included in the documentation process to facilitate automated generation of reference documentation for APIs and SDKs.

Quality assurance processes should be established for other types of documentation, e.g. tutorials, how-to guides, deployment guides etc. These allow the effectiveness of documentation to be reviewed, evaluated and updated, according to measurable success criteria. This involves testing that explanatory guides are followed correctly, via peer and user reviews. For example, can tutorials, how-to and deployment guides be effectively followed to achieve a target goal?

Finally, for complex problem domains and large projects, a centralised repository could be useful to maintain meta documents such as: procedures, style guides, domain glossaries and tracking documentation assets.

Think about the users of documentation. What are the needs they have, that documentation must serve?

With respect to users, it is advisable that the following documentation principles are established:

• Explanatory guides: Provide how-to guides and tutorials to explain how to use key software features. Furthermore, include documentation that describes how to install and configure the software.
• Frequently Asked Questions (FAQs): FAQs and troubleshooting guides should be documented to outline solutions to common issues and problems. 
Forums and discussion groups: Offer discussion groups using discord, IRC and slack to establish user communities.
• Change log: Use a change log to summarise new features and breaking changes as documentation evolves.
• Consistency and simplicity: To aid understanding, documentation should use domain vocabulary. Additionally, a glossary of key terms and concepts could be included. Furthermore, documents should be clearly structured with a consistent style.
• Accessibility: Documentation should be easily accessible to the users in a variety of formats, e.g. Wiki guides. PDFs and web sites.
• Vary learning content: Include a variety of learning content, including: videos, tooltips, alerts, code blocks and images. These are useful mechanisms to explain and summarise complex concepts, procedures and processes.
• Language translation: Documentation should be accessible to wide international audiences. Subsequently, it should be easy for users to translate documentation and search results into different languages via the use of navigation links and search tools.
• User feedback: A process must be established for users to offer feedback, e.g. via GitHub issues and discussions. This allows user needs to be captured, tracked and managed.

How should complex product documentation be structured?

Well structured documentation offers a practical experience for customers, with the benefit that it is easy to read and simple to navigate.

A software product has different types of documentation. Technical reference documentation focuses on APIs and SDKs, outlining the software interfaces that integrate with other systems. Examples are RESTful API endpoints and library functions. Conversely, user documentation focuses on knowledge transfer to bridge the gap between product development and customers. Examples are user guides, how-to guides, tutorials, FAQs and explanatory materials.

Large projects with APIs or SDKs should be decomposed across source code repositories, to reduce and maintain complexity. Decomposition should be done by functional and non-functional requirements. To facilitate maintenance, the documentation should accompany the source code in each repository. This provides version controlled documentation that is synchronised with each source code release.

Complex large user documentation should be structured to bridge the knowledge gap between product development and users. One way of providing structure is to decompose the product documentation into categories. For example, user documentation could be split into categories of glossaries, how-to guides, FAQs, tutorials, troubleshooting guides etc. Within a document, content may be categorised by the functionality, workflows and operations that users perform.

Provide a landing page for users that presents categorised information. The landing page is the first point of entry for customers. It helps users understand the purpose of the product and provides clear navigation links to different types of product documentation. Examples are:

• Overview: Explain the overall purpose of the product.
• Setup: Explain how to setup and configure the product.
• Usage: Provide navigation links to user guides, how-to guides, tutorials, API and SDK reference documents etc.
• Resources:  Provide explanatory material with examples and real world case studies. These are useful to showcase product usage within other projects and businesses.
• Support: Explain how users can give feedback. Provide links to support mechanisms, such as forums and user communities.
• License: Provide a link to license summary.
• Contribution: If applicable, explain how users can contribute to the product and documentation.

In summary, a monolithic, flat structured document should be avoided. This makes content harder to understand and navigate. A document that is decomposed into functional categories is easier to understand. Furthermore, use image and video content, code blocks, tips, alerts and callouts to enhance understanding for the reader.

What are some good principles for the on-going maintenance of documentation?

The following are recommended principles that can facilitate continuous maintenance of documentation:

• Simplicity: Keep documentation simple and structured to decrease maintenance effort. Technical documentation could be developed using Markdown. This is lightweight and can easily be converted to different formats, such as HTML and PDF. An example conversion tool is pandoc.
• Automation: Use automation where possible, e.g. incorporate building documentation using CI/CD pipelines. For example, tools such as hugo, docsify and docusaurus could be used to generate and publish a static documentation website. Additionally for APIs, use swagger document generation tools to create documents from comments and/or the API specification. This facilitates maintenance, since documentation is generated from code for versioned release.
• Style guides: Create and publish a style guide to provide consistency and enhanced quality. Style guides should specify standards for formatting and establish usage of key terminology. This enhances the quality of documentation, reducing errors and mistakes, thus increasing accuracy.
• User feedback: Establish a mechanism for users to provide feedback. For example, if the documentation is held in a GitHub or GitLab repository then issues could be recorded, tracked and updated. Example issues include feature requests and bugs.
• Quality assurance: Establish quality assurance procedures. This ensures that documentation meets defined acceptance criteria and is reviewed, prior to release.

What are the most important insights into the art of documentation that you have gained from your experiences?

I have written a variety of technical guides, learning materials, user guides and blog posts for a range of audiences and problem domains. This is within the software development and teaching sectors.

During the course of my career, I have found that understanding the domain and product is a vital collaborative process. This means actively engaging with developers, customers and marketing to understand the product and the needs of target users. Furthermore, I have always installed and used the product to gain first-hand experience in order to produce effective documentation.

To this end, I see that producing technical documentation is an active, iterative and continuous process. Documentation is refined as the product evolves, with the changing needs of users. These include: customers, system administrators and developers. Documentation is more effective when reviewed by peers and then updated accordingly.

I have gained valuable insights into tools that can be used to support the continuous evolution of documentation. The principles of software tools, such as source code repositories and CI/CD build pipelines, can also be applied to reference documentation. During my time as a Python developer, I have found it beneficial to integrate reference documentation with the source code and apply automation using swagger documentation tools. This way the reference documentation is versioned and evolves with the underlying source code.