New docs and search – switching away from Read The Docs

Last week we significantly revamped our docs and docs search. Which do you prefer, old or new?

As part of that switch, we migrated from away from Read The Docs to hosting them ourselves. Read The Docs is great but our requirements changed and we needed to move, and we think in the switch we’ve got a few immediate wins:

  • Unscientifically, our search function is much better, using elasticsearch rather than in-page JavaScript search
  • The look-and-feel of the docs is much more consistent with the rest of our web site
  • We made a couple of little macros to handle things like API method signatures - yes, Sphinx has these, but they’re tailored towards Python, not JavaScript, and we think our version is much better for our needs.

Background

When we first launched our Trigger.io Forge development platform, our documentation needs were pretty straightforward.

At the time, we only supported browser extensions, had a fairly limited number of “modules” (standardised cross-platform APIs to native features), and we were working on 2-4 week release schedule.

Read the Docs was a great tool for us at that time. It handled versioning for us, it meant there was one less thing for us to have to think about deploying and serving, and we already wrote a bunch of internal documentation in ReST: we started throwing new documentation at it and were happy for a good year or so.

After a while, however, the cracks started to show. Our codebase became much more modular and our release cycle accelerated, meaning it became a headache to keep our documentation in sync with our platform… We had to keep the documentation in a separate open-source repository from our private platform so that Read the Docs could consume from it… Read the Docs versioning only seems (seemed?) to work with branches on a git repo, but all of our other repos were mercurial… Using Sphinx to generate HTML from the ReST does let you create decent-looking output very quickly, but gets quite cumbersome when you want to step outside the normal workflow (adding extensions and including pre-built HTML for example)…

Preparing for Trigger 2.0 – “It’s not you: it’s me”

When we started working on a native plugin system for our platform, we knew things had to change. “It’s not you: it’s me”, we said to Read the Docs, and started converting our own documentation to Markdown, using pandoc for a first pass, then lots of manual touch-ups.

The thing is, we want the plugins our customers are writing to be properly documented, so that they can be effectively shared and used between and among teams and users. But, we can’t expect everyone to grasp the vagaries of ReST syntax, and the implicit assumption that Read the Docs projects are monolithic and homogeneous wasn’t really holding for us any more.

However, the benefits of having full control over how our docs are stored, processed and delivered are yet to be fully realised. We’re looking forward to being able to easily give our users per-module, per-version documentation, to easily tweak the appearance of all the docs to track our website, and maybe even include runnable examples for API methods.

Of course, taking control over our documentation like this has taken some effort. The biggest pieces were probably the conversion from ReST to Markdown (although pandoc did accelerate that quite a bit), and the generation of the navbar on the left side of our docs pages. But, once we had those in place, the clean, semantic HTML output from Markdown instantly looked pretty good with our Bootstrap styling. Integration of elasticsearch via Haystack was straightforward, and deployment just becomes a case of putting some HTML files in the right place.

The level of maturity in those ancillary libraries and tools has reached a point where NIH-ing your own documentation system may make a lot of sense: it did for us!

What’s next?

As the list of native modules keeps growing we ‘ll split that out from the core docs. But other than that we expect the basic docs structure and the way we host it to be stable for a while to come.

We’re always trying to improve the content and make it easier to discover. So if you have suggestions or feedback, please do get in touch at support@trigger.io. We’d love to hear from you!