How Forge works and why we’re proud of it

We think a lot about how to make the process of creating mobile apps using our Forge framework easier. The result of that is it takes just 3 commands to create a build and run it in a simulator, and it takes seconds to get started:

> forge create

> forge build

> forge run ios

There’s a lot of heavy lifting that goes on behind the scenes to make it that simple, and one unique aspect of our Forge framework is how our build service compiles our customers’ apps – by splitting the build between local tools and a remote server. The approach keeps users’ local development environments as simple and platform-agnostic as possible, and allows our servers to run multiple platform compiles in parallel to deliver the builds faster.

End result: the normal development cycle is orders of magnitude faster than it is with other solutions, with builds taking seconds at the command-line rather than minutes to compile or requiring you to zip up and upload code through a web interface.

Here’s a run-down of how Forge builds work behind the curtain.

The app creation process

Once you’ve signed up to Trigger, the first thing you’ll do is download our Python build tools, which you’ll set up locally at the command line. (The build tools are completely open source, and you can find them on Github here.)

With these, you set up your basic app architecture. Running forge create, you’ll check in with our primary server to verify your account, and create a unique ID for your app. The local tools then set up a directory structure on your local filesystem, which will be the basic scaffolding for your app: within that structure you can place all your HTML, CSS and JavaScript files, and configure your options and features in the
config.json file.

The clever part comes when you run your first build:

On executing the build, Forge will (1) take the contents of your config file and fire it off to our server – we run Django and nginx, all hosted on EC2. Receiving the build request, our server matches your ID and config.json contents up against your account credentials to determine which platforms need to be built, and then compiles a template app for each of the given platforms – creating a specific template that will reflect the features you’ve enabled and disabled in the config.

Template wrapped apps will vary platform to platform: most browser add-ons don’t need to be compiled (it’s just a matter of organising JavaScript into place), but for Android we’ll compile the Java and for the iOS we’ll compile the Objective-C. Of course, we can’t do all those builds with the same back-end build servers (and you wouldn’t want to): you need to run iOS builds on a Mac, and so on.

On our end, the build is then (2) farmed out to separate build servers: both to run the compile with the appropriate back-end hardware, and so that builds can run in parallel and so return  much faster. Speeding up template creation is an ongoing goal for us: we have recently had good performance gains by creating a caching minification service to avoid JVM startup lag, and we’re trialling the use of RAM disks to cut down our I/O times.

(3) The template apps are returned to your machine, along with build instructions that dictate to the local build tools how to complete the final stages of the build locally. At this stage, the “Your code” green hatched section is effectively empty.

Structure of a Forge template app

(4) The contents of the src directory – your HTML, CSS andJavaScript – will be injected into the “Your code” green hatched section of the app, and various packaging and signing processes run to complete the build.

Advantages of local / remote hybrid build approach

By compiling on the server, we save the user from setting themselves up locally for compiling for different platforms. We take on all that complexity, which means we don’t dictate what you need to write and work on your apps – whether that’s in TextMate, WebStorm, Eclipse, XCode, Aptana, whatever. Our framework remains platform-agnostic, and just a text editor and command line will do.

Once the first build has been run there is no need to run the whole build again after making changes to the HTML, CSS and JavaScript. When you haven’t enabled or disabled any major features or functionality in the config file, you can re-inject with the local build tools. This cuts down the time to get your code running on a device from minutes to seconds.

By returning instructions for the final stage of the build to your local tools (rather than encoding them permanently in the tools), we ensure that there are as few as possible version clashes between your build tools and the Forge platform level you’re building against. We’ve already pushed out dozens of fixes and improvements to the Forge platform which take advantage of this loose coupling to avoid imposing constant build tool updates upon our customers.

Testing and CI

All this shifts much of the complexity to our back-end, with lots of moving parts to be kept in check. Whenever we release a new API, it’s a challenge to ensure changes don’t break functionality across platforms and devices. Unit testing is extremely important to us. Every time we make a change, we trigger off all our automatic tests with our unit test suite to look for problems. We do this continuous integration testing using a custom test aggregation
service (which we keep meaning to integrate with our Hudson setup…):

Internal Forge continuous integration dashboard

We also run interactivity tests based on the excellent QUnit JavaScript testing framework: these tests target the Forge features which are awkward to test automatically – buttons in the browser window, statusbar notifications, etc…

Lastly, we run Node.js-based end-to-end system tests before pushing new code out to the platform – to check that apps that have already been published through our platform will continue to function just the same after pushing the update.

Final word

All this to say that we’re proud of our back-end build system, and hope you see the value in the way we approach it. It offers our customers faster builds, less hassle, and let’s them push out streamlined, unbloated apps from unbloated environments. Interested? Give us a try.