Building wxWidgets on all platforms

nScope is built on wxWidgets, a cross-platform GUI library written in C++. The library is essentially an abstraction of native graphics toolkits for existing platforms, and that allows us, the developers, to write a single codebase that compiles and runs on many operating systems with a native look and feel. It’s also open-source, and well-maintained. Although binaries of the libraries are distributed, we chose to build from source on each one of our build systems. Below is a table of all the systems and architectures we want to support with nScope.

OS GUI Toolkit Architecture Triple
Windows wxMSW 32-bit i686-w64-mingw32
Windows wxMSW 64-bit x86_64-w64-mingw32
OS X Cocoa 64-bit x86_64-apple-darwin15
Linux GTK x86 i386-linux-gnu
Linux GTK x86_64 x86_64-linux-gnu
Linux GTK ARMv7 arm-linux-gnueabihf
Linux GTK ARMv8 aarch64-linux-gnu
As you can see, we need to maintain 7 builds to support as many systems as we need. At first, that’s a lot to think about. Initially to support just the main three builds (Mac/Win/Linux), we spun up virtual machines in Virtualbox and ran builds manually in each system. This method worked to stand up nScope up to v0.7. But as we expanded architectures and moved towards stable support, we felt the need to graduate to a more production-level environment.

Vagrant VMs

Next, we started bringing up headless virtual machines with Vagrant. Vagrant is a neat tool that lets you build VMs from pre-made “boxes” which are essentially existing virtual machines that you can download from their repository. Vagrant can provision machines from script, so it’s very easy to recreate any development environment, and run it either locally with Virtualbox, VMWare, or remotely on AWS or RackSpace. Although it’s still running virtual machines, the machines themselves are much lighter weight (since they don’t run graphic interfaces) and they store all information to recreate the development environment in a simple bootstrap script, or if you want you can package your VM to a “box”

The plan was to bring up three machines, Mac, Linux, Windows. The Mac machine would need to build a single configuration, because Apple has used exclusively 64-bit intel processors for at least 6 years. The windows machine would need two build configurations, and the Linux machine would need 4, include two cross-compilations to ARM processors. Starting with Ubuntu because of their readily available packages for cross-compilation, we set out to compile wxWidgets for all four of the above linux configurations.

However, we ran into a stumbling block when Ubuntu’s package management system would not allow co-existing architectures for GTK libraries. That means we’d need to run 4 independent VMs just to support the Linux build. Although we did want the build infrastructure to be lighter, we were primarily motivated to keep the number of build machines to three or less, for reasons that we’ll discuss in a future blog post. How do we emulate many different Linux environments as lightweight as possible? Enter Docker.

Docker

Docker is a confusing piece of software, I’ll admit, and because of that we spent a long time avoiding it. But after getting to know it, it’s pretty awesome. To put it in the simplest terms possible, instead of creating virtual machines, Docker creates “containers” which are essentially everything in an Linux OS except the kernel. Docker “images” are analogous to Vagrant boxes, and Docker containers are like mini-VMs. But, there is a major difference. Because Docker uses the kernel of the host machine, Docker cannot create containers of an OS other than Linux. Docker can run on Mac and Windows, but to do so, it installs a mini Linux VM and runs Docker containers inside the mini-VM. So if we wanted to run Visual Studio, or some other windows executable, Docker is not the right tool.

On to our solution. We had already built a shell script to bring up an Ubuntu box to compile and run wxWidgets for each architecture. The only issue was that the package manager was placing a mutual exclusivity on the GTK dev files for multiple architectures. With Docker, it was simple (and fast!) to create four Ubuntu containers, and in each one install a different architecture build environment. Now, each of these environments exist on their own, and we eliminate any conflicts.

During this process, we stumbled upon an interesting Docker image called crossbuild/multiarch. With this image, it is possible to build for all the systems we need to support. Using this image as an example, we created two more Docker containers to build for Windows using the MinGW compiler. With 6 of the 7 necessary builds running in in Docker, we’re pushing to get the OSX build working that way too, but we’re not quite there yet.

Once we’ve figured out how to get the MacOS build up and running, we’ll update this page with our finished Dockerfile, and explain how to create a wxWidgets build machine for yourself.