Linuxstamp:Buildroot

Building a Toolchain and Kernel for the Linux Stamp using Buildroot

Before we begin, it helps to understand the sequence of events that occur on your Linux Stamp hardware when you power it up.

The Startup Process on the Linux Stamp

It is easiest to think of the Linux startup process on the Stamp as happening in 4 distinct steps: (i) Loader, (ii) U-Boot, (iii) Linux Kernel and (iv) Root File System. If you are accustomed to using Grub or Lilo on your Linux workstation, this may seem a little convoluted, but it is really quite straightforward.

When the Linux Stamp boots, a tiny little loader called the Darren Loader fires up, which gives you the option of reflashing portions of the Linux Stamp's flash memory (more on that later!). If you aren't updating the flash, the loader passes control to U-Boot. U-Boot's job is to locate and pass control to the Linux Kernel, which is the piece of software that sticks around, interfacing to the hardware. When the kernel starts up, one of the key jobs it is responsible for is locating a root file system (i.e. all the files that are under "/" on the machine), mounting it, and then creating the initial userland processes. These initial processes will typically end up with allowing people to login somehow - for example via a "login:" prompt on a tty or via ssh or telnet.

Since the Darren Loader and U-Boot are already supplied for you on the Linux Stamp, you won't need to supply those. Therefore, there are two remaining distinct "pieces" of code that you need to supply to the hardware so that the Stamp can be up and running with Linux: these two are (i) the Linux Kernel and (ii) the Linux Root File System (i.e. all the files under "/").

To be very precisely, the "root file system" is really a set of executables plus a set of data files that are laid out in a file system hierarchy (e.g. under /, /usr, /etc, /usr/bin ...). This set of executables (e.g. rm, sh) and data files (/etc/fstab, /etc/passwd) can either be laid out as separate files in a directory or tar'd or cpio'd or otherwise combined in some way to create a single file. We colloquially refer to this collection of executables and data files as "the root file system software". While it is okay to think of it as "a piece of software", it is important to remember that it is really a collection of several executables and files.

In order to build these two pieces of software though, you'll need yet another piece of software - the cross compiler "toolchain". Let's take a look at that before we proceed to make a toolchain.

The Cross Compiler Toolchain

To compile the kernel and the executables that go on your root file system, you can't use the compiler you have on your x86 machine. The gcc on your x86 machine will normally generate x86 code targeted at your x86 machine, not at your Linux Stamp. Unfortunately, the Linux Stamp doesn't have an x86 processor - it has an ARM processor. To be able to get gcc to create code that will run on the ARM processor on the Linux Stamp, you will need a cross compiler and other tools (e.g. an assembler and linker) which are collectively called a toolchain. A toolchain is a suite of programs run on your HOST machine and compiles to create executables that will run to a different architecture, called the TARGET architecture. In your case, the HOST architecture is presumably x86 and your TARGET architecture is arm (which is what the Linux Stamp has on it).

Therefore, you'll need a toolchain to be able to compile the kernel and the root file system.

That brings our list of things we have to create up to three: (i) the cross compiling toolchain (ii) the linux kernel and (iii) the root file system.

Your host operating system

For purposes of this discussion, I'm going to assume that your desktop or server (called the "HOST") where you are going to run the build process is an x86 machine running Ubuntu (or some other Debian based) Linux. Specifically, these worked on my workstation, which is running Ubuntu 8.04 (Hardy Heron) with all the latest patches and updates. To be accurate, I am actually running it on a virtual machine using the fabulous and free VirtualBox (available here: [1] ) from Sun, running on my MacBook Pro. I've found VirtualBox to be more stable and much faster than either VMWare's Fusion or Parallel's Desktop for Mac. What's more, VirtualBox is completely OpenSource, which means that support for it is outstanding (far better than I ever received from Parallels who still don't (as of 15 August 2008) have full support for Ubuntu 8.04 Desktop even though the final version of Ubuntu 8.04 was released in April 2008, nearly 4 months ago).

Getting to Hello World on your own custom kernel and file system

Before we actually do anything, let's get an overview of what this page will help you do.

We are to going to stop at getting your Linux Stamp running your custom kernel and root file system ... we will try and get to the point where you will be able create a "Hello World" executable that you can run on your newly customized and compiled Kernel and Root File System.

Of course, before we can do that, recall that you will need to solve three problems: (i) toolchain (ii) kernel and (iii) root file system. Once you have all three, you will use that toolchain to (cross) compile your HelloWorld.c file into a HelloWorld ARM executable, which you can then transfer over to the Stamp's new file system and run it on the Stamp!

Getting and building using Buildroot

The secret to doing this with minimal pain is to use a wonderful timesaver called Buildroot.

The beauty of Buildroot

Buildroot is a very cool set of makefiles and scripts that can do all three of these for you. For a variety of reasons though, we are going to use Buildroot to do just two of the three ... we'll use it to build the toolchain and to build the root file system. We'll build the kernel separately, because, well ... once you have the toolchain ready, recompiling the kernel is pretty straightforward and the tools that come with the kernel to build it are perfectly adequate, and it's nice to keep things simple when we can :-)

Buildroot, as the name suggests, allows you to build a root file system. What's nice about it is that it allows you to build the root file system in a self contained directory, and you wont need to build it as root! The only point where you may need to become root or use 'sudo' is to install any packages that you may not already have installed. One of the nicest things about buildroot is that it will create a corss compiler that uses the uClibc library, which is a super-slimmed down version of the usual glibc library. uClibc is designed for embedded systems, and dramatically reduces the size of the binaries that you create.

So, lets get started with that ... before you can use buildroot, you need to have a bunch of packages installed. On Ubuntu/Debian, you can do this as follows:

$ sudo apt-get install build-essential libncurses-dev bison flex texinfo zlib1g-dev gettext libssl-dev wget autoconf

While we're at it, let's grab a couple of utilities that will come in handy later on, though they aren't really needed for getting buildroot to work:

$ sudo apt-get install autoconf dropbear minicom lrzsz

Grabbing Buildroot

Getting Buildroot is easy ... provided you have subversion installed. Before you begin, create a new directory that will contain all the files and build environment for the stamp. I'm going to assume you will create a directory called stamp in your home directory, and we'll do all our work from within that directory.

$ mkdir ~/stamp
$ cd ~/stamp
$ svn co svn://uclibc.org/trunk/buildroot

You should see a LOT of output as the files are pulled across the network. When all is done, and if all goes well, you should have a new subdirectory called buildroot ... let's switch to that directory:

$ cd buildroot

You are now ready to configure and build the toolchain and your root file system!

Building the Toolchain and Root File System

Buildroot allows you to very carefully finetune just what goes on your root file system. You can build a very comprehensive root file system that includes a compiler and a full development toolchain, or, you can build a very minimal root file system that can boot linux and do very little else!

The decisions on what to include and what not to include, are up to you, but the process of deciding that can be a little overwhelming to begin with. Fortunately, buildroot is designed so it saves the specific choices on what and how to build in a file called '.config'. To save yourself the hassle of building a configuration from scratch, I have created and saved a few config files for you. I have tried most of them, so they should work on your system as well.

Getting the Linux Stamp Custom Configurations for Buildroot and Busybox

While Buildroot needs a custom configuration file to build for the Linux Stamp, the appropriately named 'busybox' utility could also do with a linux-stamp custom configuration, so we'll use a slightly customized version of that config utility as well.

Grab the collection of config files that I have created for the Linux Stamp so far (I'll keep updating the tar files as I create other interesting configurations) and untar it

$ wget http://homepage.mac.com/vvaidy/linuxstamp/downloads/linuxstamp-br-configs.tar.gz
$ wget http://homepage.mac.com/vvaidy/linuxstamp/downloads/linuxstamp-bb-configs.tar.gz
$ tar xzvf http://homepage.mac.com/vvaidy/linuxstamp/downloads/linuxstamp-br-configs.tar.gz
$ tar xzvf http://homepage.mac.com/vvaidy/linuxstamp/downloads/linuxstamp-bb-configs.tar.gz

Configuring Buildroot

If you are doing this for the first time, I'd suggest you start out with the simplest of configurations, usually called something like Stamp-config-Stable001 (it may be 002 or higher by the time you read this). Let's use this as our starting config file, and update the Buildroot config with it.

$ cp Stamp-config-Stable001 .config
$ make oldconfig

If you want to take a look at what options this config file includes, you can look at it using the 'menuconfig' or 'xconfig' utilities. I'd suggest you resist the temptation to make any changes though, you'll have plenty of time to do that later. Even seemingly simple changes can case the build to fail, so at least when you are starting out, try and use the simple configuration to get you going.

$ make menuconfig

The only item that you may want to change, depending on your setup, is under 'Build Options' and then 'Number of jobs to run simultaneously'. I have set the default to 4, which should work well on most dual core cpus. If you are compiling on a relatively low-horse-power cpu, you may want to change this back to 1

Once you have had a chance to look around, exit the menuconfig utility.

Note: if you made unintended changes, make sure you do NOT save your changes. Even if you do inadvertently change something and save it, you can always copy back the Stamp-config-stable001 file and run 'make oldconfig' again.

Running Buildroot to create a toolchain and Root File System

Now you are ready to build ... and all that takes is doing a 'make'

$ make

This would be a good time for you to take a break and grab a cup of coffee or a different beverage of your choice, since this could take a little while. Buildroot will figure out what files it needs to download, grab them, unpack them and compile them. Depending on your machine and the bandwidth you have available, it can take anything from as little as 15 minutes up to an hour or even longer. Don't worry - it'll take a lot longer the first time around. The next time you do a rebuild, things will go a lot faster even if you start from scratch, since the Buildroot downloads the files just once and keeps them around in the 'dl' subdirectory.

If all goes well, it should complete without any errors and would have created your shiny new root file system in your 'binaries/Stamp' directory. For instance, here is what my directory looks like:

$ ls binaries/Stamp/
rootfs.arm.cpio
$ 

Congratulations! You have built a toolchain and used it to build your own custom version of Linux. Before we can use this root file system though, we'll need to do a little customization before we proceed. We'll handle that next.

Note: If something went wrong, check out the Troubleshooting section below. Unless you've been successful in creating this root file system, there really is little point in continuing, so you're going to have to take the time to fix it now.

Taking stock of what we just built

Let's take a moment to see what we just built. Clearly, we built a root file system and stuck it into a single cpio archive. Before we could do 1 that though, we also built the complete toolchain that we are going to use to compile our own programs as well as the custom Linux kernel.

Customizing the Linux Root File System

(this is a work in progress ... to be continued)