Distcc
For those who have tried to compile on the N900, you might notice it is a bit slow since an ARM CPU is great for general use, but not---for example---compiling a kernel. It, however, is often useful to compile natively on the architecture you will be using. Originally this process was used to configure a distcc farm for the mini2440 ARM(el) board---and this is directly applicable to the N900 platform---and it works on my phone!
This assumes familiarity with qemu-system-arm, gcc (and possibly ccache+distcc), chroot installs, bridged networking and perhaps a few other concepts.
Where possible I distinguish the different system root-prompts as:
n900-chroot# The easydebian chroot prompt (you can ssh in too if you like) host-root# The high-power desktop system to run the ARM VM's under qemu guest-root# The ARM VM's themselves.
Contents |
Install Easy Debian
Before continuing, you will need to install Easy Debian on your phone. This is the fastest way to get a working native gcc environment on the N900. Once it is installed, install ccache, distcc, and build-essential:
n900-chroot# apt-get install ccache distcc build-essential
Then configure /usr/bin/gcc-ccache the same way as below in the easydebian chroot.
Install a few QEMU VMs
On your host computer (not the phone), download and install the Lenny armel qcow2, vmlinuz and initrd images from here:
I used the _small version so I can be as specific as possible with packages. I assume you use the _small version from here.
Install qemu-arm on your system, usually:
apt-get install qemu
and boot the VM:
host-root# qemu-system-arm -M versatilepb -kernel vmlinuz-2.6.26-1-versatile \ -initrd initrd.img-2.6.26-1-versatile \ -hda debian_lenny_armel_small.qcow2 \ -append "root=/dev/sda1"
Install the required packages on the VM
First update /etc/apt/sources.list for the country of your choice (defaults to .de). I recommend commenting the volatile repo unless you intend to use it. Then add the packages:
guest-root# apt-get update guest-root# apt-get upgrade guest-root# apt-get install build-essential ccache distcc
Configure distcc and ccache
-
Add the hosts to
/etc/distcc/hosts -
Configure
/etc/default/distccwith at least:STARTDISTCC="true" # configure your subnet ALLOWEDNETS="127.0.0.1 192.168.0.0/16"
-
create a ccache/distcc wrapper
root# mv /usr/bin/gcc /usr/bin/gcc-orig root# cat<<EOT > /usr/bin/gcc-ccache #!/bin/sh export CCACHE_PREFIX=distcc # List the distcc hosts here; I had trouble with /etc/distcc/hosts working export DISTCC_HOSTS='192.168.99.101 192.168.99.102 192.168.99.103' exec ccache gcc-orig "$@" EOT root# chmod 755 /usr/bin/gcc-ccache
Be sure to do this on the N900 too!
-
link the wrapper as the new gcc:
root# ln -s /usr/bin/gcc-ccache /usr/bin/gcc
-
test the distcc-ccache-gcc wrapper:
root# gcc -version gcc-orig: no input files distcc[2486] ERROR: compile (null) on localhost failed
The distcc error is expected
Configure the network to DHCP
guest-root# echo -e "auto eth0\niface eth0 inet dhcp" >> /etc/network/interfaces guest-root# echo -e "auto eth1\niface eth1 inet dhcp" >> /etc/network/interfaces
Halt the vm and launch multiple instances
- Halt the guest and close the window: guest-root# shutdown -h now
-
Configuring a network bridge is beyond the scope of this document, but roughly, this is my config on the *host* system. Here we assume that the N900 can route to the ARM VM farm (my network)---or that the bridge lives on the same network (more common).
root# cat<<EOT > /usr/local/bin/qemu-ifup-br0 #!/bin/sh iface="$1" bridgeName=`basename "$0" | cut -f3 -d-` echo "bridgeName=$bridgeName iface=$1" /sbin/ifconfig "$iface" 0.0.0.0 up /usr/sbin/brctl addif "$bridgeName" "$iface" EOT chmod 755 /usr/local/bin/qemu-ifup-br0 host-root# brctl addbr br0 host-root# brctl addif br0 eth0 host-root# ifconfig br0 192.168.99.222 host-root# ip route replace default via 192.168.99.1
-
Configure your local dhcp server to hand out 192.168.99.101--103 for mac addresses 00:16:3e:00:00:01--03. If you are using ISC's dhcpd, the config looks something like this:
host arm-vm1 { hardware ethernet 00:16:3e:00:00:01; fixed-address 192.168.99.101; } host arm-vm2 { hardware ethernet 00:16:3e:00:00:02; fixed-address 192.168.99.102; } host arm-vm3 { hardware ethernet 00:16:3e:00:00:03; fixed-address 192.168.99.103; } -
Launch the vm's:
host-root# for arm in 1 2 3; do qemu-system-arm -name armvm$arm \ -M versatilepb -kernel vmlinuz-2.6.26-1-versatile \ -initrd initrd.img-2.6.26-1-versatile \ -hda debian_lenny_armel_small.qcow2 \ -append "root=/dev/sda1" \ -net nic,macaddr=00:16:3e:00:00:0$arm \ -net tap,script=/usr/local/bin/qemu-ifup-br0 \ -daemonize \ -snapshot ; done -
The bridge listing should look something like this:
host-root# brctl show br0 8000.001cc0b1dadf no eth0 tap1 tap2 tap3
Run a distributed build on your N900
-
Download a recent vanilla kernel:
cd /usr/src wget -O- http://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.34.tar.bz2 | tar xjv cd linux-2.6.34 [...] make -j3 defconfig *** Default configuration is based on 'versatile_defconfig' # # configuration written to .config # make -j4 [...] -
Open top and watch your ARM VM's spin at 100% CPU:
USER PR NI VIRT RES SHR S %CPU %MEM COMMAND root 20 0 556m 362m 1492 R 101 6.0 qemu-system-arm -name armel-vm1 ... root 20 0 539m 278m 1472 R 101 4.6 qemu-system-arm -name armel-vm2 ... root 20 0 530m 216m 1472 R 99 3.6 qemu-system-arm -name armel-vm3 ... root 20 0 538m 208m 1472 R 101 3.5 qemu-system-arm -name armel-vm4 ...
As you can see, I have 4 VM's on my system, one for each core on the i7. Since the ARM architecture under qemu (currently) only supports a single core, you need to run a VM for each core.
You *could* configure distcc+ccache to cross-compile to armel on your native x86/64 hardware, however, that is a different tutorial (it would probably be much faster too, but perhaps less flexible). This might be extended to run on the N900's localhost to spin the native 600 MHz CPU too---and I leave this as an exercise for the reader.