Building Brillo: first impressions

Update: Brillo was abandoned by Google in December 2016 in favour of Android Things

Brillo (https://developers.google.com/brillo/) is Google's operating system for devices that are connected to the IoT. They communicate with each other, with the cloud and with mobile apps using a system called Weave (https://developers.google.com/weave/). Brillo and Weave are still under wraps: you have to request an invite from the two links in order to get access to documentation, sample code, and so on. However, there is enough information publicly available to build and test a device running Brillo, which is what this blog is about. I hope to cover Weave in a later post.

Brillo is actually a cut-down version of Android. Basically, all the Java components have been stripped away leaving a much simpler embedded Linux operating system. When building Brillo, some things pop out immediately, which may give a preview of future Android builds:

  • GNU make is out: the build is run by kati/ninja (same for Android N preview)
  • GCC is out: almost all C/C++ components are compiled using clang (same for Android N preview)
  • Linux kernel source is in: the AOSP tree has vendor kernels in hardware/bsp/kernel. The kernel is built using GCC, as it is not (yet) possible to build it with clang
  • DBUS is in: I have not yet found out why
  • Toybox is in: leading to a much more Linux-like command line experience

Below, I show how to build and run the Brillo emulator

Setting up the build system

This is basically the same as building Android from AOSP: the instructions are here http://source.android.com/source/initializing.html. I used a fairly old octocore AMD machine with 16 GiB RAM running Ubuntu 14.04.4 64-bit desktop. Note that although Brillo targets do not contain any Java code, the build fails at the very beginning if you don't have OpenJDK 1.7 installed.

Getting the code

The Brillo code was pushed to AOSP in November 2015. I will start by syncing the master branch:

Allow a while for that to complete because it has 34 GiB to download.

Building the emulator

The procedure is basically the same as for any AOSP build:

$ . build/envsetup.sh
$ lunch

Select brilloemulator_x86-eng (or, run lunch brilloemulator_x86-eng). Then build it using make, giving the number of CPU cores on the build machine as the -j parameter:

$ make -j8

The first thing you notice is this:

Running kati to generate build-brilloemulator_x86.ninja...

Kati is a GNU make clone which converts the many thousands of makefile snippets in an Android build into a single, huge, ninja file:

$ ls -lh out/build-brilloemulator_x86.ninja 
-rw-rw-r-- 1 chris chris 145M Mar 21 15:27 out/build-brilloemulator_x86.ninja

The main goal, apparently, is to speed-up incremental build of Android. There is some info in the README.md file at https://github.com/google/kati. To speed up incremental builds, and make them more robust, kati writes the timestamps of all the files and directories to a file, so that it only has to re-read the makefiles if something has changed:

$ ls -lh out/.kati_stamp-brilloemulator_x86 
-rw-rw-r-- 1 chris chris 12M Mar 21 15:27 out/.kati_stamp-brilloemulator_x86 

The ninja file is processed by Ninja, which is a simple build system that builds according to the rules in the input file. See https://ninja-build.org/

One nice touch is that when the build gets going, you see a nice progress indicator:

[ 44% 7550/16800] target .......................... 

Towards the end, it builds a kernel:

[ 95% 15960/16800] Building i386 4.4 kernel... 

This is a big deal. This step is optional, in that you can set up the device build files to copy a prebuilt kernel binary as with “old” AOSP builds, but the clear intention is that the kernel source is part of the archive. In hardware/bsp/kernel there are kernels for freescale, imagination, intel, marvell qcom, and rockchip. I just can't exaggerate how pleased I am about this.

Finally, you should see this message :

#### make completed successfully (34:37 (mm:ss)) #### 

The build time is quite a lot shorter than for contemporary Android builds, mostly because there is less to build, but also because of the new build system.

Running the emulator

The emulator a version of QEMU. You start it like this (assuming that you have previously sourced build/envsetup.sh and selected the target via the lunch command):

$ brilloemulator-x86

Before I go on, note that quitting the emulator is, maybe, not obvious. Remember that it is just a copy of QEMU that was started without a graphics screen (-nographic). So, to quit you type Ctrl-A X. That is, you press the Ctrl and A keys together, let go, and then press the X key, and you will see:

QEMU: Terminated 

Using Brillo at the command line feels much more Linux-y than earlier versions of Android, courtesy of Toybox (http://www.landley.net/toybox/). Toybox largely replaces the earlier command line tool called Toolbox. Now, commands such as “find”, “which”, and “uname” all work as on regular Linux. Even “ifconfig” works as it should, replacing the oddly crippled version in Toolbox. The shell is still the BSD Korn shell, but a later version. Both adb and logcat work as expected.

The basic directory layout is the same as Android: the init program and init*.rc scripts are in the root, which is an initramfs (ramdisk). The main system files are stored in the read-only system directory, which is much cut-down from Android:

# ls -p /system
bin/ build.prop etc/ lib/ usr/ xbin/

Run-time data storage is in /data, as normal. In here you find:

# ls -p /data
adb/      app-private/  data/       mediadrm/        resource-cache/
anr/      backup/       drm/        misc/            ss/
app/      bootchart/    local/      nativebenchmark/ system/
app-asec/ bugreports    lost+found/ nativetest/      tombstones/ 
app-lib/  dalvik-cache/ media/      property/        user/

There are some Dalvik related directories which I take to be atavistic vestiges from Android, but I could be missing something here, who knows?

Finally, this is the list of processes (with kernel threads removed for brevity):

# ps 
USER      PID   PPID  VSIZE  RSS   WCHAN            PC  NAME 
root      1     0     7820   1820           0 08143195 S /init 
[...]
root      981   1     3548   1476           0 0812f640 S /sbin/ueventd 
[...]
logd      1254  1     12304  2972           0 b75b1df1 S /system/bin/logd 
dbus      1257  1     6124   3204           0 b738fbd0 S /system/bin/dbus-daemon 
system    1258  1     6400   2024           0 b755bad6 S /system/bin/servicemanager 
shell     1261  1     4864   1720           0 b74bcdf1 S /system/bin/sh 
root      1262  1     7548   1024           0 0811d050 S /sbin/adbd 
keystore  1264  1     9384   4164           0 b714aad6 S /system/bin/keystore 
media     1265  1     32144  11788          0 b70d9ad6 S /system/bin/mediaserver 
system    1266  1     9104   4084           0 b730b8d5 S /system/bin/nativepowerman 
root      1267  1     9208   3860           0 b75498d5 S /system/bin/peripheralman 
root      1268  1     11280  4236           0 b7325ad6 S /system/bin/sensorservice 
wifi      1269  1     10900  4380           0 b7347c55 S /system/bin/wpa_supplicant 
audioserver 1270  1     18608  8568           0 b6af28d5 S /system/bin/brilloaudioservice 
root      1272  1     4864   2432           0 b744fc55 S /system/bin/sh 
metrics_coll 1273  1     11840  5772           0 b70ff8d5 S /system/bin/metrics_collector 
metricsd  1274  1     12804  5808           0 b74ec8d5 S /system/bin/metricsd 
root      1275  1     5408   1828           0 b74d9da1 S /system/xbin/perfprofd 
trunks    1277  1     13268  5112           0 b70938d5 S /system/bin/trunksd 
root      1278  1     15048  7852           0 b71f58d5 S /system/bin/update_engine 
webserv   1280  1     12848  6968           0 b71bf8d5 S /system/bin/webservd 
root      1300  1     13640  8760           0 b72c18d5 S /system/bin/shill 
firewall  1328  1     10524  6628           0 b73aa8d5 S /system/bin/firewalld 
dhcp      1340  1300  6208   1900           0 b75c58d5 S /system/bin/dhcpcd-6.8.2 
system    1342  1     6408   2100           0 b7353bd0 S avahi-daemon: running [linux.local] 
tlsdate   1343  1     6836   3584           0 b72b98d5 S /system/bin/tlsdated 
system    1344  1     16740  8888           0 b6e9b8d5 S /system/bin/weaved 
root      1345  1343  6708   368            0 b72bafa6 S /system/bin/tlsdated 
system    1347  1300  5172   1756           0 b7572c55 S /system/bin/dnsmasq 
root      1352  1261  4864   2476           0 b7580df1 S /system/bin/sh 
root      1370  1352  6208   2268           0 b7538fa6 R ps

Among the daemons you can see running, many are to do with network connectivity: wpa_supplicant, shill, firewalld, dhcpcd-6.8.2, avahi-daemon and dnsmasq. Then there is the Weave protocol manager, weaved, and a web server, webservd. Some components are obviously using binder to communicate: servicemanager and sensorservice, for example. And there is dbus-daemon...

Plainly there is still a lot to learn here. I will be following up with more blogs as time goes by.