Building Android and Linux Kernel for Nexus One

System Requirements

Google's Android build system only supports Linux (Ubuntu) and Mac OS. Examples below all use Ubuntu. The two most important tools from the Android SDK are "adb" and "fastboot". But "fastboot" is not available in the Linux distribution of the SDK. We will build it from the source.

For the Impatient

If you already know everything, you can find required steps for Nexus One in summary from this thread. But your mileage may vary.

Compiling Android Platform

VCS

Google uses Git to as the VCS for both Android and its own Linux kernel. Their source is published on Git web at http://android.git.kernel.org/.

Here are a couple of Git tutorials for CVS users:

• Cogito - CVS Crash Course
  
• CVS to git Transition Guide
  

Google publishes its repo tool for managing Android source. Repo does not seem to work on the Linux Kernel source tree though.

Checking-out Source

1. Create two separate directories to hold Andorid platform and its Linux kernel source trees:

laptop-2:~/myandroid$ ls
kernel  platform

2. Next, change to the platform directory and and initialize the repo.

$ cd platform
$HOME repo init -u git://android.git.kernel.org/platform/manifest.git -b froyo

The branch name "froyo" can be found on the git web at:

http://android.git.kernel.org/?p=platform/manifest.git;a=summary

Remote branch names can be found in the "heads" section on that web page.

3. Sync with the source.

$HOME repo sync

This makes "~/myandroid/platform" the root of the platform source tree. We will use "platform" for short in the following examples.

Building

1. Set up the build environment from the root of the source tree as the following:

$ . build/envsetup.sh

2. If building a phone-specific Android, then build adb first. Otherwise, skip to step six. But note that you can only build a generic Android if you skip device steps. If you flash a generic Android image to your phone, your phone will not even get a radio signal. Nor will you be able to use any buttons on the device. So skip this step only if you just want to play with the emulator.

At the root of the source tree:

$ make adb

3. Make adb available on the path.

$ export $HOME/myandroid/platform/out/host/linux-x86/bin:$PATH
$ which adb
$HOME/myandroid/platform/out/host/linux-x86/bin/adb

4. We need to extract device specific files from Nexus One. Connect the phone to the laptop using the USB cable. The phone needs to be powered on. If the phone has not been set to USB debugging. Set it at:

Settings --> Applications --> Development --> USB debugging

Then check that adb can see the device.

/out/host/linux-x86/bin$ sudo ./adb devices
[sudo] password for userone: 
* daemon not running. starting it now *
* daemon started successfully *
List of devices attached 
HTXXXXXXXXX device

5. Google Nexus One is HTC Passion. So we now run the Passion-specific extraction script under the "device/htc/passion" directory.

Note that the device files must be extracted from a Android device that matches to your build. Here we extract from a Nexus One with 2.2 update because we are going to build the Froyo branch of the code.

$ cd device/htc/passion
/device/htc/passion$./extract-files.sh

Note that adb must be on the path in order for the extraction script to succeed. The extraction script will pull HTC proprietary files from the device and store them under "vendor/htc/passion".

/vendor/htc/passion/proprietary$ ls
01_qcomm_omx.cfg        libEGL_adreno200.so        libOmxVdec.so
AdieHWCodecSetting.csv  libGLESv1_CM_adreno200.so  libOmxVidEnc.so
akmd                    libGLESv2_adreno200.so     libopencorehw.so
AudioBTID.csv           libgps.so                  libq3dtools_adreno200.so
bcm4329.hcd             libgsl.so                  libqcomm_omx.so
default.acdb            libhtc_acoustic.so         libstagefrighthw.so
default_att.acdb        libhtc_ril.so              mm-venc-omx-test
default_france.acdb     liblvmxipc.so              parse_radio_log
fw_bcm4329_apsta.bin    libmm-omxcore.so           vpimg
fw_bcm4329.bin          liboemcamera.so            yamato_pfp.fw
libcamera.so            libOmxCore.so              yamato_pm4.fw

6. Change back to the root of the source tree and use "lunch" to set up appropriate build environment variables.

$ lunch

You're building on Linux

Lunch menu... pick a combo:
     1. generic-eng
     2. simulator
     3. full_dream-userdebug
     4. full_passion-userdebug
     5. full_sapphire-userdebug

Which would you like? [generic-eng] 4

The "generic-eng" build option will allow you to build an Android that is not device specific. But then you can't really use a generic Android image on a real phone because the generic image does not know how to use device-specific features such as buttons and camera. Nor can the generic image read radio signals. Therefore, the generic-eng build is only useful for emulator purpose. The main difference between generic-eng and userdebug is if the file system is read-write or read-only by default. We will select "full_passion-user-debug" here because we want to flash our images to the device.

7. Build the Android platform from the root of the source tree.

$ make

Note: When building "full_passion-userdebug" the first time, I hit a compiler error on "librpc.so not found" which caused references in HTC's proprietary libgps to fail compiling. After googling a solution, I found the following workaround:

$ mm librpc
$ make

Do not run "make clean" when performing the two steps above. I have since checked out the froyo branch many times and have never encountered this problem again.

8. Inspect build output.

The build artifacts are located under the out subdirectory.

$ ls out
casecheck.txt  CaseCheck.txt  debug  host  target  tmp  versions_checked.mk

The Android tools for Linux/X86 are located under "host". The device images are under "target".

Compiling Linux Kernel for ARM

Checking-out Source

I used Git instead of Google's repo for kernel building.

1. Clone the MSM family of the kernel tree from the Android Git to a working directory. As of July 2010, all HTC Android phones use Qualcomm's MSM family of the CDMA/GSM/UMTS chipsets. Nexus One is no exception.

userone@userone-laptop-2:~/myandroid/kernel$ git clone git://android.git.kernel.org/kernel/msm.git .

If building the emulator kernel, use "git://android.git.kernel.org/kernel/common.git" instead.

2. Check out the latest 2.6.32 kernel branch for Nexus One.

userone@userone-laptop-2:~/myandroid/kernel$ git checkout --track -b userone-msm-2.6.32-nexusonec remotes/origin/android-msm-2.6.32-nexusonec

When the check out is done, you can use git to verify the branch info.

userone@userone-laptop-2:~/myandroid/kernel$ git branch -a
  android-msm-2.6.27
  userone-msm-2.6.32
* userone-msm-2.6.32-nexusonec
  remotes/origin/HEAD -> origin/android-msm-2.6.27
  remotes/origin/android-msm-2.6.25
  remotes/origin/android-msm-2.6.27
  remotes/origin/android-msm-2.6.29
  remotes/origin/android-msm-2.6.29-donut
  remotes/origin/android-msm-2.6.29-nexusone
  remotes/origin/android-msm-2.6.32
  remotes/origin/android-msm-2.6.32-nexusonec
  remotes/origin/android-msm-htc-2.6.25
  remotes/origin/android-msm8k-2.6.29
  remotes/origin/msm-2.6.25

The root of the kernel tree is now "userone@userone-laptop-2:~/myandroid/kernel".

Building

1. Setup the build environment using the "build/envsetup.sh" script as shown in the platform build. This sets up important cross compiling environment variables for ARM build.

2. Set up environment variables for cross compiling.

$ export ARCH=arm
$ CROSS_COMPILE=arm-eabi-

3. Pull the kernel configuration from the device and then build the source. This step is the same for building a kernel for Android emulator.

$ adb pull /proc/config.gz
$ gunzip config.gz
$ mv config .config
$ make

The arm target can be found under ..kernel/arch/arm/boot.

$ ls arch/arm/boot
bootp  compressed  Image  install.sh  Makefile  zImage

The newly built kernel is "zImage".

4. Change to root of the Android platform tree and build the new Android with the new kernel.

$ rm -rf out
$ make -j4 TARGET_PREBUILT_KERNEL=$HOME/myandroid/kernel/arch/arm/boot/zImage

The TARGET_PREBUILT_KERNEL will instruct the Android build system to use our custom-built kernel.

Flash Android Image to the Device

Warning!! Flashing images will erase all your user data on the phone!

Warning!! Attempting this at your own Risk!! Your phone will be bricked!!

Warning!! Always have factory images handy so you can recover in case of bricked phones!

An unlocked bootloader allows flashing custom images to an Anroid Phone. Android Dev Phone 1 and 2 ship with unlocked bootloaders. But the bootloader on Nexus One needs to be unlocked first.

Both adb and fastboot can write files to the device. I use fastboot for flashing images:

Unlock the Bootloader in Nexus One

from the (#References):

# Reboot phone into fastboot: Power off device and hold down trackball while powering back on. (The fastboot screen is the one with the Androids on skateboards)
   # Open a command prompt and navigate to your Android SDK tools folder.
   # Type ‘fastboot devices‘ to make sure your phone is recognized.
   # Type ‘fastboot oem unlock‘ to unlock the bootloader.
   # Use volume keys to navigate to yes and press the power button to confirm.

Flash Android Images to Nexus One

We now flash android images that we just built to the device.

$ sudo $HOME/myandroid/platform/out/host/linux-x86/bin/fastboot -p passion -w flashall
[sudo] password for userone: 

< waiting for device >
--------------------------------------------
Bootloader Version...: 0.33.0012
Baseband Version.....: 4.06.00.12_7
Serial Number........: HTXXXXXXXXX
--------------------------------------------
                  checking mid... OKAY [  0.003s]
              checking product... OKAY [  0.009s]
   checking version-bootloader... OKAY [  0.002s]
       checking version-microp... OKAY [  0.004s]
     checking version-baseband... OKAY [  0.012s]
      sending 'boot' (2336 KB)... OKAY [  0.366s]
                writing 'boot'... OKAY [  0.917s]
  sending 'recovery' (2562 KB)... OKAY [  0.405s]
            writing 'recovery'... OKAY [  1.048s]
   sending 'system' (75301 KB)... OKAY [ 10.943s]
              writing 'system'... OKAY [ 25.634s]
            erasing 'userdata'... FAILED (remote: not allowed)
finished. total time: 39.418s

The error about failing to erase userdata is because we selected user build (i.e. "userdebug") when building android. We can flash it manually:

$ sudo $HOME/myandroid/platform/out/host/linux-x86/bin/fastboot flash userdata $HOME/myandroid/platform/out/target/product/passion/userdata.img
     sending 'userdata' (2 KB)... OKAY [  0.022s]
            writing 'userdata'... OKAY [  4.872s]
finished. total time: 4.894s

If the phone hangs after rebooting, we can restore it back to factory images.

Restore Nexus One to Factory Settings

1. Download HTC Nexus One images from HTC Dev Center. HTC currently only provides Android 2.1 images for Nexus One. You can use your Google account to get the official 2.2 OTA update after restoring the phone to factory images.

2. Unzip images to a directory.

userone@userone-laptop-2:~/Downloads/nexusone$ ls
android-info.txt  NexusOne_ERE36B_TMOUS.zip  system.img
boot.img          recovery.img               userdata.img

3. Flash all images to the device using fastboot.
This is the method I used to restore my Nexus One to factory settings.

$ sudo $HOME/myandroid/platform/out/host/linux-x86/bin/fastboot flash boot ./boot.img
$ sudo $HOME/myandroid/platform/out/host/linux-x86/bin/fastboot flash recovery ./recovery.img
$ sudo $HOME/myandroid/platform/out/host/linux-x86/bin/fastboot flash system ./system.img
$ sudo $HOME/myandroid/platform/out/host/linux-x86/bin/fastboot flash userdata ./userdata.img

I did not use "fastboot -w flashall" and "fastboot update [some].zip" because they complained about missing files.

Tips

• Use adb to boot the device into the recovery mode.
  

$ adb reboot recovery

• Use adb to store recovery image.
  

$ adb push <recovery-image-package>.zip /sdcard/update.zip

• Boot into the Nexus One recovery console.
  

After selecting "recovery" from the fastboot screen on Nexus One, a screen will show up with a little android and a caution sign. Press "Volume Up" and the "power" button at the same time to go into the recovery console.

• Test kernel boot without flashing. Very useful in determining if a new kernel image can boot.
  

$ sudo fastboot boot arc/arm/boot/zImage

• Pull kernel compiling option from an existing phone.
  

$ adb pull /proc/config.gz

References

• Getting Anroid Source
  
• Android Build System
  
• Building AOSP for the Nexus One
  
• Compile Android kernel from the source
  
• Fedora ARM/HowToQemu
  
• Kernel Hackers' Guide to git
  
• HTC Android Dev Phone 1
  
• Video: How to Unlock and Root a Nexus One
  
• HTC Kernel Source and Binaries
  
• Motorola Kernel Source
  
• How to: Rooting your Nexus One - a definitive guide to what’s involved
  
• Android Fastboot
  
• MSM/QSD Android Enablement Project
  
• Android Boot Process from Power-on
  
• How to Build the LINUX KERNEL for the ANDROID EMULATOR (Eclair version)
  
• Download and Build the Google Android
  
• Anroid Porting on Real Target
  
• OpenBinder
  
• Technote android-2.6.31-001: How to regenerate bionic/libc/common/kernel headers for linux kernel v2.6.31
  
• How to dump boot message without adb