Chapter 7. Tools and Techniques

Patching

This is the process of modifying a set of files with a 'patch' file containing a list of differences between two sets of files. This is how changes to source code are normally distributed. Patch files are generated by the diff command. You need at least version 2.5 of patch for patching the kernel. Here we use patching the binutils source as an example.

A patch is normally downloaded as a gzipped patch file (denoted by it ending in .gz). If so, gunzip it:

gunzip binutils-patch.gz

You can now patch binutils. Go into your unpacked binutils source directory, and run patch with your downloaded patch file, with argument -p1:

cd /usr/src/binutils-version
patch -p1 < binutils-patch

If you aren't sure exactly how a patch should be applied then the --dry-run option is very useful. Adding this means that patch will do its work, giving all the usual output but no changes are made to the files. To check out the effects of the above patch command, for example, you do:

patch -p1 --dry-run < binutils-patch

You should see a plethora of messages like:

Hunk #1 succeeded at 112

After patching has finished, you can check whether things have worked by searching for files ending in .rej (rejected patches):

% find . -name '*.rej'

In theory, there should be none of these files; all of the patches should have succeeded. However, if there are one or two failed files, it doesn't necessarily mean it won't build especially if the failed files are documentation or have names like README or endings like .info or .texi.

If you have several of these files, however, it could indicate something seriously wrong, and that it won't build. Check that the version of the source the patch is intended to be applied to matches the version of code you have. Also sometimes patches are 'reversed' - i.e. they have the original and modified source version the wrong way round in the diff command that generated the patches. Patch is smart and if it looks like this is the case it will offer to do them all 'backwards' for you.

The -p1 in the above example refers to how many directory '/' separators patch should strip off the filenames before patching. This is to allow for differences in the depth of nesting between the source tree the patch was created on and the one it is being applied to. Normally you will find that -p1 or -p0 is correct. If a patch doesn't seem to work then looking at the depth of filename nesting in it and using a different -p option is the first thing to try.

Once you understand how this works you can save some commands by combining the unzipping and patching commands into one line like this:

zcat binutils-patch.gz | patch -p1 

or, if the patch you have is bzipped instead of gzipped, then do:

bzcat binutils-patch.bz2 | patch -p1

Compiling a Kernel

Obtaining the Kernel Source

To compile your own kernel you need the source and the right set of patches. For all cases you need the base source tree and the ARM kernel patch. The SA1100 patches are usually a good idea too, and are necessary for SA1100 or later processors. For some devices, especially newly-supported ones, you may need a further patch for that device.

Kernel Source

Base source tree

linux-2.4.1-tar.bz2

or linux-2.4.1-tar.gz

ARM kernel (Russell King) patch

patch-2.4.1-rmk1.gz

SA1100 (Nico Pitre) patch

diff-2.4.0-rmk1-np2.gz

Each applied patch adds a suffix to the kernel name, so a kernel made with the 2.4.1 source with the rmk1 ARM kernel patch and the np2 SA1100 patch would be called zImage-2.4.1-rmk1-np2. The letters indicate the patch source. In the above cases they are the names of the patch maintainers Russell King and Nico Pitre respectively. Note that although they use different naming conventions, both patch files are of the same type.

The kernel source is normally available in both gzipped and bzip2 form. The latter is significantly smaller so that is normally the better one to download. You will need the bzip2 utility to decompress bzip2 files.

All these are on the CD but you will need to copy them into your development directory. Alternatively, obtain the latest version of kernel base, ARM kernel and SA1100 patches from these ftp sites but remember to use your local mirror, e.g. ftp://ftp.uk.kernel.org/ for the UK mirror.

Kernel Source - Current Version

Kernel Base

ftp://ftp.kernel.org/pub/linux/kernel/v2.4/

ARM kernel Patch

ftp://ftp.arm.linux.org.uk/pub/linux/arm/source/kernel-patches/v2.4/

SA1100 Patch

ftp://ftp.arm.linux.org.uk/pub/linux/arm/people/nico/

The source tree can then be unpacked into your development source directory. You can put this anywhere you like, but it must be kept separate from your native source tree in /usr/src/linux. This is because many systems expect the version of headers in /usr/src/linux to correspond to the version of the kernel and glibc being run on the host machine. This is usually different from the version you are using for ARMLinux development. We recommend putting your ARM kernel source tree in /usr/arm-linux/linux (this is alongside the toolchain in a cross-compilation environment). Note that you need a fair amount of disk space free. The unpacked 2.4.1 source is a little over 100MB, and you should have two copies of it, and configuring and compiling only makes it bigger.

In your development directory enter the following command to unpack the source:

tar -xzvf linux-2.4.1.tar.gz

Or if you have the bzipped version, use these options:

tar -xIvf linux-2.4.1.tar.bz2

Finally, unzip/untar the source before applying the ARM and SA1100 patches to give you a new kernel source tree. Using patch -p0 can confuse various versions of patch so its best to cd into the linux directory and use p1, thus:

cd linux
zcat ../patch-2.4.0-rmk1.gz | patch -p1 
zcat ../diff-2.4.0-rmk1-np2.gz | patch -p1 

Kernel source mangement

If you are intending to work on the kernel, the first thing you should do is take a copy of the pristine tree. This is so that you can generate patches against it after you have made changes. The simplest way to do this is simply to do:

cp -a linux linux-2.4.1.orig

However, as in practice you will only ever change a few files and having two copies of the whole thing, 99% of which are going to remain untouched, means that you may prefer to use hard links to create the copy as then only the changed files take up space. This method relies on the editor creating a new file after every save so that the original is preserved. If you have need of such an editor, take a look at MicroEmacs or GNU Emac's back-up-when-copying-when-linked facility. To create a hard-linked source copy do this:

cp -a -l linux linux-2.4.1.orig

Alternatively, you can use CVS to control your source versions, but that is not covered here.

Making a Kernel

First you should tidy up the source to get rid of old files and stale dependencies, especially if you have previously configured for a different device with make mrproper. You should then issue the appropriate make target-device_config command to set up a default configuration file for your target device. The list of possibilties for target-device is found in linux/arch/arm/def-configs/. You will then need to set up a kernel configurations file using the defaults which have just been created. This can be accomplished with the make oldconfig command. Finally, make dep sets up kernel dependencies and sub-configuration files on the basis of the specified config whilst make zImage builds the kernel image. The kernel should consequently appear in linux/arch/arm/boot/zImage. So, to summarise:

make mrproper
make target-device_config
make oldconfig
make dep
make zImage

Making a RAMdisk

The RAMdisks that are available for popular devices are usually fine for getting started, but sooner or later you will need to make our own. The steps involved are:

  1. Put all the files you want to put on the disk in a suitable directory with the correct directory structure;

  2. Zero out a block of memory (this is so that the spare space in your RAMdisk compresses as much as possible - if it was full of whatever random stuff was in ram at th time it would waste space in the final RAMdisk);

  3. Make a filesystem in this memory;

  4. Mount it;

  5. Copy the files you prepared into it;

  6. Unmount it;

  7. Compress it.

So here's an example. The size of the RAMdisk is 2MB, your prepared files are in the directory preparedfiles, and we are making a conventional ext2 RAMdisk, rather than something more exotic. /mnt/ramdisk should already have been created as a mount point. You will need to be root.

dd if=/dev/zero of=/dev/ram bs=1k count=2048
mke2fs -vm0 /dev/ram 2048
mount -t ext2 /dev/ram /mnt/ramdisk
cp -av preparedfiles /mnt/ramdisk
umount /mnt/ramdisk
dd if=/dev/ram bs=1k count=2048 | gzip -v9 ramdisk.gz

The -m0 option to mke2fs specifies that no extra space for the super-user is reserved - again to minimise the size of the filesystem.

Looking at existing RAMdisks

To see what is in an existing RAMdisk you need to gunzip it and loop-mount it as a filesytem. Your kernel need loop support for this to work, but a desktop kernel will nromally have this (it's really useful). Lets say you have a RAMdisk called ramdisk_ks.gz (from the Assabet distribution). It's a good idea to work on a copy of the disk, not the original. Do this (/mnt/ramdisk should already have been created as a mount point and you will need to be root):

gunzip ramdisk_ks.gz 
mount -o loop ramdisk_ks /mnt/ramdisk

Now you can see the contents by browsing /mnt/ramdisk like any other filesystem. You can even add files by copying them in if there is space left in the RAMdisk. The kernel needs to have support for the filesystem used in the RAMdisk.

Making your own patches

When you have changed some bit of the kernel, e.g. supported a new device or fixed a bug, you need to make a patch of your changes in order to distribute it to others and put it back in the kernel. This is easy to do. Say you have the patched version of the kernel in linux and the original version in linux.orig, then issue these commands.

make mrproper
diff -urN linux.orig linux > my_changes.diff

The make mrproper tidies up all the extra files lying around so that the diff is only of the actual relevant files. Note that the order of the linux.orig and linux arguments above is significant. If you do it the wrong way round you will have made a 'reversed' patch, which is at best confusing. You now have linux/ and linux.orig/ as well as a patch - my_changes.diff. You may now want to rename linux/ to linux.orig so that future patches do not duplicate the chages thus far - this enables each patch to be specific to a change, rather than monolithic. This helps the kernel maintainer (see below).

Russell King maintains an automated 'Patch State System' on his website at http://www.arm.linux.org.uk/developer/patches/ which tracks patches applied to the ARM Linux kernel. It is intended that all ARMLinux developers who wish their patch to be integrated into the ARMLinux kernel (aka -rmk kernels) should submit their patch to this system. These patches will then find their way to Linus.

However, this does not give the full story of patches applied; changes that go into Linus kernels are outside the scope of this patch system.

Please note that by submitting a patch or code to the patch system, you implicitly agree that it is suitable for redistribution with the kernel source. In other words, it is up to you to ensure that the material you submit to the patch system is suitable for inclusion in my kernel, and Linus' kernel. If you have any doubts about this point, please discuss it with Russell in private email.

There are certain restrictions on the system, so please follow these instructions carefully.

Sending in a patch

If you wish to send in a patch, please use the following procedure. It contains some important points that need to be followed:

  1. Generate the patch. Please tell diff to generate unified patches, recursing and new files if necessary. The options are diff -urN. Also, please generate them using a path relative to the directory directly above the main linux/ source tree. If you only want a small area of the kernel, then use diff -urN linux.org/arch/arm/kernel linux/arch/arm/kernel. In the extremely rare case that you absolutely must generate the patch using a different path, please make this obviously visible in the textual part of the patch mail (not following this will result in delays with application of your patch).

  2. Give the mail a descriptive subject line. This will appear on the web page, and in the release notes for the next -rmk version of the kernel.

  3. Include some text in the message explaining what the new feature is, the bug, or why the patch is needed.

  4. Put on a blank line "PATCH FOLLOWS". There must be no space before or after these words on this line.

  5. On a separate line, add a tag "KernelVersion: " followed by the kernel version that the patch was generated against.

  6. Append your patch as plain unwrapped text after this line to the end of the message. Note that the patches must be precisely what diff generates. It is not acceptable for TABs to be converted to spaces, or extra newlines or spaces to be added into the file. If you are unsure about the behaviour of your mailer, send the patch to yourself and examine it.

  7. Mail it to

When you receive a reply, you may wish to supply a detailed follow up article that explains exactly what your patch does. You can do this by replying to the mail you receive from the patch system.

If at any other time you wish to follow up on the patch, please use the subject line as the key (it should include the exact string "(Patch #number)", where 'number' is the patch number that you wish to follow up to).

Please absolutely do not send MIME encoded emails, even quoted-printable MIME encoded emails to the above address. If quoted-printable emails are sent, then you will receive an error message. All mails to should be plain unwrapped text.

Advice

  • If you have developed a device driver that uses a major/minor pair, please get it allocated according to the linux/Documentation/devices.txt file. Major/minor numbers are device driver specific, not device specific. Any device driver which re-uses major/minor numbers without an extremely good reason (included along with the patch) do not stand any chance of being accepted.

  • To ease the problems of applying patches, I recommend that you send a patch that adds exactly one feature or bug fix in a single email. Mails containing zero or less, or two or more features stand a greater chance of not being applied if any one part of the patch is not acceptable for some reason.

  • Patches that are inter-dependent should not be sent - these significantly increase the probability of both patches being rejected, unless both get applied at the same time. My recommendation is: don't do it.

  • All patches should be complete, that is applying it should not cause any breakage. i.e., adding a new architecture inline function in include/asm-arm/arch-* should be done such that all existing architectures will still at the very least compile correctly. If you're not sure how it should be handled on a particular architecture, put in a GCC #warning statement.