Roll your Own

Mounting the rootfs will be done by the kernel. Usually, the kernel gets an initramfs passed as a pointer from the Bootloader, which is then running in RAM, starting daemons and mounting the actual rootfs. The kernel can also directly mount a block device via the root= commandline parameter.

Staging directory

To successfully build a rootfs, it first has to be assembled on the host system like so:

mkdir ~/rootfs
cd ~/rootfs
mkdir bin dev etc home lib proc sbin sys tmp usr var
mkdir usr/bin usr/lib usr/sbin
mkdir -p var/log

So that tree -d should produce the following output, when called inside the just created rootfs skeleton:

tree -d
.
├── bin
├── dev
├── etc
├── home
├── lib
├── proc
├── sbin
├── sys
├── tmp
├── usr
│   ├── bin
│   ├── lib
│   └── sbin
└── var
    └── log

File permissions

In a POSIX conforming system, every file has a 32-bit long description of which users are allowed to do what. There are 3 main flags:

  • read ®

  • write (w)

  • execute (x)

3 permission groups:

  • the owner himself

  • the group

  • everyone else

Permissions

and 3 extra bits:

  • SUID (4): “If the file is executable, it changes the effective UID of the process to that of the owner of the file when the program is run.”

  • SGID (2): “Similar to SUID, this changes the effective GID of the process to that of the group of the file.”

  • Sticky (1): “In a directory, this restricts deletion so that one user cannot delete files that are owned by another user. This is usually set on /tmp and /var/tmp.”

Permissions can be viewed with the ls command:

ls -l bin/ping
-rwxr-xr-x 1 root root 72344 Feb  5 05:37 /bin/ping

To set the permission flags, use the chmod command:

sudo chmod 755 bin/ping
-rwxr-xr-x 1 root root 72344 Feb  5 05:37 /bin/ping

755 is 7 (=4+2+1) rwx for the owner, 5 (=4+1) r-x for the group and 5 (=4+1) r-x for everyone. To change the leading octal digit (SUIC, SGID, Sticky), prepend the code to the permission bits. In this case: 4755 for 755 with the SUID flag:

sudo chmod 4755 bin/ping
-rwsr-xr-x 1 root root 72344 Feb  5 05:37 /bin/ping

Note

The x in the owners section has been altered to be an s.

Changing the ownership of a file can be done with the chown command: To also change the group ownership, use the -R option:

sudo chown -R root:root *

The -R option also changes the group owner.

Populating the rootfs

Executables

  • init: This is the first script to be executed. Here we are using the basic init script provided by BusyBox.

  • shell: To run scripts and create a command prompt, a shell is essential. There is a variety of shells to choose from:

    • bash: Standard Desktop Linux shell

    • ash: Much smaller than bash, thus a very popular choice for embedded systems.

    • hush: Very small shell, often used in systems with very little memory.

BusyBox

BusyBox was created to perform functions of essential Linux utilities. It is a collection of applets, which are combined into a single binary. E.g. reading a file:

busybox cat FILE

Busybox usually isn’t used like this. Usually, a symbolic link is created between the program you want to execute and busybox. In the example above, the /bin/cat file would be a symlink to /bin/busybox, so that everytime that the cat command is called, it is actually BusyBox that is executed.

Building BusyBox

Building busybox uses the same Kconfig and Kbuild system as the kernel.

git clone git://busybox.net/busybox.git
cd busybox
git checkout 1_9_stable
make distclean
make defconfig
make ARCH=arm CROSS_COMPILE=arm-cortex_a8-linux-gnueabihf-
make ARCH=arm CROSS_COMPILE=arm-cortex_a8-linux-gnueabihf- install

Libraries

Libraries are files that are connected to programms. Since linking them all statically would use a large amount of space, using shared Libraries is a more reasonable choice. Choosing the correct libraries can be done in two ways:

  • Copy all .so files from the sysroot directory of the toolchain

  • Cherry-picking only the .so files we need

Note

A full glibc is quite large, so cherry picking will be our choice:

cd ~/rootfs
arm-cortex_a8-linux-gnueabihf-readelf -a bin/busybox | grep "program interpreter"
arm-cortex_a8-linux-gnueabihf-readelf -a bin/busybox | grep "Shared library"

This displays all the libraries we need. To find them in the sysroot dir of the toolchain, run

arm-cortex_a8-linux-gnueabihf-gcc -print-sysroot ~/x-tools/arm-cortex_a8-linux-gnueabihf/arm-cortex_a8-linux-gnueabihf/sysroot

To shorten the writing time with these commands, export the SYSROOT variable:

export SYSROOT=$(arm-cortex_a8-linux-gnueabihf-gcc -print-sysroot)

Now copy all needed libaries with the cp -a command:

cd ~/rootfs
cp -a $SYSROOT/lib/ld-linux-armhf.so.3 lib
cp -a $SYSROOT/lib/ld-2.22.so lib
cp -a $SYSROOT/lib/libc.so.6 lib
cp -a $SYSROOT/lib/libc-2.22.so lib
cp -a $SYSROOT/lib/libm.so.6 lib
cp -a $SYSROOT/lib/libm-2.22.so lib

Repeat this process for each program

Stripping: Programs and libraries are often compiled with some information stored in symbol tables. Stripping this information from the library llok like this:

arm-cortex_a8-linux-gnueabihf-strip LIBRARY

where LIBRARY is the .so file you want to strip. Be careful stripping kernel modules: Some symbols are required by the module loader to relocate the module code. Use this command to remove debug symbols while keeping those used for relocation:

strip --strip-unneeded MODULE_NAME.

Device nodes

“Everything is a file” also represents the devices connected to your system. Device nodes are created using the program named mknod (short for make node):

mknod NAME TYPE MAJOR MINOR

where

  • name is the name of the device node

  • type is either c for character devices or b for a block device.

  • major and minor are a pair of numbers that are used by the kernel to route file requests to the appropriate device driver code. There is a list of standard major and minor numbers in the kernel source in the Documentation/ devices.txt file.

Device nodes must be created for every device that is planned to be used. That can be done manually or by using device managers to automate this process. Here is the manual approach shown:

cd ~/rootfs
sudo mknod -m 666 dev/null c 1 3
sudo mknod -m 600 dev/console c 5 1

Removing device nodes is rather easy, since everything is a file, thus can be removed by the rm command.

Better Device-Node-Management

There are three main ways of dynamically creating device nodes:

  • devtmpfs

  • mdev

  • udev

The only disadvantage on using those programs is, that they will eventually take a little time for each device node to create during boot time.

Proc and Sysfs

/proc and /sys are (as mentioned above) pseudo file systems, created to display the status of the kernel, its modules and so on. There is even functionality to write into some of these files, thus creating an information input to the kernel.

Both are being created on the fly by the kernel. The proc and sysfs filesystems should be mounted onto /proc ans /sys:

# mount -t proc proc /proc
# mount -t sysfs sysfs /sys

More information about the proc and sysfs can be found here.

Getting the Rootfs to the Target

There are multiple ways to mount the rootfs.

The most commonly used is via initramfs. The initramfs is a minimalistic file system, that is loaded into ram by the bootloader. It contains the init script and will mount the rootfs later on.

There is also the possibility to use a Disk image. It is a copy of the rootfs formatted and ready to be loaded onto a mass storage device onto the target. Often times, this is used on Installer images.

Last, but not least, there is the Network filesystem. Instead of flashing the image onto an sdcard, the filesystem is mounted by the target at boot time via nfs. In development, this can save a lot of time!

Systemd

Systemd is an important collection of background processes (daemons), such as wifi and syslogd, and libraries. These processes can be started at boot time, forming a chain of dependencies that are executed after each other or (if not dependend) in parallel.

To run a daemon process directly, add the deamon to the /etc/inittab:

::respawn:/sbin/syslogd -n

With the respawn flag, the process will restart after termination. The -n option lets it run as a foreground process. In the same way, klogd should be added to the inittab file.

User Accounts

Of course, every system needs user accounts. To create those, the /etc/passwd, /etc/group and /etc/shadow files are needed.

In /etc/passwd, there is unsensitive information, since everyone can read it.

root:x:0:0:root:/root:/bin/sh
daemon:x:1:1:daemon:/usr/sbin:/bin/false

Every line consists of

  • The login name

  • A hash code to verify the password

  • The UID

  • The GroupID

  • A Comment field (often blank)

  • The home directory (~)

  • The shell for this user

The password hash usually isn’t stored in the passwd file, since only unsensitive information shall be written down here. Some programs have to read users, so there is just an “x” in the password hash spot. This means that the password is stored in /etc/shadow, a file that is only readable by root.

root::10933:0:99999:7:::
daemon:*:10933:0:99999:7:::

This file consists of

  • The login name

  • A password hash code

  • Seven fields for password aging

If the password hash is empty, the user doesn’t need a password for login.

Group names are stored in, you guessed it, /etc/group.

root:x:0:
daemon:x:1:

Just like the passwd file, everyone can read this file. It is built like this:

  • Group name

  • Group password (x -> There is none)

  • GroupID

  • Optional list of users that belong to this group (comma seperated)

To activate all this, these files have to be in the staging directory. Note, that the shadow file needs permissions 600!

There is a programm named “getty” in BusyBox. It starts the login procedure, so we add it to the inittab file with the respawn flag, so it always starts up again, after a login shell is terminated:

# Previous stuff like "::sysinit:/etc/init.d/rcS"
::respawn:/sbin/getty 115200 console

Configuring Network

Assumptions:

  • Communication over eth0 interface

  • Only iPv4 configuration needed

The main network config is stored in /etc/network/intefaces. Since there are more needed for status checking etc. with ifup and ifdown, these directories also have to be created in the staging directory:

etc/network
etc/network/if-pre-up.d
etc/network/if-up.d
var/run

To configure a static IP address, the interfaces file shall be filled with

auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
    address STATIC_ADRESS
    netmask 255.255.255.0
    network 192.168.1.0

where STATIC_ADRESS is the wanted IP adress. For dynamic adress allocation via DHCP:

auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp

Since BusyBox already comes with udhcpc, there is not much to do, to have basic functionality. Just create a shell script in /usr/share/udhcpc/default.script or copy the one from busybox in the examples/udhcp/simple.script directory.

There are also some network components needed for glibc:

cd ~/rootfs
cp -a $SYSROOT/lib/libnss* lib
cp -a $SYSROOT/lib/libresolv* lib
echo "127.0.0.1 localhost" > /etc/hosts

source: Mastering Embedded Linux Programming - Third Edition By Frank Vasquez, Chris Simmonds