A tour on Linux startup process

In the previous article, we looked at how hardware starts, found the bootloader, loaded the kernel, searched and configured the drivers and started the user space.

In this article we must deepen in the user space initialization process.

The art of jumping pulling on your boots

The most simple Linux user space consists only of one program statically linked (I.E. without dynamic library load) and no more. This process takes control of all things in the system and access everything from userspace, even, can directly access the physical memory via memory mapping. In this case, the Linux development is not different to bare metal or RTOS programming. In other hand, this must be create all device files and mount the most necessary filesystems like /proc or /sys

White this approach is plausible, the cost and care to take make it very impractical. Instead, the majority of Linux systems use a set of standard tools to initialize the userland and take advantage of multiprocessor programming.

The init system responsibilities

Linux ecosystems have many implementations of the init tools but all of them have the same responsibilities and take similar actions in similar ways varying only in configuration methods, side services provided to the userland, memory footprints, initialization time optimization and other non essential gifts.

But the task of an init system must be:

Prevent own crash: This is important because the first process is the father of all other process and a fatal failure on init is fatal for all of the system
Mount kernel instrumentation filesystems like /proc, /sys and (in some cases) /dev or other temporal filesystems linke /tmp or /run
Create initial device files on /dev or launch some method of device creation (like udev, mdev or other process that listen kernel events and create /dev archives representing the devices in the kernel)
Read configuration files and coordinate other process and scripts startup
Optionally, launch one or many programs that present an user interface like command terminal, serial line, graphics user interface or web server
Respawn killed or process that fail in a critical way

Keeping this in mind, the init process will be simple like a shell script or more complex like systemd or initsysv infrastructure.

The shell init script

This option uses the capability of the kernel to load and execute all types of text files that start with a shebang standard line.

This trick consists of a text file, marked as an executable file with the first line specifying the executable that is required to run these.

By example, a bash script start with:

The file with this initial text and the executable flag set, is suitable to use as a /bin/init file. The kernel read the first line of the file, find /usr/bin/bash and execute it with this filename as the first parameter

In the script you are free to put anything of your need… if, else, while, other process calls, include other scripts, etc.

The original systemV init system was this way and, by example, slackware init system resembled this method.

SystemD init style

In the other extreme, we found the de-facto init system on modern Linux systems called systemd. A modular program (really a set of process) that read various system configuration and make a startup graph with dependencies and conditions (ot start this process until the network is done, start these scripts before disk mounting, and so on)

For example, the ssh server is started after the network is up and operative, and requires the security system to be up and working (in parallel with network load).

SystemD is extremely big but modular and, with care, is very suitable to embedded systems with some limitations (the memory footprint and processor required is slightly higher than other systems) but the results in terms of performance and flexibility is the best in the Linux ecosystem.

Busybox init system

Busybox is a collection of programs that provide many unix/linux system utilities in a single executable, making it suitable for very small embedded systems.

To achieve them, busybox combine all single utilities in an one binary file containing all required code (cp, mv, ls, dd, etc) can share code without the shared library requirement (originally busybox is designed with non-mmu linux system in mind)

You can invoke the required utility using the name as the first parameter:

But, for convenience, busybox tries to inspect the name of the current command and invoke the correct utility based on it. Then, if you make a symbolic link with the correct name pointing to the busybox binary, the binary acts as the required utility.

Now, you can write cp, mount, ls, etc normally in your command line or script and all work as expected.

Busy box, bundle a little init system that simply calls /sbin/init (or you can specify another link called /bin/init or even /linuxrc) This init system is extremely simple but powerful for embedded devices. The only configuration required is a file called /etc/inittab with simple context describing actions, precedences, and the console it runs on. If init not found this file, a default inittab with generic commands is used (internally bundled in the binary)

The inittab for busybox init system is pretty easy text file formatted line by line, ignoring blank lines, or lines starting with a # character.

Other lines need to have an specific format with four fields separated by double dots ‘:’ characters in specific format (malformed lines are ignored with warning)

The tty may be empty, in this case, the program executed shared input and output from init.

The runlevel is not implemented in busybox and is only for compatibility reasons. In sysvinit system, this indicate one of various boot profiles (by example, runlevel 3 is console multiuser, runlevel 5 is graphics multiuser, runlevel 6 is shutdown and so on)

A minimal boot inittab is like this:

This minimal inittab make various interesting things to mention:

Mount proc filesystem in /proc directory: This filesystem is essential to normal work for linux utilities. From this directory, the kernel exports many information about the process (including current process) and internal information for various device drivers.
Mount dev filesystem on /dev directory: This provides a simple interface to devices populating in the kernel using hard coded names. By example, all serial are populated as ttyS<n> and all usb serial are populated as ttyUSB<n>
Mount sysfs in /sys. The sysfs is essentially a complement of /proc directory with more internal structures on the kernel (export low level driver information, module status, etc)
Then, a temporary filesystem in ram is mounted in the /tmp directory. This is useful to maintain temporary files like logs or partial file copy. Use with care because this file system consumes RAM to store the data.
The next step is to remount the root filesystem (/ directory) with read/write permissions. Normally, the kernel starts the root filesystem in read-only mode for security reasons.
After this, start the getty program over serial 0 (/dev/ttyS0 device) at 115200 bauds that present a login prompt over the console.
Finally, declare a ctrl+alt+del action as reboot command and shutdown action with an “unmount all” command.

Device files and devfs

Says a popular proverb: “In Unix (and Linux) all devices are files”. Even if it’s not entirely true, these show an interesting philosophy of Unix (and Linux), the natural way to expose kernel functionality is through a special file visible for the system. This enables the system to manage permissions and access to various devices like a simple file in a regular Unix directory.

Originally, in early Unix systems, the device file was a regular file in a regular file system and, when the kernel intercept the call to open/close/write or read, detect the file access and redirect these operations to the device driver.

In Linux, this approach is maintained but the inherent complexity of the system with many filesystems and drivers supports require a smart approach. This approach is implemented as multiple virtual filesystems that do not live in any real device but in temporal memory. When the linux kernel is starting, the init process needs to make these virtual filesystems accessible to the userland.

The /proc, /sys and /dev directories is reserved to these virtual filesystems (strictly, /dev directory can operate in a old-Unix-way using fixed files in a real filesystem but is not practically at this days)

While /proc and /sys filesystems are populated by internal structures in the kernel (/proc contains the process information and other kernel public info, and /sys contain low level inter structures), the /dev directory has many ways to handle the creation and destruction of the file devices.

In kernel devtmpfs: The most simple way to obtain all devices in the kernel visibles in the /dev directory. This filesystem creates device files using hard coded information when a device is created internally (for a module, for a driver or programmatically). The big advantage of this is that it requires a low footprint and no user space programs. In the bad hand, the hard coded names may be not suitable for any situations and the lack of permission customization makes the security a big challenge.
Busybox provided, mdev utility: busybox provides an application calls mdev that intercept the kernel events in device creation and populate the corresponding files in /dev directory (may be mounted as a tmpfs) using a rule file (or default in-kernel hard coded name and permissions)
Advance eudev application: Eudev is a more capable and advanced version of mdev device management system that is part of systemd but many times is bundled in a separated package for use with other init systems. This is the most capable and most expensive device manager daemon. The flexibility is the same as the desktop infrastructure and the footprint and memory consumption is higher than the other options.

For little linux systems without bigger security pretentious, devtmpfs is very convenient and suitable, but if you need more flexibility, the mdev or eudev options is the way even if you waste more memory and cpu on it.

A pray by the security

The explained way is inherently low-resource oriented and not security-oriented… if you need a very secure system, the normal way to do it is follow the recommendation of big linux systems with systemd init and SELinux subsystem.

You can secured a low-resource oriented system like busybox but require planning with careful attention to all areas and opened doors.

Next steps

Now that we have covered bootstrap systems, kernel linux operation and minimal init sequence, we are ready to mount our minimal linux system and see it working…

The target device will be very simple and fully supported by actual boot systems, mainstream Linux kernel and user space… The Allwinner v3s chip covers all of these requirements. Additionally is really simple to make a self-designed PCB with it because is presented in LQFP package and only require two power sources for operate (and incorporate a big 64MB memory in the package, enough to run a simple Linux kernel)

‍

See you later!

‍

Martín Ribelotta

Embedded Linux Developer at Emtech S.A

‍

Any Comments or questions, please feel free to contact us: info@emtech.com.ar

‍