1. 2023-04-21. Apparently, on other GNU/Linux distributions the syslinux files are located elsewhere. In the case of Ubuntu, they seem to be at /usr/lib/syslinux/modules/bios/*.c32 and /usr/lib/syslinux/mbr/mbr.bin
2. 2023-04-21. This post has now a part two [A]
Last night I revisited tldrlfs [1], a GitHub repo about making a very simple installation of Linux From Scratch, which, if followed as intented by the book, is really bloated. Turns out you really don't need much to build a Linux distribution: a kernel (ehem, Linux), a bootloader, and a shell. In order to make it a little bit more usable, you'd want an init system which can handle daemons and right-after-boot configurations, and some coreutils.
The last time I tried to do this I failed miserably. However, some time has passed, and with the development of Strife [2] I've reached a much deeper level of understanding on how operating systems work and are able to boot. So I thought I could write a script that builds a distro from scratch, and see how short it can get. After many hours of painful debugging this afternoon, I got it working, so I published it as a gist [3]. In this post, I'm going to describe how it works, because I'm sure that many people can learn a lot from this.
The finished script is only ~150 lines long.
When any PC-compatible computer boots, if it follows the "classic" route, the BIOS prepares the rudimental software (RAM, mostly), loads the first sector of the hard disk to the address 0x7C00, and jumps to it. This is known as the MBR, Master Boot Record, of the disk. In this sector resides the beginning of the bootloader, which is responsible for loading the rest of the bootloader, and finally the kernel. This can be explained much more in depth, and I do so in my Bachelor Thesis [4], for those interested.
From a big picture perspective, the Linux boot procedure is the following: bootloader -> kernel -> init -> shell. You must have this roadmap in mind throughout the whole reading of this post.
The most known bootloader is, by far, Grub. However, it's pretty big, could even be considered an OS per se, so I wanted to use something lighter. At first I wanted to use Limine [5], which both Strife and Daisogen use, since it supports the Linux boot protocol, but I couldn't get it working, so I gave up and went to the classic lightweight bootloader: syslinux [6].
The kernel is Linux, obviously. However, since I'm not a Gentoo user and don't really enjoy compiling things, the script steals the latest build from the Alpine Linux project. Alpine offers some variants of Linux, which mainly differ in how many drivers they have. The smallest, oriented towards being used in VMs, is the "virt" variant, so that's my choice. The kernel itself is 7 MB, and the modules are 22 MB in total.
For the init system, I've chosen the simplest I know, which coincidentally is the one recommended by tldrlfs: hummingbird [7]. I wouldn't daily-drive it, but nevertheless it's a very interesting project. My second option would've been OpenRC, but systemd is an overkill for this project.
For the shell I could've gone with bash, but I decided the best choice is busybox, a sort of an umbrella project that, in a single 2.3 MB binary, implements a POSIX-compatible shell (ash) and a lot of coreutils. While its utilities are not as feature rich as GNU's, it's a go-to for embedded systems.
Using busybox instead of bash, since hummingbird is an independent project, and syslinux is part of The Linux Foundation (it could be considered the "official" Linux bootloader), there's no GNU software in this distribution, which makes it a Linux distribution, but not a GNU/Linux system. This is no advantage over others, but it's a fun point to bring up.
Everything that needs to be compiled is done so statically, so that there's no need to handle dynamic libraries (".so" files) in the system.
The script begins by generating an img file which will be used as the hard disk image. I wanted to make it 64 MB, but due to the amount of modules in Alpine's linux-virt, I had to make it 128 MB. However, it still can be compressed using, for example, xz, down to less than 50 MB. The file is filled with zeros copying from /dev/zero.
Then, the disk is partitioned using "fdisk". I emulate its commands using "echo" so I create one single partition with the default values (ID #1, bootable, and with the whole disk size). In order to access this partition, I use a loopback block device which I set up with "losetup" on line 24. This assigns a loopback file to the image (generally /dev/loop0), discovers the partitions, and creates block devices to them too (/dev/loop0p1).
Once I can access the partition, I format it with ext4, the de-facto Linux filesystem, and get its Universally Unique Identifier (UUID) in order to be able to refer to it later. The script mounts the partition and creates the most basic directory hierarchy in it: proc, sys, run, dev, etc, and root.
- /proc contains a "procfs" filesystem, which contains some files created by the kernel about processes
- /sys, a "sysfs" one, regarding devices and drivers
- /run is for runtime files, such as PID files and sockets for interconnecting programs
- /dev contains device files, including block devices (/dev/sda) and TTYs
- /etc should be known by the reader, it contains configuration files
- /root is just the home of the root user, UID 0, in order to have a place to fall into after log in when the system boots
The kernel is downloaded (stolen) from the Alpine repositories, extracted, and its files are moved into the root of the filesystem at line 51. This creates /boot/vmlinuz-virt, which contains the actual kernel, and /lib/modules, with the modules to be loaded at runtime depending on the discovered hardware. Line 52 does nothing but I'm too lazy to remove it and commit and force-push.
Then, hummingbird is downloaded, compiled statically, and installed onto the root. While this copies some files, the most important is /usr/bin/hummingbird, which contains the entry point: the first userspace code that's executed after the kernel is done.
The same is done with busybox. The binary itself gets copied to /bin/busybox, but a bunch of soft links are made to it with different names: busybox detects the name of the executable from which is being executed (via argv[0]) and behaves accordingly. So, if you rename "busybox" to "ls", it will behave like "ls". This is kept with soft links, so no need for copies.
Next, the initramfs. When Linux boots and initializes the hardware and kernel data structures, sometimes it's not that easy to just "mount the root". It might be necessary to load additional drivers (modules), or even decrypt a partition if it's encrypted. For this reason, it's become common to set up a temporal filesystem (initialization RAM filesystem) with a very simple environment, whose whole goal is to mount the root. My initramfs is written as a shell script, so I bring busybox with me in line 89 to this alternate reality, as well as the modules. This script is based on the indications of the Gentoo wiki [8].
- It mounts /proc and /sys
- Using /sys, it identifies the hardware and load the necessary modules at line 98 [9]. This should just be doable with "mdev -s", but apparently there's a bug around that messes it up [10]
- It mounts the root partition, identified by the UUID saved before
- Unmounts the rest of filesystems
- And finallly switches the root, executing hummingbird
The initramfs is packaged with cpio, and compressed with gzip. It's then saved at /boot/initramfs.cpio.gz for the kernel to find.
In order to be able to log in when the system boots up, I write the files /etc/passwd and /etc/shadow, which define the user "root" and give it an empty password, so it's not even asked when logging in.
Syslinux is installed starting from line 127. The late stages of the bootloader are copied, and a very simple config is built for it. This file lets Linux know which is the root partition and where to find the initramfs archive. Note that the root identification might not be necessary since it's hardcoded in the initramfs script at line 101; it's just there for good measure.
Finally, the filesystem is unmounted, and with the file being free, the script finishes by writing the MBR, the stage 1 of syslinux.
When you run "./µlfs.sh", if everything goes fine, you'll have a bootable "hdd.img" file. Try it out with your hypervisor of choice. I recommend qemu:
qemu-system-x86_64 -hda hdd.img -enable-kvm
The "-enable-kvm" just makes it faster. If it gives any problem, remove it.
It might output some errors. Ignore them. One of them is a hummingbird bug; I've PRd a fix [11]. Hopefully by the time you're reading this it has been merged.
This could very well be the starting point of a serious distribution, which maybe someone who reads this wants to make. I would recommend a few things to make this process cleaner:
1. Do not compile statically. For any serious system, it's worth to dedicate some time to get dynamic libraries working
2. A real, non-stolen, Linux build is mandatory for any non-joke distro, so you can include what you need and nothing else
3. Do have a look at hummingbird to see if it fits your purposes. It's a very indie init, which means that it has not been extensively tested. Try OpenRC as an alternative. Or, for faster (although dubious) boot, use systemd.
4. Busybox is a toy for embedded systems and it lacks many commonly used options in its coreutils. Besides, the shell falls a bit too short for any real daily-driving purposes. Consider bash (or even better, zsh) and actual GNU coreutils.
5. Consider implementing a package manager in the lines of "apt" or "pacman". At some point you'll want to install stuff, and a package manager is the most user-friendly way to do it. You could instead go the LFS route and compile everything yourself, but this is tedious and hard to maintain.
I hope you have learned something from here, and good luck in your future Linux endeavours.
Thanks for reading.
-- jlxip