Ubuntu平台采用Qemu搭建ARM虚拟机环境

1. Busybox编译

下载代码：

查看Busybox Source Control网站，有代码控制说明，我们采用git clone下载代码：BusyBox

git clone git://busybox.net/busybox.git

编译Busybox

在INSTALL文件中有编译说明，根据说明进行编译：

The BusyBox build process is similar to the Linux kernel build:

  make menuconfig     # This creates a file called ".config"
  make                # This creates the "busybox" executable
  make install        # or make CONFIG_PREFIX=/path/from/root install 

The full list of configuration and install options is available by typing: 

  make help

• make menuconfig编译环境配置，配置对应的交叉编译环境和安装路径等

配置交叉编译工具前缀，如arm64平台，需要在Settings栏配置 Cross compiler prefix(为aarch64-linux-gnu-)

• make编译
• make install安装编译结果

2. Rootfs生成

使用mkfs.sh脚本如下：

#!/bin/bash
sudo rm -rf rootfs
sudo rm -rf tmpfs
sudo rm -rf ramdisk*
sudo mkdir rootfs
sudo cp /media/hugo/kston/source/busybox-1.30.1/_install/*  rootfs/ -raf
sudo mkdir -p rootfs/proc/
sudo mkdir -p rootfs/sys/
sudo mkdir -p rootfs/tmp/
sudo mkdir -p rootfs/root/
sudo mkdir -p rootfs/var/
sudo mkdir -p rootfs/mnt/
sudo cp initfs/etc rootfs/ -arf
sudo cp initfs/sbin rootfs/ -arf
sudo cp initfs/bin rootfs/ -arf
sudo mkdir -p rootfs/lib
sudo cp -arf /usr/aarch64-linux-gnu/lib/* rootfs/lib/
sudo rm rootfs/lib/*.a
sudo /usr/bin/aarch64-linux-gnu-strip rootfs/lib/*
sudo cp initfs/lib rootfs/ -arf
sudo mkdir -p rootfs/dev/
sudo mknod rootfs/dev/tty1 c 4 1
sudo mknod rootfs/dev/tty2 c 4 2
sudo mknod rootfs/dev/tty3 c 4 3
sudo mknod rootfs/dev/tty4 c 4 4
sudo mknod rootfs/dev/console c 5 1
sudo mknod rootfs/dev/ram0 b 1 0
sudo mknod rootfs/dev/null c 1 3
sudo dd if=/dev/zero of=ramdisk bs=1M count=16
sudo mkfs.ext4 -F ramdisk
sudo mkdir -p tmpfs
sudo mount -t ext4 ramdisk ./tmpfs/  -o loop
sudo cp -raf rootfs/*  tmpfs/
sudo umount tmpfs
sudo gzip --best -c ramdisk > ramdisk.gz
sudo mkimage -n "ramdisk" -A arm -O linux -T ramdisk -C gzip -d ramdisk.gz ramdisk.img

fstab文件

cgroup /sys/fs/cgroup/ nodev 0 0

rcS文件

#!/bin/sh
mount -t proc proc /proc
mount -t sysfs sysfs /sys
mount -t tmpfs tmpfs /sys/fs/cgroup/


mkdir /sys/fs/cgroup/blkio
mkdir /sys/fs/cgroup/blk
mkdir /sys/fs/cgroup/cpu,cpuacct
mkdir /sys/fs/cgroup/cpuset
mkdir /sys/fs/cgroup/devices
mkdir /sys/fs/cgroup/freezer
mkdir /sys/fs/cgroup/hugetlb
mkdir /sys/fs/cgroup/memory
mkdir /sys/fs/cgroup/net_cls,net_prio
mkdir /sys/fs/cgroup/perf_event
mkdir /sys/fs/cgroup/pids
mkdir /sys/fs/cgroup/rama
mkdir /sys/fs/cgroup/rdma
mkdir /sys/fs/cgroup/systemd
mkdir /sys/fs/cgroup/unified

cd /sys/fs/cgroup/
ln -s cpu,cpuacct cpu
ln -s cpu,cpuacct cpuacct
ln -s net_cls,net_prio net_cls
ln -s net_cls,net_prio net_prio

/bin/sh

• 注意这里rcS是内核调用的用户空间初始化脚本，需要在最后执行/bin/sh进入shell才可以登录到shell界面进行命令行操作 
• rcS需要有可执行权限才能执行

3. 编译内核

make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- defconfig menuconfig Image

start_vm.sh

#!/bin/sh

qemu-system-aarch64 \
    -M  virt \
    -cpu cortex-a53 \
    -smp 2 \
    -m 256M \
    -kernel /media/hugo/kston/source/linux-stable/arch/arm64/boot/Image \
    -append "root=/dev/ram0 rw rootfstype=ext4 console=ttyAMA0 init=/linuxrc ignore_loglevel" \
    -initrd /media/hugo/kston/source/tools/fs/ramdisk.img \
    -nographic \
    -device e1000,netdev=dev0,mac='00:10:4c:00:22:ee' \
    -netdev tap,ifname=tap0,id=dev0,script=no,downscript=no

[    3.692809] VFS: Cannot open root device "ram0" or unknown-block(0,0): error -6

[    3.681683] uart-pl011 9000000.pl011: no DMA platform data
[    3.692809] VFS: Cannot open root device "ram0" or unknown-block(0,0): error -6
[    3.698246] Please append a correct "root=" boot option; here are the available partitions:
[    3.700281] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    3.702569] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.16.8 #5
[    3.703885] Hardware name: linux,dummy-virt (DT)
[    3.707895] Call trace:
[    3.708541]  dump_backtrace+0x0/0x1c0
[    3.709452]  show_stack+0x14/0x20
[    3.710532]  dump_stack+0x9c/0xbc
[    3.711323]  panic+0x130/0x2a0
[    3.712097]  mount_block_root+0x1a0/0x28c
[    3.717135]  mount_root+0x130/0x164
[    3.719369]  prepare_namespace+0x138/0x180
[    3.719975]  kernel_init_freeable+0x208/0x22c
[    3.720353]  kernel_init+0x10/0x108
[    3.720671]  ret_from_fork+0x10/0x18
[    3.721302] SMP: stopping secondary CPUs
[    3.725707] Kernel Offset: disabled
[    3.733074] CPU features: 0x0802004
[    3.733925] Memory Limit: none
[    3.734924] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
 

• 是因为在内核配置不对，没有配置支持ram blockdevice，打开 CONFIG_BLK_DEV_RAM
• 并且配置 CONFIG_BLK_DEV_RAM_SIZE 大小到较大，比如 40960

出现下面问题，是因为libssl-dev没有安装，后来安装libssl-dev有问题，可能是package依赖的问题

scripts/extract-cert.c:21:10: fatal error: openssl/bio.h: No such file or directory

安装libssl-dev出现问题：

The following packages have unmet dependencies:
 libssl1.0-dev : Depends: libssl1.0.0 (= 1.0.2n-1ubuntu5) but 1.0.2n-1ubuntu5.1 is to be installed
E: Unable to correct problems, you have held broken packages.

然后卸载软件包重新安装就好了：

sudo apt remove libssl1.0.0

sudo apt install libssl1.0-dev

然后就可以编译通过了