qemu-user-static [1] is a software to enable qemu [2] consumable to build and run multi architecture (ARM 64-bit, ARM 32-bit, Intel 32-bit and etc) on x86_64 Intel processor environment.
If you run docker or a docker compatible software, you can run commands like this.
$ docker run --rm --privileged multiarch/qemu-user-static:register --reset $ docker run --rm -t multiarch/ubuntu-debootstrap:arm64-bionic uname -a Linux 28c784e9c7bc 4.4.0-101-generic #124~14.04.1-Ubuntu SMP Fri Nov 10 19:05:36 UTC 2017 aarch64 aarch64 aarch64 GNU/Linux
$ docker run --rm --privileged multiarch/qemu-user-static:register --reset $ docker run --rm -it multiarch/ubuntu-debootstrap:arm64-bionic bash root@1e773d5d74af:/# uname -a Linux 1e773d5d74af 5.0.5-200.fc29.x86_64 #1 SMP Wed Mar 27 20:58:04 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux root@1e773d5d74af:/# exit exit
Why is this software important now?
Because of high growth of IoT that is to use a kind of mobile embedded device. HPC (super comuputing) on not only Intel CPU (x86_64), but also on ARM 64-bit CPU (aarch64), ARM 32-bit, IBM ppc64le, and etc. Open source project could be bulit on the multi-architecture environments.
But everyone does not have a environment for ARM and IBM architecture.
Not only that, every CI (Continuous Integration) Service that is important for open source development does not support multi architecture.
qemu-user-static enables users to run their software on the environments on local or CI "easily".
My motivaion to write this blog is that I want to spread this software to many people to promte softwares supporting the multi-architecture environments.
How does it work? (Reading the source code)
I did read the souce code to understand what it is doing.
Current latest version's commit hash is 20674ec (version v3.1.0-3 + 2 commits)
https://github.com/multiarch/qemu-user-static/commit/20674ec
$ git clone https://github.com/multiarch/qemu-user-static.git $ cd qemu-user-static $ git chckout 20674ec <= if you like reproducing below steps exactlly.
See .travis.yml. It is a top level file to understand it.
At first, install necessary packages.
If you are using Debian base Linux like Ubuntu, like this.
$ sudo apt-get install jq rpm2cpio cpio
If you are using RPM base Linux such as Fedora and CentOS, like this.
$ sudo dnf install jq cpio
or
$ sudo yum install jq cpio
Then download qemu RPM (pre-built binaries) file from Fedora Project.
You can see all the built files from here.
Right now f30 (Fedora 30) is the latest stable (beta) version.
$ wget --content-disposition https://kojipkgs.fedoraproject.org/packages/qemu/3.1.0/6.fc30/x86_64/qemu-user-static-3.1.0-6.fc30.x86_64.rpm
Extract downloaded RPM file.
$ rpm2cpio qemu-user-static-3.1.0-6.fc30.x86_64.rpm | cpio -dimv ./usr/bin/qemu-aarch64-static ./usr/bin/qemu-aarch64_be-static ./usr/bin/qemu-alpha-static ...
See extracted files under usr.
$ ls usr/ bin/ lib/ share/ $ ls usr/bin/ qemu-aarch64_be-static qemu-mips64-static qemu-riscv64-static ...
Then remaining steps are as following.
generate_tarballs.shpublish.shupdate.shdocker push
generate_tarballs.sh
generate_tarballs.sh copies ./usr/bin/qemu-*-static in the above extracted files, and creates tar.gz files to releases directory.
Run below command with a shell debug mode sh -x to see each commands in the script file.
$ sh -x ./generate_tarballs.sh + rm -rf releases + mkdir -p releases + cp ./usr/bin/qemu-aarch64_be-static ./usr/bin/qemu-aarch64-static ... releases/ + cd releases/ + for file in * + tar -czf qemu-aarch64_be-static.tar.gz qemu-aarch64_be-static + cp qemu-aarch64_be-static.tar.gz x86_64_qemu-aarch64_be-static.tar.gz ... + tar -czf qemu-xtensa-static.tar.gz qemu-xtensa-static + cp qemu-xtensa-static.tar.gz x86_64_qemu-xtensa-static.tar.gz
See below files created under releases directory.
qemu-foo-static.tar.gz and x86_64_qemu-foo-static.tar.gz is same content right now.
$ ls releases | grep arch64 qemu-aarch64_be-static qemu-aarch64_be-static.tar.gz qemu-aarch64-static qemu-aarch64-static.tar.gz x86_64_qemu-aarch64_be-static.tar.gz x86_64_qemu-aarch64-static.tar.gz
publish.sh
DO NOT RUN publish.sh by yourself.
This script uploads all the file under the releases directory to GitHub release page like https://github.com/multiarch/qemu-user-static/releases/tag/v3.1.0-3 .
Check the uploaded command version. It is QEMU 3.1.
$ wget https://github.com/multiarch/qemu-user-static/releases/download/v3.1.0-3/qemu-aarch64-static $ chmod +x qemu-aarch64-static $ ./qemu-aarch64-static --version qemu-aarch64 version 3.1.0 (qemu-3.1.0-6.fc30) Copyright (c) 2003-2018 Fabrice Bellard and the QEMU Project developers
update.sh
You can run update.sh on your local.
This script creates container images. The each container images has /usr/bin/x86_64_qemu-foo-static.tar.gz
A content built by docker build is small. Just adding the command on the scratch (no content) container.
FROM scratch
ADD https://github.com/${REPO}/releases/download/v${VERSION}/${from_arch}_qemu-${to_arch}-static.tar.gz /usr/bin
Also the script creates a container by register/Dockerfile.
busybox is a very small sized Linux distribution often used a container.
register/Dockerfile is like this.
About what the file is doing, I will dig it on next blog.
FROM busybox ENV QEMU_BIN_DIR=/usr/bin ADD ./register.sh /register ADD https://raw.githubusercontent.com/qemu/qemu/master/scripts/qemu-binfmt-conf.sh /qemu-binfmt-conf.sh RUN chmod +x /qemu-binfmt-conf.sh ENTRYPOINT ["/register"]
$ docker build -t ${REPO}:register register
So, actually build it on local.
$ sh -x ./update.sh -v "3.1.0-3" -r "multiarch/qemu-user-static" ... + from_arch=x86_64 + to_archs='aarch64 alpha arm armeb cris hppa i386 m68k microblaze microblazeel mips mips64 mips64el mipsel mipsn32 mipsn32el nios2 or1k ppc ppc64 ppc64abi32 ppc64le s390x sh4 sh4eb sparc sparc32plus sparc64 x86_64' ... + for to_arch in $to_archs + '[' x86_64 '!=' aarch64 ']' + docker build -t multiarch/qemu-user-static:x86_64-aarch64 - Sending build context to Docker daemon 2.048kB Step 1/2 : FROM scratch ---> Step 2/2 : ADD https://github.com/multiarch/qemu-user-static/releases/download/v3.1.0-3/x86_64_qemu-aarch64-static.tar.gz /usr/bin Downloading 1.61MB/1.61MB ---> 58b61c68fb02 Successfully built 58b61c68fb02 Successfully tagged multiarch/qemu-user-static:x86_64-aarch64 + docker tag multiarch/qemu-user-static:x86_64-aarch64 multiarch/qemu-user-static:aarch64 ... + docker build -t multiarch/qemu-user-static:register register Sending build context to Docker daemon 3.584kB Step 1/6 : FROM busybox latest: Pulling from library/busybox fc1a6b909f82: Pull complete Digest: sha256:954e1f01e80ce09d0887ff6ea10b13a812cb01932a0781d6b0cc23f743a874fd Status: Downloaded newer image for busybox:latest ---> af2f74c517aa Step 2/6 : ENV QEMU_BIN_DIR=/usr/bin ---> Running in 19568e18a181 Removing intermediate container 19568e18a181 ---> 9861f53c0ee0 Step 3/6 : ADD ./register.sh /register ---> cc357a1c64ec Step 4/6 : ADD https://raw.githubusercontent.com/qemu/qemu/master/scripts/qemu-binfmt-conf.sh /qemu-binfmt-conf.sh Downloading 13.24kB ---> 88c740e9bbe6 Step 5/6 : RUN chmod +x /qemu-binfmt-conf.sh ---> Running in 25444a779a58 Removing intermediate container 25444a779a58 ---> d22f87248659 Step 6/6 : ENTRYPOINT ["/register"] ---> Running in 32872dd66444 Removing intermediate container 32872dd66444 ---> be079407a8a9 Successfully built be079407a8a9 Successfully tagged multiarch/qemu-user-static:register
Below is the creatd images. The 1st line is register tag's image.
$ docker image ls -a REPOSITORY TAG IMAGE ID CREATED SIZE multiarch/qemu-user-static register be079407a8a9 58 seconds ago 1.23MB <none> <none> d22f87248659 59 seconds ago 1.23MB <none> <none> 88c740e9bbe6 About a minute ago 1.21MB <none> <none> cc357a1c64ec About a minute ago 1.2MB <none> <none> 9861f53c0ee0 About a minute ago 1.2MB multiarch/qemu-user-static sparc64 f57ba065081f About a minute ago 1.31MB multiarch/qemu-user-static x86_64-sparc64 f57ba065081f About a minute ago 1.31MB multiarch/qemu-user-static sparc32plus fde067bc03dc About a minute ago 1.31MB multiarch/qemu-user-static x86_64-sparc32plus fde067bc03dc About a minute ago 1.31MB multiarch/qemu-user-static sparc e074528ec6d1 About a minute ago 1.28MB multiarch/qemu-user-static x86_64-sparc e074528ec6d1 About a minute ago 1.28MB multiarch/qemu-user-static sh4eb 81703749fd3e About a minute ago 1.26MB multiarch/qemu-user-static x86_64-sh4eb 81703749fd3e About a minute ago 1.26MB multiarch/qemu-user-static sh4 66881cd7937e About a minute ago 1.26MB multiarch/qemu-user-static x86_64-sh4 66881cd7937e About a minute ago 1.26MB multiarch/qemu-user-static s390x 1898e07e0156 About a minute ago 1.33MB multiarch/qemu-user-static x86_64-s390x 1898e07e0156 About a minute ago 1.33MB multiarch/qemu-user-static ppc64le c3823a9099f8 About a minute ago 1.58MB multiarch/qemu-user-static x86_64-ppc64le c3823a9099f8 About a minute ago 1.58MB multiarch/qemu-user-static ppc64abi32 41a04c8f61a9 About a minute ago 1.58MB multiarch/qemu-user-static x86_64-ppc64abi32 41a04c8f61a9 About a minute ago 1.58MB multiarch/qemu-user-static ppc64 3ee6a9a7322e About a minute ago 1.57MB multiarch/qemu-user-static x86_64-ppc64 3ee6a9a7322e About a minute ago 1.57MB multiarch/qemu-user-static ppc bc5b2536b632 About a minute ago 1.56MB multiarch/qemu-user-static x86_64-ppc bc5b2536b632 About a minute ago 1.56MB multiarch/qemu-user-static or1k 490ae085bfd2 2 minutes ago 1.24MB multiarch/qemu-user-static x86_64-or1k 490ae085bfd2 2 minutes ago 1.24MB multiarch/qemu-user-static nios2 163d3382c615 2 minutes ago 1.24MB multiarch/qemu-user-static x86_64-nios2 163d3382c615 2 minutes ago 1.24MB multiarch/qemu-user-static mipsn32el cd52dc63103e 2 minutes ago 1.45MB multiarch/qemu-user-static x86_64-mipsn32el cd52dc63103e 2 minutes ago 1.45MB multiarch/qemu-user-static mipsn32 75beaed6101d 2 minutes ago 1.46MB multiarch/qemu-user-static x86_64-mipsn32 75beaed6101d 2 minutes ago 1.46MB multiarch/qemu-user-static mipsel 3945f7b45a9a 2 minutes ago 1.43MB multiarch/qemu-user-static x86_64-mipsel 3945f7b45a9a 2 minutes ago 1.43MB multiarch/qemu-user-static mips64el aa170c25943f 2 minutes ago 1.45MB multiarch/qemu-user-static x86_64-mips64el aa170c25943f 2 minutes ago 1.45MB multiarch/qemu-user-static mips64 45482a0aca0f 2 minutes ago 1.45MB multiarch/qemu-user-static x86_64-mips64 45482a0aca0f 2 minutes ago 1.45MB multiarch/qemu-user-static mips a59b3106ea87 2 minutes ago 1.43MB multiarch/qemu-user-static x86_64-mips a59b3106ea87 2 minutes ago 1.43MB multiarch/qemu-user-static microblazeel b42dc2b63e2d 2 minutes ago 1.26MB multiarch/qemu-user-static x86_64-microblazeel b42dc2b63e2d 2 minutes ago 1.26MB multiarch/qemu-user-static microblaze 70a9418a4069 2 minutes ago 1.26MB multiarch/qemu-user-static x86_64-microblaze 70a9418a4069 2 minutes ago 1.26MB multiarch/qemu-user-static m68k 98065a9eea54 2 minutes ago 1.34MB multiarch/qemu-user-static x86_64-m68k 98065a9eea54 2 minutes ago 1.34MB multiarch/qemu-user-static i386 2587bc7b0b86 2 minutes ago 1.65MB multiarch/qemu-user-static x86_64-i386 2587bc7b0b86 2 minutes ago 1.65MB multiarch/qemu-user-static hppa ccb03e044476 3 minutes ago 1.27MB multiarch/qemu-user-static x86_64-hppa ccb03e044476 3 minutes ago 1.27MB multiarch/qemu-user-static cris 2ae51540cb3c 3 minutes ago 1.27MB multiarch/qemu-user-static x86_64-cris 2ae51540cb3c 3 minutes ago 1.27MB multiarch/qemu-user-static armeb 87e1e3f5d99b 3 minutes ago 1.46MB multiarch/qemu-user-static x86_64-armeb 87e1e3f5d99b 3 minutes ago 1.46MB multiarch/qemu-user-static arm 1a362b4aea82 3 minutes ago 1.45MB multiarch/qemu-user-static x86_64-arm 1a362b4aea82 3 minutes ago 1.45MB multiarch/qemu-user-static alpha 1f0b8b105e86 3 minutes ago 1.26MB multiarch/qemu-user-static x86_64-alpha 1f0b8b105e86 3 minutes ago 1.26MB multiarch/qemu-user-static aarch64 58b61c68fb02 3 minutes ago 1.61MB multiarch/qemu-user-static x86_64-aarch64 58b61c68fb02 3 minutes ago 1.61MB busybox latest af2f74c517aa 4 days ago 1.2MB
docker push
DO NOT RUN the commanddocker push multiarch/qemu-user-static by yourself.
docker push multiarch/qemu-user-static uploads the all the image to the DockerHub repository [4]
On next blog, I will dig the compatible images [5] such as ubuntu-core and multiarch/ubuntu-debootstrap and etc.
References
- [1] qemu-user-static: https://github.com/multiarch/qemu-user-static
- [2] qemu: http://www.qemu.org/
- [3] ci-multi-arch-test: https://github.com/junaruga/ci-multi-arch-test
- [4] qemu-user-static DockerHub: https://hub.docker.com/r/multiarch/qemu-user-static/tags
- [5] https://github.com/multiarch/qemu-user-static#compatible-images
