Skip to content
“Zero setup” cross compilation and “cross testing” of Rust crates
Shell Rust
Branch: master
Clone or download

Latest commit

bors and anupdhml Merge #431
431: Allow users to pick cargo for custom target builds r=reitermarkus a=anupdhml

This is an attempt to fix #325.

We want to use the target `x86_64-alpine-linux-musl` (available from alpine's rustc) to build musl-based binaries and when using cross with something not in the standard rustc/rustup targets, cross errors out as detailed in the issue above. 

For a non-standard target, cross defaults to using `xargo` right now (behavior introduced in #217), but in our case, since we want `cargo` to run ultimately, I had to also ensure that setting `xargo = false` actually works from the cross config file. If there's a better way of handling this without breaking the current default behavior, please let me know.

Also updated the docs on xargo usage accordingly (including the parts that were lagging even without this PR).

---

A working example of cross use based on the changes here are available at:
https://github.com/wayfair-tremor/tremor-runtime/blob/f72e133f971adca67f6e6b6e265a4f5d5e96380d/Cross.toml#L30-L37

---

Thanks for providing a tool like cross btw! It's been really useful to us as we are looking to streamline our release process 😃 

Co-authored-by: Anup Dhamala <anupdhml+git@gmail.com>
Latest commit 095870a Jun 12, 2020

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Run ShellCheck on CI. May 8, 2020
assets update the README Dec 26, 2016
ci Run ShellCheck on CI. May 8, 2020
docker Prevent pkg-config from being removed in image May 15, 2020
src Try to install only available toolchains via rustup Jun 4, 2020
.gitignore initial commit Dec 25, 2016
CHANGELOG.md Remove support for `i686-apple-darwin`. Mar 12, 2020
CODE_OF_CONDUCT.md README: mention the CoC and who maintains this repo Aug 7, 2018
Cargo.lock Add support for docker in docker Apr 11, 2020
Cargo.toml Add support for docker in docker Apr 11, 2020
LICENSE-APACHE add LICENSE files Dec 25, 2016
LICENSE-MIT add LICENSE files Dec 25, 2016
README.md Update docs on xargo usage Jun 4, 2020
azure-pipelines.yml Add riscvgc-unknown-linux-gnu support Apr 24, 2020
build-docker-image.sh Run ShellCheck on CI. May 8, 2020
build.rs Refactor Dockerfiles. Aug 25, 2019
triagebot.toml Add triagebot configuration Apr 13, 2020

README.md

crates.io crates.io Build Status Docker Pulls

cross

“Zero setup” cross compilation and “cross testing” of Rust crates

This project is developed and maintained by the Tools team.

`cross test`ing a crate for the aarch64-unknown-linux-gnu target
`cross test`ing a crate for the aarch64-unknown-linux-gnu target

Features

  • cross will provide all the ingredients needed for cross compilation without touching your system installation.

  • cross provides an environment, cross toolchain and cross compiled libraries, that produces the most portable binaries.

  • “cross testing”, cross can test crates for architectures other than i686 and x86_64.

  • The stable, beta and nightly channels are supported.

Dependencies

One of these container engines is required. If both are installed, cross will default to docker.

  • Docker. Note that on Linux non-sudo users need to be in the docker group. Read the official post-installation steps. Requires version 1.24 or later.
  • Podman. Requires version 1.6.3 or later.

Installation

$ cargo install cross

Usage

cross has the exact same CLI as Cargo but as it relies on Docker you'll have to start the daemon before you can use it.

# (ONCE PER BOOT)
# Start the Docker daemon, if it's not already running
$ sudo systemctl start docker

# MAGIC! This Just Works
$ cross build --target aarch64-unknown-linux-gnu

# EVEN MORE MAGICAL! This also Just Works
$ cross test --target mips64-unknown-linux-gnuabi64

# Obviously, this also Just Works
$ cross rustc --target powerpc-unknown-linux-gnu --release -- -C lto

Configuration

You can place a Cross.toml file in the root of your Cargo project to tweak cross's behavior:

Custom Docker images

cross provides default Docker images for the targets listed below. However, it can't cover every single use case out there. For other targets, or when the default image is not enough, you can use the target.{{TARGET}}.image field in Cross.toml to use custom Docker image for a specific target:

[target.aarch64-unknown-linux-gnu]
image = "my/image:tag"

In the example above, cross will use a image named my/image:tag instead of the default one. Normal Docker behavior applies, so:

  • Docker will first look for a local image named my/image:tag

  • If it doesn't find a local image, then it will look in Docker Hub.

  • If only image:tag is specified, then Docker won't look in Docker Hub.

  • If only tag is omitted, then Docker will use the latest tag.

It's recommended to base your custom image on the default Docker image that cross uses: rustembedded/cross:{{TARGET}}-{{VERSION}} (where {{VERSION}} is cross's version). This way you won't have to figure out how to install a cross C toolchain in your custom image. Example below:

FROM rustembedded/cross:aarch64-unknown-linux-gnu-0.1.16

RUN dpkg --add-architecture arm64 && \
    apt-get update && \
    apt-get install --assume-yes libfoo:arm64
$ docker build -t my/image:tag path/to/where/the/Dockerfile/resides

Docker in Docker

When running cross from inside a docker container, cross needs access to the hosts docker daemon itself. This is normally achieved by mounting the docker daemons socket /var/run/docker.sock. For example:

$ docker run -v /var/run/docker.sock:/var/run/docker.sock -v .:/project \
  -w /project my/development-image:tag cross build --target mips64-unknown-linux-gnuabi64

The image running cross requires the rust development tools to be installed.

With this setup cross must find and mount the correct host paths into the container used for cross compilation. This includes the original project directory as well as the root path of the parent container to give access to the rust build tools.

To inform cross that it is running inside a container set CROSS_DOCKER_IN_DOCKER=true.

A development or CI container can be created like this:

FROM rust:1

# set CROSS_DOCKER_IN_DOCKER to inform `cross` that it is executed from within a container
ENV CROSS_DOCKER_IN_DOCKER=true

# install `cross`
RUN cargo install cross

...

Limitations: Finding the mount point for the containers root directory is currently only available for the overlayfs2 storage driver. In order to access the parent containers rust setup, the child container mounts the parents overlayfs. The parent must not be stopped before the child container, as the overlayfs can not be unmounted correctly by Docker if the child container still accesses it.

Passing environment variables into the build environment

By default, cross does not pass any environment variables into the build environment from the calling shell. This is chosen as a safe default as most use cases will not want the calling environment leaking into the inner execution environment.

In the instances that you do want to pass through environment variables, this can be done via build.env.passthrough in your Cross.toml:

[build.env]
passthrough = [
    "RUST_BACKTRACE",
    "RUST_LOG",
    "TRAVIS",
]

To pass variables through for one target but not others, you can use this syntax instead:

[target.aarch64-unknown-linux-gnu.env]
passthrough = [
    "RUST_DEBUG",
]

Use Xargo instead of Cargo

By default, cross uses xargo to build your Cargo project only for all non-standard targets (i.e. something not reported by rustc/rustup). However, you can use the build.xargo or target.{{TARGET}}.xargo field in Cross.toml to force the use of xargo:

# all the targets will use `xargo`
[build]
xargo = true

Or,

# only this target will use `xargo`
[target.aarch64-unknown-linux-gnu]
xargo = true

xargo = false will work the opposite way (pick cargo always) and is useful when building for custom targets that you know to work with cargo.

Supported targets

A target is considered as “supported” if cross can cross compile a “non-trivial” (binary) crate, usually Cargo, for that target.

Testing support (cross test) is more complicated. It relies on QEMU emulation, so testing may fail due to QEMU bugs rather than bugs in your crate. That said, a target has a ✓ in test column of the table below if it can run the compiler-builtins test suite.

Also, testing is very slow. cross test runs units tests sequentially because QEMU gets upset when you spawn multiple threads. This means that, if one of your unit tests spawns threads, then it's more likely to fail or, worst, never terminate.

Target libc GCC C++ QEMU test
*-apple-ios [1] N/A N/A N/A N/A
aarch64-linux-android [2] N/A 4.9 N/A
aarch64-unknown-linux-gnu 2.19 4.8.2 4.1.0
aarch64-unknown-linux-musl 1.1.20 6.3.0 4.1.0
arm-linux-androideabi [2] N/A 4.9 N/A
arm-unknown-linux-gnueabi 2.19 4.8.2 4.1.0
arm-unknown-linux-gnueabihf 2.27 7.3.0 4.1.0
arm-unknown-linux-musleabi 1.1.20 6.3.0 4.1.0
arm-unknown-linux-musleabihf 1.1.20 6.3.0 4.1.0
armv5te-unknown-linux-gnueabi 2.27 7.5.0 4.2.0
armv5te-unknown-linux-musleabi 1.1.20 6.3.0 4.1.0
armv7-linux-androideabi [2] N/A 4.9 N/A
armv7-unknown-linux-gnueabihf 2.15 4.6.2 4.1.0
armv7-unknown-linux-musleabihf 1.1.20 6.3.0 4.1.0
i586-unknown-linux-gnu 2.23 5.3.1 N/A
i586-unknown-linux-musl 1.1.20 6.3.0 N/A
i686-linux-android [2] N/A 4.9 N/A
i686-unknown-linux-gnu 2.15 4.6.2 N/A
i686-unknown-linux-musl 1.1.20 6.3.0 N/A
mips-unknown-linux-gnu 2.23 5.3.1 4.1.0
mips-unknown-linux-musl 1.1.20 6.3.0 4.1.0
mips64-unknown-linux-gnuabi64 2.23 5.3.1 4.1.0
mips64el-unknown-linux-gnuabi64 2.23 5.3.1 4.1.0
mipsel-unknown-linux-gnu 2.23 5.3.1 4.1.0
mipsel-unknown-linux-musl 1.1.20 6.3.0 4.1.0
powerpc-unknown-linux-gnu 2.19 4.8.2 3.0.1
powerpc64-unknown-linux-gnu 2.19 4.8.2 3.0.1
powerpc64le-unknown-linux-gnu 2.19 4.8.2 3.0.1
riscv64gc-unknown-linux-gnu 2.27 7.5.0 4.2.0
s390x-unknown-linux-gnu 2.23 5.3.1 4.1.0
sparc64-unknown-linux-gnu [3] 2.23 5.3.1 4.1.0
sparcv9-sun-solaris [4] 2.11 5.3.0 N/A
thumbv6m-none-eabi [5] 2.2.0 5.3.1 N/A
thumbv7em-none-eabi [5] 2.2.0 5.3.1 N/A
thumbv7em-none-eabihf [5] 2.2.0 5.3.1 N/A
thumbv7m-none-eabi [5] 2.2.0 5.3.1 N/A
wasm32-unknown-emscripten [6] 1.1.15 1.37.13 N/A
x86_64-linux-android [2] N/A 4.9 N/A
x86_64-pc-windows-gnu N/A 7.3.0 N/A
x86_64-sun-solaris [4] 2.11 5.3.0 N/A
x86_64-unknown-dragonfly [4] [3] 4.6.0 5.3.0 N/A
x86_64-unknown-linux-gnu 2.15 4.6.2 N/A
x86_64-unknown-linux-musl 1.1.20 6.3.0 N/A
x86_64-unknown-netbsd [4] 7.0 5.3.0 N/A

[1] iOS cross compilation is supported on macOS hosts.

[2] Only works with native tests, that is, tests that do not depends on the Android Runtime. For i686 some tests may fails with the error assertion failed: signal(libc::SIGPIPE, libc::SIG_IGN) != libc::SIG_ERR, see issue #140 for more information.

[4] For *BSD and Solaris targets, the libc column indicates the OS release version from which libc was extracted.

[3] No std component available as of 2017-01-10

[5] libc = newlib

[6] libc = musl, gcc = emcc; Some projects that use libc may fail due to wrong definitions (will be fixed by https://github.com/rust-lang/libc/pull/610)

Debugging

QEMU_STRACE (v0.1.9+)

You can set the QEMU_STRACE variable when you use cross run to get a backtrace of system calls from “foreign” (non x86_64) binaries.

$ cargo new --bin hello && cd $_

$ QEMU_STRACE=1 cross run --target aarch64-unknown-linux-gnu
9 brk(NULL) = 0x0000004000023000
9 uname(0x4000823128) = 0
(..)
9 write(1,0xa06320,14)Hello, world!
 = 14
9 sigaltstack(0x4000823588,(nil)) = 0
9 munmap(0x0000004000b16000,16384) = 0
9 exit_group(0)

Caveats

  • path dependencies (in Cargo.toml) that point outside the Cargo project won't work because cross use docker containers only mounts the Cargo project so the container doesn't have access to the rest of the filesystem.

License

Licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Code of Conduct

Contribution to this crate is organized under the terms of the Rust Code of Conduct, the maintainer of this crate, the Tools team, promises to intervene to uphold that code of conduct.

You can’t perform that action at this time.