Embedding a Rust binary in another Rust binary

For reasons¹ I need to embed a Rust binary in another Rust binary, however out of the box Cargo doesn’t support this easily. RFC 3028 will fix this but as of Rust 1.52 it is not implemented. This blog post will go though an example, show where Cargo does not work today and how to overcome this with a custom script.

An Example

A basic example would be having two crates, inner and outer in the same Cargo workspace. Both are binary crates, but we want to embed the entire inner binary into the outer binary. An example for this would be that when outer starts up, it needs to extract the binary in a special location and ask another application to launch inner.

A naive approach

A Cargo workspace could look like:

.
├── Cargo.lock
├── Cargo.toml
├── inner
│   ├── Cargo.toml
│   └── src
│       └── main.rs
└── outer
    ├── Cargo.toml
    └── src
        └── main.rs

At the root we have our Cargo.toml which declares the two crates:

[workspace]

members = [
  "inner",
  "outer",
]

In our inner crates’ Cargo.toml we have:

[package]
name = "inner"
version = "0.1.0"
edition = "2018"

And the entire code for the crate is:

fn main() {
    println!("Hello, from inner!");
}

Our outer crate could look like

[package]
name = "outer"
version = "0.1.0"
edition = "2018"

[dependencies]
inner = {path = "../inner"}

fn main() {
    let inner_binary_bytes = include_bytes!("../../target/debug/inner");
    println!("The inner binary size: {}", inner_binary_bytes.len());
}

In the above approach, we declare a dependency from outer to inner and hard code the location of the debug binary.

From a clean workspace a build doesn’t work

$ cargo b

...

error: couldn't read outer/src/../../target/debug/inner: No such file or directory (os error 2)
 --> outer/src/main.rs:2:30
  |
2 |     let inner_binary_bytes = include_bytes!("../../target/debug/inner");
  |                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)

...

However a subsequent build does work:

$ cargo b
...
    Finished dev [unoptimized + debuginfo] target(s) in 0.86s

And we can run outer with:

$ cargo run --bin outer
    Finished dev [unoptimized + debuginfo] target(s) in 0.00s
     Running `target/debug/outer`
The inner binary size: 442752

There are three problems here:

• Cargo does not respect the dependency from outer to inner and proceeds to build them in parallel.
• Subsequent builds ‘work’ but you might be getting a stale binary embedded which can lead to a lot of confusion.
• There is no easy way for outer to know were the desired inner binary is. The above code assumes a debug build for the current architecture but the path would be different if it were a release build or a different architecture.

A custom build script

The only way I have found to do this is to have a custom build.rs build script for outer which expects the desired binary in a specific place, and another script to coordinate the builds of inner and outer.

One benefit of this approach is that we can get rid of the dependency between inner and outer which will prevent us from accidentally referencing code in another binary.

The build.rs for outer looks like:

use std::result::Result;
use std::error::Error;
use std::env;
use std::path::Path;

fn main() -> Result<(), Box<dyn Error>> {
    let out_dir = env::var_os("OUT_DIR").unwrap();
    let out_dir = Path::new(&out_dir);
    std::fs::create_dir_all(&out_dir)?;

    let inner_bin = get_env_var("INNER_BIN")?;
    let inner_bin = Path::new(&inner_bin);

    let target_bin = out_dir.join("inner");

    std::fs::copy(inner_bin, target_bin)?;

    Ok(())
}

fn get_env_var(var: &str) -> Result<String, env::VarError> {
    println!("{}", "cargo:rerun-if-env-changed=".to_owned() + var);
    return env::var(var);
}

We have to use environment variables because that’s the only way a build script can get an argument.

Once this is done we can change ./outer/src/main.rs to read from OUT_DIR

fn main() {
    let inner_binary_bytes = include_bytes!(concat!(env!("OUT_DIR"), "/inner"));
    println!("The inner binary size: {}", inner_binary_bytes.len());
}

After this is done we just need a script to drive cargo and set the appropriate environment variables:

#!/bin/bash
set -euo pipefail

export INNER_BIN=$(cargo build  "$@" -p inner --message-format=json | jq -r 'select(.reason == "compiler-artifact") | .filenames[0]')

cargo build "$@" -p outer

With this script, we explicitly build inner first, grab the output location from cargo and persist it in the environment variable that the outer/build.rs consumes.

Invoking the script allows us to properly build inner and then embed it in outer. The design of the shell script allows us to also pass arguments to cargo build like --release or a target architecture. This script also works from a clean build and will always ensure we are embedding the latest build of inner.