*: fully hermetic builds and nix shell support
This change is a slightly more polished version of Serge's experiment:
- https://review.monogon.dev/c/monogon/+/1148
- https://bin.monogon.dev/pasta/sloth-parrot-ant
- https://bin.monogon.dev/pasta/eel-seal-wolf
There are two execution environments we have to support:
- Most builds run inside a sandbox, which is a Fedora
environment and does not require any host dependencies at all.
- Bazel itself and the tooling we require to bootstrap
the sandbox (mainly, Go and Proto toolchains). This has to
work directly on the host.
We first make the sandbox fully hermetic by setting
--experimental_use_hermetic_linux_sandbox, which set up an empty /
instead of mounting over individual directories, removing any remaining
host paths from the sandbox (except /proc and /dev/shm, which are
required by some toolchains). We also force static values for the shell,
$TMPDIR and $PATH, which would otherwise leak into the sandbox.
For the host, we use buildFHSUserEnv to build an environment which
supports our static toolchains, and well as a clean Bazel build
without all the nixpkgs patches which would otherwise break our custom
toolchains and sandbox implementation.
This allows us to use the exact same toolchains on NixOS and other
distros for perfect reproducibility.
Fixes https://github.com/monogon-dev/monogon/issues/174.
Fixes https://github.com/monogon-dev/monogon/issues/175.
Co-authored-by: Serge Bazanski <serge@monogon.tech>
Change-Id: I665471a45b315ce7e93ef16d9d056d7622886959
Reviewed-on: https://review.monogon.dev/c/monogon/+/1929
Tested-by: Jenkins CI
Reviewed-by: Serge Bazanski <serge@monogon.tech>
diff --git a/.bazelrc b/.bazelrc
index 9734f97..18253f1 100644
--- a/.bazelrc
+++ b/.bazelrc
@@ -3,11 +3,22 @@
build --incompatible_strict_action_env=true
# Run all spawns in our own hermetic sandbox sysroot.
-#build --experimental_use_hermetic_linux_sandbox
-# TODO: https://github.com/bazelbuild/rules_go/issues/1910
+build --experimental_use_hermetic_linux_sandbox
build --action_env=MONOGON_SANDBOX_DIGEST
import %workspace%/.bazelrc.sandbox
+# Hardwire all action envs to just use /usr/bin from the above sandbox. This is
+# necessary on NixOS Bazel builds, as they really like to inject /nix/store/*
+# paths otherwise. We also explicitly set it to /usr/bin only (no /bin) as
+# otherwise calling gcc from /bin/gcc breaks its own resolution of subordinate
+# commands (like cc1, as, etc.).
+build --action_env=PATH=/usr/bin
+build --host_action_env=PATH=/usr/bin
+
+# Make all shell run actions use /bin/bash instead of whatever the host might
+# have set. Again, looking at you, Bazel-on-NixOS.
+build --shell_executable=/bin/bash
+
# No local CPP toolchain resolution. In our sandbox root, it doesn't make sense -
# anything auto-detected during analysis stage is on the host instead of the sandbox.
# Sysroot rebuild is pure Go and doesn't need it either.