*: bring our own sandbox root
This change removes the build container and replaces it with a
Bazel-built Fedora 37 sysroot which is bind-mounted into the Bazel
sandbox using --sandbox_add_mount_pair. The tools/bazel wrapper script
automatically (re-)generates the sysroot when needed.
Both Bazelisk and Bazel's native wrapper automatically run the
tools/bazel script, which means that our build should now work without
extra steps on any machine with a working Bazelisk setup and unpriv ns.
This fixes all kinds of weirdness caused by the previous podman setup
("bazel run"/container pushes, log access, weird podman bugs,
breaking the IDE plugin for any non-Monogon workspaces...).
Using the sandbox hash as an action var also ensures that the cache
is invalidated whenever the ambient environment changes. Previously,
Bazel did not invalidate build steps when any host dependency changed.
To my knowledge, this was the only remaining cause for stale builds.
It also means we cannot depend on the host toolchain since it
won't be accessible in the sandbox, and anything that inspects the
host during analysis stage will fail. This currently means that
running on a non-Fedora host won't work - we fix this next.
All RPMs are pinned and the sysroot is fully reproducible.
Once we upgrade to Bazel 5.x, we can take it further by enabling
--experimental_use_hermetic_linux_sandbox and fully remove the
remaining host paths from the sandbox for full hermeticity.
In a follow-up, we can clean up the CI image to only contain the
minimum dependencies needed for Bazelisk and the agent.
Existing IntelliJ users need to remove the -Dbazel.bep.path flag
from their VM options.
Handbook/Rust rules are disabled temporarily to keep CI green
(requires a more recent rules_rust version).
Change-Id: I1f17d57d985ff9d749bf3359f259d8ef52247c18
Reviewed-on: https://review.monogon.dev/c/monogon/+/1033
Tested-by: Jenkins CI
Reviewed-by: Lorenz Brun <lorenz@monogon.tech>
diff --git a/third_party/sandboxroot/regenerate.sh b/third_party/sandboxroot/regenerate.sh
new file mode 100755
index 0000000..b9e4256
--- /dev/null
+++ b/third_party/sandboxroot/regenerate.sh
@@ -0,0 +1,108 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Tell wrapper to not touch sandbox
+export MONOGON_SYSROOT_REBUILD=1
+
+# Packages to install. Make sure to document the reason for including each package.
+PKGS=(
+ # Common base toolchain used across the tree.
+ "binutils"
+ "gcc"
+ "python3"
+ "python-unversioned-command"
+ "glibc-static"
+
+ # Kernel build
+ "flex"
+ "bison"
+ "elfutils-libelf-devel"
+ "openssl-devel"
+ "diffutils"
+ "bc"
+ "perl"
+ "lz4"
+
+ # EDK2
+ "libuuid-devel"
+ "util-linux"
+ "nasm"
+ "acpica-tools"
+
+ # TPM emulator for testing
+ "swtpm-tools"
+
+ # Clang/LLVM (for EFI toolchain)
+ "clang"
+ "llvm"
+ "lld"
+
+ # image_gcp rule
+ "tar"
+
+ # ktest
+ "qemu-system-x86-core"
+
+ # musl-host-gcc
+ "rsync"
+ "xz"
+
+ # Packages included to stabilize SAT solution when there are equal scores.
+ "fedora-release-identity-container"
+ "coreutils-single"
+ "curl-minimal"
+ "libcurl-minimal"
+ "glibc-langpack-en"
+ "selinux-policy-minimum"
+)
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+REPO=third_party/sandboxroot/repo.yaml
+BAZEL_ARGS="--noworkspace_rc --bazelrc ${DIR}/../../.bazelrc.sandboxroot"
+
+# Fetch latest repository metadata
+bazel ${BAZEL_ARGS} run //:bazeldnf -- fetch --repofile $REPO
+
+# Write BUILD.bazel template
+cat <<EOF > ${DIR}/BUILD.bazel.in
+load("@bazeldnf//:deps.bzl", "rpmtree")
+load("@bazeldnf//:def.bzl", "bazeldnf")
+
+bazeldnf(
+ name = "sandboxroot",
+ command = "sandbox",
+ tar = ":sandbox",
+)
+
+EOF
+
+touch ${DIR}/repositories.bzl.in
+
+# Create new sandbox root
+bazel ${BAZEL_ARGS} \
+ run //:bazeldnf -- rpmtree \
+ --repofile third_party/sandboxroot/repo.yaml \
+ --name sandbox \
+ --nobest \
+ --buildfile third_party/sandboxroot/BUILD.bazel.in \
+ --workspace third_party/sandboxroot/repositories.bzl.in \
+ ${PKGS[@]}
+
+# Verify package signatures
+bazel ${BAZEL_ARGS} run //:bazeldnf -- verify \
+ --repofile third_party/sandboxroot/repo.yaml \
+ --workspace third_party/sandboxroot/repositories.bzl.in
+
+# Write out repositories.bzl and clean up.
+#
+# Ideally, bazeldnf would support the format natively:
+# https://github.com/rmohr/bazeldnf/issues/26
+cat <<EOF > ${DIR}/repositories.bzl
+load("@bazeldnf//:deps.bzl", "rpm")
+
+def sandbox_dependencies():
+$(cat ${DIR}/repositories.bzl.in | sed 's/^/ /')
+EOF
+
+mv ${DIR}/BUILD.bazel.in ${DIR}/BUILD.bazel
+rm ${DIR}/repositories.bzl.in