*: fully hermetic builds and nix shell support

This change is a slightly more polished version of Serge's experiment:
- https://review.monogon.dev/c/monogon/+/1148
- https://bin.monogon.dev/pasta/sloth-parrot-ant
- https://bin.monogon.dev/pasta/eel-seal-wolf

There are two execution environments we have to support:

- Most builds run inside a sandbox, which is a Fedora
  environment and does not require any host dependencies at all.

- Bazel itself and the tooling we require to bootstrap
  the sandbox (mainly, Go and Proto toolchains). This has to
  work directly on the host.

We first make the sandbox fully hermetic by setting
--experimental_use_hermetic_linux_sandbox, which set up an empty /
instead of mounting over individual directories, removing any remaining
host paths from the sandbox (except /proc and /dev/shm, which are
required by some toolchains). We also force static values for the shell,
$TMPDIR and $PATH, which would otherwise leak into the sandbox.

For the host, we use buildFHSUserEnv to build an environment which
supports our static toolchains, and well as a clean Bazel build
without all the nixpkgs patches which would otherwise break our custom
toolchains and sandbox implementation.

This allows us to use the exact same toolchains on NixOS and other
distros for perfect reproducibility.

Fixes https://github.com/monogon-dev/monogon/issues/174.
Fixes https://github.com/monogon-dev/monogon/issues/175.

Co-authored-by: Serge Bazanski <serge@monogon.tech>
Change-Id: I665471a45b315ce7e93ef16d9d056d7622886959
Reviewed-on: https://review.monogon.dev/c/monogon/+/1929
Tested-by: Jenkins CI
Reviewed-by: Serge Bazanski <serge@monogon.tech>
4 files changed
tree: acaf5e1981fc98101f25924a6fb44cf827f826c5
  1. .github/
  2. build/
  3. cloud/
  4. go/
  5. intellij/
  6. metropolis/
  7. net/
  8. third_party/
  9. tools/
  10. .bazelignore
  11. .bazelproject
  12. .bazelrc
  13. .bazelrc.sandboxroot
  14. .bazelversion
  15. .git-ignore-revs
  16. .gitignore
  17. BUILD.bazel
  18. CODING_STANDARDS.md
  19. go.mod
  20. go.sum
  21. LICENSE
  22. README.md
  23. SETUP.md
  24. shell.nix
  25. WORKSPACE
README.md

Monogon Monorepo

This is the main repository containing the source code for the Monogon Platform.

This is pre-release software - take a look, and check back later!

Environment

Our build environment is self-contained and requires only minimal host dependencies:

  • A Linux machine or VM.
  • Bazelisk >= v1.15.0
  • A reasonably recent kernel with user namespaces enabled.
  • Working KVM with access to /dev/kvm (if you want to run tests).

Our docs assume that Bazelisk is available as bazel on your PATH.

Refer to SETUP.md for detailed instructions.

Monogon OS

Run a single node demo cluster

Build CLI and node image:

bazel build //metropolis/cli/dbg //:launch -c dbg

Launch an ephemeral test node:

bazel test //:launch -c dbg --test_output=streamed

Run a kubectl command while the test is running:

bazel-bin/metropolis/cli/dbg/dbg_/dbg kubectl describe node

Test suite

Run full test suite:

bazel test -c dbg //...