scripts: refactor for reliability
This is a small refactor/rewrite pass to scripts/create_container.sh and
scripts/bin/bazel.
First of all, it moves the IntelliJ aspect overriding logic from parsing
command line flags in scripts/bin/bazel to replace the path that the
aspect is loaded from to crafting the build container in a way that the
aspect is already located where it's expected.
This allows us to vastly simplify the scripts/bin/bazel wrapper, and
most notably fixes an issue where it wasn't able to handle Bazel flags
that contained spaces and were wrapped in ""/'' delimited strings.
Instead of adding more and more flag replacing logic in this script, we
got rid of that logic entirely, and now the only thing that the wrapper
script does is set the output root and run Bazel within the container.
At the same time, we also fix discovering newer IntelliJ versions. On my
machine, it was still discovering my 2019 installation and using an old
version of the aspect.
We also generally dust off the scripts/create_container.sh script by
ensuring all variables are always used within double quotes (the
previous implementation could've accidentally removed some directory from
the host if the user homedir contained spaces) and moving all of the
logic into a function that uses local variables.
Tested on my local workstation, IntelliJ sync works.
Change-Id: I63ae042ffe0fc7f384b98768ed66b7426b3f5df4
Reviewed-on: https://review.monogon.dev/c/monogon/+/63
Reviewed-by: Leopold Schabel <leo@nexantic.com>
diff --git a/scripts/create_container.sh b/scripts/create_container.sh
index c5c97d9..fcf0787 100755
--- a/scripts/create_container.sh
+++ b/scripts/create_container.sh
@@ -1,55 +1,120 @@
#!/usr/bin/env bash
set -euo pipefail
+shopt -s nullglob
-# Our local user needs write access to /dev/kvm (best accomplished by
-# adding your user to the kvm group).
-if ! touch /dev/kvm; then
- echo "Cannot write to /dev/kvm - please verify permissions."
- exit 1
-fi
+main() {
+ # Our local user needs write access to /dev/kvm (best accomplished by
+ # adding your user to the kvm group).
+ if ! touch /dev/kvm; then
+ echo "Cannot write to /dev/kvm - please verify permissions." >&2
+ exit 1
+ fi
+
+ # The KVM module needs to be loaded, since our container is unprivileged
+ # and won't be able to do it itself.
+ if ! [[ -d /sys/module/kvm ]]; then
+ echo "kvm module not loaded - please modprobe kvm" >&2
+ exit 1
+ fi
-# The KVM module needs to be loaded, since our container is unprivileged
-# and won't be able to do it itself.
-if ! [[ -d /sys/module/kvm ]]; then
- echo "kvm module not loaded - please modprobe kvm"
- exit 1
-fi
+ local dockerfile="build/ci/Dockerfile"
+ if ! [[ -f "${dockerfile}" ]]; then
+ echo "Dockerfile not found at path ${dockerfile}. Make sure to run this script from the root of the Monogon checkout." >&2
+ exit 1
+ fi
+
+ # Rebuild base image purely with no build context (-) ensuring that the
+ # builder image does not contain any other data from the repository.
+ podman build -t monogon-builder - < "${dockerfile}"
-# Rebuild base image
-podman build -t monogon-builder build/ci
+ # TODO(serge): stop using pods for the builder, this is a historical artifact.
+ podman pod create --name monogon
+
+ # Mount bazel root to identical paths inside and outside the container.
+ # This caches build state even if the container is destroyed.
+ #
+ # TODO(serge): do not hardcode this path? This breaks if attempting to use
+ # the build container from multiple Monogon checkouts on disk.
+ local bazel_root="${HOME}/.cache/bazel-monogon"
+ mkdir -p "${bazel_root}"
+
+ # When IntelliJ's Bazel plugin uses //scripts/bin/bazel to either build targets
+ # or run syncs, it adds a --override_repository flag to the bazel command
+ # line that points @intellij_aspect into a path on the filesystem. This
+ # external repository contains a Bazel Aspect definition which Bazel
+ # executes to provide the IntelliJ Bazel plugin with information about the
+ # workspace / build targets / etc...
+ #
+ # We need to make this path available within the build container. However,
+ # instead of directly pointing into that path on the host, we redirect through
+ # a patched copy of this repository. That patch applies fixes related to the
+ # Monogon codebase in general, not specific to the fact that we're running in a
+ # container.
+ #
+ # What this ends up doing is that the path mounted within the container
+ # looks as if it's a path straight from the host IntelliJ config directory,
+ # but in fact points into a patched copy. It looks weird, but this setup
+ # allows us to let IntelliJ's Bazel integration to trigger Bazel without us
+ # having to replace the override_repository flag that it passes to Bazel.
+ # We at some point used to do that, but parsing and replacing Bazel flags
+ # in the scripts/bin/bazel wrapper script is error prone and fragile.
-# Keep this in sync with ci.sh:
+ # Find all IntelliJ installation/config directories.
+ local ij_home_paths=("${HOME}/.local/share/JetBrains/IntelliJIdea"*)
+ # Get the newest one, if any.
+ local ij_home=""
+ if ! [[ ${#ij_home_paths[@]} -eq 0 ]]; then
+ # Reverse sort paths by name, with the first being the newest IntelliJ
+ # installation.
+ IFS=$'\n'
+ local sorted=($(sort -r <<<"${ij_home_paths[*]}"))
+ unset IFS
+ ij_home="${sorted[0]}"
+ fi
-podman pod create --name monogon
+ # If we don't have or can't find ij_home, don't bother with attempting to patch anything.
+ # If we do, podman_flags will get populated with extra flags that it will
+ # run with to support IntelliJ/Bazel integration.
+ declare -a podman_flags
+ if [[ -d "${ij_home}" ]]; then
+ echo "IntelliJ found at ${ij_home}, patching and mounting aspect repository."
+ # aspect_orig is the path to the aspect external repository that IntelliJ will
+ # inject into bazel via --override_repository. It is the path that it expects
+ # to be available to Bazel within the container, and also the path that is
+ # directly visible in the host.
+ local aspect_orig="${ij_home}/ijwb/aspect"
+ # aspect_patched is the path to our patched copy of the aspect
+ # repository. We keep this in bazel_root for convenience, as that's a
+ # path that we control on the host anyway, so we can be sure we're not
+ # trampling some other process.
+ local aspect_patched="${bazel_root}/ijwb_aspect"
-# Mount bazel root to identical paths inside and outside the container.
-# This caches build state even if the container is destroyed, and
-BAZEL_ROOT=${HOME}/.cache/bazel-monogon
-mkdir -p ${BAZEL_ROOT}
+ # If we already have a patched version of the aspect, remove it.
+ if [[ -d "${aspect_patched}" ]]; then
+ rm -rf "${aspect_patched}"
+ fi
-# The Bazel plugin injects a Bazel repository into the sync command line,
-# We need to copy the aspect repository and apply a custom patch.
+ # Copy and patch the aspect.
+ cp -r "${aspect_orig}" "${aspect_patched}"
+ patch -d "${aspect_patched}" -p1 < scripts/patches/bazel_intellij_aspect_filter.patch
-# TODO(leo): the IntelliJ path changed to ~/.config on new setups, we should look for that as well
+ # Make podman mount the patched aspect into the original aspect path in the build container.
+ podman_flags+=(-v "${aspect_patched}:${aspect_orig}")
+ else
+ echo "No IntelliJ found, not patching/mounting aspect repository."
+ fi
-IJ_HOME=$(echo ${HOME}/.IntelliJIdea* | tr ' ' '\n' | sort | tail -n 1)
-ASPECT_ORIG=${IJ_HOME}/config/plugins/ijwb/aspect
-ASPECT_PATH=${BAZEL_ROOT}/ijwb_aspect
+ podman run -it -d \
+ -v $(pwd):$(pwd):z \
+ -w $(pwd) \
+ --volume="${bazel_root}:${bazel_root}" \
+ --device /dev/kvm \
+ --privileged \
+ --pod monogon \
+ --name=monogon-dev \
+ --net=host \
+ "${podman_flags[@]}" \
+ monogon-builder
+}
-if [[ -d "$IJ_HOME" ]]; then
- echo "IntelliJ found, copying aspect file to Bazel root"
- rm -rf "$ASPECT_PATH"
- cp -r "$ASPECT_ORIG" "$ASPECT_PATH"
- patch -d "$ASPECT_PATH" -p1 < scripts/patches/bazel_intellij_aspect_filter.patch
-fi
-
-podman run -it -d \
- -v $(pwd):$(pwd):z \
- -w $(pwd) \
- --volume=${BAZEL_ROOT}:${BAZEL_ROOT} \
- --device /dev/kvm \
- --privileged \
- --pod monogon \
- --name=monogon-dev \
- --net=host \
- monogon-builder
+main