build/ci: move Dockerfile, document new CI
This moves the Builder Imager Dockerfile into //build/ci, adds some
small changes to make it usable as a Jenkins agent base, documents its
usage, and adds a script which builds and pushes that image into an
external container registry.
We also remove the old Phabricator-based CI scripting.
Change-Id: I332608f7d7105f675104db3ee2d787b2412fcbe9
Reviewed-on: https://review.monogon.dev/c/monogon/+/28
Reviewed-by: Leopold Schabel <leo@nexantic.com>
diff --git a/build/Dockerfile b/build/ci/Dockerfile
similarity index 67%
rename from build/Dockerfile
rename to build/ci/Dockerfile
index 8935d1f..e319026 100644
--- a/build/Dockerfile
+++ b/build/ci/Dockerfile
@@ -1,8 +1,3 @@
-#
-# The CI only rebuilds this Dockerfile if its hash changes.
-# Do not reference any external files, since modifications to them won't trigger a rebuild.
-#
-
FROM fedora:32
RUN dnf -y upgrade && \
@@ -33,7 +28,16 @@
grpc-cli \
nc \
python-unversioned-command \
- openssl-devel
+ openssl-devel \
+ java-11-openjdk
+
+# Create CI build user. This is not used by scripts/bin/bazel, but instead only
+# used by CI infrastructure to run build agents as.
+# The newly created user will have a UID of 500, and a corresponding CI group
+# of GID 500 will be created as well. This UID:GID pair's numeric values are
+# relied on by the CI infrastructure and must not change without coordination.
+RUN set -e -x ;\
+ useradd -u 500 -U -m -d /home/ci ci
# Install Bazel binary
RUN curl -o /usr/local/bin/bazel \
diff --git a/build/ci/README.md b/build/ci/README.md
new file mode 100644
index 0000000..45dcd3f
--- /dev/null
+++ b/build/ci/README.md
@@ -0,0 +1,64 @@
+Monogon CI
+==========
+
+Monogon has a work-in-progress continuous integration / testing pipeline.
+Because of historical reasons, some parts of this pipeline are defined in a
+separate non-public repository that is managed by Monogon Labs.
+
+In the long term, the entire infrastructure code relating to this will become
+public and part of the Monogon repository. In the meantime, this document
+should serve as a public reference that explains how that part works and how it
+integrates with `//build/ci/...` and the project as a whole.
+
+Builder Image & Container
+-------------------------
+
+`//build/ci/Dockerfile` describes a 'builder image'. This image contains a
+stable, Fedora-based build environment in which all Monogon components should
+be built. It has currently two uses:
+
+1. The build scripts at
+ `//scripts/{create_container.sh,destroy_container.sh,/bin/bazel}`. These are
+ used by developers to run Bazel against a controlled environment to develop
+ Monogon code. The `create_container.sh` script builds the Builder image and
+ starts a Builder container. The `bin/bazel` wrapper script launches Bazel in
+ it. The `destroy_container.sh` script cleans everything up.
+
+2. The Jenkins based CI uses the Builder image as a base to run Jenkins agents.
+ A Monogon Labs developer runs `//build/ci/build_ci_image`, which builds the
+ Builder Image and pushes it to a container registry. Then, in another
+ repository, that image is used as a base to overlay a Jenkins agent on top,
+ and then used to run all Jenkins actions.
+
+As Monogon evolves and gets better build hermeticity using Bazel toolchains,
+the need for a Builder image should subdue. Meanwhile, using the same image
+ensures that we have the maximum possible reproducibility of builds across
+development and CI machines, and gets us a base level of build hermeticity and
+reproducibility.
+
+CI usage
+--------
+
+When a change on https://review.monogon.dev/ gets opened, it needs to either
+be owned by a 'trusted user', or be vouched by one. This is because our current
+CI setup is not designed to protect against malicious changes that might
+attempt to take over the CI system, or change the CI scripts themselves to skip
+tests.
+
+Currently, all Monogon Labs employees (thus, the core Monogon development team)
+are marked as 'trusted users'. There is no formal process for community
+contributors to become part of this group, but we are more than happy to
+formalize such a process when needed, or appoint active community contributors
+to this group. Ideally, though, the CI system should be rebuilt to allow any
+external contributor to run CI in a secure and sandboxed fashion.
+
+CI implementation
+-----------------
+
+The CI system is currently made of a Jenkins instance running on
+https://jenkins.monogon.dev/. It runs against open changes that have the
+Allow-Run-CI label evaluated to 'ok' Gerrit Prolog rules, and executes the
+`//build/ci/jenkins-presubmit.groovy` script on them.
+
+Currently, the Jenkins instance is not publicly available, and thus CI logs are
+not publicly available either. This will be fixed very soon.