Simple CI pipeline for Smalltown Core
Since we run as root on the build server, we can properly mount a tmpfs
as the shared build volume with exec support (this is still broken for
unprivileged create_container.sh). We'll have to see when and if this
blows up, and whether we want to use a disk-backed volume instead.
The pipeline has two stages that run the following commands:
- `scripts/run_ci.sh ${build.id} ${target.phid} bazel build //core/scripts:launch`
- `timeout 30 scripts/run_ci.sh ${build.id} ${target.phid} bazel run //core/scripts:launch; true` (for visual inspection)
Those are placeholders - we will want to integrate any and all
tests in Bazel, only trigger tests whose dependencies have been
modified in a given build step, and report individual results back
to Habormaster.
What works:
- Persistent working copies on the build server. Drydocks caches a number of
persistent repository copies to avoid a full clone on each build, and
uses a leasing mechanism to allocate them. Of course, this means we
have to be careful about not polluting the repo, but Bazel takes care of that.
- Shared build cache with fast incremental rebuilds
(a build with no changes takes ~15s including the podman build step).
- Full rebuild after volume deletion takes ~4m.
- Build output shows up in Phabricator in real time.
- Aborting a build properly cancels the running build and clean up the pod.
- Launching the QEMU VM.
- Reporting build status back to Harbormaster (noop at the moment, can
be used to report unit test states later). This uses the awesome undocumented
SSH conduit transport so we don't have to deploy a separate token on the host.
- Phabricator revisions are drafts until all tests complete successfully.
Test Plan: See tests :-)
Bug: T483
X-Origin-Diff: phab/D242
GitOrigin-RevId: 64eca996c8704cb0cd4f1cbb4f88f71a6fdca1eb
diff --git a/scripts/run_ci.sh b/scripts/run_ci.sh
new file mode 100755
index 0000000..fa62a9b
--- /dev/null
+++ b/scripts/run_ci.sh
@@ -0,0 +1,73 @@
+#!/bin/bash
+# This script is executed by our CI
+set -euo pipefail
+
+BUILD_ID=$1;
+BUILD_PHID=$2;
+shift; shift;
+
+TAG=nexantic-build-${BUILD_ID}
+POD=nexantic-build-${BUILD_ID}
+
+# New image for each build - the Dockerfile might have changed.
+# Rely on the build step cache to avoid costly rebuilds.
+podman build -t ${TAG} build
+
+# Keep this in sync with create_container.sh:
+
+function cleanup {
+ rc=$?
+ ! podman pod rm $POD --force
+ ! podman rmi $TAG --force
+ exit $rc
+}
+
+trap cleanup EXIT
+
+! podman volume create \
+ --opt device=tmpfs \
+ --opt type=tmpfs \
+ --opt o=nodev,exec \
+ bazel-shared-cache
+
+podman pod create --name ${POD}
+
+podman run -d \
+ --pod ${POD} \
+ --ulimit nofile=262144:262144 \
+ --name=${POD}-cockroach \
+ cockroachdb/cockroach:v19.1.5 start --insecure
+
+podman run \
+ -v $(pwd):/work \
+ -v bazel-shared-cache:/user/.cache/bazel/_bazel_root \
+ --device /dev/kvm \
+ --privileged \
+ --pod ${POD} \
+ --name=${POD}-bazel \
+ ${TAG} \
+ $@
+
+cat <<EOF
+{
+ "buildTargetPHID": "${BUILD_PHID}",
+ "type": "pass"
+}
+EOF
+
+env
+
+function conduit() {
+ # Get Phabricator host from Git origin
+ local pattern='ssh://(.+?):([0-9]+)'
+ [[ "$(git remote get-url origin)" =~ $pattern ]];
+ local host=${BASH_REMATCH[1]}
+ local port=${BASH_REMATCH[2]}
+
+ ssh "$host" -p "$port" conduit $@
+}
+
+# Report build results if we made it here successfully
+conduit harbormaster.sendmessage <<EOF
+{"params": "{\"buildTargetPHID\": \"${BUILD_PHID}\", \"type\": \"pass\"}"}
+EOF