Manually invalidate image cache in CI

Our objective is to ensure that the build always uses the latest
Dockerfile for a given build, while minimizing rebuilds.

A counter is explicit and can be used to intentionally invalidate
the image cache, even if the the file's content did not change.

The caching performed by `podman build` is not very clever -
it's not shared between repositories, and is easily invalidated
by successive builds with different versions of the Dockerfile.

Garbage collection is not trivial, since there may be multiple
in-flight revisions with different counters.

Ref T506

Fixes T616

Test Plan: CI ;)

Bug: T616, T506

X-Origin-Diff: phab/D373
GitOrigin-RevId: 5bfb8cd8b98175d645c904aee8e45402d0c049c7
diff --git a/scripts/run_ci.sh b/scripts/run_ci.sh
index e3efd27..c95ed6e 100755
--- a/scripts/run_ci.sh
+++ b/scripts/run_ci.sh
@@ -3,11 +3,13 @@
 # https://phab.monogon.dev/harbormaster/plan/2/
 set -euo pipefail
 
+DOCKERFILE_HASH=$(sha1sum build/Dockerfile | cut -c -8)
+
 BUILD_ID=$1;
 BUILD_PHID=$2;
 shift; shift;
 
-TAG=nexantic-build-${BUILD_ID}
+TAG=nexantic-version-${DOCKERFILE_HASH}
 POD=nexantic-build-${BUILD_ID}
 
 # We keep one Bazel build cache per working copy to avoid concurrency issues
@@ -20,16 +22,19 @@
 
 CACHE_VOLUME=bazel-cache-$(getWorkingCopyID)
 
-# New image for each build - the Dockerfile might have changed.
-# Rely on the build step cache to avoid costly rebuilds.
-podman build -t ${TAG} build
+# We do our own image caching since the podman build step cache does
+# not work across different repository checkouts and is also easily
+# invalidated by multiple in-flight revisions with different Dockerfiles.
+if ! podman image inspect "$TAG" >/dev/null; then
+  echo "Could not find $TAG, building..."
+  podman build -t ${TAG} build
+fi
 
 # Keep this in sync with create_container.sh:
 
 function cleanup {
   rc=$?
   ! podman pod rm $POD --force
-  ! podman rmi $TAG --force
   exit $rc
 }