| commit | 20312b40ed4644c64581b4cc8d93a0fc0035fc71 | [log] [tgz] |
|---|---|---|
| author | Serge Bazanski <serge@monogon.tech> | Wed Apr 19 13:49:47 2023 +0200 |
| committer | Serge Bazanski <serge@monogon.tech> | Wed Apr 19 15:42:44 2023 +0000 |
| tree | 5b812982e0b5bfd7ef0c10659393d37ad0062d62 | |
| parent | ff619354a126df5acffd78ee2f072a17d074e7ab [diff] |
cloud/bmaas: implement exponential backoffs This lets work fail with a proper exponential backoff. This is important not just to not hammer external systems, but also to not end up with exteremely long and verbose historical logs for repeatedly failing processes. To implement these, we have to slightly alter our model: instead of always persisting backoffs for a given machine, we only persist them as long as the last work for a given process has failed, deleting pertinent backoffs (if any) on success. Then, the existence of a backoff item is used to calculate the value of the next backoff. The change also introduces and explicit period, in seconds, to the backoff item. It is currently implemented as a nullable field, but a future migration/update might make them non-nullable (and delete any straggling backoffs that still don't have the period set). Change-Id: I958fcd957dae1156349224f07fb8d4836955d375 Reviewed-on: https://review.monogon.dev/c/monogon/+/1565 Tested-by: Jenkins CI Reviewed-by: Lorenz Brun <lorenz@monogon.tech>
This is the main repository containing the source code for the Monogon Platform.
This is pre-release software - take a look, and check back later!
Our build environment is self-contained and requires only minimal host dependencies:
/dev/kvm (if you want to run tests).Our docs assume that Bazelisk is available as bazel on your PATH.
Refer to SETUP.md for detailed instructions.
Build CLI and node image:
bazel build //metropolis/cli/dbg //:launch -c dbg
Launch an ephemeral test node:
bazel test //:launch -c dbg --test_output=streamed
Run a kubectl command while the test is running:
bazel-bin/metropolis/cli/dbg/dbg_/dbg kubectl describe node
Run full test suite:
bazel test -c dbg //...