| commit | 62f1d3680947e1d78bacf2a7277fb4b2007ebacb | [log] [tgz] |
|---|---|---|
| author | Lorenz Brun <lorenz@monogon.tech> | Tue Nov 14 16:18:24 2023 +0100 |
| committer | Lorenz Brun <lorenz@monogon.tech> | Tue Nov 14 16:44:03 2023 +0000 |
| tree | 38c2fe1d57b68788f79ae018075b246f228310cc | |
| parent | 60461b2b23eb57319525a3e00d7ae57e51598ebc [diff] |
treewide: stop using LZ4 for initrd compression There are two issues at play here: One is a bug in pierrec/lz4 when using the legacy framing format [1]. This bit us when we hit a broken size region with CL:2130, taking hours to debug. The other is the fact that the Linux LZ4 frame format has significant design issues [2], especially with concatenanted initrds. The first issue could be fixed by switching to a different LZ4 implementation (we do even have the reference impl in the monorepo) but there is no API to generate the legacy frame format and things like [3], a patch carried by Ubuntu to fix more edge cases just do not inspire confidence in such a solution. Thus, this CL switches over to using zstd for compressing initrds. Zstd is slower than LZ4 for decompressing, but it still decompresses at multiple GB/s per core while having a much better compression ratio. It also doesn't have any Linux-specific bits and Linux uses the reference implementation for decoding, which should make it much more robust. So overall I think this is a good tradeoff. [1] https://github.com/pierrec/lz4/issues/156 [2] https://github.com/lz4/lz4/issues/956#issuecomment-736705712 [3] https://launchpadlibrarian.net/507407918/0001-unlz4-Handle-0-size-chunks-discard-trailing-padding-.patch Change-Id: I69cf69f2f361de325f4b39f2d3644ee729643716 Reviewed-on: https://review.monogon.dev/c/monogon/+/2313 Tested-by: Jenkins CI Reviewed-by: Serge Bazanski <serge@monogon.tech>
This is the main repository containing the source code for the Monogon Platform.
This is pre-release software - take a look, and check back later!
Our build environment is self-contained and requires only minimal host dependencies:
/dev/kvm (if you want to run tests).Our docs assume that Bazelisk is available as bazel on your PATH.
Refer to SETUP.md for detailed instructions.
Build CLI and node image:
bazel build //metropolis/cli/dbg //:launch --config dbg
Launch an ephemeral test node:
bazel test //:launch --config dbg --test_output=streamed
Run a kubectl command while the test is running:
bazel-bin/metropolis/cli/dbg/dbg_/dbg kubectl describe node
Run full test suite:
bazel test --config dbg //...