| commit | 57b4375dc2763dbf8444a4786bd41b7ec1a8172b | [log] [tgz] |
|---|---|---|
| author | Serge Bazanski <serge@nexantic.com> | Mon Jul 13 19:17:48 2020 +0200 |
| committer | Serge Bazanski <serge@nexantic.com> | Mon Jul 13 19:17:48 2020 +0200 |
| tree | 96c6ec6648426bd51bbf82573b2fbe28f2044868 | |
| parent | 73fc59541abfc457598cc5e62ae4d2c3b84065a1 [diff] |
core/internal/cluster: implement multi-node clusters with 'golden ticket'. As we have fully ripped out all traces of the node management service or integrity checks, we implement a stopgap system that allows us to continue developing multi-node clusters. This mechanism is enrolment using 'golden tickets', which are protobuf messages that can be generated via the debug service on an existing cluster, and set on a new node's EnrolmentConfig to bring that enrol that node into the cluster. As this is a stopgap measure (waiting for better cluster lifecycle design), this is somewhat poorly implemented, with known issues: - odd enrolment flow that creates all certificates off-node and results in some code duplication in the cluster manager and node debug service - (more) assumptions that every node is both a kubernetes and etcd member. - absolutely no protection against consensus loss due to even quorum membership, repeated issuance of certificates - dependence on knowing the IP address of the new node ahead of time, which is not something that our test harness supports well (or that we want to rely on at all) Test Plan: part of existing multi-node tests X-Origin-Diff: phab/D591 GitOrigin-RevId: 8f099e6ef37f8d47fb2272a3a14b25ed480e377a
This is the monorepo storing all of nexantic's internal projects and libraries.
We assume a Fedora host system provisioned using rW, and IntelliJ as the IDE.
For better reproducibility, all builds are executed in containers.
Spinning up: scripts/create_container.sh
Spinning down: scripts/destroy_container.sh
Running commands: scripts/run_in_container.sh <...>
Using bazel using a wrapper script: scripts/bin/bazel <...> (add to your local $PATH for convenience)
Launch the node:
bazel run //:launch
Run a kubectl command:
bazel run //core/cmd/dbg -- kubectl describe
This repository is compatible with the IntelliJ Bazel plugin. All commands run inside the container, and necessary paths are mapped into the container.
We check the entire .ijwb project directory into the repository, which requires everyone to use the latest version of both IntelliJ and the Bazel plugin, but eliminates manual setup steps.
The following steps are necessary:
Install Google's official Bazel plugin in IntelliJ.
Add the absolute path to your ~/.cache/bazel-nxt folder to your idea64.vmoptions (Help → Edit Custom VM Options) and restart IntelliJ:
-Dbazel.bep.path=/home/leopold/.cache/bazel-nxt
Set "Bazel Binary Location" in Other Settings → Bazel Settings to the absolute path of scripts/bin/bazel. This is a wrapper that will execute Bazel inside the container.
Open the .ijwb folder as IntelliJ project.
Disable Vgo support for the project.
Run a non-incremental sync in IntelliJ
The plugin will automatically resolve paths for generated files.
If you do not use IntelliJ, you need to use the scripts/bazel_copy_generated_for_ide.sh script to copy files locally.