)]}' { "log": [ { "commit": "fa5c2fccc528b40f216687e02f0c1cd004e013d6", "tree": "f39c24f681176b7bbf36fe6af304c6902124f552", "parents": [ "4efaa019244db96128941965aa72c0e1371b0d2d" ], "author": { "name": "Lorenz Brun", "email": "lorenz@nexantic.com", "time": "Mon Sep 28 13:32:12 2020 +0200" }, "committer": { "name": "Lorenz Brun", "email": "lorenz@nexantic.com", "time": "Mon Sep 28 13:32:12 2020 +0200" }, "message": "Use CoreDNS for everything and make directives dynamic\n\nThis moves CoreDNS from Kubernetes to the network tree and uses\nit for OS-side resolution too. For this to work together with Kubernetes it now\ncontains a dynamic directive system which allows various parts of the OS\nto register and unregister directives at runtime. This system is used to hook\nKubernetes and DHCP-supplied DNS servers into the configuration.\n\nThis also enables the hosts plugin to resolve the local hostname from within\nCoreDNS to avoid querying external DNS servers for that (T773).\n\nTest Plan:\nCTS covers K8s-related tests, external resolution manually tested from\na container.\n\nBug: T860, T773\n\nX-Origin-Diff: phab/D628\nGitOrigin-RevId: f1729237f3d17d8801506f4d299b90e7dce0893a\n" }, { "commit": "ca24cfaef52b388438f06e69352643a4ee0185ca", "tree": "7333d3472dcf2f57cd7ed73349c5fc224749c296", "parents": [ "339582bb8d52b930c15cee77548f11794bb3b362" ], "author": { "name": "Lorenz Brun", "email": "lorenz@nexantic.com", "time": "Tue Aug 18 13:49:37 2020 +0200" }, "committer": { "name": "Lorenz Brun", "email": "lorenz@nexantic.com", "time": "Tue Aug 18 13:49:37 2020 +0200" }, "message": "Fixups after Node refactor\n\nTest Plan: Tested in the CTS revision.\n\nX-Origin-Diff: phab/D618\nGitOrigin-RevId: 4c591d463a0709fc944f52e32069cb7ababd55ca\n" }, { "commit": "57b4375dc2763dbf8444a4786bd41b7ec1a8172b", "tree": "96c6ec6648426bd51bbf82573b2fbe28f2044868", "parents": [ "73fc59541abfc457598cc5e62ae4d2c3b84065a1" ], "author": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Mon Jul 13 19:17:48 2020 +0200" }, "committer": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Mon Jul 13 19:17:48 2020 +0200" }, "message": "core/internal/cluster: implement multi-node clusters with \u0027golden ticket\u0027.\n\nAs we have fully ripped out all traces of the node management service or\nintegrity checks, we implement a stopgap system that allows us to\ncontinue developing multi-node clusters. This mechanism is enrolment\nusing \u0027golden tickets\u0027, which are protobuf messages that can be\ngenerated via the debug service on an existing cluster, and set on a new\nnode\u0027s EnrolmentConfig to bring that enrol that node into the cluster.\n\nAs this is a stopgap measure (waiting for better cluster lifecycle\ndesign), this is somewhat poorly implemented, with known issues:\n - odd enrolment flow that creates all certificates off-node and results\n in some code duplication in the cluster manager and node debug\n service\n - (more) assumptions that every node is both a kubernetes and etcd\n member.\n - absolutely no protection against consensus loss due to even quorum\n membership, repeated issuance of certificates\n - dependence on knowing the IP address of the new node ahead of time,\n which is not something that our test harness supports well (or that\n we want to rely on at all)\n\nTest Plan: part of existing multi-node tests\n\nX-Origin-Diff: phab/D591\nGitOrigin-RevId: 8f099e6ef37f8d47fb2272a3a14b25ed480e377a\n" }, { "commit": "73fc59541abfc457598cc5e62ae4d2c3b84065a1", "tree": "3eed74792f885733069eedefa2de5dfaaa92052a", "parents": [ "1ebd1e133bac1a7fe0d667ec2ac95f87f63c3701" ], "author": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Tue Jul 21 12:50:54 2020 +0200" }, "committer": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Tue Jul 21 12:50:54 2020 +0200" }, "message": "core/proto: remove NodeDebugService.GetCondition\n\nThis stopped being used after D590 where we moved the debug service to\nstart late enough that we\u0027re sure we already have the prerequisite\nconditions to continue testing. In the future, the debug service might\ngrow some introspection methods into the supervisor - if so, that will\nsomewhat replace this bespoke condition API.\n\nTest Plan: no behavioural changes\n\nX-Origin-Diff: phab/D604\nGitOrigin-RevId: a7edf8a45467fb2be602323b612abe054acf2b11\n" }, { "commit": "1ebd1e133bac1a7fe0d667ec2ac95f87f63c3701", "tree": "c84bca5f68d4bbe959006215bf4711050af04288", "parents": [ "c2c7ad97b50194a550e77b875570ece90259f4ea" ], "author": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Mon Jul 13 19:17:16 2020 +0200" }, "committer": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Mon Jul 13 19:17:16 2020 +0200" }, "message": "core/internal/cluster: add new single-node cluster code\n\nThis adds a cluster library, that consists of:\n - a Node object that can be loaded from and saved into etcd,\n representing a node of the cluster that can have different \u0027role\n tags\u0027 assigned to it\n - a cluster Manager, that is responsible for bringing up the local node\n into a cluster (by creaating a new cluster, enrolling into or joining a\n cluster)\n\nThis also gets wired into core/cmd/init, and as such completes a chunk\nof The Refactor. This code should pass tests.\n\nTest Plan: this should work! should be covered by existing e2e tests.\n\nX-Origin-Diff: phab/D590\nGitOrigin-RevId: e88022164e4353249b29fc16849a02805f15dd49\n" }, { "commit": "c2c7ad97b50194a550e77b875570ece90259f4ea", "tree": "cc0d43c49c5d1cb787adf5c548c589fa50e9e72e", "parents": [ "efdb6e9da9ed4d575afe72fde02a27817eca37c4" ], "author": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Mon Jul 13 17:20:09 2020 +0200" }, "committer": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Mon Jul 13 17:20:09 2020 +0200" }, "message": "core/internal: move containerd and kubernetes to localstorage\n\nThis moves the last users of the old \u0027storage\u0027 library onto \u0027localstorage\u0027. We move a lot of \u0027runtime\u0027 directories to a single `/ephemeral` root. This could be called `/run`, but that might imply FHS compliance - which we don\u0027t have, nor want to have.\n\nWe also slightly refactor Kubernetes services to be a bit nicer to spawn. But generally, this is a pure refactor, with no functional changes.\n\nTest Plan: this should fail. part of a larger stack. D590 is the first tip of the stack that should work.\n\nX-Origin-Diff: phab/D589\nGitOrigin-RevId: d2a7c0bb52c2a7c753199221c609e03474936c22\n" }, { "commit": "70f65b237aa29f2e9aced8a4a1e1739b6544cb92", "tree": "35f9b86b2e8b33572495935039e3b9c0a65d30c0", "parents": [ "52f7f291c1987fe98bd10d3ad79d4a0c8772ad03" ], "author": { "name": "Lorenz Brun", "email": "lorenz@nexantic.com", "time": "Wed Jul 08 17:02:47 2020 +0200" }, "committer": { "name": "Lorenz Brun", "email": "lorenz@nexantic.com", "time": "Wed Jul 08 17:02:47 2020 +0200" }, "message": "Add init debugging support\n\nThis adds Delve into the initramfs and a conditional hook which attaches Delve to our init\nafter the network is up. This allows for breakpoint-debugging the init itself, at least after the\nvery early node bringup.\n\nTest Plan:\n`bazel run -c dbg //:launch`, then use IDEA\u0027s Go Remote target to connect to localhost:2345\nand set a breakpoint.\n\nBug: T786\n\nX-Origin-Diff: phab/D581\nGitOrigin-RevId: f6b32e7b7f4d36c8492df3e11ee97588817dbd8e\n" }, { "commit": "8da5377d65930ff0a4085449c61f09fcfe64ec02", "tree": "1bed5948776792298347e315341d199fc7bdcaa6", "parents": [ "6acfc323aa74a424220907218cfa7f303b6992cc" ], "author": { "name": "Leopold Schabel", "email": "leo@nexantic.com", "time": "Tue Jun 02 15:56:51 2020 +0200" }, "committer": { "name": "Leopold Schabel", "email": "leo@nexantic.com", "time": "Tue Jun 02 15:56:51 2020 +0200" }, "message": "core/cmd/init: ignore SIGURG\n\nTest Plan: Error is gone.\n\nBug: T743\n\nX-Origin-Diff: phab/D526\nGitOrigin-RevId: ad080bd3299c2c32208ba7553b750b16282c5656\n" }, { "commit": "878f5f9e5f9de93b09d354db7d116fd3d558dbfa", "tree": "994b67ea5264f7e38bb67e9043a369454eaab75d", "parents": [ "9a741a861a4cb5c52b0251a4abf3a2c606b06198" ], "author": { "name": "Lorenz Brun", "email": "lorenz@nexantic.com", "time": "Tue May 12 16:15:39 2020 +0200" }, "committer": { "name": "Lorenz Brun", "email": "lorenz@nexantic.com", "time": "Tue May 12 16:15:39 2020 +0200" }, "message": "Add Kubernetes Worker and infrastructure\n\nAdds Kubernetes Kubelet with patches for syscall-based mounting and\nsyscall-based (and much faster) metrics. fsquota patches have been\ndeferred to a further revision (for robust emptyDir capacity isolation).\n\nChanges encoding of the node ID to hex since Base64-URL is not supported\nas a character set for K8s names. Also adds `/etc/machine-id` and\n`/etc/os-release` since Kubernetes wants them. `os-release` is generated\nby stamping, `machine-id` is the hex-encoded node ID derived from the\npublic key.\n\nAlso includes a primitive reconciler which automatically ensures a set of\nbuilt-in Kubernetes objects are always present. Currently this includes\na PSP and some basic RBAC policies that are elementary to proper cluster\noperations.\n\nAdds an additional gRPC service (NodeDebugService) to cleanly\ncommunicate with external debug and test tooling. It supports reading\nfrom logbuffers for all externally-run components, checking conditions\n(for replacing log matching in testing and debugging) and getting\ndebug credentials for the Kubernetes cluster.\n\nA small utility (dbg) is provided that interfaces with NodeDebugService\nand provides access to its functions from the CLI. It also incorporates\na kubectl wrapper which directly grabs credentials from the Debug API\nand passes them to kubectl\n(e.g. `bazel run //core/cmd/dbg -- kubectl describe node`).\n\nTest Plan:\nManually tested.\nKubernetes:\n`bazel run //core/cmd/dbg -- kubectl create -f test.yml`\n\nChecked that pods run, logs are accessible and exec works.\n\nReading buffers:\n`bazel run //core/cmd/dbg -- logs containerd`\n\nOutputs containerd logs in the right order.\n\nAutomated testing is in the works, but has been deferred to a future\nrevision because this one is already too big again.\n\nX-Origin-Diff: phab/D525\nGitOrigin-RevId: 0fbfa0c433de405526c7f09ef10c466896331328\n" }, { "commit": "c88c82db8b1a7f8a07782c970e1d0dfb453f9f66", "tree": "22072c4f18e4aaa855577ff0b42a86ef77a9c4cb", "parents": [ "60febd9db40970a31a2f49bdb969897a37c11cc6" ], "author": { "name": "Lorenz Brun", "email": "lorenz@nexantic.com", "time": "Fri May 08 14:35:04 2020 +0200" }, "committer": { "name": "Lorenz Brun", "email": "lorenz@nexantic.com", "time": "Fri May 08 14:35:04 2020 +0200" }, "message": "Add containerd \u0026 gVisor support\n\nThis adds containerd, CNI, gVisor and all the necessary shims\nand supporting infrastructure. It also enables all relevant features in\nthe Linux kernel. containerd is designed as a simple supervisor.Runnable.\nIt is not being started yet, this will happen in D497.\n\nSplit out from feature/kubelet.\n\nTest Plan:\nHas been tested in conjunction with the rest of D497, will be\ncovered by a K8s E2E test there.\n\nX-Origin-Diff: phab/D509\nGitOrigin-RevId: 92523516b7e361a30da330eb187787e6045bfd17\n" }, { "commit": "b1b742f91489cafa199bf5dd6e83d965cb23f63f", "tree": "11ad76da23350c0d70ee2f21aa50e56cb1421b57", "parents": [ "9c09c4e9a637dedc1643e32419f56f789e79fec8" ], "author": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Tue Mar 24 13:58:19 2020 +0100" }, "committer": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Tue Mar 24 13:58:19 2020 +0100" }, "message": "core/internal/network: move to supervisor\n\nTest Plan: behaviour shouldn\u0027t change, covered by existing tests\n\nBug: T653\n\nX-Origin-Diff: phab/D430\nGitOrigin-RevId: b92f0953daba6da84bad96120cde2021c4a82e5c\n" }, { "commit": "581b0bd6386a077e29107710e008983b62233ccf", "tree": "85cf721d9711e7adc88c744c55ee12a96ee7114d", "parents": [ "79d7a625709242204993cffbd99ed734dc1c50a5" ], "author": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Thu Mar 12 13:36:43 2020 +0100" }, "committer": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Thu Mar 12 13:36:43 2020 +0100" }, "message": "init: remount to tmpfs\n\nrunsc needs to be able to pivot_root. According to @lorenz this does not\nwork from initramfs. This introduces a temporary fix to re-mount and\nre-exec into a new root based on tmpfs.\n\nA proper fix would be to use a real filesystem instead of initramfs\n(like squashfs), but this will do for now.\n\nWe also use this opportunity to use devtmpfs instead of manually\nmanaging /dev. This collides with the storage manager that tries to\ncreate all storage nodes - we just remove that.\n\nTest Plan: shouldn\u0027t change behaviour\n\nX-Origin-Diff: phab/D433\nGitOrigin-RevId: aa59fec6551bab1b1b9c2fe037dce410e550981b\n" }, { "commit": "cdb8c78eb7d29e6595053c455141007cb1c13a83", "tree": "db17ef01058c8185887e26e31131d62c168a23c7", "parents": [ "6c8d5f9319706be576563b990c875afc0d60d02d" ], "author": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Mon Feb 17 12:34:02 2020 +0100" }, "committer": { "name": "Serge Bazanski", "email": "serge@nexantic.com", "time": "Mon Feb 17 12:34:02 2020 +0100" }, "message": "Revamp DHCP, add basic context management\n\nThis started off as a small change to make the network service DHCP client a bit nicer, and ended up basically me half-assedly starting to add context within Smalltown.\n\nIn my opionion a simple OnStart/OnStop lifecycle management for services will stop working once we have to start handling failing services. I think taking inspiration from Erlang\u0027s OTP and implementing some sort of supervision tree is the way to go. I think this also ties nicely together with Go\u0027s context system, at least partially. Implementing the full supervision tree system is out of scope for this change, but at least this introduces .Context() on the base service struct that service implementations can use. Currently each service has its own background context, but again, this should tie into some sort of supervision tree in the future. There will be a design document for this.\n\nI also rejigger the init code to have a context available immediately, and use that to acquire (with timeout) information about DHCP addresses from the network service.\n\nI also fix a bug where the network service is started twice (once by init, once by the smalltown node code; now the smalltown node code takes in a dependency injected network service instead).\n\nI also fix a bug where OnStop would call OnStart. Whoops.\n\nTest Plan: no new functionality, covered by current tests\n\nBug: T561\n\nX-Origin-Diff: phab/D396\nGitOrigin-RevId: adddf3dd2f140b6ea64eb034ff19533d32c4ef23\n" }, { "commit": "aa6b7346a87a5512fbdd5b39db766000c0e10415", "tree": "8b7665934b854d4d2ee18e90a289752f8cd85942", "parents": [ "5e0bd2d43ab72cf4091e7689d02f95e07b1c1010" ], "author": { "name": "Lorenz Brun", "email": "lorenz@nexantic.com", "time": "Thu Dec 12 02:55:02 2019 +0100" }, "committer": { "name": "Lorenz Brun", "email": "lorenz@nexantic.com", "time": "Thu Dec 12 02:55:02 2019 +0100" }, "message": "Attestation \u0026 Identity \u0026 Global Unlock \u0026 Enrolment\n\nThis changes the node startup sequence significantly. Now the following three startup procedures replace the old setup/join mechanic:\n* If no enrolment config is present, automatically bootstrap a new cluster and become master for it.\n* If an enrolment config with an enrolment token is present, register with the NodeManagementService.\n* If an enrolment config without an enrolment token is present, attempt a normal cluster unlock.\n\nIt also completely revamps the GRPC management services:\n* NodeManagementService is a master-only service that deals with other nodes and has a cluster-wide identity\n* NodeService is only available in unlocked state and keyed with the node identity\n* ClusterManagement is now a master-only service that\u0027s been spun out of the main NMS since they have very different authentication models and also deals with EnrolmentConfigs\n\nThe TPM support library has also been extended by:\n* Lots of integrity attestation and verification functions\n* Built-in AK management\n* Some advanced policy-based authentication stuff\n\nAlso contains various enhancements to the network service to make everything work in a proper multi-node environment.\n\nLots of old code has been thrown out.\n\nTest Plan: Passed a full manual test of all three startup modes (bootstrap, enrolment and normal unlock) including automated EnrolmentConfig generation and consumption in a dual-node configuration on swtpm / OVMF.\n\nBug: T499\n\nX-Origin-Diff: phab/D291\nGitOrigin-RevId: d53755c828218b1df83a1d7ad252c7b3231abca8\n" }, { "commit": "a4516f9887e43b774e49c22db93cdf289dc9cfb1", "tree": "8a0761a3480074b01d5584a1cd5c111a69f76594", "parents": [ "6e8f69c53a2c82f5a760ab2e8152218cc86f3430" ], "author": { "name": "Leopold Schabel", "email": "leo@nexantic.com", "time": "Wed Dec 04 20:27:05 2019 +0000" }, "committer": { "name": "Leopold Schabel", "email": "leo@nexantic.com", "time": "Wed Dec 04 20:27:05 2019 +0000" }, "message": "Add minimal functionality test for k8s control plane\n\nBasic functionality test that sends the bootstrap RPC call,\nwaits for the k8s control plane to come up and runs a simple\nkubectl command (that is expected to fail).\n\nAdds reflection to the server to make grpc_cli easier to use.\n\nTest Plan:\nRan `:launch` (because we modified its config) and `:test_boot`,\nsaw a nicely booted k8s cluster:\n\n{P90}\n\nX-Origin-Diff: phab/D275\nGitOrigin-RevId: fe01e3f3ed09877aa76c15946664c9d9bdc4751b\n" }, { "commit": "68c58755e0a56e1b1c565d80f99056ec4948fbec", "tree": "f122ab392769d33620077c65ddf0f0a3aed43d1c", "parents": [ "5ed291ea1833ffd07665b6194f7b6db2b7c1c4aa" ], "author": { "name": "Leopold Schabel", "email": "leo@nexantic.com", "time": "Thu Nov 14 21:00:59 2019 +0100" }, "committer": { "name": "Leopold Schabel", "email": "leo@nexantic.com", "time": "Thu Nov 14 21:00:59 2019 +0100" }, "message": "Improve documentation, remove dead code plus some minor refactorings\n\nThis improves our code-to-comments ratio by a lot.\n\nOn the refactorings:\n\n- Simplify the cluster join mode to just a single protobuf message -\n a node can either join an existing cluster or bootstrap a new one.\n All of the node-level setup like hostname and trust backend is done\n using the setup call, since those are identical for both cases.\n\n- We don\u0027t need a node name separate from the hostname. Ideally, we would\n get rid of IP addresses for etcd as well.\n\n- Google API design guidelines suggest the `List` term (vs. `Get`).\n\n- Add username to comments for consistency. I think the names provide\n useful context, but git blame is a thing. What do you think?\n\n- Fixed or silenced some ignored error checks in preparation of using\n an errcheck linter. Especially during early boot, many errors are\n obviously not recoverable, but logging them can provide useful debugging info.\n\n- Split up the common package into smaller subpackages.\n\n- Remove the audit package (this will be a separate service that probably\n uses it own database, rather than etcd).\n\n- Move storage constants to storage package.\n\n- Remove the unused KV type.\n\nI also added a bunch of TODO comments with discussion points.\nAdded both of you as blocking reviewers - please comment if I\nmisunderstood any of your code.\n\nTest Plan: Everything compiles and scripts:launch works (for whatever that\u0027s worth).\n\nX-Origin-Diff: phab/D235\nGitOrigin-RevId: 922fec5076e8d683e1138f26d2cb490de64a9777\n" }, { "commit": "0d7c91e331022831a974c2e34d32bb5b89ddc89c", "tree": "5b822873c015053f4b697d60c33fa3b1ef9a3a4b", "parents": [ "043daa57020dd36e074488dcb432114a548a3d2a" ], "author": { "name": "Hendrik Hofstadt", "email": "hendrik@certus.one", "time": "Wed Oct 23 21:44:47 2019 +0200" }, "committer": { "name": "Hendrik Hofstadt", "email": "hendrik@certus.one", "time": "Wed Oct 23 21:44:47 2019 +0200" }, "message": "Implement monorepo layout\n\nImplemented the nexantic monorepo.\n\nSmalltown code was moved to `core`. From now on all code will live in top level directories named after the projects with the exception for general purpose libraries which should go to `\u003clang\u003elibs`.\n\nGeneral build and utility folders are underscore prefixed.\n\nThe repo name will from now on be rNXT (nexantic). I think this change makes sense since components in this repo will not all be part of Smalltown, the Smalltown brand has been claimed by Signon GmbH so we need to change it anyway and the longer we wait the harder it will be to change/move it.\n\nTest Plan: Launched Smalltown using `./scripts/bin/bazel run //core/scripts:launch`\n\nX-Origin-Diff: phab/D210\nGitOrigin-RevId: fa5a7f08143d2ead2cb7206b4c63ab641794162c\n" } ] }