tree 3a020de589b3931d069a29deebddbb47e1f41d7f
parent 961a7cc7f4794833f821a368b68d12ec70353d6c
author Serge Bazanski <serge@monogon.tech> 1706187714 +0100
committer Serge Bazanski <serge@monogon.tech> 1706204602 +0000

metropolis/node/curator: restart listener on status changes

This change ensures that a Curator's gRPC listener restart not only when
it switches from leader to follower (and vice versa), but also when the
curator instance keeps being a leader/follower but the leadership status
changes in an important way.

Restarting the listener is important when the Curator is a re-elected
leader, as otherwise the listener will continue running with the old
leadership data which includes the old leadership election key/revision,
subsequently failing any leadership transactions when serving gRPC
requests.

Similarly, we need to restart the follower listener when the we the
leader changes to some other node, otherwise we end up serving old
leadership data in GetCurrentLeader.

An alternative fix would be to move the leadership status to an Event
value that gets updated and passed through to the listeners. This would
avoid somewhat unnecessary listener restarts on leadership changes, but
this will do for now.

This should fix issue #276 and perhaps some related flakiness. I'll try
to write a test for this on top of this CR, but this will require a new
kind of Curator test which we don't have yet.

Change-Id: I20ef147adeb0c976b904ef55dbadb9b461111e94
Reviewed-on: https://review.monogon.dev/c/monogon/+/2722
Tested-by: Jenkins CI
Reviewed-by: Lorenz Brun <lorenz@monogon.tech>