package yb.consensus
Get desktop application:
View/edit binary Protocol Buffers messages
A Raft implementation.
rpc UpdateConsensus (, )
consensus.proto:627Analogous to AppendEntries in Raft, but only used for followers.
Similar to UpdateConsensus but takes a batch of ConsensusRequestPB and returns a batch of ConsensusResponsePB.
message
consensus.proto:388repeated consensus_request = 1
message
consensus.proto:392repeated consensus_response = 1
RequestVote() from Raft.
message
consensus.proto:227A request from a candidate peer that wishes to become leader of the configuration serving tablet with 'tablet_id'. See RAFT sec. 5.2.
optional bytes dest_uuid = 6
UUID of server this request is addressed to.
required string tablet_id = 1
Identifies the tablet configuration a the vote is being requested for.
required bytes candidate_uuid = 2
The uuid of the sending peer.
required int64 candidate_term = 3
The term we are requesting a vote for. If this term is higher than the callee's term, the callee will update its own term to match, and if it is the current leader it will step down.
required candidate_status = 4
The candidate node status so that the voter node can decide whether to vote for it as LEADER. In particular, this includes the last OpId persisted in the candidate's log, which corresponds to the lastLogIndex and lastLogTerm fields in Raft. A replica must vote no for a candidate that has an OpId lower than them.
optional bool ignore_live_leader = 5
Normally, replicas will deny a vote with a LEADER_IS_ALIVE error if they are a leader or recently heard from a leader. This is to prevent partitioned nodes from disturbing liveness. If this flag is true, peers will vote even if they think a leader is alive. This can be used for example to force a faster leader hand-off rather than waiting for the election timer to expire.
optional bool preelection = 7
true if it is pre-election request and state of voter should not be changed.
message
consensus.proto:263A response from a replica to a leader election request.
optional bytes responder_uuid = 1
The uuid of the node sending the reply.
optional int64 responder_term = 2
The term of the node sending the reply. Allows the candidate to update itself if it is behind.
optional bool vote_granted = 3
True if this peer voted for the caller, false otherwise.
repeated int32 remaining_leader_lease_duration_ms = 4
An upper bound on the remainder of the old leader's lease that the new leader has to wait out before it can start serving read and write requests.
repeated bytes leader_lease_uuid = 7
UUID of node that has lease.
repeated fixed64 leader_ht_lease_expiration = 5
An upper bound on the deadline of the old hybrid time leader's lease that the new leader has to wait out before it can start serving read and write requests. Contains only physical part of hybrid time.
repeated bytes leader_ht_lease_uuid = 8
UUID of node that has ht lease. Lease and lease ht have different lifetime, so set of leases and corresponding UUIDs could be different.
optional bool preelection = 6
true if it is response to pre-election request.
optional consensus_error = 998
TODO: Migrate ConsensusService to the AppStatusPB RPC style and merge these errors. Error message from the consensus implementation.
optional error = 999
A generic error message (such as tablet not found).
rpc ChangeConfig (, )
consensus.proto:642Implements all of the one-by-one config change operations, including AddServer() and RemoveServer() from the Raft specification, as well as an operation to change the role of a server between VOTER and PRE_VOTER. An OK response means the operation was successful.
message
consensus.proto:79A configuration change request for the tablet with 'tablet_id'. These requests are restricted to one-by-one operations, as specified in Diego Ongaro's Raft PhD thesis. This is the RPC request, but it does not end up in the log. See also ChangeConfigRecordPB.
optional bytes dest_uuid = 4
UUID of server this request is addressed to.
required bytes tablet_id = 1
optional ChangeConfigType type = 2
The type of config change requested. This field must be specified, but is left as optional due to being an enum.
optional server = 3
The peer to add or remove. When 'type' == ADD_SERVER, both the permanent_uuid and last_known_addr fields must be set. For REMOVE_SERVER, either the permanent_uuid field or last_known_addr should be set - 'use_host' is for using the latter.
optional int64 cas_config_opid_index = 5
The OpId index of the committed config to replace. This optional parameter is here to provide an atomic (compare-and-swap) ChangeConfig operation. The ChangeConfig() operation will fail if this parameter is specified and the committed config does not have a matching opid_index. See also the definition of RaftConfigPB.
optional bool use_host = 6
For REMOVE case, when this is set, we assume that the peer is dead and will not affect the quorum anymore. So we skip checking/using its server uuid and remove it based on the ip/port info from 'server'.
message
consensus.proto:110The configuration change response. If any immediate error occurred the 'error' field is set with it.
optional error = 1
optional fixed64 hybrid_time = 2
The hybrid_time chosen by the server for this change config operation. TODO: At the time of writing, this field is never set in the response. TODO: Propagate signed hybrid_times. See KUDU-611.
Implements unsafe config change operation for manual recovery use cases.
message
consensus.proto:574An unsafe change configuration request for the tablet with 'tablet_id'.
optional bytes dest_uuid = 1
UUID of server this request is addressed to.
required bytes tablet_id = 2
required bytes caller_id = 3
Sender identification, it could be a static string as well.
required new_config = 4
The raft config sent to destination server. Only the 'permanent_uuid' of each peer in the config is required (address-related information is ignored by the server). The peers specified in the 'new_config' are required to be a subset of (or equal to) the peers in the committed config on the destination replica.
message
consensus.proto:592The unsafe change configuration response. 'error' field is set if operation failed.
optional error = 1
rpc GetNodeInstance (, )
consensus.proto:647message
consensus.proto:410(message has no fields)
message
consensus.proto:413optional node_instance = 1
optional registration = 2
rpc RunLeaderElection (, )
consensus.proto:650Force this node to run a leader election.
message
consensus.proto:420Message that makes the local peer run leader election to be elected leader. Assumes that a tablet with 'tablet_id' exists.
optional bytes dest_uuid = 2
UUID of server this request is addressed to.
required bytes tablet_id = 1
the id of the tablet
optional committed_index = 3
The id of the last operation the current leader has committed to its log. When specified, the peer must wait till the operation has been committed to its log before starting the election to be qualified as a leader.
optional bytes originator_uuid = 4
UUID of old leader, that requested this election. If we lost election, we would notify originator about this fact. So it would reset its withold timeout and try to become a leader again.
optional bool suppress_vote_request = 5
optional bool initial_election = 6
message
consensus.proto:442optional error = 1
A generic error message (such as tablet not found).
Notify originator about lost election, so it could reset its timeout.
message
consensus.proto:447optional bytes dest_uuid = 1
UUID of server this request is addressed to. It is originator_uuid from RunLeaderElectionRequestPB
optional bytes tablet_id = 2
the id of the tablet
optional bytes election_lost_by_uuid = 3
UUID of server that lost election
message
consensus.proto:459optional error = 1
A generic error message (such as tablet not found).
rpc LeaderStepDown (, )
consensus.proto:656Force this node to step down as leader.
message
consensus.proto:464optional bytes dest_uuid = 2
UUID of server this request is addressed to.
required bytes tablet_id = 1
The id of the tablet.
optional bytes new_leader_uuid = 3
UUID of the server that should run the election to become the new leader.
optional bool force_step_down = 4
Used in tests to ignore check for new leader conditions.
optional bool disable_graceful_transition = 5
If new_leader_uuid is not specified, the current leader will attempt to gracefully transfer leadership to another peer. Setting this flag disables that behavior.
message
consensus.proto:484optional error = 1
A generic error message (such as tablet not found).
optional int64 time_since_election_failure_ms = 2
Time in milliseconds since previous election failure with the same intended ("protege") leader. This is used to track the remaining amount of time during which the load balancer should not attempt to ask (any) leader of this tablet to step down in favor of the same peer.
rpc GetLastOpId (, )
consensus.proto:659Get the latest committed or received opid on the server.
message
consensus.proto:494optional bytes dest_uuid = 2
UUID of server this request is addressed to.
required bytes tablet_id = 1
the id of the tablet
optional OpIdType opid_type = 3
Whether to return the last-received or last-committed OpId.
optional OperationType op_type = 4
message
consensus.proto:507optional opid = 1
optional error = 2
A generic error message (such as tablet not found).
rpc GetConsensusState (, )
consensus.proto:662Returns the committed Consensus state.
message
consensus.proto:513optional bytes dest_uuid = 2
UUID of server this request is addressed to.
required bytes tablet_id = 1
The id of the tablet.
optional ConsensusConfigType type = 3
Whether to fetch the committed or active consensus state.
message
consensus.proto:524optional cstate = 1
optional error = 2
A generic error message (such as tablet not found).
optional LeaderLeaseStatus leader_lease_status = 3
Allows returning the leader lease status in the same RPC. Useful for waiting for the leader to be allowed to serve requests.
Instruct this server to remotely bootstrap a tablet from another host.
message
consensus.proto:535optional bytes dest_uuid = 5
UUID of server this request is addressed to.
required bytes tablet_id = 1
optional bytes split_parent_tablet_id = 8
required bytes bootstrap_source_peer_uuid = 2
Identification for the host we are bootstrapping from.
repeated bootstrap_source_private_addr = 3
repeated bootstrap_source_broadcast_addr = 6
optional bootstrap_source_cloud_info = 7
optional int64 caller_term = 4
The caller's term. In the case that the target of this request has a TOMBSTONED replica with a term higher than this one, the request will fail.
optional bool is_served_by_tablet_leader = 9
When the remote bootstrap request is served by a closest follower to the new peer, the new peer needs to update the leader on progessing the log anchor.
optional bytes tablet_leader_peer_uuid = 10
repeated tablet_leader_private_addr = 11
repeated tablet_leader_broadcast_addr = 12
optional tablet_leader_cloud_info = 13
optional uint32 clone_source_seq_no = 14
If this tablet was created by cloning, the sequence number of the operation that created it and the clone source tablet's id. This is used to reject RBS requests if the source tablet is present on the destination and has not applied the clone yet.
optional string clone_source_tablet_id = 15
message
consensus.proto:569optional error = 1
message
consensus.proto:596optional error = 1
message
consensus.proto:64A configuration change request for the tablet with 'tablet_id'. This message is dynamically generated by the leader when AddServer() or RemoveServer() is called, and is what gets replicated to the log.
Used in:
required bytes tablet_id = 1
required old_config = 2
The old committed configuration config for verification purposes.
required new_config = 3
The new configuration to set the configuration to.
Used in:
UNKNOWN_CHANGE = 0
ADD_SERVER = 1
REMOVE_SERVER = 2
CHANGE_ROLE = 3
message
consensus.proto:609Used in:
optional fixed64 client_id1 = 1
optional fixed64 client_id2 = 2
repeated range = 3
Used in:
CONSENSUS_CONFIG_UNKNOWN = 999
CONSENSUS_CONFIG_COMMITTED = 1
Committed consensus config. This includes the consensus configuration that has been serialized through consensus and committed, thus having a valid opid_index field set.
CONSENSUS_CONFIG_ACTIVE = 2
Active consensus config. This could be a pending consensus config that has not yet been committed. If the config is not committed, its opid_index field will not be set.
message
consensus_types.proto:58Consensus-specific errors use this protobuf
Used in: ,
required ConsensusErrorPB.Code code = 1
The error code.
required status = 2
The Status object for the error. This will include a textual message that may be more useful to present in log messages, etc, though its error code is less specific.
The codes for consensus responses. These are set in the status when some consensus internal error occurs and require special handling by the caller. A generic error code is purposefully absent since generic errors should use tserver.TabletServerErrorPB.
Used in:
UNKNOWN = 0
INVALID_TERM = 2
Invalid term. Sent by peers in response to leader RPCs whenever the term of one of the messages sent in a batch is lower than the the term the peer is expecting.
LAST_OPID_TOO_OLD = 3
For leader election. The last OpId logged by the candidate is older than the last OpId logged by the local peer.
ALREADY_VOTED = 4
For leader election. The local replica has already voted for another candidate in this term.
NOT_IN_QUORUM = 5
The replica does not recognize the caller's request as coming from a member of the configuration.
PRECEDING_ENTRY_DIDNT_MATCH = 6
The responder's last entry didn't match the caller's preceding entry.
LEADER_IS_ALIVE = 7
The local replica is either a leader, or has heard from a valid leader more recently than the election timeout, so believes the leader to be alive.
CONSENSUS_BUSY = 8
The local replica is in the middle of servicing either another vote or an update from a valid leader.
CANNOT_PREPARE = 9
The local replica was unable to prepare a single transaction.
TABLET_SPLIT = 10
Split operation has been added to this Raft group log.
message
metadata.proto:145This PB is used to serialize all of the persistent state needed for Consensus that is not in the WAL, such as leader election and communication on startup.
required committed_config = 1
Last-committed peership.
required int64 current_term = 2
Latest term this server has seen. When a configuration is first created, initialized to 0. Whenever a new election is started, the candidate increments this by one and requests votes from peers. If any RPC or RPC response is received from another node containing a term higher than this one, the server should step down to FOLLOWER and set its current_term to match the caller's term. If a follower receives an UpdateConsensus RPC with a term lower than this term, then that implies that the RPC is coming from a former LEADER who has not realized yet that its term is over. In that case, we will reject the UpdateConsensus() call with ConsensusErrorPB::INVALID_TERM. If a follower receives a RequestConsensusVote() RPC with an earlier term, the vote is denied.
optional string voted_for = 3
Permanent UUID of the candidate voted for in 'current_term', or not present if no vote was made in the current term.
optional bytes split_parent_tablet_id = 4
optional uint32 clone_source_seq_no = 5
optional string clone_source_tablet_id = 6
message
consensus.proto:303A consensus request message, the basic unit of a consensus round.
Used as request type in: ConsensusService.UpdateConsensus
Used as field type in:
optional bytes dest_uuid = 7
UUID of server this request is addressed to.
required string tablet_id = 1
required bytes caller_uuid = 2
The uuid of the peer making the call.
required int64 caller_term = 3
The caller's term. As only leaders can send messages, replicas will accept all messages as long as the term is equal to or higher than the last term they know about. If a leader receives a request with a term higher than its own, it will step down and enter FOLLOWER state (see Raft sec. 5.1).
optional preceding_id = 4
The id of the operation immediately preceding the first operation in 'ops'. If the replica is receiving 'ops' for the first time 'preceding_id' must match the replica's last operation. This must be set if 'ops' is non-empty.
required committed_op_id = 5
The id of the last committed operation in the configuration. This is the id of the last operation the leader deemed committed from a consensus standpoint (not the last operation the leader applied). Raft calls this field 'leaderCommit'.
repeated ops = 6
Sequence of operations to be replicated by this peer. These will be committed when committed_index advances above their respective OpIds. In some cases committed_index can indicate that these operations are already committed, in which case they will be committed during the same request.
optional int32 leader_lease_duration_ms = 8
Leader lease duration in milliseconds. A leader is not allowed to serve up-to-date reads until it is able to replicate a lease extension. A new leader cannot assume its responsibilities until this amount time has definitely passed since the old leader sent the consensus request. Due to potential clock skew, we are not sending a timestamp, but an amount of time followers have to wait.
optional fixed64 ht_lease_expiration = 9
Leader lease expiration, physical part of hybrid time. A new leader cannot add new entries to RAFT log until hybrid time passes this expiration.
optional fixed64 propagated_safe_time = 10
Safe time measured on the leader at the time this request was generated.
optional fixed64 propagated_hybrid_time = 11
Hybrid time on the leader when this request was generated.
optional bool trace_requested = 12
If enabled, a trace will be collected for this RPC and returned in the response.
message
consensus.proto:362Used as response type in: ConsensusService.UpdateConsensus
Used as field type in:
optional bytes responder_uuid = 1
The uuid of the peer making the response.
optional int64 responder_term = 2
The current term of the peer making the response. This is used to update the caller (and make it step down if it is out of date).
optional status = 3
The current consensus status of the receiver peer.
optional error = 999
A generic error message (such as tablet not found), per operation error messages are sent along with the consensus status.
optional int64 num_sst_files = 5
Number of SST files in this follower regular db.
optional fixed64 propagated_hybrid_time = 6
Hybrid time on the follower when this request was processed.
optional string trace_buffer = 7
The trace for the RPC as a string, returned only if requested.
message
metadata.proto:109Represents a snapshot of a configuration at a given moment in time.
Used in: , , , ,
required int64 current_term = 1
A configuration is always guaranteed to have a known term.
optional string leader_uuid = 2
There may not always be a leader of a configuration at any given time. The node that the local peer considers to be leader changes based on rules defined in the Raft specification. Roughly, this corresponds either to being elected leader (in the case that the local peer is the leader), or when an update is accepted from another node, which basically just amounts to a term check on the UpdateConsensus() RPC request. Whenever the local peer sees a new term, the leader flag is cleared until a new leader is acknowledged based on the above critera. Simply casting a vote for a peer is not sufficient to assume that that peer has won the election, so we do not update this field based on our vote. The leader listed here, if any, should always be a member of 'configuration', and the term that the node is leader of _must_ equal the term listed above in the 'current_term' field. The Master will use the combination of current term and leader uuid to determine when to update its cache of the current leader for client lookup purposes. There is a corner case in Raft where a node may be elected leader of a pending (uncommitted) configuration. In such a case, if the leader of the pending configuration is not a member of the committed configuration, and it is the committed configuration that is being reported, then the leader_uuid field should be cleared by the process filling in the ConsensusStatePB object.
required config = 3
The peers. In some contexts, this will be the committed configuration, which will always have configuration.opid_index set. In other contexts, this may a "pending" configuration, which is active but in the process of being committed. In any case, initial peership is set on tablet start, so this field should always be present.
message
consensus.proto:196Status message received in the peer responses.
Used in: ,
required last_received = 1
The last message received (and replicated) by the peer.
optional last_received_current_leader = 4
The id of the last op that was replicated by the current leader. This doesn't necessarily mean that the term of this op equals the current term, since the current leader may be replicating ops from a prior term. Unset if none currently received. In the case where there is a log matching property error (PRECEDING_ENTRY_DIDNT_MATCH), this field is important and may still be set, since the leader queue uses this field in conjuction with last_received to decide on the next id to send to the follower.
optional int64 last_committed_idx = 2
The last committed index that is known to the peer.
optional last_applied = 5
The last operation applied by the peer.
optional error = 3
When the last request failed for some consensus related (internal) reason. In some cases the error will have a specific code that the caller will have to handle in certain ways.
The transaction driver type: indicates whether a transaction is being executed on a leader or a replica.
UNKNOWN_DRIVER = 0
LEADER = 1
REPLICA = 2
message
consensus.proto:120Required by HistoryCutoffOperation.
Used in:
optional fixed64 primary_cutoff_ht = 1
History cutoff of all the tables of this tablet. On the master, this field is used for the sys catalog table. For cotables i.e. ysql system tables, the cotables_cutoff_ht field is used. On tservers, this field is used everywhere i.e. for both colocated and non-colocated tables. In such cases, the cotables_cutoff_ht is invalid.
optional fixed64 cotables_cutoff_ht = 2
Set only on the master for cotables.
Used in: ,
HAS_LEASE = 0
OLD_LEADER_MAY_HAVE_LEASE = 1
NO_MAJORITY_REPLICATED_LEASE = 2
message
consensus.proto:177NoOp requests, mostly used in tests.
Used in:
optional bytes payload_for_tests = 1
Allows to set a dummy payload, for tests.
message
consensus.proto:183NoOp responses, mostly used in tests.
optional bytes payload_for_tests = 1
Allows to set a dummy payload, for tests.
Used in:
UNKNOWN_OPID_TYPE = 0
RECEIVED_OPID = 1
COMMITTED_OPID = 2
message
consensus.proto:397A message reflecting the status of an in-flight transaction.
required op_id = 1
required OperationType operation_type = 2
required int64 running_for_micros = 3
Time the transaction has been in flight.
required string description = 4
Quick human-readable description (e.g., ToString() output).
optional string trace_buffer = 6
If tracing is enabled when viewing the transaction, the trace buffer is copied here.
The types of operations that need a commit message, i.e. those that require at least one round of the consensus algorithm.
Used in: , ,
UNKNOWN_OP = 0
NO_OP = 1
WRITE_OP = 3
CHANGE_METADATA_OP = 4
CHANGE_CONFIG_OP = 5
UPDATE_TRANSACTION_OP = 6
SNAPSHOT_OP = 7
TRUNCATE_OP = 8
HISTORY_CUTOFF_OP = 9
SPLIT_OP = 10
CHANGE_AUTO_FLAGS_CONFIG_OP = 11
CLONE_OP = 12
Used in: ,
UNKNOWN_MEMBER_TYPE = 999
PRE_VOTER = 0
Any server added into a running consensus with the intention of becoming a VOTER, should be added as a PRE_VOTER. Such a server stays as a PRE_VOTER (for a specific tablet) till it is remote bootstrapped, after which it is promoted to VOTER. While in this mode, a server will not vote nor try to become a leader.
VOTER = 1
PRE_OBSERVER = 2
Any server added into a running consensus with the intention of becoming an async replica (OBSERVER), Such a server stays as a PRE_OBSERVER (for a specific tablet) till it is remote bootstrapped, after which it is promoted to OBSERVER. While in this mode, a server will not vote nor try to become a leader.
OBSERVER = 3
Async replication mode. An OBSERVER doesn't participate in any decisions regarding the consensus configuration. It only accepts update requests and allows read requests.
message
consensus.proto:188required id = 1
The id of the operation that failed in the other peer.
required status = 2
The Status explaining why the operation failed.
message
metadata.proto:94A set of peers, serving a single tablet.
Used in: , , , , ,
optional int64 opid_index = 1
The index of the operation which serialized this RaftConfigPB through consensus. It is set when the operation is consensus-committed (replicated to a majority of voters) and before the consensus metadata is updated. It is left undefined if the operation isn't committed.
optional bool unsafe_config_change = 4
Flag to allow unsafe config change operations.
repeated peers = 2
The set of peers in the configuration.
message
metadata.proto:67A peer in a configuration.
Used in: , ,
optional bytes permanent_uuid = 1
Permanent uuid is optional: RaftPeerPB/RaftConfigPB instances may be created before the permanent uuid is known (e.g., when manually specifying a configuration for Master/CatalogManager); permament uuid can be retrieved at a later time through RPC.
optional PeerMemberType member_type = 2
repeated last_known_private_addr = 3
repeated last_known_broadcast_addr = 4
optional cloud_info = 5
message
consensus.proto:135A Replicate message, sent to replicas by leader to indicate this operation must be stored in the write-ahead log.
The Raft operation ID (term and index) being replicated.
Used in: ,
required id = 1
required fixed64 hybrid_time = 2
The hybrid time assigned to this message.
optional int64 monotonic_counter = 9
A counter that is forever increasing in a tablet (like hybrid time). Used for list indexing.
required OperationType op_type = 4
optional write = 5
optional change_metadata_request = 6
optional transaction_state = 10
optional snapshot_request = 11
optional truncate = 12
optional split_request = 14
optional change_config_record = 7
optional history_cutoff = 13
optional auto_flags_config = 15
optional clone_tablet = 16
optional committed_op_id = 8
The Raft operation ID known to the leader to be committed at the time this message was sent. This is used during tablet bootstrap for RocksDB-backed tables.
optional noop_request = 999
message
consensus.proto:601Retryable requests.
Used in:
optional int64 first_id = 1
optional int64 last_id = 2
optional min_op_id = 3
optional uint64 min_time = 4
optional uint64 max_time = 5
message
consensus.proto:615optional last_op_id = 1
repeated client_requests = 2
optional fixed64 min_replay_txn_start_ht = 4