Proto commits in databendlabs/openraft

These 14 commits are when the Protocol Buffers files have changed:

2026-08-03

change: move `snapshot_id` out of `SnapshotMeta` # Summary A snapshot is identified by the position it covers, so the `snapshot_id`, which identified a transfer rather than the snapshot, leaves the metadata: state machines no longer invent an id when building a snapshot. The id now lives only in the chunked v1 protocol, the one place that needs it, and both serialized layouts — the stored metadata and the v1 request — remain exactly as in 0.9. # Details The v1 chunked receiver compares the id against the in-flight stream to tell a new transfer from a continuation; a v2 full-snapshot transfer completes in one call and never needs one. That makes the id part of the transfer protocol, not of the snapshot. Breaking: `SnapshotMeta` loses its `snapshot_id` field and now carries only `last_log_id` and `last_membership`. The v1 protocol keeps its 0.9 wire layout via a new `openraft_legacy::network_v1::SnapshotMeta`: the 0.9-shaped type with the id declared last, embedded in `InstallSnapshotRequest`, which therefore still serializes as the same five fields. `new(meta, id)` and `into_meta()` convert to and from the core type; a `From` impl is ruled out by the orphan rule. A positional format (`bincode`, `postcard`, `rmp-serde::to_vec`) encodes a struct as a bare sequence, so the field count is the compatibility contract; tests pin the exact JSON of the request and of both metadata types. The stored form of the core `SnapshotMeta` also keeps the 0.9 layout: it writes three fields with an always-empty `snapshot_id` and ignores the field when reading, so snapshots stored by 0.9 load in 0.10 and vice versa with no data migration. The v1 sender now generates a fresh id per transfer session instead of reusing one baked into the snapshot at build time, so a retransmitted snapshot is never mistaken for a continuation of an aborted attempt. The rocksstore example named snapshot files by `meta.snapshot_id`. Both writers — the snapshot builder and `install_snapshot()` — now derive the name from the snapshot position through one shared helper, because `get_current_snapshot()` resolves the latest snapshot by greatest file name, which requires every writer to share one scheme. The gRPC example drops the field from its `SnapshotRequestMeta` message and reserves the proto tag so it cannot be reused with a different meaning. tests-turmoil is adapted too; it is not a workspace member, so `cargo check --workspace` does not cover it. Upgrade tip: Construct `SnapshotMeta` from the two remaining fields: let meta = SnapshotMeta { last_log_id, last_membership, }; A `RaftNetworkV2` (full-snapshot) implementation needs no change. A chunked v1 implementation builds `InstallSnapshotRequest` as in 0.9, with the id inside the meta, importing `SnapshotMeta` from `openraft_legacy::network_v1`.

The documentation is generated from this commit.

2026-06-12

change: leadership-transfer vote request overrides leader lease # Summary A vote request sent by a leadership-transfer election now carries a `leadership_transfer` flag, and a voter grants it even when its leader lease has not expired. Previously the transfer relied on the `TransferLeaderRequest` broadcast disabling the lease on every voter before the target's vote request arrived; when the vote request overtook the broadcast, the vote was rejected and, with elections disabled, the cluster stayed leaderless. # Details - The race: `broadcast_transfer_leader()` sends `TransferLeaderRequest` to each voter as an independent task, and the target starts its election as soon as its own copy arrives. The election's vote request can reach another voter before that voter's lease-disabling broadcast does; the voter then rejects the vote by the leader-lease check. With `enable_elect: false` the candidate never retries, so the cluster stays leaderless permanently. Hit by the merge queue under `OPENRAFT_NETWORK_SEND_DELAY=30`: https://github.com/databendlabs/openraft/actions/runs/27390569655 - The fix is the special flag from the Raft dissertation, section 4.2.3: `VoteRequest.leadership_transfer` claims the election is authorized by the current Leader, and the voter exempts such a request from the lease check. The flag travels with the vote request itself, so it is immune to ordering between separate RPC paths. Disabling the lease on receiving `TransferLeaderRequest` is kept as a fallback. - Public API: the new `pub` field on `VoteRequest` is a source-breaking change for code that constructs it with a struct literal or matches it exhaustively; `VoteRequest::new()` is unchanged. On the wire the field deserializes to `false` when absent (`serde(default)`), so requests from older peers behave as before. The `raft-kv-memstore-grpc` example protobuf carries the new field. Upgrade tip: Construct `VoteRequest` with `VoteRequest::new()`, which sets the new field to `false`, or add the field to struct literals: VoteRequest { vote, last_log_id, leadership_transfer: false, } If the network layer maps `VoteRequest` to a custom wire format (e.g. protobuf) instead of serializing the whole struct, add the field to that format and forward it. Dropping it silently disables the lease override, and a transfer-leader election can again be rejected by a voter whose leader lease has not expired.

2025-12-15

feat: add pipeline mode for streaming replication Implement pipeline replication where the leader continuously streams log entries to followers after finding the match point, replacing the request-response pattern with efficient pipelined I/O. ## Replication Phases Replication to a follower has two phases: 1. **Binary search phase**: The leader runs a binary search to find the exact matching position of log entries on the follower. 2. **Pipeline mode**: After finding the match point, the leader calls the `stream_append` method on the network and continuously generates AppendEntries requests. The network implementation should pipeline all requests to the follower and yield responses. Note that responses and requests don't have to be 1-to-1 mapped - the number of responses can be smaller than the number of requests. `stream_append` provides a default implementation that calls the existing `append_entries` method to emulate streaming replication. A mature implementation should run in real pipeline mode instead of request-response manner. When a request is received by `stream_append`, it is responsible for sending all content of the request to the follower - partial success is not allowed. ## I/O Progress Synchronization Add watch channels for replication tasks to synchronize with leader I/O progress: - `io_accepted_tx`: Notifies observers before I/O operations are submitted to storage, enabling preparation for upcoming I/O events - `io_submitted_tx`: Notifies replication tasks when log entries have been submitted to storage and are safe to read The replication stream monitors these channels to detect leader changes and wait for log availability without polling. ## Replication Stream Reuse When rebuilding replication streams after a membership change, reuse existing streams instead of destroying all and recreating. This avoids unnecessary stream teardown and maintains in-flight replication state. - Add `close_old_streams: bool` field to `RebuildReplicationStreams` command - `become_following()` emits `RebuildReplicationStreams { targets: vec![], close_old_streams: true }` - Membership changes use `close_old_streams: false` to preserve existing streams - Properly join and cleanup only removed replication streams ## Data Structure Changes - Add `Payload` enum for log replication specifications (`LogIdRange`, `LogsSince`) - Add `LogsSince` variant to `Inflight` for unbounded log streaming - Add `Replicate` struct combining `inflight_id` with `Payload` - Add `ReplicationProgress` to track local committed and remote matched state - Add `is_logs_since()` method to `Inflight` for type checking - Add `get_partial_success()` method to `AppendEntriesResponse` - Remove obsolete `request.rs`, `replication_state.rs`, `log_state.rs` ## API Changes - `RaftNetworkV2::stream_append` now accepts `'static` stream lifetime - Use `stream_append` for heartbeat instead of `append_entries` - Use `stream_append` for linearizable read confirmation ## gRPC Example Updates - Add `StreamAppend` RPC with bidirectional streaming to proto - Implement `stream_append` server handler and client - Remove chunked `append_entries` fallback logic - Add `From<StreamAppendResult> for pb::AppendEntriesResponse` conversion ## Documentation - Update getting-started guide to use `RaftNetworkV2` as primary trait - Add documentation for optional `stream_append()` method for pipelined replication ## Other Changes - Remove unused `_committed_rx` field from `RaftCore` - Remove stale TODO comments and commented-out code - Remove unused inflight-id - Consolidate I/O progress broadcast after `initiate_replication()`

2025-02-04

Doc: Finish gRPC example raft-kv-memstore-grpc A rust test `tests/test_cluster.rs` brings up a 3 nodes cluster and executes write and read on it. - Fix: #1287

2025-02-03

Refactor: example raft-kv-memstore-grpc Move protobuf types to 3 files: - `app.proto` defines application API and types. - `raft.proto` defines Raft-protocol API and types. - `management.proto` defines non-app management API and types, such as membership config API. - Part of #1287

Change: Refine Log Entry Traits This commit refines the `RaftEntry` and related traits, to better support application defined log `Entry` type. Key Changes: 1. Remove `FromAppData` trait: - The `FromAppData` trait, which was used to create log `Entry` from application data, is removed. - Applications should now implement the new `RaftEntry::new()` method to create log entries directly. 2. Remove `RaftLogId` trait, it becomes an internal trait. 3. Update `RaftEntry` trait: - The `RaftEntry` trait no longer requires `RaftLogId` (due to its redefinition) and now mandates the implementation of: - `new()`: For creating a log `Entry`. - `log_id_parts()`: To return references to the log ID's committed leader ID(term) and index. - `set_log_id()`: To update the log entry's ID. - Default methods are provided: - `new_blank()`, `new_normal()`, `new_membership()` for creating different types of log entries. - `log_id()` returns an owned `LogId` instance. - `index()` returns the index of the log entry. 4. Introduce `RefLogId`: - `RefLogId` is a reference-based representation of a log ID, complementing the existing `LogIdOf<C>` (a storage-based implementation). - `RefLogId` adds system-defined properties (e.g., `Ord` implementation) while referencing an existing `LogIdOf<C>`. - Internal components now use `RefLogId` where possible, improving flexibility and consistency. 5. Update example `raft-kv-memstore-grpc`: - Updated to implement log `Entry` and related types using protobuf, including state machine and RPC message types. - Added snapshot streaming transmission implementation. - Removed `serde` dependency from the example. - Part of #1278. --- Upgrade tips: 1. For Applications with Custom `RaftEntry` Implementations: If you've declared a custom `RaftEntry` (e.g., `declare_raft_types!(MyTypes: Entry = MyEntry)`): - Remove the implementation of `FromAppData`. - Implement the following new methods: - `new()` - `log_id_parts()` - `set_log_id()` 2. For Applications Using OpenRaft's Default `Entry`: - No changes are required.

2025-01-03

Refactor: update gRPC example to adapt to `LeaderId` changes - Update the `raft-kv-memstore-grpc` example to use the protobuf-defined `LeaderId`. - Automatically implement `CommittedLeaderId` for all types. - Add `openraft::vote::LeaderIdCompare` to provide comparison functions for both single-leader-per-term and multi-leader-per-term implementations.

2024-12-14

Refactor: Add gRPC network kv-memstore example (#1274) * CI: add raft-kv-memstore-grpc in ci * CI: add install protoc in steps examples

2019-07-14

Very large scale refactor based on experience so far. Very large overhaul to streamline API and interfaces. Removed protobuf in favor of standard Rust types with serde. Protobuf and any other serialization types can be added as extensions or very easily implemented by users of the crate. Serde allows for broad serialization options as well. Finally found a solid pattern for being able to remove the need for allocating application specific errors from the RaftStorage layer. These are now expressed as generic constraints. Using PhantomData for the various types to keep track of the type parameter where it is otherwise not being used. All of the logic of the Raft system has been cleanly separated into submodules for more easier reasoning about the system. Previous implementation of the system overall algorithm has been reinstated.

This commit does not contain any .proto files.

2019-06-23

Finished up the election timeout system. README updates on AdminCommand structure for controlling the Raft node from outside of the Raft protocol itself. A few new node states have been introduced for more clearly handling the Raft node startup lifecycle. This provides more visibility into the state of the system for the app which is using this system. A bit of refactoring to remove unnecessary copy ops.

2019-06-19

AppendEntries impl is complete. The storage interface for dealing with snapshots is pretty much completely spec'ed out.

2019-06-13

Raft API has been a little further pinned down. Raft API has been segmented into 4 layers. All in terms of message handling. Still need to pin down the storage interface.

Good progress being made.

2019-06-12

Initial commit.