These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)
| Commit: | d1d4bdd | |
|---|---|---|
| Author: | Mathieu Baudet | |
Carry the owner authorization in block proposals, replacing the OriginalProposal enum
| Commit: | bf0812b | |
|---|---|---|
| Author: | Mathieu Baudet | |
Merge remote-tracking branch 'origin/ma2bd/456-retain-owner-signature' into ma2bd/decouple-proposer-from-owner # Conflicts: # linera-chain/src/data_types/mod.rs # linera-chain/src/manager.rs # linera-core/src/client/chain_client/mod.rs # linera-core/src/client/mod.rs
| Commit: | 0957b38 | |
|---|---|---|
| Author: | Mathieu Baudet | |
Merge remote-tracking branch 'origin/main' into ma2bd/456-retain-owner-signature # Conflicts: # linera-chain/src/certificate/confirmed.rs # linera-chain/src/certificate/generic.rs # linera-chain/src/certificate/lite.rs # linera-chain/src/certificate/validated.rs # linera-core/src/client/mod.rs # linera-core/src/unit_tests/worker_tests.rs # linera-rpc/proto/rpc.proto # linera-rpc/src/grpc/conversions.rs # linera-rpc/tests/snapshots/format__format.yaml.snap # linera-sdk/src/test/block.rs
| Commit: | fe5aa27 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Make safety faults attributable based on conflicting certificates. (#6572) ## Motivation Only including the final `ConfirmedBlock` votes (_C-votes_) isn't enough in general: A validator that signed two conflicting blocks in two _different_ rounds isn't necessarily violating the rules of the protocol! ## Proposal `ValidatedBlock` votes (_V-votes_) in round s now include an optional round number l < s that means: * I am voting to validate this block in round s. * If l is specified: It got a quorum of V-votes in round l and I have not signed a C-vote for a different block in any round r in between (l ≤ r < s). * If there is no l (`None`): I have not signed any C-vote for a different block in any r < s. A `ConfirmedBlock` certificate now contains not only a quorum of C-votes in some round s, but also a quorum of V-votes in the same round. If those V-votes have a round l, it also contains a quorum of V-votes from l, etc., until it reaches a quorum without l. That way, each pair of conflicting `ConfirmedBlock` certificates will contain evidence that a third of the validators violated either of two rules: * They signed two conflicting C-votes in the same round. * They signed a V-vote with l in round s, but did sign a conflicting C-vote in a round l < r < s. As an optimization for the happy path (confirmation in the first round), the C-vote payload now contains a `first_round` flag, which is set simply if these C-votes are in the first round. (Which one the first round is depends on the chain's ownership config.) That way, in the first round we can always omit the V-vote quorum. However, a new type of equivocation is: * A validator signed a C-vote with `first_flag` in round s, but _also_ a C-vote in an earlier round. ## Test Plan Tests were added and updated. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - This is preliminary work for #6237 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 9b86e95 | |
|---|---|---|
| Author: | Mathieu Baudet | |
Allow any round-eligible owner to propose a block authorized by its authenticated owner
| Commit: | 7f1ba9e | |
|---|---|---|
| Author: | Mathieu Baudet | |
Make certificate comments precise about the owner authorization semantics
| Commit: | 395f471 | |
|---|---|---|
| Author: | Mathieu Baudet | |
Remove issue references from code comments
| Commit: | 9e6d2e2 | |
|---|---|---|
| Author: | Mathieu Baudet | |
| Committer: | Mathieu Baudet | |
Retain the block owner's proposal signature alongside certificates and require it for blocks with an authenticated owner (#456)
| Commit: | de5bacf | |
|---|---|---|
| Author: | Mathieu Baudet | |
| Committer: | GitHub | |
Batch validator catch-up via an aggregated MissingCrossChainUpdates error (#6556) ## Motivation A client that owns a busy hub chain (one receiving cross-chain messages from many sender chains) periodically froze for minutes and tripped the chain-idle liveness probe, restarting the process. (Likely) Root cause: when a validator is behind on the sender chains a block consumes, it rejects the proposal **one missing sender at a time** (`MissingCrossChainUpdate`), and the client brings it up to date with **one blocking round-trip per sender**. Accepting a single high-fan-out block therefore takes ~N sequential `propose → sync-one-sender → retry` round-trips per validator, and because per-chain block production is a single task gated on a quorum, this serializes into multi-minute stalls (observed `process_inbox` p99 ≈ 8.7 min during a freeze). An operational mitigation (`maxPendingMessageBundles=1`) is already deployed; this is the protocol-level fix. ## Proposal Add an aggregated error, `MissingCrossChainUpdates { chain_id, bundles }` — the batched form of `MissingCrossChainUpdate` — that lets a validator declare **every** missing incoming bundle it needs to validate a block proposal in a single response, and have the client fetch them all in one batch before retrying. Concretely: - When validating a proposal, the validator now **collects all** missing cross-chain bundles instead of bailing on the first, and returns them together as `(origin, height)` pairs. - The client handles `MissingCrossChainUpdates` on the block-proposal path, syncing every reported origin chain in a single batch (via `send_chain_info_up_to_heights`) and retrying once. The local node's "download missing sender blocks from a validator" path is batched the same way: all reported senders are fetched, then the proposal is retried once. This deliberately targets **proposals only**. Confirmed certificates don't have the same pathology: missing incoming bundles are tolerated (`must_be_present=false`), missing blobs are already discovered up front from `required_blob_ids` and returned as a single `BlobsNotFound`, and the only relevant missing events (the committee/epoch stream) are already batched into one `EventsNotFound`. So there is no O(N)-round-trip problem to fix there, and the confirmed-certificate path is left on its existing batched errors. Because clients and validators are upgraded independently, the change is **backward compatible**. The wire error is appended without disturbing existing ones, and a new `supports_aggregated_missing` field on the proposal / confirmed-certificate requests lets a client advertise that it understands the aggregated error. The validator adapts its reply accordingly at the gRPC boundary: capable clients receive every missing sender as `MissingCrossChainUpdates`; older clients are downgraded to the legacy per-sender `MissingCrossChainUpdate` (the first missing sender) and recover one round-trip at a time, exactly as before. All four old/new client/validator combinations interoperate. A separate follow-up will retire the legacy one-at-a-time path once every deployed node understands the aggregated error. ## Test Plan - A worker-level test asserts that a proposal consuming several not-yet-received bundles is rejected with a single `MissingCrossChainUpdates` listing **all** of them (rather than bailing on the first). - A client-level integration test (`test_proposal_batches_missing_dependency_catch_up`) drives the full path end to end: a block consuming bundles from three sender chains is proposed to a validator that is behind on **all** of them; the validator reports one aggregated error, the client catches every sender up in a single batch, and the block is confirmed. The test also confirms the loop terminates. - Unit tests cover the gRPC error adaptation in both directions: capable clients get `MissingCrossChainUpdates` unchanged; older clients get it downgraded to the legacy per-sender `MissingCrossChainUpdate`. - Unit tests cover the wire round-trip of the new error (bincode-in-protobuf) and the chain-to-node error mapping. - `cargo clippy --all-targets` (incl. `--features server`), `cargo +nightly fmt`, and the affected crates' test suites pass. ## Release Plan Targets `testnet_conway` for now. The new error variant is additive and inert until validators are upgraded; the capability handshake keeps the rollout safe for any mix of upgraded and non-upgraded clients and validators. Changing the wire error set requires a coordinated validator/client deployment.
| Commit: | 15c5496 | |
|---|---|---|
| Author: | Mathieu Baudet | |
Trim the aggregated error to bundles only and rename it to MissingCrossChainUpdates.
| Commit: | a790b06 | |
|---|---|---|
| Author: | Mathieu Baudet | |
Negotiate aggregated-error support over gRPC: validators downgrade MissingDependencies to legacy errors for clients that do not advertise support, and lift legacy missing-event/blob errors into the aggregated shape for those that do.
| Commit: | 28d32e5 | |
|---|---|---|
| Author: | Andre da Silva | |
| Committer: | Andre da Silva | |
Backport EventBlockHeights RPC and client index path to testnet_conway
| Commit: | 9ca9695 | |
|---|---|---|
| Author: | Andre da Silva | |
| Committer: | GitHub | |
[backport] Add streaming DownloadBlobs RPC endpoint (#5976) (#6156) ## Motivation Backport of #5976 to `testnet_conway`. Adds the streaming `DownloadBlobs` gRPC and simple-transport endpoints that yield blobs as a stream (one `BlobContent` per requested ID, in order). Callers are not rewired in this PR; they continue to use the parallel unary `download_blob` path. The endpoint must land and be deployed on the validators first, so the follow-up client-side rewire can be tested against validators that actually serve it. ## Proposal Adds the new RPC method to: - `linera-rpc/proto/rpc.proto` (gRPC method definition) - `linera-rpc/src/grpc/client.rs` and `linera-service/src/proxy/grpc.rs` (gRPC client + server) - `linera-rpc/src/simple/client.rs`, `simple/server.rs`, `simple/transport.rs`, `message.rs`, `client.rs` (simple transport: dedicated connection per streaming request) - `linera-core/src/node.rs` (`ValidatorNode` trait method) - `linera-core/src/remote_node.rs` (`RemoteNode::download_blobs`) - `linera-storage/src/db_storage.rs` (server-side blob fetching helper) - `linera-rpc/tests/snapshots/format__format.yaml.snap` (wire-format snapshot adds the new variant) Per the variant-index constraint of the simple transport, the new `DownloadBlobs` variant of `RpcMessage` is placed at the end of the enum on `testnet_conway` instead of in its main-branch position (between `DownloadBlob` and `DownloadPendingBlob`). This preserves the wire-format indices of all existing variants. A comment on the variant documents this constraint for future backporters. ## Test Plan CI. ## Release Plan - Nothing to do / These changes follow the usual release cycle.
| Commit: | b3985c8 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Checkpoints MVP (#6275) ## Motivation Long chains take too long to synchronize. We need checkpoints so that new clients and validators don't need the full chain. ## Proposal As a first step, implement an MVP: Checkpointing, but only for chains with no messages or events, whose execution state fits into a single blob. ## Test Plan Tests were added; in particular a `client_tests` case where a new client synchronizes using a checkpoint, and doesn't download the block before the checkpoint. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Part of #460 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
| Commit: | b835ec6 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Remove the per-chain committee map. (#6236) ## Motivation Revoking an epoch means the committee is not trusted anymore. This should not be delayed by each chain having to acknowledge the revocation. ## Proposal Remove the per-chain committee map; revocation takes effect immediately. ## Test Plan CI; tests have been updated where necessary. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - First step of https://github.com/linera-io/linera-protocol/issues/6237. - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
| Commit: | 05a8013 | |
|---|---|---|
| Author: | Andre da Silva | |
| Committer: | GitHub | |
Add streaming DownloadBlobs RPC endpoint (#5976) ## Motivation During chain sync, each blob referenced by a batch of certificates is downloaded via a separate `DownloadBlob(BlobId) -> BlobContent` unary RPC call. Batches with hundreds of new blobs fire hundreds of sequential round trips before the certificates can be processed, which becomes the dominant bottleneck: measured sync logs show ~63% of wall-clock time is spent waiting for blob downloads to complete. A batch, streaming RPC lets the validator serve multiple blobs on a single connection and lets the caller process blobs as they arrive instead of waiting for the full batch. ## Proposal Add a new streaming RPC endpoint on validators. No caller code is switched over in this PR — the existing singular `DownloadBlob` is kept for backwards compatibility, so validators can be deployed before clients start using the new endpoint. - Proto: `rpc DownloadBlobs(BlobIds) returns (stream BlobContent)` - `ValidatorNode` trait: `async fn download_blobs(&self, blob_ids: Vec<BlobId>) -> Result<BlobStream, NodeError>` where `BlobStream = BoxStream<'static, Result<BlobContent, NodeError>>` - gRPC transport: tonic native server streaming - Simple transport: one dedicated connection per request, server sends one `RpcMessage::DownloadBlobResponse` per blob, client filters them into a stream (same pattern as `subscribe`) - Server-side `DbStorage::read_blobs` (and `read_blob_states`) are changed from a sequential loop into `futures::future::try_join_all` of per-blob reads. Each blob lives under its own `root_key` (its own ScyllaDB partition), so cross-partition reads can't be coalesced into a single `IN` query — the ScyllaDB-recommended approach is concurrent per-partition queries via the shard-aware driver, which spreads the load across all shards and nodes instead of funnelling through a single coordinator. RocksDB also benefits: tokio can overlap the per-blob cache/SST lookups. Callers will migrate to the new RPC in a follow-up PR (with retry-on-mid-am-error logic that replays only the remaining blob ids on another validator). ## Test Plan CI. The new RPC is additive and no existing caller code is changed, so existing tests exercise the unchanged `DownloadBlob` path. The new endpoint will be validated end-to-end in the follow-up PR that switches callers over. ## Release Plan - These changes should be backported to the latest `testnet` branch
| Commit: | 98e9f20 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Extract linera-exporter from linera-service (#6041) ## Motivation Backport the `linera-exporter` extraction from `linera-service` (originally #5946 on `testnet_conway`) to `main`. ## Proposal Cherry-pick of `8733dc9bcb` (Extract linera-exporter from linera-service crate) with conflict resolution to adapt to main's diverged APIs: - **New crates**: `linera-exporter` and `linera-storage-runtime` added to the workspace - **`linera-service/src/storage.rs`**: Replaced full implementation with re-exports from `linera-storage-runtime` - **`linera-storage-runtime`**: Adapted to main's type names (`StorageCacheConfig` instead of `StorageCacheSizes`, `certificate_cache_size` instead of `lite_certificate_cache_size`, added`cache_cleanup_interval_secs`) - **`linera-exporter`**: Fixed API mismatches against main (sync vs async methods, `Committee::new` returning `Result`, `ExportersTracker::new` arity, `Storage` trait bound fixes, tracing init) - **Cargo.toml files**: Only structural additions (new workspace members/deps), no version downgrades from testnet_conway ## Test Plan - `cargo clippy --locked --all-targets --all-features` passes (matching pre-push hook) - No conflict markers remain - No version downgrades from testnet_conway (`0.15.15`) in any Cargo.toml ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Original PR: #5946
| Commit: | de31612 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Don't fetch blob when retrying x-chain requests for sender chains (#5998) (#5999) Port of #5998. ## Motivation `LocalNodeClient::retry_pending_cross_chain_requests` went through `handle_chain_info_query`, which unconditionally calls `initialize_and_save_if_needed`. For a sender chain whose `ChainDescription` blob was never needed locally (e.g. only non-height-0 blocks preprocessed), the init step would fail with `BlobsNotFound` and the caller would download the `ChainDescription` from validators — once per sender per sync. ## Proposal Add a dedicated worker entry point (`cross_chain_network_actions`) that computes pending cross-chain requests from the outbox without touching execution state, and route the retry through it. ## Test Plan A test was extended to assert that a sparse sender chain's description wasn't downloaded. ## Release Plan - Backport, but without the change to the format. ## Links - `testnet_conway` version: #5998. - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| Commit: | 8733dc9 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Extract linera-exporter from linera-service crate. (#5946) ## Motivation The `linera-exporter` binary lives inside the `linera-service` crate, pulling in all of its dependencies (wasmer, wasmtime, etc.) despite needing only a fraction. This inflates build times and Docker image sizes unnecessarily. The exporter is a distinct infrastructure component (~5600 lines, own gRPC proto, metrics, persistent state, multiple export backends) that deserves its own crate for cleaner builds, code organization, and ownership boundaries. It also creates problems when we want to add a dependency on `linera-bridge` to the exporter (the proof generator part) as then we get into another circular dependency problem. ## Proposal Extract the block exporter into two new crates: - **`linera-storage-runtime`** — Shared storage configuration and runtime infrastructure (`CommonStorageOptions`, `StorageConfig`, `Runnable` trait, `StorageMigration`, etc.) previously in `linera-service/src/storage.rs`. This layer sits between `linera-storage` (core traits) and service binaries that need to parse configs and instantiate storage backends (used in `linera-service`, `linera-exporter` and `linera-bridge`). - **`linera-exporter`** — Standalone block exporter crate with its own `[[bin]]` target. Contains all exporter source code, config types (`BlockExporterConfig`, `DestinationConfig`, etc.), gRPC proto, and metrics. Has zero dependency on `linera-service`. `linera-service` is slimmed down: its `src/exporter/` directory and `[[bin]] linera-exporter` are removed. It re-exports the moved types from the new crates for backward compatibility. The e2e exporter test is relocated to `linera-service/tests/exporter_tests.rs` since it depends on the `cli_wrappers` test infrastructure. Also fixes a pre-existing `E0275` overflow in `linera-bridge` by adding `recursion_limit = "256"`, and fixes a Docker build path conflict where the `linera-exporter/` crate directory shadowed the binary during `mv`. Thanks to this work, we can also bring back the EVM exporter part https://github.com/linera-io/linera-protocol/pull/5946/changes/ddab01725e2679608a6883419b16dc7cea481e89 And re-enable `linera-bridge` e2e tests (https://github.com/linera-io/linera-protocol/pull/5946/commits/a05b1c1711c7cf3f145ae1a2b9f2fe91c1bc2103) ## Test Plan - `cargo build --workspace --exclude linera-web` — full workspace build passes - `cargo clippy --all-targets --all-features -- -D warnings` — clean - `cargo test -p linera-storage-runtime` — 1 test passing - `cargo test -p linera-exporter` — 15 tests passing - `cargo test -p linera-service --lib` — 10 tests passing - `cargo build -p linera-exporter --bin linera-exporter` — standalone binary builds - Docker: `docker build -f docker/Dockerfile.exporter -t linera-exporter .` — builds successfully - Docker: `docker build -f docker/Dockerfile.bridge -t linera-bridge .` — builds successfully (with recursion_limit fix) ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 221680e | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Add opt-in recovery from inbox gaps and execution outcome mismatch (#5907) Port of https://github.com/linera-io/linera-protocol/pull/5876, https://github.com/linera-io/linera-protocol/pull/5881, https://github.com/linera-io/linera-protocol/pull/5882, https://github.com/linera-io/linera-protocol/pull/5886, https://github.com/linera-io/linera-protocol/pull/5896, https://github.com/linera-io/linera-protocol/pull/5890 and https://github.com/linera-io/linera-protocol/pull/5900. ## Motivation On the testnet, we added mechanisms to recover from failures to persist inbox updates and from `IncorrectOutcome` errors. These should not be reachable in theory, other than due to database corruptions, but it might still be useful to keep these recovery mechanisms. ## Proposal Add `RevertConfirm` cross-chain request to recover from state inconsistencies where a recipient chain lost persisted inbox state after a confirmation was sent. When enabled via `--allow-revert-confirm`, the recipient detects inbox gaps and requests the sender to re-add outbox entries and resend bundles. Also add `--reset-on-incorrect-outcome-mins` to reset and re-execute a chain's entire block history when an `IncorrectOutcome` error is detected (guarded by a minimum cooldown to prevent loops). ## Test Plan Some tests were added. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Testnet PRs: https://github.com/linera-io/linera-protocol/pull/5876, https://github.com/linera-io/linera-protocol/pull/5881, https://github.com/linera-io/linera-protocol/pull/5882, https://github.com/linera-io/linera-protocol/pull/5886, https://github.com/linera-io/linera-protocol/pull/5896, https://github.com/linera-io/linera-protocol/pull/5890, https://github.com/linera-io/linera-protocol/pull/5900 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| Commit: | 76a8a11 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
More complete `RevertConfirm` requests. (#5900) ## Motivation In some cases we send `RevertConfirm` with the lowest height that we _know_ we are missing. We should instead request the lowest height we _don't_ know we are _not_ missing, to make it more likely that we really get _all_ events. ## Proposal Rename the field and request the next expected height. ## Test Plan CI ## Release Plan - Hotfix validators - Port to `main`. ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| Commit: | af6267d | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
[testnet] Add opt-in recovery from inbox gaps and exec outcome mismatch. (#5876) ## Motivation If a chain confirms the receipt of a cross-chain message and the sender chain updates its outbox accordingly but the recipient chain's update of its inbox somehow fails to be persisted, the recipient chain may never recover because it rejects later cross-chain updates. ## Proposal Add an `--allow-revert-confirm` config option to the server. If this is enabled, the recipient chain will request that the sender chain resets its outbox and retries the messages. In addition, add a `--reset-on-incorrect-outcome` option: Servers that encounter an incorrect block execution outcome for a _confirmed_ block (should only be possible due to DB corruption in theory) will re-execute the chain in question. ## Test Plan Tests were added. ## Release Plan - These changes should be ported to `main`, and - be released in a validator hotfix. - Affected validators should be restarted with the `--allow-revert-confirm` option. ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| Commit: | 9cc4fd8 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Make `read_event` work even if the event is not locally known yet. (#5710) Port/rewrite of #5638 and #5656. ## Motivation We have a use case for reading events without having subscribed and seen `UpdateStreams` yet. ## Proposal Add an index of all event-publishing blocks. Make the client download the publishing block whenever it is missing an event, then retry. ## Test Plan A test was added. Instead of writing a new example just for this, `UpdateStream` is used manually, without the client having the streams yet. ## Release Plan - Nothing to do. ## Links - Testnet version: #5638 - Follow-up with cleanups: #5656 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| Commit: | d8cd1ce | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Sparse-sync publisher chains on subscription (#5737) Port of #5653, and adds chain info query field. ## Motivation When subscribing to a notification stream, chains should immediately get notified about the most recent events, even if they happened before the subscription. ## Proposal Add a `request_previous_event_blocks` field to `ChainInfoQuery`. Use it to do a sparse sync for publisher chains on subscription. ## Test Plan Tests were extended. The social end-to-end test became flaky and had to be fixed by cherry-picking https://github.com/linera-io/linera-protocol/pull/5768 and applying the same logic in one more place. ## Release Plan - Nothing to do. ## Links - Fixes https://github.com/linera-io/linera-protocol/issues/5651 - Testnet version: #5653 - Endpoint on testnet (replaced by `ChainInfoQuery` here): https://github.com/linera-io/linera-protocol/pull/5659 - Test fix on the testnet: https://github.com/linera-io/linera-protocol/pull/5768 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Mathieu Baudet <1105398+ma2bd@users.noreply.github.com>
| Commit: | 73b80a9 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
[testnet] Add `previous_event_blocks` validator endpoint (#5659) ## Motivation https://github.com/linera-io/linera-protocol/pull/5653 introduces and uses a new endpoint, so it fails the compatibility tests. ## Proposal Add the new endpoint first, in this separate PR. ## Test Plan See #5653 for tests that use this endpoint. ## Release Plan - Hotfix the validators. - Then merge #5653. ## Links - Part of #5653 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
| Commit: | 3df357b | |
|---|---|---|
| Author: | Andre da Silva | |
| Committer: | Mathieu Baudet | |
Send notifications in batches (#5054) Sending notifications one at a time creates high RPC overhead. Batching notifications reduces overhead and improves throughput. - Add `NotifyBatch` RPC method for sending multiple notifications at once - Implement `BatchForwarder` for collecting and sending batched notifications - Configure batch sizes and timeouts - This _does not_ batch when sending from proxy -> client - Deploy and verify notifications are batched - Before, even when making the notification size grow with the network size, and using a very aggressive multiplier, I would still eventually start to get back-pressured, and would see it in the logs. After this change, that did not happen anymore, even at less aggressive multipliers. - Nothing to do / These changes follow the usual release cycle.
| Commit: | ef5d7f3 | |
|---|---|---|
| Author: | Andre da Silva | |
| Committer: | GitHub | |
Send notifications in batches (#5054) ## Motivation Sending notifications one at a time creates high RPC overhead. Batching notifications reduces overhead and improves throughput. ## Proposal - Add `NotifyBatch` RPC method for sending multiple notifications at once - Implement `BatchForwarder` for collecting and sending batched notifications - Configure batch sizes and timeouts - This _does not_ batch when sending from proxy -> client ## Test Plan - Deploy and verify notifications are batched - Before, even when making the notification size grow with the network size, and using a very aggressive multiplier, I would still eventually start to get back-pressured, and would see it in the logs. After this change, that did not happen anymore, even at less aggressive multipliers. ## Release Plan - Nothing to do / These changes follow the usual release cycle.
| Commit: | 35d2440 | |
|---|---|---|
| Author: | Andre da Silva | |
| Committer: | Andre da Silva | |
Move download_sender_certificates_for_receiver to proxy
| Commit: | 3876afe | |
|---|---|---|
| Author: | Andre da Silva | |
| Committer: | Andre da Silva | |
Send notifications in batches
| Commit: | 0646f41 | |
|---|---|---|
| Author: | Andre da Silva | |
| Committer: | Andre da Silva | |
Move download_sender_certificates_for_receiver to proxy
| Commit: | 6e01efd | |
|---|---|---|
| Author: | Andre da Silva | |
| Committer: | Andre da Silva | |
Send notifications in batches
| Commit: | 09c4b9c | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Bump tonic dependencies (#5148) ## Motivation We are debugging exporter's issues with sending blocks to indexer destination but tonic errors don't contain any details about the root cause. ## Proposal Latest releases of tonic are supposed to contain more error details. Bump tonic versions to 0.14.2 Note this is already done on `main`, it just wan't backported as it didn't seem necessary at the time. ## Test Plan CI. ## Release Plan - These changes should - be released in a validator hotfix. ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 2074493 | |
|---|---|---|
| Author: | Andre da Silva | |
| Committer: | Andre da Silva | |
Move download_sender_certificates_for_receiver to proxy
| Commit: | 742eb97 | |
|---|---|---|
| Author: | Andre da Silva | |
| Committer: | Andre da Silva | |
Send notifications in batches
| Commit: | 4ee4f79 | |
|---|---|---|
| Author: | Mathieu Dutour Sikiric | |
| Committer: | GitHub | |
Implement the access to the shard_id (#4781) ## Motivation For debugging purposes, it is useful to access the shard index of the validators for a fixed chain. ## Proposal The implementation is straightforward: * Add it as an endpoint to the proxy. * Add it as a feature to the "linera" client. ## Test Plan No test has been added to the CI. However, the function has been tested locally with the following steps. A: Starting the validators ```bash ./target/debug/linera storage delete-all --storage service:tcp:localhost:1235:table_a rm -rf /tmp/WORK && mkdir -p /tmp/WORK export RUST_BACKTRACE=full ./target/debug/linera net up --policy-config testnet --storage service:tcp:localhost:1235:table_a --validators 4 --shards 4 --path /tmp/WORK ``` B: Running the faucet ```bash export LINERA_WALLET="/tmp/WORK/wallet_0.json" export LINERA_KEYSTORE="/tmp/WORK/keystore_0.json" export LINERA_STORAGE="rocksdb:/tmp/WORK/client_0.db" ./target/debug/linera faucet --amount 1000 --port 8079 --storage-path /tmp/WORK/faucet_storage.sqlite ``` C: Creating a chain, showing ```bash rm -rf /tmp/WORK_B && mkdir -p /tmp/WORK_B export LINERA_WALLET="/tmp/WORK_B/wallet_0.json" export LINERA_KEYSTORE="/tmp/WORK_B/keystore_0.json" export LINERA_STORAGE="rocksdb:/tmp/WORK_B/client_0.db" export FAUCET_URL=http://localhost:8079 ./target/debug/linera wallet init --faucet $FAUCET_URL ./target/debug/linera wallet request-chain --faucet $FAUCET_URL ./target/debug/linera wallet show ``` D: Showing up the result from the obtained `ChainId` ```bash ./target/debug/linera query-shard-info 42a1c60a969b9c6ce893fd3746ec6084e0c336820f1336fa40e39e4aa5708c5b Querying validators for shard information about chain 42a1c60a969b9c6ce893fd3746ec6084e0c336820f1336fa40e39e4aa5708c5b. Chain ID: 42a1c60a969b9c6ce893fd3746ec6084e0c336820f1336fa40e39e4aa5708c5b Validator Shard Information: Validator: 033f3e155e7c0c7fae94692df5501eeab55e98a8c086e568cb672d7a8270e7c08f Address: grpc:localhost:13004 Total Shards: 4 Shard ID for chain: 0 Validator: 02b7742453a60ad3e2d615f59aa6b00d72d0b126ea9181264f3bd459eb1e42b557 Address: grpc:localhost:13003 Total Shards: 4 Shard ID for chain: 3 Validator: 03b99ffb3804db5fa05534a5eb0bc890add528acb66b39931bca878ee2b68f72a7 Address: grpc:localhost:13001 Total Shards: 4 Shard ID for chain: 2 Validator: 02e33d49cc2279adfdfea6cafa523d739f95969ebddd13a96720f0dffafb2dceb2 Address: grpc:localhost:13002 Total Shards: 4 Shard ID for chain: 1 ``` ## Release Plan If this is feasible, then we should backport it to the TestNet Conway branch. ## Links None.
| Commit: | 3f7d7fb | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
[testnet] Add `BlobLastUsedByCertificate` endpoint to the validator proxy. (#4526) ## Motivation This is a Conway backport of https://github.com/linera-io/linera-protocol/pull/4420 ## Proposal In this PR we add the validator side only of the new endpoint. Importantly, clients do not use it yet (see [download_certificate_for_blob](https://github.com/linera-io/linera-protocol/blob/main/linera-core/src/remote_node.rs#L187) on `main`). This will be added after validators are updated. ## Test Plan CI. ## Release Plan - This is already a testnet branch - updating validators. - After validators' update we update clients. ## Links https://github.com/linera-io/linera-protocol/pull/4420 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | d802346 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Bump versions of `tonic` dependencies. (#4446) ## Motivation Originally the motivation was to use latest `tonic-web-wasm-client` (version `0.8.0`) which introduced the timeout on web clients using JS `fetch` method (https://github.com/devashishdxt/tonic-web-wasm-client/pull/84). In order to do so, update of all tonic dependencies was required. ## Proposal Update `tonic`. Unfortunately, we don't use the `tonic-web-wasm-client` latest feature as it also cancels the long-living streaming requests (like our `Subscribe` endpoint) which triggers the re-subscription (repeatedly). This slows down the clients but also could lead to potentially loosing some events emitted while it was un-subscribed. ## Test Plan CI. ## Release Plan - This change follows usual release cycle. ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) --------- Signed-off-by: deuszx <95355183+deuszx@users.noreply.github.com> Co-authored-by: James Kay <james.kay@linera.io>
| Commit: | 272bc98 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Control creation of network actions when handling `ChainInfoQuery`. (#4523) ## Motivation Creating network actions requires reading certificates – this takes a lot of CPU time (reading from storage, deserializing certificates, etc.) but it's not always needed. Network actions are created in couple of places but one surprising one was when handling a `ChainInfoQuery`. ## Proposal Add a boolean field to `ChainInfoQuery` struct that controls whether the caller wants to create network actions. By default it is set to `true` to maintain backwards compatibility but clients can call `no_network_actions` to set it to false. ## Test Plan CI. ## Release Plan - Nothing to do / These changes follow the usual release cycle. This is already backported to testnet (https://github.com/linera-io/linera-protocol/pull/4518) ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 771ce59 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
[testnet] Clients update validators about missing committees. (#4493) (#4515) ## Motivation If a validator doesn't have the latest committee and receives a block signed by that committee, it will fail with `EventsNotFound`. Also, the validator updater is sending the wrong blocks in some cases for sparse event publisher chains. ## Proposal Handle that in the client and update the validator about the admin chain. Use the correct block heights. Almost all the credit for this goes to @bart-linera! ## Test Plan A test was added that reproduced the issue. ## Release Plan - These changes should be backported to the latest `testnet` branch, then - be released in a new SDK. ## Links - Closes #4490. - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) --------- Co-authored-by: Andreas Fackler <afck@users.noreply.github.com> Co-authored-by: Bartłomiej Kamiński <bartlomiej.kaminski@linera.io>
| Commit: | d5b5915 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
[testnet] Control creation of network actions when handling `ChainInfoQuery`. (#4518) ## Motivation Creating network actions requires reading certificates – this takes a lot of CPU time (reading from storage, deserializing certificates, etc.) but it's not always needed. Network actions are created in couple of places but one surprising one was when handling a `ChainInfoQuery`. ## Proposal Add a boolean field to `ChainInfoQuery` struct that controls whether the caller wants to create network actions. By default it is set to `true` to maintain backwards compatibility but clients can call `no_network_actions` to set it to false. ## Test Plan CI. ## Release Plan - Nothing to do / These changes follow the usual release cycle. This is already a testnet backport. ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 83bdb4d | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add bool to ChainInfoQuery to trigger create_network_actions
| Commit: | 538df63 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add bool to ChainInfoQuery to trigger create_network_actions
| Commit: | 33ef5cd | |
|---|---|---|
| Author: | Mathieu Baudet | |
| Committer: | GitHub | |
[testnet] Add new endpoint DownloadRawCertificatesByHeight (#4492) ## Motivation This is the first push-safe half of #4483 We want to release this asap but not break clients in the meantime. ## Proposal Split #4483 ## Test Plan CI + tested on main ## Release Plan testnet branch --------- Co-authored-by: deuszx <95355183+deuszx@users.noreply.github.com>
| Commit: | 206c631 | |
|---|---|---|
| Author: | deuszx | |
Revert "[testnet] Add new DownloadRawCertificatesByHeight endpoint and use it. (#4483)" This reverts commit 651cc812e49a1af2680d8792cf08f06ad9fabf1a.
| Commit: | 651cc81 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
[testnet] Add new DownloadRawCertificatesByHeight endpoint and use it. (#4483) ## Motivation During our performance tests it was found that proxies' bottleneck is (de)serialization of certificates when responding to `DownloadCertificatesByHeights` request. ## Proposal It is a waste of CPU cycles to deserialize data (after loading from the DB) only to serialize it for transporting over gRPC protocol. This PR: - adds a new method on the storage `read_certificates_raw` which returns raw bytes of the requested certificates - adds a new endpoint to `rpc.proto/ValidatorNode` – `DownloadRawCertificatesByHeight` that responds with (bcs) bytes of the requested certificates - updates proxy code to serve the new endpoint - the gRPC _client_ does NOT get a new method, instead the old `download_certificates_by_height` is modified to use the new endpoint. **This is safe:** b/c we will release this as a new SDK version meaning old clients still use old code paths while next version uses new. ## Test Plan CI ## Release Plan - These changes should be backported to the latest `devnet` branch, then - be released in a new SDK, - be released in a validator hotfix. - These changes should be backported to the latest `testnet` branch, then - be released in a new SDK, - be released in a validator hotfix. ## Links Testnet backport of https://github.com/linera-io/linera-protocol/pull/4478 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 6f94385 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Add new DownloadRawCertificatesByHeight endpoint and use it. (#4478) ## Motivation During our performance tests it was found that proxies' bottleneck is (de)serialization of certificates when responding to `DownloadCertificatesByHeights` request. ## Proposal It is a waste of CPU cycles to deserialize data (after loading from the DB) only to serialize it for transporting over gRPC protocol. This PR: - adds a new method on the storage `read_certificates_raw` which returns raw bytes of the requested certificates - adds a new endpoint to `rpc.proto/ValidatorNode` – `DownloadRawCertificatesByHeight` that responds with (bcs) bytes of the requested certificates - updates proxy code to serve the new endpoint - the gRPC _client_ does NOT get a new method, instead the old `download_certificates_by_height` is modified to use the new endpoint. **This is safe:** b/c we will release this as a new SDK version meaning old clients still use old code paths while next version uses new. ## Test Plan CI ## Release Plan - These changes should be backported to the latest `devnet` branch, then - be released in a new SDK, - be released in a validator hotfix. - These changes should be backported to the latest `testnet` branch, then - be released in a new SDK, - be released in a validator hotfix. ## Links closes https://github.com/linera-io/linera-protocol/issues/4479 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | c394922 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add bool to ChainInfoQuery to trigger create_network_actions
| Commit: | 1348655 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add new DownloadRawCertificatesByHeight endpoint and use it.
| Commit: | 462bcd8 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add bool to ChainInfoQuery to trigger create_network_actions
| Commit: | 548acea | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add bool to ChainInfoQuery to trigger create_network_actions
| Commit: | 93ba6c9 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add new DownloadRawCertificatesByHeight endpoint and use it.
| Commit: | a364252 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add a new endpoint for downloading certificates without deserializing them.
| Commit: | 3a431aa | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Bump tonic dependencies
| Commit: | b78d65a | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Bump tonic dependencies
| Commit: | 57014c1 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Fetch certificate last used the blob (#4420) ## Motivation Continuation of the work on improving performance of clients. Trying to decrease # of roundtrips between clients and validators. ## Proposal It was noted that we often make two requests to download a certificate for blob: 1. `BlobLastUsedBy` – returning a certificate hash 2. `DownloadCertificate` – returning the certificate. Here we introduce a new endpoint – `BlobLastUsedByCertificate(blob_id) -> ConfirmedCertifiate` – which does both things in one query. Note that the new endpoint is introduced in proxy only. This decreases number of necessary network roundtrips from 9 to 6: <img width="314" height="264" alt="Screenshot 2025-08-27 at 12 07 08" src="https://github.com/user-attachments/assets/1e6bc42c-276b-45a1-bf77-b6a304c57b17" /> ## Test Plan CI ## Release Plan - Nothing to do / These changes follow the usual release cycle. **OR** - These changes should be backported to the latest `devnet` branch, then - be released in a new SDK, - be released in a validator hotfix. - These changes should be backported to the latest `testnet` branch, then - be released in a new SDK, - be released in a validator hotfix. ## Links <!-- Optional section for related PRs, related issues, and other references. If needed, please create issues to track future improvements and link them here. --> - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | fce62e5 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Fetch certificate last used the blob
| Commit: | 8d76fa9 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Return the blob alongside the certificate
| Commit: | 4e2dd2e | |
|---|---|---|
| Author: | deuszx | |
| Committer: | Mathieu Baudet | |
Improve syncing on incoming message (#4373) ## Motivation When inspecting the performance of our web demos (namely hosted-fungible) we noticed that an incoming transfer triggers 7 network requests while only 3 (last ones) are about accepting an incoming transfer (proposing a block that includes the incoming message). After closer research, it was discovered that we make: - `HandleChainInfoQuery` - first request to learn about the sender chain tip - `HandleChainInfoQuery` - second time we request via `fetch_sent_certificate_hashes` with a specific `BlockRange` (after checking the `info.requested_received_log`) - `DownloadCertificates` - to download the certififcates (by hash) - `DownloadCertificates` - unnecessary query with empty payload (this was a bug). - and finally three requests to propose the block. ## Proposal This PR proposes an improvement of the situation – decreasing # of requests from 7 to 5: - `HandleChainInfoQuery` – to learn about the missing certificates for block heights that we might be missing - `DownloadCertificatesByHeights` – new endpoint that we use to download certificates at specific block heights - and finally three requests to propose a block. I don't think we can get it any lower with the way `ChainClient` is currently structured. This also improves all calls to `query_certificates_from` which are now making 1 network query instead of two. ## Test Plan CI ## Release Plan - Nothing to do / These changes follow the usual release cycle. **OR** - These changes should be backported to the latest `devnet` branch, then - be released in a new SDK, - be released in a validator hotfix. ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | ae116ad | |
|---|---|---|
| Author: | deuszx | |
Fetch certificate last used the blob
| Commit: | ed661c5 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Improve syncing on incoming message (#4373) ## Motivation When inspecting the performance of our web demos (namely hosted-fungible) we noticed that an incoming transfer triggers 7 network requests while only 3 (last ones) are about accepting an incoming transfer (proposing a block that includes the incoming message). After closer research, it was discovered that we make: - `HandleChainInfoQuery` - first request to learn about the sender chain tip - `HandleChainInfoQuery` - second time we request via `fetch_sent_certificate_hashes` with a specific `BlockRange` (after checking the `info.requested_received_log`) - `DownloadCertificates` - to download the certififcates (by hash) - `DownloadCertificates` - unnecessary query with empty payload (this was a bug). - and finally three requests to propose the block. ## Proposal This PR proposes an improvement of the situation – decreasing # of requests from 7 to 5: - `HandleChainInfoQuery` – to learn about the missing certificates for block heights that we might be missing - `DownloadCertificatesByHeights` – new endpoint that we use to download certificates at specific block heights - and finally three requests to propose a block. I don't think we can get it any lower with the way `ChainClient` is currently structured. This also improves all calls to `query_certificates_from` which are now making 1 network query instead of two. ## Test Plan CI ## Release Plan - Nothing to do / These changes follow the usual release cycle. **OR** - These changes should be backported to the latest `devnet` branch, then - be released in a new SDK, - be released in a validator hotfix. ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 0dab8b7 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Fix backwards incompatibility in protobuf def
| Commit: | 210080a | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Remove unused ChainInfoQuery field
| Commit: | fc8f7c1 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add support for selective cert request ChainInfoQuery
| Commit: | 2d7710f | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Change the API to expect list of block heights rather than ranges
| Commit: | c51443d | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add endpoint for reading certs by chainId and block range
| Commit: | cfb89c1 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add support for selective cert request ChainInfoQuery
| Commit: | c299ea5 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Change the API to expect list of block heights rather than ranges
| Commit: | 36ce154 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add endpoint for reading certs by chainId and block range
| Commit: | 965b064 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Improve "missing vote" errors. (#4327) ## Motivation We are seeing "missing vote" errors in some scenarios, although it's not clear it can currently be reproduced on `main`. ## Proposal Include in the error message whether it was a missing vote due to a proposed block, validated block or requested timeout. Make validators return errors instead of just skipping in more cases. Now we only skip signing and return the chain info if we either aren't a validator (don't have signing keys) or we already have signed the requested vote. ## Test Plan CI should catch regressions. Otherwise these changes should both make "missing vote" errors less likely (impossible, with non-faulty validators) and easier to debug. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 1ebe688 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Add gRPC based indexer (#4285) ## Motivation One of the destinatinos the block exporter is capable of pushing data to is an indexer. Up until now only a client part (of the gRPC interface) was implemented. ## Proposal Add a server side of the indexer destination. Uses SQLite backed indexer with four tables at the moment: - blocks - blobs - incoming bundles - and posted messages for deduplication of incoming bundles. ## Test Plan CI and manual. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 63a1d56 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add gRPC based indexer
| Commit: | fb255a9 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Add gRPC based indexer
| Commit: | 91466af | |
|---|---|---|
| Author: | Mathieu Baudet | |
| Committer: | GitHub | |
Rename a few structures in storage service (#4276) ## Motivation * Facilitate the future refactoring to address https://github.com/linera-io/linera-protocol/issues/4275 * Make the names more uniform ## Proposal * rename `ServiceStoreServerInternal` into `LocalStore` (implements the `KeyValueStore` trait) * rename `ServiceStoreServer` into `StorageServer` (just a server state) * rename `ServiceStoreClient` into `StorageServiceStore` (implements the `KeyValueStore` trait + "Service" is a bit vague) * rename `StoreConfig::Service` into `StoreConfig::StorageService` (idem) * rename `StoreProcessor` into `StorageService` (consistency) ## Test Plan CI ## Release Plan - Nothing to do / These changes follow the usual release cycle.
The documentation is generated from this commit.
| Commit: | f59cbcb | |
|---|---|---|
| Author: | usagi32 | |
| Committer: | GitHub | |
testnet backport of the indexer exporter (#4187) ## Motivation This PR just backports the indexer destination #3949 to the babbage testnet that's already part of the main branch.
| Commit: | 7be4ef1 | |
|---|---|---|
| Author: | usagi32 | |
| Committer: | GitHub | |
Indexer destination (#3949) ## Motivation Generic indexer destination for the block exporter. To be rebased on top of #3896 . ## Proposal - Integration with the local net. - Global indexing for the blobs for each destination kind. - Also Closes #3669 ## Test Plan Unit + integration tests ## Release Plan - Nothing to do / These changes follow the usual release cycle.
| Commit: | 6cfc444 | |
|---|---|---|
| Author: | Bartłomiej Kamiński | |
| Committer: | GitHub | |
Remove committees from the chain description (#4131) ## Motivation `ChainDescription` contains committees that were active at the time of creation of the chain. This unnecessarily duplicates information that is already available in other places, like the committee blobs. ## Proposal Replace the `committees` field in the `ChainDescription` that maps epochs to committees by just the set of active epochs. ## Test Plan Regressions should be caught by CI. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) --------- Co-authored-by: Andreas Fackler <andreas.fackler@linera.io>
| Commit: | ba002bb | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Remove get_public_key method from Signer. (#4088) ## Motivation The `get_public_key` method in the `Signer` API is a blocker for implementing that trait for EVM wallets - like MetaMask - which do not expose the public key to the users. ## Proposal Remove `Signer::get_public_key`. There were few palces where we used `get_public_key()` via `ChainClient` only to map that into `AccountOwner` - these were updated to call `client.identity()` directly instead. The biggest change is in the `BlockProposal` struct, which also carried `AccountPublicKey` used for verifying the signature later on. Here the public keys were "inlined" into the `AccountSignature`: for `Ed25519 and `Secp256k1` cases this meant simply adding the key to those variants: ```rust pub enum AccountSignature { Ed25519 { signature: ed25519::Ed25519Signature, public_key: ed25519::Ed25519PublicKey, }, Secp256k1 { signature: secp256k1::Secp256k1Signature, public_key: secp256k1::Secp256k1PublicKey, }, EvmSecp256k1(secp256k1::evm::EvmSignature) ``` and use the `public_key`s to verify the signatures later. For the EVM case we recover the `AccountOwner` (via recovering EVM address and turning that into `AccountOwner::Address20`) from the signature and the message. ## Test Plan CI ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links Closes https://github.com/linera-io/linera-protocol/issues/4077 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 76e46ae | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Move public key to AccountSignature; for EVM use Address
| Commit: | 7d91384 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Move `admin_id` from `ChainDescription` into `NetworkDescription` (#3961) ## Motivation There is no use case for different chains having different admin chain IDs, so admin ID should be a network-wide value, not a per-chain value. ## Proposal Move the admin chain ID from `ChainDescription` into `NetworkDescription`. ## Test Plan Regressions should be caught by CI. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - This is @bart-linera's https://github.com/linera-io/linera-protocol/pull/3946, rebased and with a few minor test fixes. - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist) --------- Co-authored-by: Bartłomiej Kamiński <bartlomiej.kaminski@linera.io>
| Commit: | ec7202a | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Fix authenticated signer when re-proposing a fast block. (#3920) ## Motivation When re-proposing a validated block from an earlier round there is a validated block certificate, signed by a quorum of validators, attesting the correctness of the outcome and the validity of the oracle responses—_and_ that the `authenticated_signer` is actually the one who originally submitted the block proposal to the validators! This last part I missed in the special case where we re-propose an earlier proposal by a super owner from the fast round! In that case, there is no validated block certificate, which is why we disallow oracles. But we erroneously compare the `authenticated_signer` to the owner _re-proposing_ the block, rather than the original super owner. Of course comparing it to the super owner is not enough: The regular owner must not be able to do something in the super owner's name without permission. So we need to also verify the super owner's signature (and super ownership!) again. Super owners by design take greater responsibility for a chain's liveness than regular owners, and in the case of re-proposing a _fast_ block, the super owner's signature needs to play a similar role to the validators' signatures when re-proposing a _regular_ block. ## Proposal Properly distinguish the _three_ cases of a block proposal: * The current proposer is the one who originally created this block. They are the authenticated signer. * This is a retry of an earlier proposal by a super owner in the _fast_ round: The super owner is the authenticated signer, but their signature must be included in the proposal and verified again. * This is a retry of an earlier proposal in regular round: The original proposer is the authenticated signer, but we don't need to verify their signature again; the validators' signatures of the earlier round's certificate already proves that the proposal is valid (in addition to proving that the included oracle responses are, too). ## Test Plan `test_re_propose_fast_block` was added. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 1820109 | |
|---|---|---|
| Author: | Mathieu Baudet | |
| Committer: | GitHub | |
Store network information in DB (#3755) ## Motivation We currently pass around a genesis configuration when deploying a validator but this is not really needed. This is the first step to simplify things. ## Proposal * Create a notion of network description to be persisted in DB * Use it in the proxy * Upgrade the RPC to return the full `NetworkInformation` while we're at it. Incidentally, we won't compute the genesis config hash over and over again. ## Test Plan CI ## Release Plan - Nothing to do / These changes follow the usual release cycle.
| Commit: | c5ccaad | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Remove pub-sub channels. (#3793) ## Motivation With #3784, pub-sub channels are no longer needed. ## Proposal Remove them, and the types `ChannelName`, `Origin`, `Target`, `Medium`, `Destination`, etc. ## Test Plan CI ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Part of #365. - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | a5b31fa | |
|---|---|---|
| Author: | Eric Norberg | |
| Committer: | GitHub | |
Fix minor typos in code (#3673)
| Commit: | 9f7f56c | |
|---|---|---|
| Author: | Mathieu Baudet | |
| Committer: | GitHub | |
Remove the type Owner (#3625) ## Motivation * Introduce a uniform 0x syntax similar to Ethereum * Prepare 20-byte addresses for EVM users * fixes #1713 ## Proposal This is based on the work of @deuszx with #3594 but trying to minimize the changes. Note: * This PR doesn't try to rename `AccountOwner` into `Address` because this may require a lot of work to also rename all the fields and the variables `owner` to `address`. * Next, we may want to add `0x` to the syntax of `ApplicationId`. Follows PRs https://github.com/linera-io/linera-protocol/pull/3626 and https://github.com/linera-io/linera-protocol/pull/3627 ## Test Plan CI --------- Signed-off-by: Mathieu Baudet <1105398+ma2bd@users.noreply.github.com> Co-authored-by: Andreas Fackler <afck@users.noreply.github.com>
| Commit: | 24a8b3b | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
RM Owner and use MultiAddress
| Commit: | d3df122 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Rename AccountOwner to MultiAddress
| Commit: | caabd19 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Update variable names after refactor
| Commit: | 86690ec | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Rename MultiAddress to Address
| Commit: | 26e3477 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
RM Owner and use MultiAddress
| Commit: | ac1a0e9 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | deuszx | |
Rename AccountOwner to MultiAddress
| Commit: | 5f582a7 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Make Chain a variant of AccountOwner (#3560) ## Motivation We want to fix and expand our notion of Address (currently Owner, GenericApplicationId, etc.) to different type of addresses (32-byte Linera/Solana, 20-byte EVM). In order to do that we need to prepare the code for the introduction of new variants. ## Proposal Inline a `Chain` into `AccountOwner` enum to identify cases where transactions (mostly token transfers) are targeting or using chain's account balance. Previously that case was handled with the usage of `Option<AccountOwner>`. This made refactoring more difficult. ## Test Plan CI should catch regressions. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links <!-- Optional section for related PRs, related issues, and other references. If needed, please create issues to track future improvements and link them here. --> - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 840430a | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Use ValidatorPublicKey instead of ValidatorName (#3372) ## Motivation Currently we're using `ValidatorName` (a newtype wrapper around `ValidatorPublicKey`) in many places where it's not making any quality improvements. I'd argue that it's: - unnecessary (it's just a wrapper around our own struct) - confusing (we have `ValidatorName` and, next to it, `ValidatorSignature` that we verify with `validator_name.public_key` ) - might clash (in the future) with "nicknames" - i.e. onchain human-readable aliases that validators may decide to register ## Proposal Replace usage of `ValidatorName` with `ValidatorPublicKey`. Also rename field names from `name` to `public_key` - this affects the RPC format and CLI. ## Test Plan CI should catch any regressions. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links <!-- Optional section for related PRs, related issues, and other references. If needed, please create issues to track future improvements and link them here. --> - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 8ef379f | |
|---|---|---|
| Author: | Mathieu Dutour Sikiric | |
| Committer: | GitHub | |
Implement `list_root_keys` to the `AdminKeyValueStore` (#3142) ## Motivation We need to have the feature of accessing to the list of root keys in order to have mutation function and accessing to the list of chain ids of a storage. Fixes #3085 ## Proposal The implementation causes some problems: * For ScyllaDb the implementation is very easy since the root_key is used as a partition key. * For DynamoDb we can implement the root_key as a partition key, but DynamoDb forbids iterating and determining all the partition keys. So, we need to keep track of the keys. * For RocksDb / StorageService / IndexedDb we need to keep track of the list of root_keys. * For RocksDb, this led to a simplification since the edge case of having a key of the form `[255, ..., 255]` disappears. This corrects a problem and a test is added to detect it. * The result of the `list_root_keys` will not be the same on different storage. If storage has been created with `fn create`, some keys were written but later deleted then in DynamoDb, RocksDb, storage-service, IndexedDb the root key will show up as existing while in ScyllaDb, the root key will not be visible. The writing of the root key occurs when a `write_batch` is done. ## Test Plan One test has been added for this feature. ## Release Plan No impact on the TestNet / DevNet. It can follow the normal release plan. ## Links None.
| Commit: | cec6d8f | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Upload proposals' blobs separately. (#3204) ## Motivation Including blobs with the gRPC message that contains a block proposal or certificate severely limits the total size of the blobs. (See https://github.com/linera-io/linera-protocol/issues/3048.) ## Proposal Remove the `blobs` field from block proposals, and handle blobs separately, with one message each. I added a `maximum_published_blobs` limit per block, and limited the number of pending proposals with blobs to 1 for permissionless chains. (See https://github.com/linera-io/linera-protocol/issues/3203.) ## Test Plan The tests have been updated where necessary. Otherwise they already cover different scenarios involving proposals with blobs. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Closes #3202 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 903b975 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Use owners instead of public keys. (#3160) ## Motivation For #3162 we need to stop using `ChainOwnership` as a lookup table for public keys. In general, we are using `PublicKey` in several places where we should be using the `Owner` type. ## Proposal Remove public keys from `ChainOwnership`. Add the signer's public key directly to the `BlockProposal` instead. When creating key pairs, assigning chains and changing chain ownership via CLI commands, node service mutations or system API calls, use `Owner` instead of `PublicKey`. ## Test Plan Several tests have been updated. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Closes #3165. - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 8ed5ddb | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Upload blobs for validated block certificates in separate messages. (#3153) ## Motivation Ultimately we want to transfer all blobs separately, rather than in a single message. (https://github.com/linera-io/linera-protocol/issues/3048) This PR is another step towards that goal: The blobs required by a validated block certificate are now uploaded separately, rather than in the same message as the certificate itself. ## Proposal Add a map of missing blobs for the highest-round validated block to the chain state, and a `HandlePendingBlob` endpoint to populate that map. ## Test Plan There are already tests covering different scenarios with validated blocks' blobs. Where necessary, these were updated. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Closes #3152. - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 3a61111 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Fetch locked blobs individually, not in `ChainManagerInfo`. (#3121) ## Motivation Ultimately we want to transfer all blobs separately, rather than in a single message. (https://github.com/linera-io/linera-protocol/issues/3048) This PR is one step towards that goal: When the client fetches the locked block from a validator, it now requests the corresponding blobs one by one, rather than all at once. ## Proposal Add a `DownloadPendingBlob` endpoint; remove the locked blobs from the `ChainManagerInfo`. ## Test Plan Existing tests exercise this scenario, e.g. `test_finalize_locked_block_with_blobs`. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Part of #3048. - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 8a46488 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Rename DownloadBlobContent → DownloadBlob (#3117) ## Motivation When I addressed https://github.com/linera-io/linera-protocol/pull/3108#discussion_r1909026114, I only renamed `UploadBlobContent` to `UploadBlob`. `DownloadBlobContent` has a name inconsistent with that now. ## Proposal Rename `DownloadBlobContent` to `DownloadBlob`. ## Test Plan (Only renaming.) ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Original discussion: https://github.com/linera-io/linera-protocol/pull/3108#discussion_r1909026114 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | 27923c7 | |
|---|---|---|
| Author: | Andreas Fackler | |
| Committer: | GitHub | |
Upload blobs for confirmed certificates separately. (#3108) ## Motivation Including blobs with the gRPC message that contains a block proposal or certificate severely limits the total size of the blobs. (See #3048.) ## Proposal As a first step, remove the blobs from the `handle_confirmed_certificate` functions and messages. Instead, when a validator sees a fully signed confirmed block it creates the blob states in its local storage even if it doesn't have the blobs yet. The client can then upload the blobs one by one, and the validator will accept them. Finally, the client can retry sending the certificate. We don't do this for block proposals or validated blocks yet: These will need to be handled differently, because in these cases the blob has not necessarily been successfully published yet, so we should _not_ create a blob state. Instead, we will put these blobs into a temporary cache. ## Test Plan The existing tests are now using the new flow for confirmed block certificates. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Part of #3048 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)
| Commit: | d5acb31 | |
|---|---|---|
| Author: | deuszx | |
| Committer: | GitHub | |
Cache executed blocks (#3005) * Add CertificateType to Lite* structs. * Cache ExecutedBlock instead of Confirmed* or Validated*. * Impl (Partial)Eq for Hashed * Hardcode CertificateKind u8 representation * Include CertificateKind when signing over * Rename executed_block_hash back to value_hash * Store ConfirmedBlock-s again