Proto commits in linera-io/linera-protocol

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

2025-05-02

Store network information in DB (#3755) ## Motivation We currently pass around a genesis configuration when deploying a validator but this is not really needed. This is the first step to simplify things. ## Proposal * Create a notion of network description to be persisted in DB * Use it in the proxy * Upgrade the RPC to return the full `NetworkInformation` while we're at it. Incidentally, we won't compute the genesis config hash over and over again. ## Test Plan CI ## Release Plan - Nothing to do / These changes follow the usual release cycle.

The documentation is generated from this commit.

2025-04-23

Remove pub-sub channels. (#3793) ## Motivation With #3784, pub-sub channels are no longer needed. ## Proposal Remove them, and the types `ChannelName`, `Origin`, `Target`, `Medium`, `Destination`, etc. ## Test Plan CI ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Part of #365. - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)

2025-03-28

Fix minor typos in code (#3673)

2025-03-24

Remove the type Owner (#3625) ## Motivation * Introduce a uniform 0x syntax similar to Ethereum * Prepare 20-byte addresses for EVM users * fixes #1713 ## Proposal This is based on the work of @deuszx with #3594 but trying to minimize the changes. Note: * This PR doesn't try to rename `AccountOwner` into `Address` because this may require a lot of work to also rename all the fields and the variables `owner` to `address`. * Next, we may want to add `0x` to the syntax of `ApplicationId`. Follows PRs https://github.com/linera-io/linera-protocol/pull/3626 and https://github.com/linera-io/linera-protocol/pull/3627 ## Test Plan CI --------- Signed-off-by: Mathieu Baudet <1105398+ma2bd@users.noreply.github.com> Co-authored-by: Andreas Fackler <afck@users.noreply.github.com>

2025-03-21

RM Owner and use MultiAddress

Rename AccountOwner to MultiAddress

2025-03-20

Update variable names after refactor

2025-03-18

Rename MultiAddress to Address

RM Owner and use MultiAddress

Rename AccountOwner to MultiAddress

2025-03-16

Make Chain a variant of AccountOwner (#3560) ## Motivation We want to fix and expand our notion of Address (currently Owner, GenericApplicationId, etc.) to different type of addresses (32-byte Linera/Solana, 20-byte EVM). In order to do that we need to prepare the code for the introduction of new variants. ## Proposal Inline a `Chain` into `AccountOwner` enum to identify cases where transactions (mostly token transfers) are targeting or using chain's account balance. Previously that case was handled with the usage of `Option<AccountOwner>`. This made refactoring more difficult. ## Test Plan CI should catch regressions. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links <!-- Optional section for related PRs, related issues, and other references. If needed, please create issues to track future improvements and link them here. --> - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)

2025-02-20

Use ValidatorPublicKey instead of ValidatorName (#3372) ## Motivation Currently we're using `ValidatorName` (a newtype wrapper around `ValidatorPublicKey`) in many places where it's not making any quality improvements. I'd argue that it's: - unnecessary (it's just a wrapper around our own struct) - confusing (we have `ValidatorName` and, next to it, `ValidatorSignature` that we verify with `validator_name.public_key` ) - might clash (in the future) with "nicknames" - i.e. onchain human-readable aliases that validators may decide to register ## Proposal Replace usage of `ValidatorName` with `ValidatorPublicKey`. Also rename field names from `name` to `public_key` - this affects the RPC format and CLI. ## Test Plan CI should catch any regressions. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links <!-- Optional section for related PRs, related issues, and other references. If needed, please create issues to track future improvements and link them here. --> - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)

2025-02-06

Implement `list_root_keys` to the `AdminKeyValueStore` (#3142) ## Motivation We need to have the feature of accessing to the list of root keys in order to have mutation function and accessing to the list of chain ids of a storage. Fixes #3085 ## Proposal The implementation causes some problems: * For ScyllaDb the implementation is very easy since the root_key is used as a partition key. * For DynamoDb we can implement the root_key as a partition key, but DynamoDb forbids iterating and determining all the partition keys. So, we need to keep track of the keys. * For RocksDb / StorageService / IndexedDb we need to keep track of the list of root_keys. * For RocksDb, this led to a simplification since the edge case of having a key of the form `[255, ..., 255]` disappears. This corrects a problem and a test is added to detect it. * The result of the `list_root_keys` will not be the same on different storage. If storage has been created with `fn create`, some keys were written but later deleted then in DynamoDb, RocksDb, storage-service, IndexedDb the root key will show up as existing while in ScyllaDb, the root key will not be visible. The writing of the root key occurs when a `write_batch` is done. ## Test Plan One test has been added for this feature. ## Release Plan No impact on the TestNet / DevNet. It can follow the normal release plan. ## Links None.

2025-02-01

Upload proposals' blobs separately. (#3204) ## Motivation Including blobs with the gRPC message that contains a block proposal or certificate severely limits the total size of the blobs. (See https://github.com/linera-io/linera-protocol/issues/3048.) ## Proposal Remove the `blobs` field from block proposals, and handle blobs separately, with one message each. I added a `maximum_published_blobs` limit per block, and limited the number of pending proposals with blobs to 1 for permissionless chains. (See https://github.com/linera-io/linera-protocol/issues/3203.) ## Test Plan The tests have been updated where necessary. Otherwise they already cover different scenarios involving proposals with blobs. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Closes #3202 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)

2025-01-22

Use owners instead of public keys. (#3160) ## Motivation For #3162 we need to stop using `ChainOwnership` as a lookup table for public keys. In general, we are using `PublicKey` in several places where we should be using the `Owner` type. ## Proposal Remove public keys from `ChainOwnership`. Add the signer's public key directly to the `BlockProposal` instead. When creating key pairs, assigning chains and changing chain ownership via CLI commands, node service mutations or system API calls, use `Owner` instead of `PublicKey`. ## Test Plan Several tests have been updated. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Closes #3165. - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)

2025-01-21

Upload blobs for validated block certificates in separate messages. (#3153) ## Motivation Ultimately we want to transfer all blobs separately, rather than in a single message. (https://github.com/linera-io/linera-protocol/issues/3048) This PR is another step towards that goal: The blobs required by a validated block certificate are now uploaded separately, rather than in the same message as the certificate itself. ## Proposal Add a map of missing blobs for the highest-round validated block to the chain state, and a `HandlePendingBlob` endpoint to populate that map. ## Test Plan There are already tests covering different scenarios with validated blocks' blobs. Where necessary, these were updated. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Closes #3152. - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)

2025-01-13

Fetch locked blobs individually, not in `ChainManagerInfo`. (#3121) ## Motivation Ultimately we want to transfer all blobs separately, rather than in a single message. (https://github.com/linera-io/linera-protocol/issues/3048) This PR is one step towards that goal: When the client fetches the locked block from a validator, it now requests the corresponding blobs one by one, rather than all at once. ## Proposal Add a `DownloadPendingBlob` endpoint; remove the locked blobs from the `ChainManagerInfo`. ## Test Plan Existing tests exercise this scenario, e.g. `test_finalize_locked_block_with_blobs`. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Part of #3048. - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)

Rename DownloadBlobContent → DownloadBlob (#3117) ## Motivation When I addressed https://github.com/linera-io/linera-protocol/pull/3108#discussion_r1909026114, I only renamed `UploadBlobContent` to `UploadBlob`. `DownloadBlobContent` has a name inconsistent with that now. ## Proposal Rename `DownloadBlobContent` to `DownloadBlob`. ## Test Plan (Only renaming.) ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Original discussion: https://github.com/linera-io/linera-protocol/pull/3108#discussion_r1909026114 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)

Upload blobs for confirmed certificates separately. (#3108) ## Motivation Including blobs with the gRPC message that contains a block proposal or certificate severely limits the total size of the blobs. (See #3048.) ## Proposal As a first step, remove the blobs from the `handle_confirmed_certificate` functions and messages. Instead, when a validator sees a fully signed confirmed block it creates the blob states in its local storage even if it doesn't have the blobs yet. The client can then upload the blobs one by one, and the validator will accept them. Finally, the client can retry sending the certificate. We don't do this for block proposals or validated blocks yet: These will need to be handled differently, because in these cases the blob has not necessarily been successfully published yet, so we should _not_ create a blob state. Instead, we will put these blobs into a temporary cache. ## Test Plan The existing tests are now using the new flow for confirmed block certificates. ## Release Plan - Nothing to do / These changes follow the usual release cycle. ## Links - Part of #3048 - [reviewer checklist](https://github.com/linera-io/linera-protocol/blob/main/CONTRIBUTING.md#reviewer-checklist)

2024-12-06

Cache executed blocks (#3005) * Add CertificateType to Lite* structs. * Cache ExecutedBlock instead of Confirmed* or Validated*. * Impl (Partial)Eq for Hashed * Hardcode CertificateKind u8 representation * Include CertificateKind when signing over * Rename executed_block_hash back to value_hash * Store ConfirmedBlock-s again

2024-12-05

Split APIs into per-certificate type endpoints (#2996) * Split API certifcate-handling methods into cert-specific. * Fix strings * Format the unformattable * Send full timeout certificate (not lite). * Reintroduce generic send_certificate method * Revert "Reintroduce generic send_certificate method" This reverts commit 2645b494cf16cece3692650aaaad07e3da11e236. * RM test-only method and replace with direct call

2024-12-02

RPC messages work on ConfirmedBlockCertificates (#2976) * RPC messages work on ConfirmedBlockCertificates * RM singular DownloadCertificate RPC query. * Ensure that we get correct number of certs in resposen. * Don't Box Vectors, * RM unnecessary conversion * Error contains CryptoHash type * Map ConfirmedBlockCertificate to Certificate before serializing * Obliterate CertificateValue for real this time (#2988) * Remove CertificateValue and change Certificate encoding. * Review part 1 * Update linera-chain/src/certificate/mod.rs Co-authored-by: Andreas Fackler <afck@users.noreply.github.com> Signed-off-by: deuszx <95355183+deuszx@users.noreply.github.com> --------- Signed-off-by: deuszx <95355183+deuszx@users.noreply.github.com> Co-authored-by: Andreas Fackler <afck@users.noreply.github.com> * Use ensure macro. * Update linera-rpc/src/grpc/client.rs Co-authored-by: Andreas Fackler <afck@users.noreply.github.com> Signed-off-by: deuszx <95355183+deuszx@users.noreply.github.com> * Import ensure --------- Signed-off-by: deuszx <95355183+deuszx@users.noreply.github.com> Co-authored-by: Andreas Fackler <afck@users.noreply.github.com>

2024-11-28

Allow applications to hold native tokens (#2978) * Use `AccountOwner` as the key for `balances` Allow applications to hold native tokens. * Use `AccountOwner` in `ChainInfoRequest` Allow requesting the balance of an application's account. * Allow client to query application balances Update `ChainClient` methods to be able to query an application's balance. * Use `AccountOwner` in `OwnerBalance` request Allow the runtime to request the balance of application accounts. * Update parameter in runtime's `read_owner_balance` Prepare to update the WIT API to be able to query an application's balance. * Update `read_owner_balance` WIT interface Use `AccountOwner` as the parameter instead of `Owner`, in order to support application accounts. * Derive `Hash` for `AccountOwner` Prepare to use the type in `SystemMessage`, which has `Hash` derived for it. * Use `AccountOwner` inside `MockContractRuntime` Prepare to change the runtime interface to use `AccountOwner`. * Use `AccountOwner` inside `MockServiceRuntime` Prepare to change the runtime interface to use `AccountOwner`. * Use `AccountOwner` in runtime's `owner_balance` Update the SDK's runtime API to support querying an application's balance. * Use `AccountOwner` in cached `owner_balances` Prepare the `ServiceRuntime` to use an updated WIT interface that allows querying the balance of application accounts. * Use `AccountOwner` in `OwnerBalances` request Allow the runtime to get a list the balances of application accounts. * Update return type of `read_owner_balances` Prepare to update the WIT API to be able to list application balances. * Update `read_owner_balances` WIT interface Return `AccountOwner` in order to include application accounts. * Update runtime's `owner_balances` signature Include application balances in the result. * Use `AccountOwner` in cached `balance_owners` Prepare the `ServiceRuntime` to use an updated WIT interface that allows querying the balance of application accounts. * Use `AccountOwner` in `BalanceOwners` request Allow the runtime to get a list the accounts including application accounts. * Update return type of `read_balance_owners` Prepare to update the WIT API to be able to list application accounts. * Update `read_balance_owners` WIT interface Return `AccountOwner` in order to include application accounts. * Update runtime's `balance_owners` signature Include application accounts in the result. * Remove redundant authentication check The authentication is checked before the message is sent, so it does not need to be checked again when the message is received, because only the system application can send its own messages. Also prepares to allow applications to initiate transfers and claims, which can't be authorized by the message receiving code. * Use `AccountOwner` in `SystemMessage` Prepare to allow applications to transfer native tokens. * Change `transfer` parameter into `AccountOwner` Prepare to allow applications to transfer native tokens. * Verify application token transfers Ensure that moving an application's account can only be performed by the application that owns the account. * Send application ID to execution state actor Allow it to be used to authenticate transfers. * Accept application transfers in execution actor Update the request message to use `AccountOwner` so that the runtime can request transfers from application accounts. * Update the runtime trait to support app. transfers Replace the `Owner` parameter with `AccountOwner`, so that applications can request transfers from their account. * Update `transfer` WIT interface Use `AccountOwner` instead of `Owner` as the source parameter in order to allow applications to request transfers from their own accounts. * Update `ContractRuntime::transfer` SDK interface Use `AccountOwner` to allow applications to transfer native tokens using the SDK. * Send authenticated application ID to `claim` func. Prepare to allow claiming from application accounts. * Verify application claims Allow the source account to be an application account, and ensure only the application that owns that account can access it. * Send application ID in `ExecutionRequest::Claim` Allow it to be used to authenticate claims from applications. * Tweak `Account` constructors documentation Add documentation links and rephrase a little to provide more details. * Refactor `FromStr` implementation for `Account` Prepare to handle more than one colon in the input string. * Use `AccountOwner` in `Account` type Update `Account` to also represent application accounts. * Box some futures to reduce their size Prevent excessive stack usage by nested futures. * Support app. accounts in native token example Remove normalization of owners, and use `AccountOwner` directly. * Use `AccountOwner` in `SystemExecuteState` helper Update the `balances` map to support applications. * Refactor to move test helpers to `test_utils` Prepare to allow the helpers to be used by a new integration test module. Also, rename the test helpers for consistency. * Test `transfer` contract runtime API Ensure that applications can transfer from their accounts, from the shared chain account, and from the signer's account. * Test `claim` contract runtime API Ensure that applications can claim from their accounts or from the signer account in remote chains. * Derive `Arbitrary` for `Amount` Prepare to use it in property tests. * Test `read_chain_balance` contract runtime API Ensure applications can read the chain balance. * Test `read_owner_balance` contract runtime API Ensure applications can read the balances of individual accounts. * Test `read_owner_balances` contract runtime API Ensure applications can read a list of all account balances. * Test `read_balance_owners` contract runtime API Ensure applications can read a list of all accounts. * Test reading the balance of a missing account The system API should fallback and return zero in that case. * Test unauthorized transfers from applications Ensure that execution fails with the appropriate error. * Test unauthorized claims from applications Ensure that execution fails with the appropriate error. * Add a `create_dummy_query_context` helper function Allow tests to create a dummy context to test queries. * Test `read_chain_balance` service runtime API Ensure applications can read the chain balance. * Move `test_accounts_strategy` to `test_utils` Allow the helper function to be used in other tests. * Test `read_owner_balance` service runtime API Ensure applications can read the balances of individual accounts. * Test `read_owner_balances` service runtime API Ensure applications can read a list of all account balances. * Test `read_balance_owners` service runtime API Ensure applications can read a list of all accounts. * Test reading the balance of a missing account The system API should fallback and return zero in that case. * Replace `ok_or_else` with `context` Simplify the code to generate the error. Co-authored-by: Andreas Fackler <afck@users.noreply.github.com> Signed-off-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com> * Expect returned accounts to be sorted The runtime API should be deterministic, so the returned lists of accounts must be sorted. Co-authored-by: Andreas Fackler <afck@users.noreply.github.com> Signed-off-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com> --------- Signed-off-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com> Co-authored-by: Andreas Fackler <afck@users.noreply.github.com>

2024-11-26

Certificate handler methods work with generic certificate type. (#2921) * Generalize functions to work on GenericCertificate * RM HashedCertificateValue from value_cache_tests.rs * Fix compilation * HashedCertificateValue constructors map to new types * Fix tests * Use Either in full_certificate fn * Replace Has with CertificateValueT * Fix compilation under benchmark feature * RM unnecessary trait bounds * Fix name * RM asyn_trait, replace with correct approach * Pin it on me * Fix tests * clippy * Update validators after submitting proposal. * Rename inner to executed_block * Reformat in usage of instrument macro * RM unnecessary intermediate variables * Fix comment * s/CertificateProcessor/ProcessableCertificate/g * fmt * Return ConfirmedBlock value from Storage (#2950) * Return ConfirmedBlock value from Storage * s/read_hashed_certificate_value/read_hashed_confirmed_block/g * RM unused TryFrom impl * Simplify ValidatorNode API. (#2960) * RM unused endpoint * Fix field numbering in proto definitions * download_certificate API returns ConfirmedBlockCertifiate

2024-11-15

Turn ChainManager's pending_blobs into proposed/locked specific pending blobs

Update the `updater.rs` code. (#2894) * Remove the `blobs_last_used_by` from `ValidatorNode`. * Add `missin_glob_ids` to `ValidatorNode`. * Use the `missing_blob_ids` in the `updater.rs` code.

2024-11-11

Implements `blobs_last_used_by` for grouped exchanges. (#2830) * Implements blobs_last_used_by for grouped exchanges. * Remove some "use RpcMessage::*" statements.

2024-10-20

Add download certificates endpoint (#2644) * Add DownloadCertificates endpoints * Use download_certificates where it was easy. * New snapshot * Adapt response to message size on the server side. * Adapt to chunked response on the client side. * Remove CHUNK_SIZE const. Hide chunking in GrpcClient * Popping removes the last element... * Add test try_proto_serialize and fix bug. Vec::with_capacity(_).len() returns 0 and when encoding protobuf message the implementation of `impl BufMut for Vec` would subtract len (0) from isize::MAX. * Change approach to how we track certs that fit into the limit * @afck review.

2024-10-18

Backports to `testnet_boole` (#2641) * Retry all gRPC requests, not just notification streams. (#2565) * Retry on gRPC errors. * Rename retry parameters. * Refactor delegate_client, make all functions retry. * Refactor client_delegate; make all methods retry. * Fix missed rename. * Address review comments. * marshal → unmarshal; remove unused import * FUT → Fut * Update outdated comments and identifiers (#2448) * Remove an obsolete comment. * Update comments, identifiers related to message bundles. * `linera-client`: relax unnecessary `Send` bound on `P` (#2450) * Upgrade to 1.81.0; fix a few allow attributes (#2463) * Upgrade to Rust 1.81.0 * Remove some clippy allow attributes. * Minor cleanups in linera-core. (#2468) * Add CommunicateAction::round. * Improve doc comments in client.rs. * Remove some unused functions and unnecessary code. * Update comment to make clear that pending_block is an option. * Simplify notification handling in client.rs (#2469) * Simplify notification handling in client.rs * Revert stricter check, since it fails in the tests. * Remove pub; move trait bounds to where they are needed. * Make synchronize_received_certificates_from_validator a method. (#2472) * In the node service, don't unnecessarily call prepare_chain. (#2479) * Add latency metrics for client functions. (#2508) * Add latency metrics for client functions. * Fix error messages: counter vs. histogram. * Add struct NamedNode. (#2514) * Add struct NamedNode. * Move more functionality to NamedNode. * Box block proposal to avoid stack overflows. * Add missing large_futures check to linera-client. * Remove LocalValidatorNode(Provider) and LocalNotificationStream (#2522) * rename LocalNotificationStream into WebNotificationStream * remove LocalValidatorNode(Provider) * (nit) Rename NamedNode into RemoteNode (#2521) * named_node -> remote_node * NamedNode -> RemoteNode * Move ChainState to a submodule and make it guard its own invariant. (#2523) * Use synchronize_from_validators instead of duplicating it. * Move client.rs → client/mod.rs * Make ChainState guard its own invariant * Add an error message for pending block at wrong height. * Cleanup: Use convenience methods. * Fix deadlock. * Misc minor fixes. (#2539) * Fix cargo machete error: remove unused dep. * Fix grammar: first_nth → first_n * Move attributes to after the comments. * Remove redundant prepare_chain calls. * Lock mutex in more client methods. (#2567) * Lock mutex in more client methods. * Make mutating ChainState methods pub(super). * Don't keep the client mutex longer than necessary. (#2581) * Sync admin chain only when needed (#2538) * Sync admin chain only when needed * Don't forget old committees after syncing the admin chain. * Allow creating a ChainClient if the chain is already in the map. (#2597) * Allow creating a ChainClient if the chain is already in the map. * Add the storage-service feature to run the benchmark test in CI. * Add TODO for #2600. * Make the other benchmark test stable, too. Rename it. * `linera-client`: add IndexedDB persistence backend for the wallet (#2420) * `linera_client::persistent`: support async persistence * `linera-client`: add IndexedDB persistence implementation * `linera_client::persistent`: refactor `mutate` into extension traits so we can use `trait-variant` * `linera-client`: remove `localStorage` support * `linera_client::persistent::dirty`: use `derive_more` * `linera-execution`: update deprecated `derive_more` syntax * `linera-client`: see if failing to save some changes is problematic * CI: use `cargo run` for the storage-service instead of building and running separately * `linera-service`: save the wallet after creation * `linera-client`: only build IndexedDB support on the Web * `linera-client`: fix benchmarking build * `linera-service`: fix `remote_net.rs` * `linera-client`: clippy fixes * Documentation tweaks Co-authored-by: Andreas Fackler <afck@users.noreply.github.com> Signed-off-by: James Kay <james.kay@linera.io> * `linera-client`: apply suggestions from code review * `linera_client::persistent::indexed_db`: simplify and generalize API * `linera-client`: don't require that the user construct a `WalletState` * `linera-base`: use the Web console for tracing output * `linera-client`: add tracing and fix a dirtiness bug * Cleanup --------- Signed-off-by: James Kay <james.kay@linera.io> Co-authored-by: Andreas Fackler <afck@users.noreply.github.com> * Remove ChainClients map. (#2604) * Remove ChainClients map. * Remove some unnecessary generic arguments. * Use Arc::clone explicitly. * Move trait bounds to ValidatorNodeProvider itself. * Move trait bounds to ValidatorNode itself. * Move trait bounds to ClientContext itself. * Reduce the number of initial chains. (#2626) * Reduce the number of initial chains. * Fix README tests; make 2 other chains by default. * Fix test_end_to_end_benchmark. * Fix CLI.md. * Retry node service query on timeouts (#2640) Treat `reqwest` timeout errors the same as errors reported through GraphQL. * Add an end-to-end test that doubles as a latency benchmark. (#2513) * Add an end-to-end test that can be used as a latency benchmark. * Make number of test iterations configurable via env variable. * Reduce timeout to 1 second in total. * Revert "Sync admin chain only when needed (#2538)" This reverts commit 74cf731f6501d1c45768b0f6faf3765e27072a41. * Increase timeout in test_end_to_end_repeated_transfers. * Refactor synchronized_received_certificates_from_validator (#2643) * Refactor synchronized_received_certificates_from_validator * s/handle_certificate/check_certificate * RM Certificate from enum variants * RM obsolete comment * Return error on validator error * Move RemoteNode to its own module; move try_query_certificates_from there. (#2623) --------- Signed-off-by: James Kay <james.kay@linera.io> Co-authored-by: James Kay <james.kay@linera.io> Co-authored-by: Mathieu Baudet <1105398+ma2bd@users.noreply.github.com> Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@linera.io> Co-authored-by: deuszx <95355183+deuszx@users.noreply.github.com>

2024-09-25

Misc minor fixes. (#2539) * Fix cargo machete error: remove unused dep. * Fix grammar: first_nth → first_n * Move attributes to after the comments. * Remove redundant prepare_chain calls.

2024-09-04

Update outdated comments and identifiers (#2448) * Remove an obsolete comment. * Update comments, identifiers related to message bundles.

2024-08-27

Replace Bytecode with Blobs (#2375) ## Motivation Now that we have blobs, there are several follow ups we can do. ## Proposal This PR replaces Bytecode with Blobs. This will clean up a lot of code and also make our blocks smaller when publishing Bytecodes. Fixes #2149 #2199 ## Test Plan CI Some tests had to be altered since we don't have to subscribe to get bytecodes anymore, and also the `ApplicationId` format has changed with this.

2024-08-05

Some leftover renames after BlobContent (#2349) ## Motivation We renamed this in a previous PR, but some names were still incorrect ## Proposal Use blob content wherever is appropriate ## Test Plan CI

2024-08-02

Blob->BlobContent HashedBlob->Blob (#2345) ## Motivation `HashedBlob` isn't really a hashed blob anymore, as the `BlobId` now has more than just the hash. ## Proposal Change `Blob` to be `BlobContent`, and change `HashedBlob` to `Blob`. Hopefully this makes the code a bit less confusing. ## Test Plan CI

2024-07-24

Add query-validator command, and make set-validator check version. (#2289) * Add query-new-validator command, and make set-validator check version. * Add an endpoint to get the genesis config hash from the proxy. * Make set-validator and query-new-validator check genesis config. * Call query-new-validator in a test. * Rename query-new-validator to query-validator. * Replace --force with --skip-online-check.

2024-07-22

Introduce the function `contains_keys` for the `KeyValueStore` trait. (#2244) * The function `contains_keys` is introduced into the `KeyValueStore` trait. * A small correction to the `read_multi_values_bytes` of ScyllaDB.

2024-06-20

Endpoint to request which certificate published a blob last in a given validator (#2158) ## Motivation Building on top of #2156, there we will need this endpoint for the client to communicate with the validators. Fixes #2030 ## Proposal Adds an endpoint to get what `Certificate` last used a blob. ## Test Plan CI for now, but will use this in following PRs

2024-06-18

Replace request_sent_certificates_in_range and use the download of certificates (#2091) ## Motivation Right now we have `request_sent_certificates_in_range` in the chain info query. We don't need to go all the way to the server to get these certificates. ## Proposal With the new endpoints, as discussed on Slack, we could modify this to return just the certificate hashes, and then download the certificates from the proxy directly using the new endpoints. ## Test Plan CI

Endpoint to download a Certificate (#2090) ## Motivation There are also cases where we need to be able to download one certificate directly from the Proxy. ## Proposal Adds an endpoint for downloading a certificate given it's hash ## Test Plan CI for now, will replace chain info queries in following PR

Certificate -> CertificateRequest on rpc.proto (#2084) ## Motivation The current proto struct is called `Certificate`, but it isn't really representing that ## Proposal Rename it to what it actually represents ## Test Plan CI

Replace certificate chain info query usage with certificate download (#2081) ## Motivation Now that we can download certificates, using the chain info query isn't needed anymore ## Proposal Use the download certificate entrypoint instead ## Test Plan CI

Endpoint to download a certificate value (#2074) ## Motivation We need to be able to download certificate values directly from the Proxy. Fixes #2029 ## Proposal Adds an entrypoint for downloading a certificate value given a hash ## Test Plan CI for now, will replace chain info queries in following PR

2024-06-13

Refactor BlockProposal. (#2126) * Refactor BlockProposal. * Improve documentation and naming. * Fix error message, remove let _.

2024-06-04

Replace blob chain info query usage with blob download (#2098) * Replace blob chain info query usage with blob download * Don't fail if a faulty validator sends a wrong blob. * Improve error message. Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com> Signed-off-by: Andreas Fackler <afck@users.noreply.github.com> --------- Signed-off-by: Andreas Fackler <afck@users.noreply.github.com> Co-authored-by: Andre da Silva <andre.dasilva@linera.io> Co-authored-by: Janito Vaqueiro Ferreira Filho <janito.vff@gmail.com>

2024-06-01

Endpoint to download certificates

Certificate -> HandleCertificateRequest on rpc.proto

Replace blob chain info query usage with blob download

Replace certificate chain info query usage with certificate download

GRPC entrypoint to download a certificate

2024-05-31

Endpoint to download a certificate

Replace certificate chain info query usage with certificate download

Certificate -> HandleCertificateRequest on rpc.proto

Replace blob chain info query usage with blob download

GRPC entrypoint to download a certificate

2024-05-28

GRPC entrypoint to download a blob (#2044) ## Motivation We need to be able to download blobs directly from the Proxy as part of the user blobs work. Fixes #2028 ## Proposal This PR does a few things: * Adds a storage connection to the Proxy * Adds an entrypoint for downloading a blob given a blob ID ## Test Plan WIP

2024-05-23

PublishBlob (#2014) ## Motivation Currently users are able to upload bytecode that can be used to create applications. Other chains have to subscribe to the publisher chain to be able to use the bytecode. We want to generalize this so that: * Users can publish arbitrary data that can be used as bytecode or read by existing applications. * One doesn’t have to subscribe to the publisher chain (and wait for it to handle the subscription request) to use the blob. * Blobs are content-addressed, so that if two users upload identical blobs, they are stored only once. ## Proposal This PR implements the `PublishBlob` operation ## Test Plan Unit test ## Release Plan Since we're adding a new `SystemOperation`, I'm guessing we have to do this: - Need to update the developer manual.

2024-05-08

blob -> hashed_certificate_value (#2003) ## Motivation With user blobs on the way, the existing usage of `blob` in the code base can be confusing ## Proposal Rename `blob` to what it actually is, a `hashed_certificate_value` ## Test Plan CI

2024-04-11

Add fallback/public mode to the chain manager. (#1899) * Fix formatting. * A fallback_delay parameter for chains. * Keep track of first seen timestamps in the inbox. * Add fallback validator weights to the chain manager. * Add Round::Validator variant. * Add fallback votes. * Add a unit test, fix leader selection and update comment. * Rename LeaderTimeout to Timeout. * Rename request_timeout to request_leader_timeout. * Fix RPC format. * Fix cargo check-all-features. * Allow ChainClient to use a validator identity. * Add links in doc comments. * Simplify timeout request handling. * Apply review comments.

2024-03-21

Make the `linera-storage-service` handle requests and replies that have size higher than 4M. (#1788) * The protocol buffer file was changed so that the batch is now taking an `append` entry. * If the batch are bigger than 4M, then they are split into smaller ones via the `append`. * The `clear_journal` is removed from the protocol buffer since it is irrelevant to memory/rocksdb that are the only used here. * When the answer to a request has more than 4M, then it is split into smaller chunks and then returned. * The GRPC streams are not used here. The code is only meant for testing purposes so that should be sufficient for the time being. Co-authored-by: Andreas Fackler <afck@users.noreply.github.com>

2024-03-06

Implement the storage service for the end-to-end tests (#1679) * Implement the AdminKeyValueStorage for the ServiceStorage. So, now we have namespaces for the tests. * Make the feature available to all the end-to-end tests along with ScyllaDb, DynamoDb, RocksDb * To make the binary of storage accessible to other crates, not just in tests, the `resolve_binary` was used. * The `resolve_binary` and `CommandExt` are moved to `linera-base` to avoid a circular dependency. * The SharedStore was renamed ServiceStore. * The child protection is extended by adding the checking of absence of endpoint before creation. Co-authored-by: Andreas Fackler <afck@users.noreply.github.com>

2024-03-05

Consistently spell ID. (#1747) * Consistently spell ID. * Fix remaining IDs.

2024-02-11

Implement the shared storage for memory/rocksDb (#1525) * Elimination of constructors taking a guard for Memory related storage. * Implementation of a shared storage system using GRPC. * Make the functions for testing storage (read, writes, etc.) usable from other crates. Co-authored-by: Mathieu Baudet <1105398+ma2bd@users.noreply.github.com>

2024-02-06

Separate version logic into `linera-version` crate (#1594) * Separate version logic into `linera-version` crate * `linera-version`: Use a `cfg` to disable some dependencies for build * `linera-version`: loosen compatibility checks * `linera-version`: add some crate documentation * `linera-version`: code more defensively for `cargo install` * `linera-service`: fix GraphQL tests against new version API * `linera-version`: make `CrateVersion` a scalar * `linera-version`: make version numbers `u32` * `linera-version`: use GitHub `/tree/` to refer to source * `docker/Dockerfile`: copy `linera-version` into the Docker context * `linera-version`: serialize via `semver::Version` * `linera-version`: branch `CrateVersion` representation depending on whether the serialization format is human-readable

2024-01-29

nit: Rename system_balance in callbacks and chain info queries (#1574) * rename request_system_balance into request_owner_balance * rename system_balance into chain_balance * rename comments as well * fix expected fuel in test

2024-01-28

Make the --version option of linera binaries print the full VersionInfo. (#1572) * nit: mark grpc field as optional for documentation purposes * Make the --version option of linera binaries print the full VersionInfo. * refactor: split FaucetService and Faucet(Url) * add GraphQL API to query version info from a faucet * add faucet version check in the linera tool * fix graphql schema * add unit test + simplify code

2024-01-27

Support funding from authenticated accounts (#1548) * nit: split doc comment * support funding from authenticated accounts and native tokens possibly changed from user apps * fix comments from previous PR * nit: use chain info response to obtain description * Support requesting authenticated balances from a chain info query * create client methods to access owner balance * fix local_balance in case of failing system messages and add test for gas paid by an owner account * Rename `ChainClient::local_balance` into `query_balance`, make `synchronize_validators` not return a balance * Support accounts where querying balance on the CLI * address comment by moving wallet-synchronization invariant * Apply suggestions from code review Co-authored-by: Andreas Fackler <afck@users.noreply.github.com> Signed-off-by: Mathieu Baudet <1105398+ma2bd@users.noreply.github.com> --------- Signed-off-by: Mathieu Baudet <1105398+ma2bd@users.noreply.github.com> Co-authored-by: Andreas Fackler <afck@users.noreply.github.com>

2024-01-26

Expose version info to end users (#1560) * `linera-base`: compile in version info * `linera-service`: print version info at startup * `linera-service`: add `version` query to get version info from the node via GraphQL * `linera-rpc`: Add an RPC call to get the version from the validator * `linera-service`: add validator version information to the result of `query-validators`

2024-01-18

Optimize cross-chain requests: don't send whole certificates (#1510) * create_cross_chain_request should fail if it can't load from storage. * Optimize cross-chain messages: don't send whole certificates. * Address comments: rename variables

2023-11-06

Merge chain managers (#1109) * Add OwnerConfig and OwnerType; super owners can make fast blocks. * Use multi- instead of single-owner manager everywhere. * Separate regular and super owners. Regular owners don't have a weight anymore. If they propose in round 0, that is a fast block, otherwise it's a slow block. * Remove other chain manager variants. * Replace RoundNumber with RoundId enum. * The fast round cannot be skipped without a timeout. * Add a test for round types. * Fix round ID logging. * Ignore failing DynamoDB test; there is a separate ticket for it. * Rename RoundId to Round. * Address review comments.

2023-09-05

Extend chain info query and chain manager with stubs for leader timeouts. (#986) * Add request_leader_timeout to chain info query. * Add stubs for handling leader timeouts.

2023-08-07

Make round number a u32. (#938) Even with one round per second, 2^32 rounds would be 136 years. Since round lengths are increasing, `u32::MAX` should be unreachable in practice.

2023-07-19

Fix liveness issue for multi-owner chain manager.

2023-06-08

Move round number to Certificate. It's possible that two `ConfirmedBlock` certificates for the same block with different round numbers get a quorum of signatures. To make sure validators agree on the correct `previous_block_hash` for the next block, the hash must not depend on the round number. This commit moves it from `CertificateValue` to `Certificate` (and `Vote` etc.). The signatures still apply to the value _and_ the round number.

2023-06-05

rename `Message` into `IncomingMessage`

2023-05-16

Normalize gRPC field numbers.

Make it optional to wait for cross-chain messages.

2023-05-04

Simplify gRPC messages, use bincode. It's not necessary to mirror all data types in the protocol definition; using bincode is faster and simpler.

2023-05-03

Send only one ConfirmUpdatedRecipient per UpdateRecipient.

2023-04-20

Use u32 as effect index.

2023-04-11

Transmit (direct) messages from different applications on the same queue The behavior of channels is unchanged. We introduce `ChannelFullName` to designate channel queues.

2023-03-20

Distinguish owners (aka addresses) from public keys

2023-03-16

make sure every proxyable gRPC message has an outer ChainId used for routing

2023-03-08

Don't send full certificates on request_received_certificates, just chain ID and height.

2023-03-07

Allow providing binary blobs with block proposals.

2023-02-24

Fetch only Value in download_blob, not certificate.

Store only values for bytecode blobs.

2023-02-20

Rename required_certificates to blob_certificates.

Add a required_certificates parameter to handle_certificate.

2023-02-10

Deduplicate cross-chain messages.

2023-01-27

Server: Handle certificates without values.

Rename HashCertificate to LiteCertificate

Use ChainManagerInfo in RPCs, with optional values.

2023-01-26

remove ApplicationDescription and BytecodeLocation from protobuf

2023-01-18

create explicit effect DeclareApplication to register application IDs

2023-01-13

nits: use bytecode_id and bytecode_location consistently

2023-01-10

Origin::chain_id into Origin::sender_id

Wire proxy in internal network (#310) * Added proxy internal host and port to InternalNetworkValidatorConfig * Moved the NotificationService under ValidatorNode * Proxy and Servers now discriminate between public and private ports * Updated kind tests to specify internal ports

2023-01-09

Use `ChannelName` instead of `String` Replace occurrences where strings were used for channel names with the new specific `ChannelName` type. The constant channel names were also changed to be `Lazy<ChannelName>` instances instead of `&'static str` references.

Wiring push notifications with rest of codebase (#288) Wires Push Notifications up with the rest of the codebase, expanding on the original `Notification` Protobuf messages and expanding `GrpcServer::handle_network_actions` to also forward `Notification`s.

2023-01-06

Push notifications initial PR. (#239) * Created the Notification gRPC service. Implemented it for GrpcProxy and implemented the Notifier which is responsible for maintaining outbound streams to clients. Created the Notifier gRPC service. Implemented it for GrpcProxy and GrpcServer. * Modified gRPC proxy to wrap an Arc so that we only have one instance at any given time. * Notifier removes dead connections and added tests * Code clean up and moving from using grpc::ChainId to messages::ChainId for notifier * Added a test for testing eviction and fixed some deadlocking behaviour * Added NotifierServiceServer to GrpcProxy * Removed ValidatorWorker implementation from GrpcProxy and ValidatorNode implementation from GrpcServer