Proto commits in MystenLabs/sui

These 85 commits are when the Protocol Buffers files have changed:

2025-04-25

list_owned_objects: expose balance for coin types Expose the balance of an object if its a coin type. Also add a test to ensure that the coins are returned in reverse sorted order by their balances.

The documentation is generated from this commit.

list_owned_objects: add support for filtering on object type

2025-04-16

grpc: use proto schema for ResolveTransaction Finish migrating the ResolveTransaction api from the old legacy JSON format to using the new proto schema.

2025-04-15

grpc sleep endpoint for infra testing

rpc: remove outdated prototype grpc service (#21767)

2025-04-14

rpc: make use of indexed epoch info

2025-04-09

rpc-index: add support for indexing more information This patch reworks the 'rpc-index' database to: - Add support for indexing information about epochs - Add support for indexing balance changes and object type information for transactions - Add support for indexing type information for dynamic fields - Reworks the owner index to support iteration over owned objects by type as well as enforcing that Coin types are reverse sorted by their balances (greater balances come first).

2025-03-31

chore: update sui-sdk-types (#21662)

chore: update sui-sdk-types (#21663)

2025-03-27

proto: rework proto messages based on feedback This patch introduces a new protobuf package 'sui.rpc.v2beta' and 'sui.rpc.v2alpha' with completely reworked protobuf messages as well as reworked rpc service definitions. In particular the changes to the messages are as follows: - Address and ObjectId are now represented as 'string' types using their canonical human-readable representation in hex. - Digest is now represented as 'string' type using its canonical human-readable representation in base58. - TypeTag and StructTag are now represented as 'string' types using their canonical human-readable representation. - i128, u128, u256 and Bn254FieldElement are now represented as 'string' types using their canonical human-readable representation in base10. - The Bcs type now includes a 'name' field that can be used to specify the type name of the serialized value. - Various types representing a sui core type have been reworked and optimized for readability and accessibility over providing as close as possible a 1-1 mapping to their BCS representation. - Many message definitions have been split into separate files for improved readability and maintainability. This patch also implements and serves the new api and removes the old sui.node.v2alpha.NodeService and sui.node.v2alpha.SubscriptionService services. The sui.node.v2.NodeService service is left as-is and will be removed in a subsequent release.

2025-02-27

Estimate commit rate in ConsensusHandler (#21318) This is built on a small system for observing state computed from N previous commits. In order to guard against latent forking bugs caused by such observations, the digest of all state thus observed is added to each commit prologue. This requires a new version (V4) of ConsensusCommitPrologue Stacked on #21310

rpc: define a default read_mask for GetObject When a read_mask is not provided for the GetObject endpoint have it default to "object_id,version,digest".

rpc: remove GetCheckpointOptions internal type and directly use a read mask

rpc: remove ExecuteTransactionOptions internal type and directly use a read mask

rpc: remove GetFullCheckpointOptions internal type and directly use a read mask

rpc: remove GetTransactionOptions internal type and directly use a read mask

2025-02-24

proto: remove options field from GetCheckpointRequest

proto: remove options field from GetFullCheckpointRequest

proto: remove options field from GetTransactionRequest

proto: remove options field from ExecuteTransactionRequest

proto: remove deprecated input/output object fields

rpc: update docs

proto: remove options field from GetObjectRequest

2025-02-13

grpc: remove remove unnecessary nesting of repeated fields

proto: fix SignatureScheme enum to match bcs serialized value

grpc: introduce FieldMasks for partial responses Instead of defining custom "options" types for each endpoint to control the amount of data being returned, leverage the standard google.protobuf.FieldMask type and inspiration from https://google.aip.dev/157 in order to provide users fine-grained control over what fields are returned. Initially the provided field masks will support parity with the existing options and in the future provided field masks may support more fine-grained control over nested fields.

2025-02-11

grpc: add protos for more well known types Add protos for: - Any - FieldMask - Status - error_details (to be used with Status)

proto: rework directory structure to conform with standards

2025-02-07

rpc: add GetGasInfo alpha endpoint

rpc: add GetCoinInfo grpc endpoint

rpc: add ListDynamicFields grpc endpoint

rpc: move GetCoinInfo and ListDynamicFields to alpha service

rpc: add GetProtocolVersion alpha endpoint

rpc: add SimulateTransaction and ResolveTransaction alpha endpoints

rpc: add ListAccountObjects alpha endpoint

2025-02-04

rpc: documentation pass on proto files (#21045)

2025-01-24

proto: adjust doc comments so that they appear when docs are built

rpc: introduce SubscriptionService Introduce a new SubscriptionService which has a single gRPC method for subscribing to a stream of checkpoints. This new service is in the 'sui.node.v2alpha' package and is intended to be experimental and subject to change until the service is promoted to the 'sui.node.v2' package.

2025-01-09

sui-rpc: rework proto files based on 'buf lint' Rework the sui package proto files based on suggestions from 'buf lint'. One of the suggestions is to ensure that services end with a common prefix `Service`. In order to gracefully transition to this for the release to mainnet, the renamed service `NodeService` and the old service `Node` will both be supported for a single release, after which the older `Node` service will be removed.

sui-rpc: rework proto files based on 'buf lint' Rework the sui package proto files based on suggestions from 'buf lint'. One of the suggestions is to ensure that services end with a common prefix `Service`. In order to gracefully transition to this for the release to mainnet, the renamed service `NodeService` and the old service `Node` will both be supported for a single release, after which the older `Node` service will be removed.

2024-12-23

rpc: change all options to default to false when not provided Change all gRPC method options to default to false when not provided. This will result push clients to explicitly ask for the fields that they want instead of relying on any defaults that could change. This also will also make it easier to rely on the generated protobuf option types where if a field isn't present, its accessor will provide the default value of `false`.

2024-12-16

rpc: support requesting and sending UserSignatures encoded as bytes

rpc: ensure strings are string types in proto files Update all proto fields that have their corresponding protocol type encoded as utf8 to `string` but were previously listed as `bytes`. This is a wire-format compatible change and is safe to make since all previous uses of these fields were as valid utf8 strings.

2024-12-09

rename sui-rest-api crate to sui-rpc-api

rpc: introduce new protobuf definitions Introduces a new set of more fleshed out protobuf definitions for both core types (and conversions to/from sui-sdk-types and the protobuf types) as well as rpc request and response types.

rpc: introduce new protobuf definitions Introduces a new set of more fleshed out protobuf definitions for both core types (and conversions to/from sui-sdk-types and the protobuf types) as well as rpc request and response types.

rename sui-rest-api crate to sui-rpc-api

2024-12-06

rpc: introduce new protobuf definitions Introduces a new set of more fleshed out protobuf definitions for both core types (and conversions to/from sui-sdk-types and the protobuf types) as well as rpc request and response types.

rename sui-rest-api crate to sui-rpc-api

2024-11-07

[fastpath] support locking transactions and returning effects without signatures (#20128) ## Description - Allow locking transactions without getting back a signed transaction. - Allow getting back unsigned effects for previously executed transactions. - Add basic types to support using finalized effects without aggregated effects sigs. - Handle `ConsensusTransactionKind::UserTransaction` in callsites that already handle `ConsensusTransactionKind::CertifiedTransaction`. - Support reconfiguring components other than `QuorumDriver`, in reconfig observers. - Remove some unused code. ## Test plan CI --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:

2024-11-01

[fastpath] support locking transactions and returning effects without signatures

[fastpath] supporting locking transactions and returning effects without signatures

2024-10-31

rest: support protobuf format for committee endpoints

rest: support protobuf format for resolve, simulate, and execute transaction endpoints

2024-10-29

rest: rework proto format to better reuse of common types

rest: introduce protobuf as an acceptable content type Given the BCS format is inherently un-evolvable (unless you utilize enums) it makes a poor wire format as any changes made lead to incompatibilities with existing clients. Today the REST api uses JSON as the default, and human-readable, format and BCS for the binary one. This PR introduces Protobuf as another possible format type and converts the `list_checkpoints` api to support requesting a page of checkpoints in a protobuf format. The plan is to introduce support for protobuf to other apis and to phase out bcs in most places.

2024-08-01

narwhal: manually build grpc interface Manually build grpc interface instead of leveraging protoc in order to eliminate needing to build the expensive protoc compiler.

This commit does not contain any .proto files.

2024-05-16

Consensus changes for SIP-19 (#17508) ## Description Please refer to this SIP draft: https://github.com/sui-foundation/sips/pull/19 The PR focuses only on consensus related changes, covering both Narwhal and Mysticeti. We will proceed to work on the remaining parts once this gets merged. The PR includes a commit from @mwtian that changes `TransactionConsumer` for ack handling in a bundle context. Special thanks to @mwtian for taking time to discussions and authoring the change above! ## Test Plan The feature is not exactly user-facing and do not break anything. ### Type of Change (Check all that apply) - [x] protocol change - [ ] user-visible impact - [ ] breaking change for a client SDKs - [ ] breaking change for FNs (FN binary must upgrade) - [ ] breaking change for validators or node operators (must upgrade binaries) - [ ] breaking change for on-chain data layout - [ ] necessitate either a data wipe or data migration --------- Co-authored-by: Mingwei Tian <mingwei@mystenlabs.com>

2023-07-11

[Narwhal] remove DAGs and network definitions for external consensus (#12950) ## Description 1. Remove two DAG implementions only used by external consensus. 2. Cleanup some network interfaces only used by external consensus. #12946 ## Test Plan CI Narwhal nightly: https://github.com/MystenLabs/sui/actions/runs/5524847205 --- If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process. ### Type of Change (Check all that apply) - [ ] protocol change - [ ] user-visible impact - [ ] breaking change for a client SDKs - [ ] breaking change for FNs (FN binary must upgrade) - [ ] breaking change for validators or node operators (must upgrade binaries) - [ ] breaking change for on-chain data layout - [ ] necessitate either a data wipe or data migration ### Release notes

2023-02-22

[Narwhal] rename send_message RPC to send_certificate (#8520) Rename the `send_message()` RPC to `send_certificate()` which should be more accurate. Add request and response types for `send_certificate()`.

2022-10-06

chore: update license header to remove year

2022-09-13

primary: convert primary-to-worker interface to quic/anemo (MystenLabs/narwhal#976) * primary: convert primary-to-worker interface to quic/anemo * config: remove unused primary-to-worker address and rename worker-to-worker to worker-address * network: remove duplicate code by removing WorkerToPrimaryNetwork * network: rename *Network2 -> *Network

network: convert worker-to-primary interface to quic/anemo (MystenLabs/narwhal#941) Fixes MystenLabs/narwhal#884

2022-09-12

worker: remove unused client-batch-request api This patch removes the unused client-batch-request api given its unused.

worker: convert worker-to-worker network to quic/anemo

2022-09-02

quic: primary-to-primary (MystenLabs/narwhal#668)

2022-08-02

Rename CollectionRetrievalResult field 'batch' to 'transaction_list' (MystenLabs/narwhal#536) * Rename field 'batch' to 'transaction_list' * Remove concept of batches in client facing code * update example docs

2022-08-01

primary: add interface for querying information about a primary's workers

2022-07-29

Add endpoint to retrieve multiaddr of nw primary (MystenLabs/narwhal#590) * add endpoint to publish multiaddr of nw primary * Fix broken test * use cluster in configuration tests * change addr to address * pass name instead of address to grpc server

2022-07-25

[Lint] Run clang-format for protobuf (MystenLabs/narwhal#485) * Add protobuf linting and switch to super-linter * clang-format protobuf * format proto file * fix * fix * . * format proto * . * indent 4 spaces * . * .

2022-06-28

Hookup NewNetworkInfo endpoint with backend (MystenLabs/narwhal#386) * hookup new_network_info endpoint with backend * fix imports * Update /examples docs & proto and check for epoch validity

2022-06-15

Add skeleton for NewEpoch endpoint (MystenLabs/narwhal#351) * Add skeleton for new epoch endpoint * add method to extract and verify public key from request similar to proposer

2022-05-31

Initial template for demo client

2022-05-25

Add NodeReadCausal Endpoint (MystenLabs/narwhal#276) * Add NodeReadCausal Endpoint * remove comments

2022-05-24

Add ReadCausal Endpoint (MystenLabs/narwhal#258) * Add RemoveCollections gRPC endpoint * address review comments * fix tests * Add protos for ReadCausal * Add read causal endpoint * add licesnse * Respond to review comments * Signed Certifcate Test Investigation * fix: reorder dependencies in primary * Remove false assert in broken test * Fix fixture and signed test * cleanup before submit Co-authored-by: François Garillot <francois@garillot.net>

2022-05-19

feat: Introduce the rounds endpoint for the ProposerAPI (MystenLabs/narwhal#253) This commit adds a new gRPC endpoint, the rounds endpoint. It retrieves the rounds that contain available certificates.

2022-05-16

Add NewNetworkInfo endpoint (MystenLabs/narwhal#244) * Add RemoveCollections gRPC endpoint * address review comments * fix tests * Add NewNetworkInfo endpoint * update endpoint doc comment * change epoch number from int32 to uint32 * fix tests after merging with main

2022-05-12

Add RemoveCollections endpoint to Consensus API (MystenLabs/narwhal#224) * Add RemoveCollections gRPC endpoint * address review comments * fix tests * respond to review comments

2022-05-06

sui-network: manually define rpc service Manually define our Validator rpc service instead of relying on a .proto file. This has the advantange that for now we can use types already defined in Rust for wire messages instead of requiring proto definitions or using the awkward double serialization we were currently using.

This commit does not contain any .proto files.

sui-network: manually define rpc service Manually define our Validator rpc service instead of relying on a .proto file. This has the advantange that for now we can use types already defined in Rust for wire messages instead of requiring proto definitions or using the awkward double serialization we were currently using.

This commit does not contain any .proto files.

2022-05-04

Add gRPC Server + get_collections endpoint to Narwhal (MystenLabs/narwhal#140) * Add simple grpc server to primary & add test to hit embedded grpc server in primary * Checkpointing gRPC Server + get_collections endpoint * Switch from build.rs to bootstrap method * Completed get_collections endpoint * Cleanup for review * small typo * fix license headers * update generated narwhal.rs * Refactoring/Changes based on review comments * Only spawn grpc server if external consensus is used & fix broken tests * fix benchmark_client: missed field name update * Add port picker for benchmarking and address other review comments * Fix small typo * rename proto bytes fields with more descriptive names

2022-05-02

worker: convert transactions endpoint to grpc Also include a streaming send to better enable the benchmark client.

types: add proto definitions for narwhal interfaces

network: convert to grpc

network: convert to grpc