Proto commits in MystenLabs/sui

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

Commit:74fa583
Author:Nick

Encapsulate event bitpacking

Commit:11ddb1d
Author:Nick

grpc-loadtest: move build context out of docker/ (meta-svc ignores .dockerignore negations)

Commit:a5c4667
Author:Nick

grpc-loadtest: add k6 build context for meta-svc

Commit:5f20508
Author:wlmyng
Committer:GitHub

[graphql] Migrate Transaction type to use gRPC cursor format (#27035)

Commit:3e0b5d4
Author:Brandon Williams
Committer:Brandon Williams

rpc-store: record package publish checkpoint for point-in-time reads Previously the `package_versions` CF mapped `(original_id, version)` to just the storage id, with no record of when each version was published — so it could not resolve the latest version of a package as of a past checkpoint, which GraphQL needs. Add an optional publish `checkpoint` to `PackageVersionInfo`. Tip indexing records the real checkpoint; the live-set restore writes its rows via the new `store_restored` with the checkpoint left unset (a restore floor), marking versions published before the available window. The new `get_package_at_checkpoint` walks a package's versions and returns the highest whose publish checkpoint is at or before the queried one, treating restore floors as having always existed. Regenerated the proto code with `codegen.rs` and added read-query tests.

Commit:83c27de
Author:Will Yang
Committer:Will Yang

Use proto. bootstrap test so ci can catch drift, and use cargo test -p sui-rpc-cursor to regen migrate cursor-related tests to rust redundant cursor deref, reduce pagination size

Commit:84aad6d
Author:Brandon Williams
Committer:Brandon Williams

rpc-store: record package publish checkpoint for point-in-time reads Previously the `package_versions` CF mapped `(original_id, version)` to just the storage id, with no record of when each version was published — so it could not resolve the latest version of a package as of a past checkpoint, which GraphQL needs. Add an optional publish `checkpoint` to `PackageVersionInfo`. Tip indexing records the real checkpoint; the live-set restore writes its rows via the new `store_restored` with the checkpoint left unset (a restore floor), marking versions published before the available window. The new `get_package_at_checkpoint` walks a package's versions and returns the highest whose publish checkpoint is at or before the queried one, treating restore floors as having always existed. Regenerated the proto code with `codegen.rs` and added read-query tests.

Commit:c143feb
Author:Brandon Williams
Committer:Brandon Williams

rpc-store: resolve checkpoint-pinned reads below the restore anchor Change the `object_version_by_checkpoint` value from a bare version varint to a `Protobuf<ObjectVersionInfo>` carrying the version plus an `optional bool from_restore`. The flag is set only on rows written by the live-set restore at the anchor checkpoint (`store_restored`); tip indexing and backfill leave it unset (`store`), so the common case pays no extra bytes. Extend `get_object_version_at_checkpoint` with a restore fallback. When no entry is at or before the queried checkpoint, the object's earliest row is the only candidate: a restore-floor row (`from_restore` set) means the object predated the available window and was live then, so return it; any other earliest row is a later change or creation, so the object did not exist as of the queried checkpoint. This is what lets a static object's restore floor and an object created in the anchor checkpoint -- which share a single row at the anchor -- be told apart. The function is correct over `[T, ...]` today, where `T` is the restore anchor. Serving `[L, T)` in embedded mode additionally needs the index backfilled over `[L, T]` with synthetic floor rows for objects that change in the window, and its `reader_lo` advertised at `L` -- left to a follow-up.

Commit:9fc9de6
Author:Brandon Williams
Committer:Brandon Williams

rpc-store: record package publish checkpoint for point-in-time reads Previously the `package_versions` CF mapped `(original_id, version)` to just the storage id, with no record of when each version was published — so it could not resolve the latest version of a package as of a past checkpoint, which GraphQL needs. Add an optional publish `checkpoint` to `PackageVersionInfo`. Tip indexing records the real checkpoint; the live-set restore writes its rows via the new `store_restored` with the checkpoint left unset (a restore floor), marking versions published before the available window. The new `get_package_at_checkpoint` walks a package's versions and returns the highest whose publish checkpoint is at or before the queried one, treating restore floors as having always existed. Regenerated the proto code with `codegen.rs` and added read-query tests.

Commit:df3c818
Author:Brandon Williams
Committer:Brandon Williams

rpc-store: resolve checkpoint-pinned reads below the restore anchor Change the `object_version_by_checkpoint` value from a bare version varint to a `Protobuf<ObjectVersionInfo>` carrying the version plus an `optional bool from_restore`. The flag is set only on rows written by the live-set restore at the anchor checkpoint (`store_restored`); tip indexing and backfill leave it unset (`store`), so the common case pays no extra bytes. Extend `get_object_version_at_checkpoint` with a restore fallback. When no entry is at or before the queried checkpoint, the object's earliest row is the only candidate: a restore-floor row (`from_restore` set) means the object predated the available window and was live then, so return it; any other earliest row is a later change or creation, so the object did not exist as of the queried checkpoint. This is what lets a static object's restore floor and an object created in the anchor checkpoint -- which share a single row at the anchor -- be told apart. The function is correct over `[T, ...]` today, where `T` is the restore anchor. Serving `[L, T)` in embedded mode additionally needs the index backfilled over `[L, T]` with synthetic floor rows for objects that change in the window, and its `reader_lo` advertised at `L` -- left to a follow-up.

Commit:3b2b6cf
Author:Brandon Williams
Committer:Brandon Williams

rpc-store: Index end-of-epoch system info into the epochs CF Previously the `epochs` CF recorded only `end_timestamp_ms` and `end_checkpoint` for an epoch's end, leaving it short of the data postgres keeps in `kv_epoch_ends`. This commit brings the CF to parity with the postgres epoch tables so the same epoch fields can be served later. Extend `StoredEpoch` with the end-of-epoch fields: `tx_hi` (the epoch's `network_total_transactions`), `safe_mode`, the `SystemEpochInfoEvent` gas and stake counters (`total_stake`, `storage_fund_balance`, `storage_fund_reinvestment`, `storage_charge`, `storage_rebate`, `stake_subsidy_amount`, `total_gas_fees`, `total_stake_rewards_distributed`, `leftover_storage_fund_inflow`), and the BCS-encoded `epoch_commitments`. The field-wise merge operator copies each new field when present, and the epochs pipeline now extracts them from the boundary checkpoint -- mirroring the postgres `kv_epoch_ends` handler -- recording `safe_mode = true` and leaving the counters unset when no `SystemEpochInfoEvent` is emitted. These fields are written and stored but not yet surfaced through a read path; `get_epoch` is unchanged. Regenerated the proto code with `codegen.rs`, and added merge-storage and event-extraction tests.

Commit:52f433d
Author:Brandon Williams

rpc-store: Index end-of-epoch system info into the epochs CF Previously the `epochs` CF recorded only `end_timestamp_ms` and `end_checkpoint` for an epoch's end, leaving it short of the data postgres keeps in `kv_epoch_ends`. This commit brings the CF to parity with the postgres epoch tables so the same epoch fields can be served later. Extend `StoredEpoch` with the end-of-epoch fields: `tx_hi` (the epoch's `network_total_transactions`), `safe_mode`, the `SystemEpochInfoEvent` gas and stake counters (`total_stake`, `storage_fund_balance`, `storage_fund_reinvestment`, `storage_charge`, `storage_rebate`, `stake_subsidy_amount`, `total_gas_fees`, `total_stake_rewards_distributed`, `leftover_storage_fund_inflow`), and the BCS-encoded `epoch_commitments`. The field-wise merge operator copies each new field when present, and the epochs pipeline now extracts them from the boundary checkpoint -- mirroring the postgres `kv_epoch_ends` handler -- recording `safe_mode = true` and leaving the counters unset when no `SystemEpochInfoEvent` is emitted. These fields are written and stored but not yet surfaced through a read path; `get_epoch` is unchanged. Regenerated the proto code with `codegen.rs`, and added merge-storage and event-extraction tests.

Commit:27474e2
Author:Brandon Williams
Committer:Brandon Williams

rpc-store: resolve checkpoint-pinned reads below the restore anchor Change the `object_version_by_checkpoint` value from a bare version varint to a `Protobuf<ObjectVersionInfo>` carrying the version plus an `optional bool from_restore`. The flag is set only on rows written by the live-set restore at the anchor checkpoint (`store_restored`); tip indexing and backfill leave it unset (`store`), so the common case pays no extra bytes. Extend `get_object_version_at_checkpoint` with a restore fallback. When no entry is at or before the queried checkpoint, the object's earliest row is the only candidate: a restore-floor row (`from_restore` set) means the object predated the available window and was live then, so return it; any other earliest row is a later change or creation, so the object did not exist as of the queried checkpoint. This is what lets a static object's restore floor and an object created in the anchor checkpoint -- which share a single row at the anchor -- be told apart. The function is correct over `[T, ...]` today, where `T` is the restore anchor. Serving `[L, T)` in embedded mode additionally needs the index backfilled over `[L, T]` with synthetic floor rows for objects that change in the window, and its `reader_lo` advertised at `L` -- left to a follow-up.

Commit:6637e98
Author:Brandon Williams
Committer:GitHub

rpc-store: introduce sui-rpc-store crate (#26978) A storage backend for `sui-rpc-api`, built on `sui-consistent-store`, that hosts the column families backing every read the RPC service performs. It consolidates raw chain data previously served by the validator's perpetual / checkpoint / committee stores together with the indexes previously served by `sui-core::rpc_index` and `sui-indexer-alt-consistent-store`. The crate is organized into: - `schema` — the column-family layout. Each CF declares its on-disk key and value types plus per-CF RocksDB options. Covers raw chain data (objects, transactions, effects, events, checkpoints) and the indexes (owner, dynamic field, coin balance, package versions, epochs, ledger history, transaction / event bitmaps). - `indexer` — sequential `sui-indexer-alt-framework` pipelines that populate the schema from observed checkpoints, the `Indexer` orchestrator that wires them against a shared `Synchronizer`, plus the `pruner` and `restore` drivers for history pruning and bulk-load from a formal snapshot. - `reader` — `RpcStoreReader`, which exposes the schema through the trait stack `sui-rpc-api` consumes (`ObjectStore`, `ReadStore`, `ChildObjectResolver`, `RpcStateReader`, and `RpcIndexes`). Generic over a `Reader` so one type serves both tip reads and reads bound to a captured snapshot. - `proto` — bespoke protobuf value messages defined under `proto/sui/rpc_store/`, generated via `codegen.rs`. - `config` — `ServiceConfig` and the layered pipeline / committer / pruner / restore configuration. `start_indexer` is the standalone-binary entry point; the embedded fullnode path constructs `Indexer::from_store` directly with its own ingestion and streaming clients.

Commit:9c245c2
Author:Brandon Williams
Committer:Brandon Williams

rpc-store: introduce sui-rpc-store crate A storage backend for `sui-rpc-api`, built on `sui-consistent-store`, that hosts the column families backing every read the RPC service performs. It consolidates raw chain data previously served by the validator's perpetual / checkpoint / committee stores together with the indexes previously served by `sui-core::rpc_index` and `sui-indexer-alt-consistent-store`. The crate is organized into: - `schema` — the column-family layout. Each CF declares its on-disk key and value types plus per-CF RocksDB options. Covers raw chain data (objects, transactions, effects, events, checkpoints) and the indexes (owner, dynamic field, coin balance, package versions, epochs, ledger history, transaction / event bitmaps). - `indexer` — sequential `sui-indexer-alt-framework` pipelines that populate the schema from observed checkpoints, the `Indexer` orchestrator that wires them against a shared `Synchronizer`, plus the `pruner` and `restore` drivers for history pruning and bulk-load from a formal snapshot. - `reader` — `RpcStoreReader`, which exposes the schema through the trait stack `sui-rpc-api` consumes (`ObjectStore`, `ReadStore`, `ChildObjectResolver`, `RpcStateReader`, and `RpcIndexes`). Generic over a `Reader` so one type serves both tip reads and reads bound to a captured snapshot. - `proto` — bespoke protobuf value messages defined under `proto/sui/rpc_store/`, generated via `codegen.rs`. - `config` — `ServiceConfig` and the layered pipeline / committer / pruner / restore configuration. `start_indexer` is the standalone-binary entry point; the embedded fullnode path constructs `Indexer::from_store` directly with its own ingestion and streaming clients.

Commit:43aa508
Author:Brandon Williams
Committer:Brandon Williams

consistent-store: introduce sui-consistent-store crate A type-safe RocksDB wrapper that will back Sui's on-disk indexes, intended to replace the ad-hoc `typed-store`-style patterns in `sui-core::rpc_index` and `sui-indexer-alt-consistent-store` over time. The crate provides: - `Encode` and `Decode` traits, plus a `Protobuf<T>` wrapper that forwards them through `prost::Message`. Encoding is append-only via `BufMut` so multiple values can share one buffer. - `Db`, a cheap-to-clone handle around an opened RocksDB database carrying a bounded in-memory snapshot buffer keyed by checkpoint. - `Schema` and `SchemaAtSnapshot` for declaring column families and re-binding a struct of typed handles at a captured snapshot. The framework's bookkeeping CFs (restore state, watermark, chain id) are auto-registered and reachable via `Db::framework`. - `DbMap<K, V, R>`, the primary read/write surface — typed point gets (with zero-copy `get_raw` backed by `bytes::Bytes`), typed iteration with prefix-typed bounds, and the staging methods `Batch` uses. Parameterized by a sealed `Reader` so the same handle serves both live-tip and snapshot-bound reads, including `&Db` / `&Snapshot` zero-`Arc`-bump variants. - `Batch`, the atomic typed write batch: `put`, `delete`, `merge` across one or more CFs, committed together. - An indexer-alt-framework adapter: `Store<S>` + `Connection<'_, S>` implement `Store` and `SequentialStore` and route pipelines through a `Synchronizer` that coordinates writes from multiple sequential pipelines and takes cross-pipeline snapshots at stride boundaries. `Restore` is the sibling shape for bulk-load drivers (formal snapshot, perpetual store). - `ColumnFamilyStatsCollector`, a `prometheus::Collector` that emits per-CF RocksDB integer properties as labeled gauges, holding a weak handle so the collector does not pin the database alive. Internal Sui dependencies are limited to the indexer-alt-framework store traits and `sui-types`; the rest is `rocksdb` (with snappy, lz4, zstd, zlib, and multi-threaded CF support), `prost`, `bytes`, `parking_lot`, and `tracing`. Tests run against real `tempfile`-backed databases.

Commit:8eccec6
Author:Brandon Williams
Committer:Brandon Williams

sui-rpc-api: replace alpha proof/event services with v2alpha Previously, sui-rpc-api shipped its own alpha `ProofService` and `EventService` to back the in-tree authenticated-events light client. With the v2alpha `ProofService` (`GetCheckpointObjectProof`, with both inclusion and non-inclusion proofs) and `LedgerService.ListEvents` (filtered by `EventStreamHeadFilter`) now defined in the `sui-rpc` SDK, the in-tree stubs are obsolete. With this commit: - the alpha proof and event services are removed from `sui-rpc-api` and the v2alpha `ProofService` is mounted instead, backed by `sui_sdk_types::merkle::MerkleTree` so the SDK-side proto conversions carry the wire representation; - the rpc-index column family `events_by_stream` plus its compaction filter, `EventIndexKey`, `event_iter`, and `authenticated_event_iter` are dropped, along with `RpcConfig::authenticated_events_indexing`; authenticated event lookups now go through the existing `ledger_history_indexing` bitmap path, whose backfill already maintains the `EventStreamHead` bitmap dimension; - the in-tree prototype `sui-light-client` is ported to the new APIs, with cursor-based pagination buffering events across pages until each containing checkpoint is fully covered before folding into the MMR. The on-chain MMR is one fold per checkpoint, so partial-checkpoint folds would never match the on-chain head; - the e2e tests for authenticated events and object inclusion proofs are rewritten against the v2alpha surface, and the `corrupt_authenticated_event` fail-point moves into the new `render_event_chunk` path so the existing MMR-mismatch simtest keeps working.

Commit:3a97028
Author:stefan-mysten
Committer:GitHub

[forking] allow impersonating a sender 17/n (#26485) ## Description This PR allows to execute a transaction as an address for which we do not have the keys. The Sui CLI now has a `--forking-mode` flag to bypass the signatures check before submitting the transaction execution. Finally, it renames the crate to `sui-fork`. ## Test plan Existing CI --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [ ] GraphQL: - [x] CLI: Sui CLI has a new `--forking-mode` flag to be used with the new `sui-fork` tool to allow impersonate a sender. - [ ] Rust SDK: - [ ] Indexing Framework:

Commit:fbb66b7
Author:Nick
Committer:Nick

Bigtable Bitmap Indexes

Commit:0af885e
Author:stefan-mysten
Committer:GitHub

[forking 10/n] add forking service grpc for interacting with the network (#26328) ## Description This PR adds a proto description for the ForkingService that enables users to interact with the forked network through the common RPC apis. The main new APIs are: ``` sui.forking.ForkingService/GetStatus sui.forking.ForkingService/AdvanceClock sui.forking.ForkingService/AdvanceCheckpoint ``` The changes to `sui-rpc-api` are now pushed in a different PR (https://github.com/MystenLabs/sui/pull/26327), so I plan to merge that first before this one, and remove the changes here. ## Test plan Existing tests. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] Indexing Framework:

Commit:45720b7
Author:Nick
Committer:Nick

Bigtable Bitmap Indexes

Commit:219dd88
Author:alex-mysten
Committer:alex-mysten

Switch ProofService and EventService from alpha to v2 proto definitions

Commit:5b9dfa1
Author:alex-mysten
Committer:Alexander Stek

Switch ProofService and EventService from alpha to v2 proto definitions

Commit:df0b148
Author:wlmyng
Committer:GitHub

[jsonrpc-alt] Migrate to read from consistent-store instead of `obj_info` (#25159) ## Description jsonrpc-alt uses `obj_info` to filter for owned objects by a type prefix, and to support queries for owned objects not of a certain type (slush requirement.) However, the latter is quite inefficient as it requires a sequential scan through the postgres db, and we have a consistent-store that can satisfy these queries, so this PR migrates jsonrpc-alt over to read from consistent-store instead of `obj_info` to unblock getting rid of the last consistent pipelines in postgres. This PR implements a basic form of `MatchNone`, which just supports "owned objects not of this type". We plan to support type inclusion/exclusion filters eventually. From consistent-store up: - `skip_prefix` in the forward and reverse iter to skip past a provided prefix - in pagination.rs, `paginate_from_front` and its equivalent additionally skips past entries starting with `exclude_prefix` - in consistent-store's `list_owned_objects.rs`, we assume a base prefix of `OwnerKind`, and a single exclusion filter on object type - `ListOwnedObjectsRequest` has new field `optional string exclude_object_type = 3;` - consistent service returns new error if both `object_type` and `exclude_object_type` are provided - consistent reader `list_owned_objects` method has additional `exclude_object_type` ## Test plan Existing tests in `jsonrpc_compound_object_filter.rs`, unit tests in `iter.rs` and `pagination.rs`. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] Indexing Framework:

Commit:68db897
Author:Andrew Schran
Committer:GitHub

Add per-commit binary logging for congestion control (#25398) ## Description - Includes per-object and per-tx cost details. - Saves logged proto files to disk. - Default size limit 10x 100MB files. ## Test plan Unit tests --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] Indexing Framework:

Commit:1e1c68e
Author:Alex Stek | Mysten Labs
Committer:GitHub

[light-client] authenticated events client (#24226) ## Description Client for authenticated events, which allows joining an event stream at the present moment and resuming an event stream from a previously validated checkpoint (in case of interruption). A few reviewer considerations: 1. At present we do not have a mechanism for replaying archival (pruned) events. This client solely allows joining (and rejoining, in case of temporary interruption) a stream of incoming events. To date, retrieving archival events still requires downloading and processing checkpoints. 2. Trust ratcheting requires that we fetch the last checkpoint of the epoch to verify the next epoch's committee, from genesis to the current epoch. This presumes that the checkpoint data is unpruned on the full node. 3. Resuming a stream requires that the last checkpoint is one where events were emitted and the EventStreamHead was updated. The use case is the scenario where a client goes down and needs to resume from the last verified checkpoint where we received events. It is not currently possible to start the stream from an arbitrary unpruned checkpoint. 4. This light client is tightly coupled to sui_types and other dependencies within the sui repo. We should consider the right long-term home for this implementation (sui-rust-sdk?) ## Test plan New e2e tests added. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK:

Commit:7267c60
Author:wlmyng
Committer:GitHub

Cherry pick #24963 exposing address balances from consistent service up to graphql (#25025) ## Description New `Balance.address_balance` and `Balance.coin_balance` fields to reflect the two possible sources of balance for an address. `Balance.total_balance` will return the sum of address and coin balances. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [x] GraphQL: `Balance.totalBalance` now returns the sum of balances from owned coins and from the accumulator object. The individual coin or address balances can be retrieved through `Balance.coinBalance` and `Balance.addressBalance` respectively. For the previous behavior, select the `Balance.coinBalance` field for coin balances only. - [ ] CLI: - [ ] Rust SDK: - [ ] Indexing Framework:

Commit:ff5565a
Author:wlmyng
Committer:GitHub

Expose address balances from consistent service up through graphql (#24963)

Commit:7780807
Author:wlmyng
Committer:GitHub

[consistent-store] Index address balances (#24713)

Commit:1baebbb
Author:Alex Stek | Mysten Labs
Committer:GitHub

[light-client] index accumulator version on events (#24304) ## Description Index accumulator version on events. When building the MMR for authenticated event verification we need to group events by commit rather than checkpoint as there may be multiple commits / settlements per checkpoint. ## Test plan Added a simtest to create multiple commits per checkpoint. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] Indexing Framework:

Commit:13b6a21
Author:stefan-mysten
Committer:GitHub

[pkg-alt] merge main into sui-pkg-alt (#24395)

Commit:0245ae7
Author:Alex Stek | Mysten Labs
Committer:GitHub

get_obj_inclusion_proof API and e2e light-client test (#23960) ## Description Add a new API to fetch an object version at a given checkpoint sequence number along with an inclusion proof the object was written during that checkpoint sequence number. ## Test plan Light client e2e tested + get_object_inclusion_proof happy / unhappy cases --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK:

Commit:9592048
Author:Brandon Williams
Committer:GitHub

chore: update tonic to 0.14 (#24032) This patch updates tonic to 0.14 as well as adds support for building the bigtable client from proto files directly (instead of just having a the generated protos checked in).

Commit:ff78ac0
Author:Alexander Stek
Committer:Alexander Stek

get_obj_inclusion_proof API and e2e light-client test

Commit:55c49bd
Author:Alexander Stek
Committer:Alexander Stek

get_obj_inclusion_proof API and e2e light-client test

Commit:0f99f92
Author:Alexander Stek
Committer:Alexander Stek

get_obj_inclusion_proof API and e2e light-client test

Commit:44fce9a
Author:Alex Stek | Mysten Labs
Committer:GitHub

Authenticated Event Streams - ListAuthenticatedEvents API & index (#23560) ## Description Created a new full node gRPC API to query authenticated events given a stream_id, a checkpoint seq, and limit. Todo: Resolve API model for an event stream inclusion proof & integrate prover, add more e2e tests including proofs & multiple events per txn ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK:

Commit:b5fe4ed
Author:alex.stek
Committer:alex.stek

Refer to stable Event proto type; index events within txn indexing

Commit:34de0e5
Author:Alexander Stek
Committer:alex.stek

paginate based on event counts. remove proofs

Commit:0a6f5f8
Author:Alexander Stek
Committer:alex.stek

authenticated event streams RPC + Index

Commit:ffa6de4
Author:Alexander Stek
Committer:Alexander Stek

Always query on a fixed range, allow optionally specifying end checkpoint

Commit:b073de0
Author:alex.stek
Committer:Alexander Stek

Refer to stable Event proto type; index events within txn indexing

Commit:bc16ba0
Author:Alexander Stek
Committer:Alexander Stek

paginate based on event counts. remove proofs

Commit:fc9ed9a
Author:Alexander Stek
Committer:Alexander Stek

authenticated event streams RPC + Index

Commit:c45799a
Author:Alexander Stek
Committer:Alexander Stek

authenticated events light client end to end demo

Commit:589140a
Author:Alexander Stek
Committer:Alexander Stek

authenticated events light client end to end demo

Commit:018fb97
Author:Alexander Stek
Committer:alex.stek

paginate based on event counts. remove proofs

Commit:5279fe7
Author:Alexander Stek
Committer:alex.stek

authenticated event streams RPC + Index

Commit:17657a5
Author:alex.stek
Committer:alex.stek

Refer to stable Event proto type; index events within txn indexing

Commit:45edcc6
Author:alex.stek
Committer:alex.stek

Refer to stable Event proto type; index events within txn indexing

Commit:e06b61a
Author:Alexander Stek
Committer:alex.stek

paginate based on event counts. remove proofs

Commit:e9dea28
Author:Alexander Stek
Committer:alex.stek

authenticated event streams RPC + Index

Commit:dba3ca9
Author:alex.stek
Committer:alex.stek

Refer to stable Event proto type; index events within txn indexing

Commit:a33e51b
Author:Alexander Stek
Committer:alex.stek

paginate based on event counts. remove proofs

Commit:889a1cf
Author:Alexander Stek
Committer:alex.stek

authenticated event streams RPC + Index

Commit:931e812
Author:Alexander Stek
Committer:Alexander Stek

paginate based on event counts. remove proofs

Commit:ecfc78d
Author:Alexander Stek
Committer:Alexander Stek

authenticated event streams RPC + Index

Commit:c6a9533
Author:Alexander Stek
Committer:Alexander Stek

paginate based on event counts. remove proofs

Commit:86a00f4
Author:Alexander Stek
Committer:Alexander Stek

authenticated event streams RPC + Index

Commit:6c01876
Author:Alexander Stek
Committer:Alexander Stek

authenticated event streams RPC + Index

Commit:12839c0
Author:Alexander Stek

switch from using transaction Digest to idx

Commit:de847e0
Author:Alexander Stek
Committer:Alexander Stek

rename api to ListAuthenticatedEvents to conform to RPC

Commit:dfcb1b2
Author:Alexander Stek

rename Event

Commit:617d9df
Author:Alexander Stek
Committer:Alexander Stek

generate rpc api within monorepo

Commit:583ea9a
Author:Tim Zakian
Committer:GitHub

Merge `main` at `91a15fe19e` into `bella-ciao` branch (#23430) ## Description Merge of `main` (at `91a15fe19e`) into the `bella-ciao` branch. NB: I _have not_ merged anything outside of `external-crates` `sui-types` and `sui-move-natives` pretty much in this PR. In particular, you will still see `HEAD` markers in e.g., the adapter or possibly other places. This will be handled in a separate PR (and we can discuss more at the bella-ciao meeting later today). ## Test plan Make sure `external-crates` is green, rest will be red. --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] gRPC: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: --------- Signed-off-by: longhutianjie <keplrnewton@icloud.com> Signed-off-by: liuchuan <582041246@qq.com> Co-authored-by: Ashok Menon <ashok@mystenlabs.com> Co-authored-by: Brandon Williams <brandon@mystenlabs.com> Co-authored-by: stefan-mysten <135084671+stefan-mysten@users.noreply.github.com> Co-authored-by: Mark Logan <103447440+mystenmark@users.noreply.github.com> Co-authored-by: wlmyng <127570466+wlmyng@users.noreply.github.com> Co-authored-by: Andrew Schran <aschran@mystenlabs.com> Co-authored-by: Todd Nowacki <tmn@mystenlabs.com> Co-authored-by: Joey Pham <joey.pham@mystenlabs.com> Co-authored-by: Xun Li <lxfind@gmail.com> Co-authored-by: Bridgerz <bridgerzoske@gmail.com> Co-authored-by: Adam Welc <adam@mystenlabs.com> Co-authored-by: Debirth <44721066+zcy1024@users.noreply.github.com> Co-authored-by: Mark Logan <mark@mystenlabs.com> Co-authored-by: Dario Russi <113150618+dariorussi@users.noreply.github.com> Co-authored-by: Alex Stek | Mysten Labs <alex.stek@mystenlabs.com> Co-authored-by: Damir Shamanaev <damirka.ru@gmail.com> Co-authored-by: Michael D. George <mike.george@mystenlabs.com> Co-authored-by: John Martin <johnjosephmartin@icloud.com> Co-authored-by: mwtian <81660174+mwtian@users.noreply.github.com> Co-authored-by: lazymio <ziqiaokong@gmail.com> Co-authored-by: Cam Swords <cameronswords@gmail.com> Co-authored-by: Deepak Maram <deepak@mystenlabs.com> Co-authored-by: phoenix <51927076+phoenix-o@users.noreply.github.com> Co-authored-by: Evan Wall <evan.wall@mystenlabs.com> Co-authored-by: akmysti <222086987+akmysti@users.noreply.github.com> Co-authored-by: Eugene Boguslavsky <eugene@mystenlabs.com> Co-authored-by: ronny-mysten <118224482+ronny-mysten@users.noreply.github.com> Co-authored-by: Rifvck Zieger <rifvckzieger@gmail.com> Co-authored-by: Jessie Mongeon <133128541+jessiemongeon1@users.noreply.github.com> Co-authored-by: Arun Koshy <97870774+arun-koshy@users.noreply.github.com> Co-authored-by: longhutianjie <keplrnewton@icloud.com> Co-authored-by: Jort <jordan.jennings@mystenlabs.com> Co-authored-by: Emma Zhong <emma@mystenlabs.com> Co-authored-by: Will Bradley <will.bradley@mystenlabs.com> Co-authored-by: Ronny Roland <ronmilton@mystenlabs.com> Co-authored-by: hoffman <bettthoffman@gmail.com> Co-authored-by: Nick Vikeras <nickvikeras@gmail.com> Co-authored-by: henryachen <henryachen93@gmail.com> Co-authored-by: cg <146006860+chris-gorham@users.noreply.github.com> Co-authored-by: Abhinav Garg <abhinavg6@gmail.com> Co-authored-by: Abhinav Garg <abhi.garg@mystenlabs.com> Co-authored-by: Radovenchyk <raddole@ukr.net> Co-authored-by: Rej Ect <99460023+rejected-l@users.noreply.github.com> Co-authored-by: Andrey Chursin <andrey@mystenlabs.com> Co-authored-by: Henry Chen <henry.chen@mystenlabs.com> Co-authored-by: Bob Liu <akagi201@gmail.com> Co-authored-by: Manolis Liolios <manos@mystenlabs.com> Co-authored-by: nikos-terzo <139557324+nikos-terzo@users.noreply.github.com> Co-authored-by: Sang Kim <nonvis@gmail.com> Co-authored-by: nikos-terzo <nikos.terzo@mystenlabs.com> Co-authored-by: leo <lc582041246@gmail.com> Co-authored-by: Mingwei Tian <mingwei@mystenlabs.com> Co-authored-by: Anastasios Kichidis <akihidis@gmail.com> Co-authored-by: Jordan Gensler <jordan@mystenlabs.com> Co-authored-by: William Robertson <williamrobertson13@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Will Yang <willyang@mystenlabs.com>

The documentation is generated from this commit.

Commit:87c6030
Author:Bridgerz
Committer:Bridgerz

Add Coin Registry System Object

Commit:9242d77
Author:Ashok Menon
Committer:Ashok Menon

consistency: naming and docs changes ## Description Clarify some names and docs in the consistent store. ## Test plan Existing tests: ``` $ cargo nextest run \ -p sui-indexer-alt-e2e-tests \ -p sui-indexer-alt-consistent-api \ -p sui-indexer-alt-consistent-store \ -p sui-indexer-alt-graphql ```

The documentation is generated from this commit.

Commit:b6497d3
Author:Ashok Menon
Committer:Ashok Menon

consistency: AvailableRange returns full high watermark ## Description Expose the extra information that has now been exposed per snapshot, to help with the GraphQL integration. ## Test plan Manually tested.

Commit:dd3bbf3
Author:Bridgerz
Committer:Bridgerz

Add Coin Registry System Object

Commit:fef7a19
Author:Bridgerz
Committer:Bridgerz

Add Coin Registry System Object

Commit:3fad59c
Author:Ashok Menon
Committer:Ashok Menon

consistency: naming and docs changes ## Description Clarify some names and docs in the consistent store. ## Test plan Existing tests: ``` $ cargo nextest run \ -p sui-indexer-alt-e2e-tests \ -p sui-indexer-alt-consistent-api \ -p sui-indexer-alt-consistent-store \ -p sui-indexer-alt-graphql ```

The documentation is generated from this commit.

Commit:a0f18d1
Author:Ashok Menon
Committer:Ashok Menon

consistency: AvailableRange returns full high watermark ## Description Expose the extra information that has now been exposed per snapshot, to help with the GraphQL integration. ## Test plan Manually tested.

Commit:0527e3d
Author:Bridgerz
Committer:Bridgerz

Add Coin Registry System Object

Commit:fd7756a
Author:Bridgerz
Committer:Bridgerz

Add Coin Registry System Object

Commit:0aef14e
Author:Ashok Menon
Committer:Ashok Menon

consistency: naming and docs changes ## Description Clarify some names and docs in the consistent store. ## Test plan Existing tests: ``` $ cargo nextest run \ -p sui-indexer-alt-e2e-tests \ -p sui-indexer-alt-consistent-api \ -p sui-indexer-alt-consistent-store \ -p sui-indexer-alt-graphql ```

The documentation is generated from this commit.

Commit:cab0e7b
Author:Ashok Menon
Committer:Ashok Menon

consistency: AvailableRange returns full high watermark ## Description Expose the extra information that has now been exposed per snapshot, to help with the GraphQL integration. ## Test plan Manually tested.

Commit:fe9f504
Author:Brandon Williams
Committer:GitHub

grpc: remove old v2beta service (#23082)

Commit:ba8c20b
Author:Ashok Menon
Committer:Ashok Menon

consistency: AvailableRange returns full high watermark ## Description Expose the extra information that has now been exposed per snapshot, to help with the GraphQL integration. ## Test plan Manually tested.

Commit:af5abda
Author:Andrew Schran
Committer:Andrew Schran

Update names of consensus-related types in TransactionEffects across the codebase Names changed to reflect that there are now non-shared consensus objects: `UnchangedSharedKind` -> `UnchangedConsensusKind` `InputSharedObject` -> `InputConsensusObject` Names also updated in APIs accordingly to match the equivalent type names in sui-core. Names in JSON RPC left alone to maintain backwards compatibility until it is eventually deprecated.

Commit:28554c6
Author:Andrew Schran

Update names of consensus-related types in TransactionEffects across the codebase Names changed to reflect that there are now non-shared consensus objects: `UnchangedSharedKind` -> `UnchangedConsensusKind` `InputSharedObject` -> `InputConsensusObject` Names also updated in APIs accordingly to match the equivalent type names in sui-core. Names in JSON RPC left alone to maintain backwards compatibility until it is eventually deprecated.

Commit:dbb5fda
Author:Ashok Menon
Committer:Ashok Menon

consistency: ListObjectsByType ## Description Add a new pipeline for indexing objects by (just) their type, and expose it through gRPC through a new RPC. ## Test plan New E2E test: ``` $ cargo nextest run \ -p sui-indexer-alt-consistent-store \ --test consistent_store_list_objects_by_type_tests ```

Commit:03a2b7a
Author:Ashok Menon
Committer:Ashok Menon

consistency: expose stride through AvailableRange ## Description The stride is the gap between checkpoints at which the service takes snapshots. If GraphQL is combining data from a consistent store with other stores, it needs to know how often the store takes snapshots in case another store is lagging behind the consistent store, and it needs to re-align all data on some earlier checkpoint. ## Test plan Manually tested.

Commit:f7e10ac
Author:Ashok Menon
Committer:Ashok Menon

consistency: BatchGetBalances ## Description Support multi-gets for balance requests. This change also - introduces an `owner` field for the `Balance` type used in all balance responses, - introduces a 'max_batch_size' configuration that is also exposed through the service config, - fixes handling for "not in range" errors from the database for `GetBalance` (and implements them correctly the first time for `BatchGetBalances`) so that they aren't treated as internal errors. Tests have been added for this behaviour. ## Test plan Updated E2E tests for batch gets ``` $ cargo nextest run \ -p sui-indexer-alt-e2e-tests \ --test consistent_store_balance_tests ```

Commit:344ab34
Author:Ashok Menon
Committer:Ashok Menon

consistency: GetBalance ## Description Point lookup for an individual balance, by owner and type. ## Test plan New E2E tests: ``` $ cargo nextest run \ -p sui-indexer-alt-e2e-tests \ --test consistent_store_balance_tests ```

Commit:8c23da9
Author:Ashok Menon
Committer:Ashok Menon

consistency: ListBalances ## Description Add an indexing pipeline for address balances, and RPCs for listing (paginated) balances. Balance indexing uses RocksDB's merge operations so that the pipeline can commit balance deltas, and leave the database to accumulate them (also creating new records and deleting empty records). An API for getting a single balance, and for multi-getting balances will follow. ## Test plan New E2E tests: ``` cargo nextest run \ -p sui-indexer-alt-e2e-tests \ --test consistent_store_balance_tests ```

Commit:7ac54ea
Author:Ashok Menon
Committer:Ashok Menon

consistency: expose stride through AvailableRange ## Description The stride is the gap between checkpoints at which the service takes snapshots. If GraphQL is combining data from a consistent store with other stores, it needs to know how often the store takes snapshots in case another store is lagging behind the consistent store, and it needs to re-align all data on some earlier checkpoint. ## Test plan Manually tested.

Commit:4a16b56
Author:Ashok Menon
Committer:Ashok Menon

consistency: GetBalance ## Description Point lookup for an individual balance, by owner and type. ## Test plan New E2E tests: ``` $ cargo nextest run \ -p sui-indexer-alt-e2e-tests \ --test consistent_store_balance_tests ```

Commit:206436c
Author:Ashok Menon
Committer:Ashok Menon

consistency: BatchGetBalances ## Description Support multi-gets for balance requests. This change also - introduces an `owner` field for the `Balance` type used in all balance responses, - introduces a 'max_batch_size' configuration that is also exposed through the service config, - fixes handling for "not in range" errors from the database for `GetBalance` (and implements them correctly the first time for `BatchGetBalances`) so that they aren't treated as internal errors. Tests have been added for this behaviour. ## Test plan Updated E2E tests for batch gets ``` $ cargo nextest run \ -p sui-indexer-alt-e2e-tests \ --test consistent_store_balance_tests ```

Commit:aeafff6
Author:Ashok Menon
Committer:Ashok Menon

consistency: ListBalances ## Description Add an indexing pipeline for address balances, and RPCs for listing (paginated) balances. Balance indexing uses RocksDB's merge operations so that the pipeline can commit balance deltas, and leave the database to accumulate them (also creating new records and deleting empty records). An API for getting a single balance, and for multi-getting balances will follow. ## Test plan New E2E tests: ``` cargo nextest run \ -p sui-indexer-alt-e2e-tests \ --test consistent_store_balance_tests ```

Commit:b23ef78
Author:Ashok Menon
Committer:Ashok Menon

consistency: ListObjectsByType ## Description Add a new pipeline for indexing objects by (just) their type, and expose it through gRPC through a new RPC. ## Test plan New E2E test: ``` $ cargo nextest run \ -p sui-indexer-alt-consistent-store \ --test consistent_store_list_objects_by_type_tests ```

Commit:b8c8a8e
Author:Ashok Menon
Committer:Ashok Menon

consistency: integrate into e2e tests ## Description Add the consistent service to the E2E test cluster. All E2E tests will now wait for the service to catch up, but none of them query the service yet. This required adding a new method to return the available checkpoint range (used by the test runner to check that the consistent store indexer has caught up). ## Test plan Existing E2E tests: ``` $ cargo nextest run -p sui-indexer-alt-consistent-store ```

Commit:570df06
Author:Ashok Menon
Committer:Ashok Menon

consistency: initial gRPC service interface ## Description Build an initial interface for a consistent service, supporting just ListOwnedObjects. ## Test plan Initial (basic) protobuf generation tests.

Commit:0b6565e
Author:Ashok Menon
Committer:Ashok Menon

consistency: configure page sizes and expose through gRPC ## Description Add configuration for default and max page size to the consistent service and expose the configuration through a new gRPC method (`ServiceConfig`). This configuration is not yet used by anything. The proto schema changes also include a minor rename (`START` and `END` become `FRONT` and `BACK`) to match the nomenclature used in GraphQL. ## Test plan Manually tested: ``` $ grpcurl -v -plaintext localhost:7001 sui.rpc.consistent.v1alpha.ConsistentService/ServiceConfig Resolved method descriptor: rpc ServiceConfig ( .sui.rpc.consistent.v1alpha.ServiceConfigRequest ) returns ( .sui.rpc.consistent.v1alpha.ServiceConfigResponse ); Request metadata to send: (empty) Response headers received: content-type: application/grpc date: Mon, 21 Jul 2025 14:11:42 GMT x-sui-rpc-version: 1.52.0-672319c9b78c-dirty Response contents: { "defaultPageSize": 50, "maxPageSize": 200 } Response trailers received: (empty) Sent 0 requests and received 1 response ```

Commit:8f98f1f
Author:Ashok Menon
Committer:Ashok Menon

consistency: expose stride through AvailableRange ## Description The stride is the gap between checkpoints at which the service takes snapshots. If GraphQL is combining data from a consistent store with other stores, it needs to know how often the store takes snapshots in case another store is lagging behind the consistent store, and it needs to re-align all data on some earlier checkpoint. ## Test plan Manually tested.

Commit:6578eb4
Author:Ashok Menon
Committer:Ashok Menon

consistency: BatchGetBalances ## Description Support multi-gets for balance requests. This change also - introduces an `owner` field for the `Balance` type used in all balance responses, - introduces a 'max_batch_size' configuration that is also exposed through the service config, - fixes handling for "not in range" errors from the database for `GetBalance` (and implements them correctly the first time for `BatchGetBalances`) so that they aren't treated as internal errors. Tests have been added for this behaviour. ## Test plan Updated E2E tests for batch gets ``` $ cargo nextest run \ -p sui-indexer-alt-e2e-tests \ --test consistent_store_balance_tests ```

Commit:548d860
Author:Ashok Menon
Committer:Ashok Menon

consistency: GetBalance ## Description Point lookup for an individual balance, by owner and type. ## Test plan New E2E tests: ``` $ cargo nextest run \ -p sui-indexer-alt-e2e-tests \ --test consistent_store_balance_tests ```

Commit:e501e7d
Author:Ashok Menon
Committer:Ashok Menon

consistency: ListBalances ## Description Add an indexing pipeline for address balances, and RPCs for listing (paginated) balances. Balance indexing uses RocksDB's merge operations so that the pipeline can commit balance deltas, and leave the database to accumulate them (also creating new records and deleting empty records). An API for getting a single balance, and for multi-getting balances will follow. ## Test plan New E2E tests: ``` cargo nextest run \ -p sui-indexer-alt-e2e-tests \ --test consistent_store_balance_tests ```

Commit:7d4788d
Author:Ashok Menon
Committer:Ashok Menon

consistency: ListObjectsByType ## Description Add a new pipeline for indexing objects by (just) their type, and expose it through gRPC through a new RPC. ## Test plan New E2E test: ``` $ cargo nextest run \ -p sui-indexer-alt-consistent-store \ --test consistent_store_list_objects_by_type_tests ```

Commit:860f7ce
Author:Ashok Menon
Committer:Ashok Menon

consistency: integrate into e2e tests ## Description Add the consistent service to the E2E test cluster. All E2E tests will now wait for the service to catch up, but none of them query the service yet. This required adding a new method to return the available checkpoint range (used by the test runner to check that the consistent store indexer has caught up). ## Test plan Existing E2E tests: ``` $ cargo nextest run -p sui-indexer-alt-consistent-store ```

Commit:1a7c96e
Author:Ashok Menon
Committer:Ashok Menon

consistency: configure page sizes and expose through gRPC ## Description Add configuration for default and max page size to the consistent service and expose the configuration through a new gRPC method (`ServiceConfig`). This configuration is not yet used by anything. The proto schema changes also include a minor rename (`START` and `END` become `FRONT` and `BACK`) to match the nomenclature used in GraphQL. ## Test plan Manually tested: ``` $ grpcurl -v -plaintext localhost:7001 sui.rpc.consistent.v1alpha.ConsistentService/ServiceConfig Resolved method descriptor: rpc ServiceConfig ( .sui.rpc.consistent.v1alpha.ServiceConfigRequest ) returns ( .sui.rpc.consistent.v1alpha.ServiceConfigResponse ); Request metadata to send: (empty) Response headers received: content-type: application/grpc date: Mon, 21 Jul 2025 14:11:42 GMT x-sui-rpc-version: 1.52.0-672319c9b78c-dirty Response contents: { "defaultPageSize": 50, "maxPageSize": 200 } Response trailers received: (empty) Sent 0 requests and received 1 response ```