Proto commits in farcasterxyz/snapchain-v0

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

2026-06-11

feat(mesh): network-wide topology crawl over the gossip port (#928) Add an admin-gated crawl that assembles a network-wide view of the validator mesh by querying each connected validator for its local mesh view and aggregating the responses. - gossip: a sibling request-response behaviour for mesh diagnostics (/snapchain/diagnostics/mesh/1.0.0), separate from the Malachite sync rpc so the wire format can evolve independently. The responder serves only validator peers (fail-closed); non-validators are refused. - proto: MeshTopology + UnreachableNode messages and a GetMeshTopology RPC; the per-node query reuses GetMeshViewRequest/MeshView verbatim. - crawl: connected-only — query every connected validator concurrently, classify each response against the local validator set, and surface unconnected/unresponsive validators as `unreachable` rather than dropping them. Traversal is always validators-only; readers are never queried. - http: /v1/mesh?crawl=true returnshe topology (ascii or json). - render: per-topic gossip-mesh display — the single-node table gains a column per topic and the crawl renders one adjacency matrix per topic (directional cells flag one-way links). Topics are chosen via ?topics=a,b (default consensus,mempool; `all` selects every topic), validated against the canonical topic list. - consensus: latest_validator_public_keys() — the latest validator set per shard, unioned — feeds both validator classification and the diagnostics gate, so a removed validator is excluded from both. resolves NEYN-11925

The documentation is generated from this commit.

feat(mesh): local mesh view endpoint `/v1/mesh` (#927) Add an admin-gated endpoint that returns a node's view of the gossip mesh and topology: connected peers, per-topic mesh membership, and per-peer/per-topic gossip rates. Validators are identified cryptographically — a peer's libp2p PeerId is derived from its Ed25519 signing key and matched against the validator set at the current height — not by a heuristic. - proto: GetMeshView RPC + MeshView/MeshSelf/MeshPeer/TopicMembership/ GossipRate messages; ContactSource and MeshNodeType enums. Reserve ttl/visited_peer_ids on the request for the future propagating crawl. - gossip: track observed connection addresses separately from peer-attested contact info; assemble the raw mesh view from gossipsub all_peers/mesh_peers + sampled gossip-rate counters. - server: classify peers against the validator set (the only place pubkey -> set mapping happens) and admin-gate via admin_rpc_auth. - http_server: /v1/mesh route with ?at=ascii|json and ?validators_only=; ASCII table + consensus-mesh graph renderer. Bundled fix: connected peers lacking collected contact info (e.g. validators, which never publish it) were dropped from /v1/currentPeers. They now appear tagged source=DERIVED with their observed address, never conflated with COLLECTED peer-attested data. Properly fixing validator contact-info publishing is tracked in #911 / NEYN-11921. resolves #911 resolves NEYN-11921 merge plan: merge after #926

2026-05-20

Add LIVE_AT user data support (#899) ## Summary Implementation of https://github.com/farcasterxyz/protocol/discussions/268 for Linear NEYN-10699. - Add `USER_DATA_TYPE_LIVE_AT = 14` with V17 feature gating and length-only validation. - Add a separate LIVE_AT mempool rate-limit budget based on storage units, with invalidation on storage rent and storage lending changes. - Coalesce pending LIVE_AT heartbeats by full LWW ordering (`timestamp + hash`) so each fid has at most one pending LIVE_AT per shard. - Add validation, mempool, rate-limit, consensus/devnet coverage, and docs/changelog updates. - Fix local Docker devnet peering so the compose stack can be used for end-to-end validation. ## Release schedule V17 enables `ProtocolFeature::LiveAt` by network schedule: - Testnet: 2026-05-21 5:00 PM UTC (`1779382800`) - Mainnet: 2026-06-04 5:00 PM UTC (`1780592400`) Nodes must deploy this PR's build before each network's `active_at`. Devnet is already V17 at `active_at = 0`, so local devnets accept LIVE_AT immediately. ## Test plan - `PATH="$HOME/.cargo/bin:$PATH" rustup run stable rustfmt --edition 2024 src/version/version.rs` - `PATH="$HOME/.cargo/bin:$PATH" rustup run stable cargo test test_live_at --lib` - `PATH="$HOME/.cargo/bin:$PATH" rustup run stable cargo test live_at_budget --lib` - `PATH="$HOME/.cargo/bin:$PATH" rustup run stable cargo test live_at_rate_limit_rejects_zero_storage_units --lib` - `PATH="$HOME/.cargo/bin:$PATH" rustup run stable cargo test live_at_activation_schedule --lib` - `PATH="$HOME/.cargo/bin:$PATH" rustup run stable cargo test test_next_version_timestamp_for --lib` - `PATH="$HOME/.cargo/bin:$PATH" rustup run stable cargo test test_live_at_consensus --test consensus_test` - Local Docker devnet smoke: `docker compose build`, `docker compose up`, verified `/v1/info` block progress and LIVE_AT set/update/clear readback through the running devnet. Made with [Cursor](https://cursor.com) --------- Co-authored-by: Cursor <cursoragent@cursor.com>

2026-05-06

feat: surface current user/app nonces on signersByFid (#888) Adds `current_user_nonce` (always populated) and `requester_fid_nonces` (map<uint64, uint32> keyed by requester FID) to SignersByFidResponse, plus a new SignersByFidRequest carrying repeated `requester_fids` for batched lookups. Exposes the gasless-key counters that previously lived only behind merge-time validation, so clients composing the next signed- key-request no longer have to track nonces locally — including across revocations, where the per-key record is gone but the counter persists.

2026-04-29

feat: add unified signer API with on-chain and off-chain support (#864) This commit introduces a new unified signer API, including the `GetSigner` and `GetSignersByFid` RPC methods, which return both on-chain and off-chain (gasless) signer records. The `Signer` message structure is enhanced to accommodate fields relevant to both sources, and the corresponding responses are defined. Additionally, tests are added to ensure the correct functionality of the new API endpoints, verifying the integration of on-chain and gasless key records. This change aims to streamline the signer retrieval process and improve overall API usability.

2026-04-25

Remove shard_id from BlocksRequest (#822) `GetBlocks` always returns blocks from shard 0 regardless of the `shard_id` value in `BlocksRequest`, making the field silently misleading to callers. ### Changes - **Proto**: Remove `shard_id` (field 1) from `BlocksRequest`; mark as `reserved` to prevent field number reuse - **Rust**: Remove `shard_id: 0` from `BlocksRequest` construction in `cli.rs` and `rpc_client.rs` - **Docs**: Update `blocks.md` to drop `shard_id` from the `BlocksRequest` table ```proto // Before message BlocksRequest { uint32 shard_id = 1; uint64 start_block_number = 2; optional uint64 stop_block_number = 3; } // After message BlocksRequest { reserved 1; reserved "shard_id"; uint64 start_block_number = 2; optional uint64 stop_block_number = 3; } ``` --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: manan19 <377624+manan19@users.noreply.github.com>

2026-04-22

feat: add KEY_ADD and KEY_REMOVE to protos (#786) add empty handlers where needed

2025-12-20

debug account root mismatch

feat: separate proto files into standalone crate (#729) ## Summary - Extracts protocol buffer definitions into a separate `snapchain-proto` crate so consumers can import protos without depending on the entire snapchain project - Converts the project to a Cargo workspace with `proto/` as a workspace member - Uses extension traits pattern to maintain functionality while respecting Rust's orphan rules ## Changes - Create `proto/` workspace member with its own `Cargo.toml` and `build.rs` - Move proto definitions from `src/proto/` to `proto/definitions/` - Move proto-only impls (Height, ShardHash, Message, FullProposal, ConsensusMessage, BlockEvent) to proto crate - Add extension traits for methods requiring main crate types: - `CommitsExt` for commit certificate conversion - `FullProposalExt` for proposer address - `HubEventExt` for event construction - `HubEventStorageExt` for storage operations - Rename `HubEvent::from()` to `HubEvent::new_event()` to avoid trait conflict with `std::convert::From` ## Test plan - [ ] Docker compose build passes - [ ] All existing tests pass - [ ] Proto crate can be imported independently

2025-10-29

chore: add more information about lent and borrowed units to rpc (#717) Update the `GetCurrentStorageLimitsByFid` rpc to include details about how many units are lent, borrowed, and purchased to figure out where there's a discrepancy between snapchain and the client.

2025-10-07

chore: add bulk rpc for querying `LendStorage` messages (#702) This rpc provides us a way to know if a `LendStorage` message has been merged into snapchain.

2025-09-29

feat: enable migrating onchain events to shard 0 and fix bugs with storage lending (#669) Add an rpc to migrate onchain events to shard 0 from each of the shard 1 and 2. In addition, this feature contains several fixes necessary to support storage lends properly. The bugs were discovered while testing in testnet. As of this feature storage lending is fully supported in testnet. Fixes: - Reset event id to the current block number when starting to propose, validate, or commit a block in the block engine. This was missing so all event ids were being produced with height 0 and we exhausted the available sequence numbers. - Force storage lenders to retain 1 unit of storage so they can revoke storage they've lent. Otherwise, the lender runs into rate limits when trying to revoke lent storage after lending all their storage out. This is easier and safer than bypassing rate limits. - Listen to confirmed blocks in the mempool and evict storage lends that have been put into blocks in shard 0. - Have rate limits in the mempool key off the fid's shard and never shard 0. The mempool was crashing trying to find a `ShardStore` for shard 0. - Add a count of onchain events to the `GetInfo` rpc so we can validate that the migration of onchain events to shard 0 has happened correctly.

2025-09-25

chore: separate out ContactInfo and GossipMessage protos (#695) The client protos fail to generate with an error because the type definition for `GossipMessage` is too big. Separate out `ContactInfo` and `GossipMessage` so we can use `ContactInfo` in rpcs on the client without having to pull in `GossipMessage`.

2025-09-22

feat: Discover nodes to sync with using gossip p2p (#682) when replication starts, we seed it with an initial bootstrap list. The replication service then spins up a lightweight gossip node to discover peers to sync against.

2025-09-18

feat: Configure or get (from public IP) announce_rpc_address (#681) Allow setting a `announce_rpc_address` in the config and share that out in the ContactInfo gossip messages. If not configured, get it from public IP

2025-09-12

feat: support storage lending in the shard and block engines (#665) Support storage lending. This feature enables support for storage lending in the `BlockEngine` and `ShardEngine` but the messages aren't handled properly in the rpc server and mempool yet.

2025-09-11

feat: Add the ability to start a node via replication (#667) These are the client changes to support replication. It's not enabled yet, once we feel confident with some manual testing, I'll turn it on.

2025-09-09

feat: implement system message to revalidate a user message with external proofs

feat: periodically revoke invalid ens proofs

2025-09-05

feat: Add profile token user data type (#666) Implement https://github.com/farcasterxyz/protocol/discussions/247. Will go live on the protocol on Sep 24 (engine version 10)

feat: Add trie-based replication server (#663) Modify the replication server to use trie-based iteration to get all messages instead of by DB.

2025-09-02

feat: read block events from shard 0 in other shards (#639) Have all other shards consume block events from shard 0.

2025-08-30

Merge remote-tracking branch 'adityapk/trie_iteration_replicator' into trie_iteration_client

return shard_chunk and blocks in metadata

2025-08-29

wip

fix panic while decoding trie key

wip

2025-08-28

Merge remote-tracking branch 'origin/main' into trie_iteration_replicator

Merge remote-tracking branch 'adityapk/trie_iteration_replicator' into trie_iteration_client

feat: Allow attaching orphaned nodes to the merkle trie (#651) When we write to the trie from parallel threads (like we do in the replicator), we will write "orphaned" nodes. These are nodes that are written to the DB, but not yet attached to the root node. This is because we want to write in parallel threads to different parts of the trie. Once the operation is done, we call `attach_to_root` and `recalculate_hashes` to add new nodes to the trie. This way, we can parallelize construction of the trie. Also remove excluded_hash() and snapshot() which we don't use anymore. The `get_trie_node_metadata` is still supported, which can be used to get info about the parts of a trie.

add onchain event support

2025-08-27

read user messages

feat: produce transactions and events in shard 0 (#634) Produce blocks from shard 0 that contain block events and transactions. We use a heartbeat event for testing the events pipeline.

2025-08-14

fixes

rough end to end impl of cross shard communication

2025-08-11

Upgrade malachite to latest

2025-08-05

Add transaction replication endpoint(s) (#606) Adds endpoints for fetching replication shapshot metadata and transactions for a given shard. --------- Co-authored-by: Paul <515868+eyeamera@users.noreply.github.com>

2025-07-29

feat: Add SubmitBulkMessages rpc/http API (#600) Add the SubmitBulkMessages API (which had been commented out). The endpoint (both grpc and http API) accepts an array of messages, and submits them sequentially. Right now, the messages need to be independent (i.e., a message that depends on a previous message in the array to succeed is not supported yet) It will attempt to merge all the messages, and return success/failure for each one individually

2025-07-25

feat: expose next engine version timestamp in GetInfo API (#595) Based on https://github.com/farcasterxyz/snapchain/pull/587 Fixes https://github.com/farcasterxyz/snapchain/issues/586 --------- Co-authored-by: christopherwxyz <christopher@unofficial.run>

2025-07-24

Read implementation for replication (#589) Adds: - Extension to Engine to support replaying individual transactions for backfilling FID state data - ReplicationStores: used to hold "snapshots" (read-only views) into individual shard stores taken at a specified block interval. These snapshots are utilized to provide a consistent view into FID data at known block height(s). - Replicator: the business logic for taking snapshots and exporting FID data - Replication Service: a new grpc service that exposes an interface in front of replicator for querying FID state. Follow-ups: - [ ] Some accounts have a lot of messages (e.g. fids 2 and 3) so we'll need to sync them over multiple requests. - [ ] Solve for message ordering at the engine level [:link:](https://github.com/farcasterxyz/snapchain/pull/589#discussion_r2216488125) - [ ] Allow db-free lookups at the engine level (e.g. when an FID doesn't exist we shouldn't error) - [ ] Don't commit transactions for FID unless the account root matches [:link:](https://github.com/farcasterxyz/snapchain/pull/589#discussion_r2221348609) Towards: #510 --------- Co-authored-by: Paul <515868+eyeamera@users.noreply.github.com>

2025-07-15

feat: Add endpoint for connected peers (#541) Add RPC endpoint for getting the currently connected peers. To do this, introduces a `BTreeMap` in `SnapchainGossip` that stores the contact info received from other peers.

2025-07-09

feat: Storage redenomination (#570) Implements the Storage Re-denomination FIP: https://github.com/farcasterxyz/protocol/discussions/229 As part of it, we're also extending the expiry of storage units that are set to expire on Aug 29 2025 by one year.

2025-07-02

feat: Add admin endpoint to retry fname transfers (#546) Admin endpoint to retry fname transfers by fid.

2025-06-24

feat: add event counts per event type to BlockConfirmed (#555) This is required to help the client determine if there are missed events with filtering applied on the event stream.

2025-06-20

feat: Add Block Confirmed event type as the first event emitted for a block (#549) #544 Adds a "block confirmed" event to signal block finalization. This event includes block metadata such as number, hash, timestamp, and total event count.

2025-06-12

feat: unified fid address type endpoint (#535) Add GetFidAddressType endpoint for unified address validation as reference in https://github.com/farcasterxyz/snapchain/issues/529 ## Summary Adds a new `GetFidAddressType` endpoint that checks if an address is valid as a custody address, auth address, or verified address in a single API call. ## Problem Currently, SIWF (Sign-In With Farcaster) authentication requires racing multiple API calls to check if an address is valid: - Check IdRegistry for custody addresses - Check KeyRegistry for auth addresses (keyType=2) - Check verified addresses This is inefficient and adds unnecessary complexity. ## Solution New endpoint `GetFidAddressType(fid, address)` returns: ```proto { is_custody: boolean, // Address owns the FID (IdRegistry) is_auth: boolean, // Address is an auth key (KeyRegistry keyType=2) is_verified: boolean, // Address is verified for this FID valid: boolean // Any of the above is true } ``` ## Changes - Added proto definitions for `FidAddressTypeRequest` and `FidAddressTypeResponse` - Implemented gRPC endpoint in `server.rs` - Added HTTP endpoint `/v1/fidAddressType` - Added comprehensive tests ## Benefits - Single API call instead of multiple racing requests - Simplified client implementation for SIWF - Clear distinction between address types - Supports farcaster.xyz using custody addresses on mobile and auth addresses on web

2025-06-06

Adds Primary Address support to Snapchain (#465) Implementation should mirror [the updated FIP](https://github.com/farcasterxyz/protocol/discussions/141) This is an initial draft of the PR, I have a hunch that it's ~80% there but it's also worth opening up for quick feedback to make sure I don't invest time in the wrong direction.

2025-06-05

feat: support pro users (#516) Add support for pro users.

2025-06-04

perf: Fix perf test script (#507) - Adds implementation to Admin APIs (`submit_user_name_proof`, `submit_on_chain_event`) for perf tests. These are only accessible on unspecified or dev networks. - Updates perftest w/ a sleep to allow for previous messages to be processed - Updates CLI utils to utilize auth for `send_message` and `send_on_chain_event` --------- Co-authored-by: Sanjay Raveendran <sanjayprabhu@gmail.com> Co-authored-by: Paul <515868+eyeamera@users.noreply.github.com>

feat: Support Basenames (#515) Implement https://github.com/farcasterxyz/protocol/discussions/234 to support basenames Todo: - [x] Unit Tests - [x] Username length validation update - [ ] Manual verification

Add validation for Ethereum and Solana primary address user data types

2025-05-17

chore: add ability to upload snapshot just for 1 shard (#496) This allows us to manually retry just 1 shard rather than all of them.

2025-05-01

feat: add timestamp to hub event (#409) This makes it much easier to determine how far behind you are. https://github.com/farcasterxyz/snapchain/issues/397

2025-04-29

feat: Add GetFids API (#407) Implement `GetFids` and `v1/fids` which was previously stubbed out. I've tried to follow the established patterns as much as possible for constructing `next_page_token` etc. I also had to modify `get_onchain_events` to allow requests that aren't tied directly to fids. Closes https://github.com/farcasterxyz/snapchain/issues/392

2025-04-26

fix: implement /v1/events HTTP API endpoint (#451) Inheriting from https://github.com/farcasterxyz/snapchain/pull/411. Merged main and cleaned up some api inconsistencies --------- Co-authored-by: gr4yha7 <jamaal.ekpenyong22@gmail.com>

2025-04-24

feat: handle slow peers in gossip

2025-04-18

fix: make GetInfo mirror hubs GetInfo

2025-04-16

feat: Introduce new unit type for FIP

[feat] Add new event type for merge failures (#374) Add a new HubEvent type for merge failures that contains the error code and reason why message failed to be merged. This is useful for clients who submit messages which were included into the mempool, but then failed later. Also updates some api error messages to be consistent with hubs, as well as return some additional fields in `GetInfo` and `HubEvent`

2025-04-14

feat: Return block number as a field in Event (#359) - Add a new `HubEventResponse` message to `request_response.proto` - Add helper for converting a `HubEvent` to a `HubEventResponse` in `event.rs` - Use helper in existing Events API endpoint (subscribe) in `server.rs` - Change return type for Events API in `rpc.proto` - Add tests to `server_tests` ### TODO - [x] Implement GetEvent with new `HubEventResponse` closes #344 When reviewing I think the biggest thing to confirm is that I'm handling the shard index and store lookup correctly. --------- Co-authored-by: Sanjay <sanjayprabhu@gmail.com>

2025-04-10

fix: Keep subscribe shard index backwards compatible (#364) Subscribe call is not backwards compatible with existing clients because the sequence id changed on the protobuf, keep it the same. And some other minor fixes.

2025-04-02

feat: support peer autodiscovery (#337) Allow nodes to find each other so that they can receive data from nodes other than the bootstrap nodes.

2025-03-26

feat: admin rpc to retry onchain events by fid and block range (#334) We dropped some onchain events during the initial sync and need an admin rpc to retrigger ingest for the range we missed. Tested locally. ``` ❯ grpcurl -plaintext -proto src/proto/admin_rpc.proto -d '{"fid" : 1030970}' -import-path src/proto 127.0.0.1:3383 AdminService/RetryOnchainEvents {} ❯ grpcurl -plaintext -proto src/proto/admin_rpc.proto -d '{"block_range": {"start_block_number" : 133577805, "stop_block_number": 133577810}}' -import-path src/proto 127.0.0.1:3383 AdminService/RetryOnchainEvents ``` ``` 2025-03-26T03:50:14.112717Z INFO snapchain::connectors::onchain_events: Starting l2 events subscription latest_block_on_chain=133681118 latest_block_in_db=133669197 2025-03-26T03:50:23.013693Z INFO snapchain::connectors::onchain_events: Retrying onchain events for fid fid=1030970 2025-03-26T03:50:23.194556Z INFO snapchain::connectors::onchain_events: Processed onchain event fid=1030970 event_type="EVENT_TYPE_STORAGE_RENT" block_number=133577808 block_timestamp=1742754393 tx_hash="9a158395ecd3ba711c0d6c7ca7de39fb6f33044b9e889029bae49b5ebfb8b88f" log_index=21 2025-03-26T03:50:23.388938Z INFO snapchain::connectors::onchain_events: Processed onchain event fid=1030970 event_type="EVENT_TYPE_SIGNER" block_number=133577808 block_timestamp=1742754393 tx_hash="9a158395ecd3ba711c0d6c7ca7de39fb6f33044b9e889029bae49b5ebfb8b88f" log_index=22 2025-03-26T03:50:23.576881Z INFO snapchain::connectors::onchain_events: Processed onchain event fid=1030970 event_type="EVENT_TYPE_ID_REGISTER" block_number=133577808 block_timestamp=1742754393 tx_hash="9a158395ecd3ba711c0d6c7ca7de39fb6f33044b9e889029bae49b5ebfb8b88f" log_index=20 2025-03-26T03:51:50.404134Z INFO snapchain::connectors::onchain_events: Retrying onchain events in range start_block_number=133577805 stop_block_number=133577810 2025-03-26T03:51:50.556316Z INFO snapchain::connectors::onchain_events: Processed onchain event fid=1030970 event_type="EVENT_TYPE_ID_REGISTER" block_number=133577808 block_timestamp=1742754393 tx_hash="9a158395ecd3ba711c0d6c7ca7de39fb6f33044b9e889029bae49b5ebfb8b88f" log_index=20 2025-03-26T03:51:50.622399Z INFO snapchain::connectors::onchain_events: Processed onchain event fid=1030970 event_type="EVENT_TYPE_STORAGE_RENT" block_number=133577808 block_timestamp=1742754393 tx_hash="9a158395ecd3ba711c0d6c7ca7de39fb6f33044b9e889029bae49b5ebfb8b88f" log_index=21 2025-03-26T03:51:50.696271Z INFO snapchain::connectors::onchain_events: Processed onchain event fid=1030970 event_type="EVENT_TYPE_SIGNER" block_number=133577808 block_timestamp=1742754393 tx_hash="9a158395ecd3ba711c0d6c7ca7de39fb6f33044b9e889029bae49b5ebfb8b88f" log_index=22 2025-03-26T03:51:50.785513Z INFO snapchain::connectors::onchain_events: Processed onchain event fid=1030971 event_type="EVENT_TYPE_ID_REGISTER" block_number=133577810 block_timestamp=1742754397 tx_hash="3597280e93945aad66d507e553ff13ae33055e2b39920f6f1b41434d9230139e" log_index=6 2025-03-26T03:51:50.848978Z INFO snapchain::connectors::onchain_events: Processed onchain event fid=1030971 event_type="EVENT_TYPE_STORAGE_RENT" block_number=133577810 block_timestamp=1742754397 tx_hash="3597280e93945aad66d507e553ff13ae33055e2b39920f6f1b41434d9230139e" log_index=7 2025-03-26T03:51:50.922974Z INFO snapchain::connectors::onchain_events: Processed onchain event fid=1030971 event_type="EVENT_TYPE_SIGNER" block_number=133577810 block_timestamp=1742754397 tx_hash="3597280e93945aad66d507e553ff13ae33055e2b39920f6f1b41434d9230139e" log_index=8 ```

2025-03-25

feat: Add mempool size to getInfo (#333) Return the size of the mempool in getInfo

2025-03-20

feat: Add auth for rpc endpoints and cleanup admin rpc (#323) Add auth for submit and subscribe rpc endpoints. And for admin endpoints. Don't mount the admin endpoint if admin rpc auth is not configured. Also remove admin endpoints that directly mutate state, we shouldn't allow this for mainnet

2025-03-07

feat: add missing methods (#291)

2025-03-04

feat: Read node documentation and related fixes (#289) Add documentation for how to run a read node and related small fixes around logging and add missing Info http endpoint

2025-02-28

support read nodes (#267) Add support for nodes that can listen to committed blocks and serve rpcs but won't participate in consensus.

2025-02-19

fix: Block production fixes (#264) - Add consistent 250ms (by default) block time via commit timeout - Don't store full shard chunks in the block, just store headers - Look up chunks from shard db instead of waiting on the channel (fixes block proposer getting stuck forever if it didn't get a chunk in time) - Fix consensus tests not correctly including messages - Delete unused sync code - set max_messages per block to 500 to compensate for slower block time (~1mbs at 200 bytes per messages * 500 * 2 shards * 4 blocks per sec at 250ms). We probably have some room to bump this higher, but need to be mindful of the number of shards running on the same node.

2025-02-06

Add shard info to getInfo RPC (#251) Add shard info to getInfo so we have more visibility into the current heights.

2025-02-05

pick up fname transfer ingest from where it left off (#246) Start ingesting fname transfer from where the node stopped before restart.

2025-02-04

clean up on chain event ingest persistence (#245) Clean up code for onchain events persistence.

pick up on chain events ingestion where it left off (#244) Store the last ingested onchain event and pick up ingestion where we left off.

2025-02-03

feat: additional read apis for grpc, fleshing out http server (#242) Adds the other read apis we can currently support for gRPC, and scaffolds out most of the http api, but isn't exposed via mod so cargo doesn't try to compile it

2025-01-31

Remove old consensus (#241) Remove code for the pre malachite 0.0.1 consensus implementation. And add a tool to generate keypairs

Rewrite malachite integration (#239) - Upgrade to malachite v0.0.1 - Rewrite integration to be at the actor level instead of re-implementing the malachite consensus actor - Implement malachite sync

2025-01-29

upload snapshots to s3 (#233) Clear existing snapshots and upload new ones to s3 via an admin rpc.

2025-01-27

add fixed validator set

populate validators on blocks

feat: Add gossip and eviction to the mempool (#200) This PR adds a basic implementation of the gossipped mempool described in #180. It uses the existing mempool implementation but changes the key to use a key derived from the timestamp and identity of a message. Additions to the mempool are broadcast using a new gossip message. Messages are also evicted from the mempool when shard chunks are emitted that contain messages contained in the mempool. Some minor changes to the channels and overall plumbing was required to setup the mempool communication.

2025-01-06

add rpc for trie inspection (#196) Port the equivalent of the `GetSyncMetadataByPrefix` rpc from hubs. Here, there a trie per shard, so we specify the shard id in the request.

2024-12-19

turbohash (#171) This change stores the hash of a trie node in the node's parent (in a hashmap). This yields an important performance improvement when recomputing the hash of a changed node. Previously, when recomputing the hash, as we are working our way up the trie, we had to load siblings of a changed node out of the db. In this version, the sibling hashes are already loaded, which dramatically reduces the number of db reads.

2024-12-18

turbohash

2024-12-13

Implement Storage Limit RPC (#163)

submit username proofs via admin rpc (#162) We need fnames in snapchain for the migration and an rpc to allow us to submit them. Tested this by running the migration with username proofs.

More read apis (#161) Implement read apis for most of the remaining stores

2024-12-12

serde

Implement GetCast read api and prepare for rest of the read rpcs (#156)

2024-12-10

add overall fid message count to stats (#150) Add overall per-fid message count to the `GetInfoByFid` rpc response. This is useful for understanding message count discrepancies in onchain events. ``` ❯ grpcurl -plaintext -proto src/proto/rpc.proto -import-path src/proto -d '{"fid": 522}' 127.0.0.1:3383 HubService/GetInfoByFid { "numMessages": "70", "numMessagesByMessageType": { "1": "14", "2": "0", "3": "0", "4": "0", "5": "49", "6": "0", "7": "0", "8": "0", "11": "3", "12": "0", "13": "0", "14": "0" } } ```

add endpoint for message counts by fid (#149) It's useful to have visibility into which message types have differing counts for reach fid to validate the migration. ``` ❯ grpcurl -plaintext -proto src/proto/rpc.proto -import-path src/proto -d '{"fid": 503}' 127.0.0.1:3383 HubService/GetInfoByFid { "numMessagesByMessageType": { "1": "100", "2": "0", "3": "386", "4": "17", "5": "97", "6": "0", "7": "0", "8": "0", "11": "0", "12": "0", "13": "0", "14": "0" } } ```

2024-12-09

implement a get info rpc (#147) This rpc is useful for understanding if messages are actually getting merged upon submit. ``` ❯ grpcurl -plaintext -proto src/proto/rpc.proto -import-path src/proto 127.0.0.1:3383 HubService/GetInfo { "dbStats": { "numMessages": "23219", "numFidRegistrations": "10", "approxSize": "47380709" } } ```

2024-12-07

shard routing (#140)

2024-12-05

remove package names from protos so that rpcs are compatible between hubs and snapchain (#120) Remove package names so that rpcs are compatible between snapchain and hubs.

Revert "add submitLegacyMessage endpoint (#124)" (#130) This reverts commit d7a2df8ef6f9e0a1547a65bee78516590331ec6f.

Revert "create separate service at root namespace for hub compatible … (#129) This reverts commit 32d935e32eb8648822433341ae928a01a41dc40d.

create separate service at root namespace for hub compatible rpcs (#127) Putting `submitLegacyMessage` in the existing rpc service doesn't work due to namespacing issues-- the service is in the "rpc" namespace. Move this endpoint into a new service at the root namespace that exactly matches the proto structure of the parallel hub rpc. Future compatible rpcs should go there too.

Support registering fnames as usernames (#125) Support for merging fname proofs and validation to ensure username can only be merged with the correct proof

add submitLegacyMessage endpoint (#124) We'll use this endpoint for submitting existing hub messages to snapchain nodes. This avoids us needing to remove qualified module names.

rename SnapchainService to HubService (#122) The service name needs to be the same as it is in the existing hubs for compatibility.

2024-12-01

add endpoint for submitting onchain events via rpc (#105) We need to have a storage allocation for an fid before we can accept a message for it. Add the ability to submit an onchain event via rpc to unblock successfully submitting messages via rpc.