Proto commits in ObolNetwork/charon

These 51 commits are when the Protocol Buffers files have changed:

2025-02-26

*: electra (#3558) Merge electra branch to main category: feature ticket: none

The documentation is generated from this commit.

2025-01-21

*: add custom target gas limit in cluster lock v1.10 (#3456) Up until now the gas limit was hardcoded to 30M. However, the community is pushing towards bumping those numbers and you can't really customise that in Charon. With those updates a `targetGasLimit` variable is introduced to the cluster definition and a new version of cluster definition is created, v1.10. The previous default version was v1.8, now the default is set to v1.10, including also the changes of consensus protocol introduced in v1.9. However, good to keep in mind there is only 1 consensus protocol enabled in charon, meaning the consensus protocol configuration in v1.9 isn't making much difference. Previously the gas limit was hardcoded to 30M, now this can be configured by supplying `--target-gas-limit` flag to both CreateDKG and CreateCluster commands. The variable is not compulsory and its default is set to 36M (this is the new value commonly used in the ecosystem). If a new charon version (i.e.: v1.3) uses old definition (and lock) file, its default will still be the previous one of 30M. To bump the number, unfortunately, a new cluster needs to be created with the updated definition version. category: feature ticket: #3419

2025-01-08

app/peerinfo: add nickname field to peerinfo protocol (#3428) Add nickname flag to `run` command, exchange nicknames using peerinfo protocol and create nickname metric to display on Grafana. Preview:  category: feature ticket: #1655

2024-12-02

Revert "blacklist" This reverts commit 45e0e8b4b81bb36b5daced89c666e223fc9ba2c8.

2024-11-29

blacklist

2024-11-15

distinguish by duty and test with kurtosis

hotstuff component and tests

2024-11-14

hotstuff integration wip

2024-11-04

*: consensus abstraction (#3327) Summary of changes: 1. Introduced & implemented `ConsensusController` interface. 2. Refactored the entire `consensus` package with increased test coverage. 3. Added `cluster run --consensus-protocol=xyz` flag. 4. Wired Priority protocol to change the current core consensus algorithm. 5. Support to run two consensus protocols at a time (one for Priority - QBFT and another is for Core). 6. Added `docs/consensus.md` category: feature ticket: #3304

2024-02-26

app: remove todos (#2903) Remove TODOs from app package. category: misc ticket: none

2024-02-20

app/peerinfo: added builder_api_enabled metric (#2890) Adds `app_peerinfo_builder_api_enabled` metric (gauge) indicating if the `peer` has builder API enabled. category: feature ticket: #2879

2024-01-25

*: implement audit fixes (#2821) Cherry-picks audit fixes tagged with `qs-audit` on `main-v0.19` category: misc ticket: none

2024-01-22

dkg/bcast: exchange full messages (#2798) Instead of passing around hashes, share the whole message with the target for them to validate. category: refactor ticket: none

2023-11-08

core: revert core interfaces (#2682) Revert changes to core interfaces. This is done since the deneb upgrade no longer requires multiple signatures per signed object. Reverting to the original implementation makes the interfaces simple and code much easier to reason about. category: refactor ticket: #2659

2023-11-07

Revert "core: update interfaces (#2635)" This reverts commit 2fe130e254febcf866b24a06b32e83041cf7b8e7.

2023-11-06

*: convert to uint64 (#2668) Convert `Duty.Slot` from int64 to uint64. This is recommended according to ethereum consensus specs: https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md#custom-types category: feature ticket: #2658

2023-10-13

core: update interfaces (#2635) Update `SignedData` interface to include: * `Signatures() []Signature` - Replaces `Signature()` * `SetSignatures([]Signature) (SignedData, error)` - Replaces `SetSignature()` * `MessageRoots() ([][32]byte, error)` - Replaces `MessageRoot()` * `HashRoot() ([32]byte, error)` Also, update `Eth2SignedData` interface to include: * `DomainNames() []signing.DomainName` - Replaces `DomainName()` These changes to the core interfaces are done in order to prepare for the upcoming deneb ethereum upgrade. category: feature ticket: #2597

2023-08-09

*: update v0.17 branch with latest code for release (#2528) The following PRs have been cherry-picked: - https://github.com/ObolNetwork/charon/pull/2490 - https://github.com/ObolNetwork/charon/pull/2494 - https://github.com/ObolNetwork/charon/pull/2496 - https://github.com/ObolNetwork/charon/pull/2509 - https://github.com/ObolNetwork/charon/pull/2510 - https://github.com/ObolNetwork/charon/pull/2504 - https://github.com/ObolNetwork/charon/pull/2511 - https://github.com/ObolNetwork/charon/pull/2514 - https://github.com/ObolNetwork/charon/pull/2516 - https://github.com/ObolNetwork/charon/pull/2518 category: misc ticket: none

2023-08-04

dkg/sync: introduce explicit sync steps (#2509) Introduces explicit sync steps in DKG, allowing all peers to wait for other to complete each step before continuing to next step. This prevents fast peers timing out sending stuff to slow peers. It also mitigates against shutdown races. category: bug ticket: #887

2023-07-12

cluster/manifest: remove mutation timestamp (#2412) Refactors the manifest mutation by removing the timestamp field since this make deterministic mutations basically impossible. Instead, move timestamps to the data of mutations created by single entities, like `NodeApproval`. category: refactor ticket: none

2023-06-27

cluster/manifest: refactor cluster hashes (#2369) Refactor `manifestpb.Cluster` to expose `InitialMutationHash` and `LatestMutationHash` explicitly. `LatestMutationHash` is required for: - Identify manifest "iteration", allowing to comparison of different manifest iterations. - Use of parent hash when adding subsequent mutations. Also regenerate all protobufs with latest protogengo version. category: refactor ticket: none

2023-06-22

p2p: check protonil for all receives (#2346) Check all libp2p protos received over the wire for nil fields. Also extend `protonil` to detect nil list eleements and nil map values. category: misc ticket: none

2023-06-21

app/protonil: implement generic nil proto checker (#2344) Implements a generic protobuf nil checker to mitigate against manual verbose or buggy protobuf nil checks. > Note this requires fields that allowed to be nil to be specified as `optional` in the proto definition. category: misc ticket: none

2023-06-19

*: rename cluster state to cluster manifest (#2331) Renames "cluster state" to "cluster manifest". category: refactor ticket: none

2023-06-16

cluster/state: refactor cluster to proto (#2322) Refactors the resulting `Cluster` state type to proto since it the a very important part of the "spec", it the thing we the cluster state protocol is building. So make it official and part of the spec. Also rename `cluster/statepb/v1` import alias from `pbv1` to `statepb` since we already using `pbv1` for `corepb/v1` alias. category: refactor ticket: none

2023-06-15

cluster/state: refactor to proto serialisation (#2320) Refactors cluster state serialisation from JSON to proto and from hashing from SSZ to simple SHA256 hashing combined with anypb. Protos provide the following benefits: - This removes complexity of SSZ hashing - Removes the complexity of JSON serialisation - Schema first protobuf definitions category: refactor ticket: none

2023-05-24

dkg: lock hash k1 signature in lock file (#2224) This commit ensures all node operators individually sign the agreed lock hash with their K1 private keys, and stores the resulting signature in the lock file itself. category: feature ticket: #2204

2023-03-20

dkg/sync: enforce version (#1901) Enforce matching peer minor version. DKG only guarantees compatibility with patch versions. category: bug ticket: #1895

2023-03-17

dkg: refactor to reliable broadcast (#1896) Refactor DKG to use reliable broadcast instead of p2p broadcast. category: feature ticket: #1893

dkg/bcast: implement reliable broadcast protocol (#1893) Implementation of a reliable-broadcast protocol for use in the DKG. All peers sign the broadcasted message hash, then the message is broadcasted with those signatures as proof. category: feature ticket: #1889

2023-03-01

core/consensus: add support for pointer values on the wire (#1838) Upgrades the QBFT wire protocol (without breaking backwards compatibility) by adding the whole value to the `ConsensusMsg` envelope and hashes of those values to the `QBFTMsg` itself. This will allow us to remove the values from the QBFT messages, which will reduce network bandwidth usage by a lot (90% for large values; beacon blocks). This increases the network bandwidth for next release v0.14, but once we remove support for v0.13, it will reduce it by a lot. category: feature ticket: #1552

2022-12-14

core/consensus: fix wire compatibility (#1556) Reverts "reserved" fields since they are required to calculate correct hash when verifying signatures. Instead just stop using fields. category: bug ticket: #1555

core/consensus: remove deprecated value fields (#1554) Removes deprecated consensus wire protocol fields. New fields were introduced in v0.11, so this removes compatibility with v0.10 and older. category: refactor ticket: #1555

2022-11-18

core/consensus: sniff qbft messages (#1452) Sniff recent qbft consensus instance messages and serve them as gzip protos from 2MB fifo buffer from `/debug/qbft`. category: feature ticket: #1450

2022-11-17

core/peerinfo: add peer starttime (#1454) Adds peer start time to peerinfo protocol and peer panel on the overview dashboard. category: feature ticket: none

2022-11-14

core/priority: refactor implementation (#1433) Refactor priority protocol implementation: - Align with consensus component architecture - Change instance to duty for improve compatibility and logging and simplicity. - Control priority instance with single context, including consensus timeout. category: refactor ticket: #1429

2022-11-11

app/log: add loki client (#1337) Add simple loki client to directly sending logs to loki instead of jumping through docker-compose hoops. Still needs to be integrated. category: feature ticket: #1406

2022-10-16

app/peerinfo: track peer git commit hash (#1297) Tracking peer version is great, but in order to debug network/wire/compatibility issues properly, you need the actual git commit hash of your peers. category: feature ticket: none

2022-10-13

core/priority: refactor priority protocol (#1271) Refactors the priority protocol: - Removes metadata from proto messages since not required anymore - Makes all types generic `anypb.Any` to support arbitrary use-cases. - Refactor static msgProvider to provide when triggered. category: refactor ticket: #1257

2022-10-05

app/peerinfo: implement protocol skeleton (#1225) This adds the skeleton for the new `peerinfo` protocol. It is aimed at sharing peer information/metadata. The supported use-cases are: - Track peer clock offset - Track peer lock hash - Track peer app version Tests will be added in a separate PR. category: feature ticket: #1224

core/consensus: fix wire compatibility issue (#1223) A second signature wasn't the correct approach for backwards compatibility. Since we marshal the proto to bytes when calculating the hash, unknown fields are actually retained. That means that hashing the whole proto always results in the same hash (and signature) even if the wire contained unknown fields. category: bug ticket: #1222

2022-10-03

core/consensus: generic protobuf values (#1214) Refactors consensus component to support arbitrary protobuf data types. This is backwards wire compatible with previous versions. category: feature ticket: #1205

2022-07-29

core/priority: add scores to output (#875) Adds explicit scores to resulting priorities. This solves the tie-breaker (if scores are identical) problem by allowing the user of the priority protocol to decide how to handle ties based on their use-case. - This simplifies the protocol, no fancy pseudo random shuffling. - Users can however still decide to implement fancy pseudo random shuffling themselves if scores are identical - Or users can use the priority result ordering defined by peers with lowest peer IDs. category: feature ticket: #798

2022-07-25

core/priority: implement cluster priority calculation (#843) Implements the prioritiser's deterministic cluster wide priority function. category: feature ticket: #798

2022-06-14

dkg: add protobuf messages for sync (#707) Adds protobuf messages for dkg sync protocol: sync.proto and sync.pb.go category: feature ticket: #684

2022-05-11

core/consensus: add message signing (#517) Adds message signing and verification helper functions for qbft consensus, category: feature ticket: #445 feature_flag: qbft_consensus

core/consensus: implement qbft component (#514) Implements a qbft consensus component that integrates qbft into p2p and the core workflow. Note this isn't wired yet. Tests will also be added in subsequent PR. category: feature ticket: #445 feature_flag: qbft_consensus

2022-05-05

dkg: implement frost p2p transport (#479) Implement frost DKG p2p transport: - Duplicates broadcast messages and send p2p. - Assumes that all nodes are connected at time of DKG - Uses protobuf for compatibility guarantees category: feature ticket: #478 feature_set: alpha

2022-04-06

core/parsigex: introduce protobuf wire messages (#372) Switches to use protobuf messages for `parsigex` wire protocol to ensure backwards compatability. Use `buf` tool chain: - `buf generate` generates go files. - `buf lint` lints the proto files - `buf breaking` checks for breaking changes. category: feature ticket: #321

2022-01-28

Delete unused legacy files and folders (#78) - Moves version to runner/version package - Delete unused folders and files - Rename cluster/testdata folder (and improve API a little) - Remove protobuf stuff since not used anymore

This commit does not contain any .proto files.

2021-12-16

Initial P2P layer (#11) - Add ENR database - Add self ENR generation - Add crude libp2p bootstrapping - Remove controller and metrics (for now) - Add PoC DV bootstrapping and manifest - Add internal API server using gRPC-Gateway