Proto commits in paritytech/polkadot-sdk

These 34 commits are when the Protocol Buffers files have changed:

2025-04-30

RFC-0008: Store parachain bootnodes in the relay chain DHT (#8072) Implement [RFC-0008 "DHT bootnodes"](https://polkadot-fellows.github.io/RFCs/approved/0008-parachain-bootnodes-dht.html). Close https://github.com/paritytech/polkadot-sdk/issues/1825. With this mechanism, every parachain node is eligible to act as a bootnode. If its peer ID is close to the parachain key for the current relay chain epoch, it becomes discoverable by other parachain nodes via the relay chain DHT. This removes the need to specify bootnodes in the parachain chainspec, eliminating a single point of failure and simplifying things for parachain operators. The mechanism is enabled by default. The embedded DHT bootnode can be disabled using the `--no-dht-bootnode` flag, and discovery of such nodes can be disabled with the `--no-dht-bootnode-discovery` flag. --------- Co-authored-by: cmd[bot] <41898282+github-actions[bot]@users.noreply.github.com>

The documentation is generated from this commit.

2025-04-10

minor: fix whitespace in protobuf schema

The documentation is generated from this commit.

2025-03-27

Downgrade to `proto2` to resolve `protoc` compatibility with `optional`

2025-02-26

Add paranode req-resp protocol protobuf schema

2024-11-26

post-state POC

2024-07-25

authorithy-discovery: Make changing of peer-id while active a bit more robust (#3786) In the case when nodes don't persist their node-key or they want to generate a new one while being in the active set, things go wrong because both the old addresses and the new ones will still be present in DHT, so because of the distributed nature of the DHT both will survive in the network untill the old ones expires which is 36 hours. Nodes in the network will randomly resolve the authorithy-id to the old address or the new one. More details in: https://github.com/paritytech/polkadot-sdk/issues/3673 This PR proposes we mitigate this problem, by: 1. Let the query for a DHT key retrieve more than one results(4), that is also bounded by the replication factor which is 20, currently we interrupt the querry on the first result. ~2. Modify the authority-discovery service to keep all the discovered addresses around for 24h since they last seen an address.~ ~3. Plumb through other subsystems where the assumption was that an authorithy-id will resolve only to one PeerId. Currently, the authorithy-discovery keeps just the last record it received from DHT and queries the DHT every 10 minutes. But they could always receive only the old address, only the new address or a flip-flop between them depending on what node wins the race to provide the record~ 2. Extend the `SignedAuthorityRecord` with a signed creation_time. 3. Modify authority discovery to keep track of nodes that sent us old record and once we are made aware of a new record update the nodes we know about with the new record. 4. Update gossip-support to try resolve authorities more often than every session. ~This would gives us a lot more chances for the nodes in the networks to also discover not only the old address of the node but also the new one and should improve the time it takes for a node to be properly connected in the network. The behaviour won't be deterministic because there is no guarantee the all nodes will see the new record at least once, since they could query only nodes that have the old one.~ ## TODO - [x] Add unittests for the new paths. - [x] Make sure the implementation is backwards compatible - [x] Evaluate if there are any bad consequence of letting the query continue rather than finish it at first record found. - [x] Bake in versi the new changes. --------- Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io> Co-authored-by: Dmitry Markin <dmitry@markin.tech> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>

2024-04-09

Integrate litep2p into Polkadot SDK (#2944) [litep2p](https://github.com/altonen/litep2p) is a libp2p-compatible P2P networking library. It supports all of the features of `rust-libp2p` that are currently being utilized by Polkadot SDK. Compared to `rust-libp2p`, `litep2p` has a quite different architecture which is why the new `litep2p` network backend is only able to use a little of the existing code in `sc-network`. The design has been mainly influenced by how we'd wish to structure our networking-related code in Polkadot SDK: independent higher-levels protocols directly communicating with the network over links that support bidirectional backpressure. A good example would be `NotificationHandle`/`RequestResponseHandle` abstractions which allow, e.g., `SyncingEngine` to directly communicate with peers to announce/request blocks. I've tried running `polkadot --network-backend litep2p` with a few different peer configurations and there is a noticeable reduction in networking CPU usage. For high load (`--out-peers 200`), networking CPU usage goes down from ~110% to ~30% (80 pp) and for normal load (`--out-peers 40`), the usage goes down from ~55% to ~18% (37 pp). These should not be taken as final numbers because: a) there are still some low-hanging optimization fruits, such as enabling [receive window auto-tuning](https://github.com/libp2p/rust-yamux/pull/176), integrating `Peerset` more closely with `litep2p` or improving memory usage of the WebSocket transport b) fixing bugs/instabilities that incorrectly cause `litep2p` to do less work will increase the networking CPU usage c) verification in a more diverse set of tests/conditions is needed Nevertheless, these numbers should give an early estimate for CPU usage of the new networking backend. This PR consists of three separate changes: * introduce a generic `PeerId` (wrapper around `Multihash`) so that we don't have use `NetworkService::PeerId` in every part of the code that uses a `PeerId` * introduce `NetworkBackend` trait, implement it for the libp2p network stack and make Polkadot SDK generic over `NetworkBackend` * implement `NetworkBackend` for litep2p The new library should be considered experimental which is why `rust-libp2p` will remain as the default option for the time being. This PR currently depends on the master branch of `litep2p` but I'll cut a new release for the library once all review comments have been addresses. --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: Dmitry Markin <dmitry@markin.tech> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Alexandru Vasile <alexandru.vasile@parity.io>

2024-04-08

Integrate litep2p into Polkadot SDK (#2944) [litep2p](https://github.com/altonen/litep2p) is a libp2p-compatible P2P networking library. It supports all of the features of `rust-libp2p` that are currently being utilized by Polkadot SDK. Compared to `rust-libp2p`, `litep2p` has a quite different architecture which is why the new `litep2p` network backend is only able to use a little of the existing code in `sc-network`. The design has been mainly influenced by how we'd wish to structure our networking-related code in Polkadot SDK: independent higher-levels protocols directly communicating with the network over links that support bidirectional backpressure. A good example would be `NotificationHandle`/`RequestResponseHandle` abstractions which allow, e.g., `SyncingEngine` to directly communicate with peers to announce/request blocks. I've tried running `polkadot --network-backend litep2p` with a few different peer configurations and there is a noticeable reduction in networking CPU usage. For high load (`--out-peers 200`), networking CPU usage goes down from ~110% to ~30% (80 pp) and for normal load (`--out-peers 40`), the usage goes down from ~55% to ~18% (37 pp). These should not be taken as final numbers because: a) there are still some low-hanging optimization fruits, such as enabling [receive window auto-tuning](https://github.com/libp2p/rust-yamux/pull/176), integrating `Peerset` more closely with `litep2p` or improving memory usage of the WebSocket transport b) fixing bugs/instabilities that incorrectly cause `litep2p` to do less work will increase the networking CPU usage c) verification in a more diverse set of tests/conditions is needed Nevertheless, these numbers should give an early estimate for CPU usage of the new networking backend. This PR consists of three separate changes: * introduce a generic `PeerId` (wrapper around `Multihash`) so that we don't have use `NetworkService::PeerId` in every part of the code that uses a `PeerId` * introduce `NetworkBackend` trait, implement it for the libp2p network stack and make Polkadot SDK generic over `NetworkBackend` * implement `NetworkBackend` for litep2p The new library should be considered experimental which is why `rust-libp2p` will remain as the default option for the time being. This PR currently depends on the master branch of `litep2p` but I'll cut a new release for the library once all review comments have been addresses. --------- Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io> Co-authored-by: Dmitry Markin <dmitry@markin.tech> Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com> Co-authored-by: Alexandru Vasile <alexandru.vasile@parity.io>

2024-01-16

Make Polkadot SDK generic over the networking implementation Implement `NetworkBackend` for `NetworkWorker`, introduce more traits related to making notification/request-response protocols generic over the underlying networking backend and start using these traits throughout the codebase. `sc_network::NetworkService` is no longer provided as an object but instead as `Arc<dyn NetworkService>` which abstracts the underlying networking backend without introducing generics across the entire codebase. Notification/request-response protocol configurations are generic over the networking backend as libp2p and litep2p have slightly different ways of providing said protocols and thus require custom ways of initializing them. The network backend be selected with `--network-backend` switch

2022-11-18

Fix the light client protocol protobuf schema (#12732) * Fix the light client protocol protobuf schema * Add another test * Remove unused protobuf struct * Ok you have to use the nightly rustfmt apparently

2022-10-12

Clarify the "direction" field of block requests (#12438)

2022-10-11

Remove the unused light client requests (#12470) * Remove the unused light client requests * Add comment about new ids

2022-10-08

Remove "to_block" field from BlockRequests (#12447) * Remove "to_block" field from BlockRequests * Maybe fix the tests

2022-09-13

Move bitswap to its own crate (#12242) * Introduce `sc-network-bitswap` Move the bitswap protocol out of `sc-network` to its own crate. * Improve test coverage * Remove mention of bitswap from `sc-network` * Fix documentation * Fix clippy and remove the void dependency * Remove unneeded trait bound and bump prost to 0.11

2022-06-17

Upgrade to libp2p 0.45.1 (#11682) * Upgrade to libp2p 0.45.1 * Limit max_negotiating_inbound_streams to 512 * Upgrade prost-build to 0.10 * Set max_negotiating_inbound_streams to 2048 Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * Fix authority discovery protobuf * Fix comments in authority-discovery schema Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * Add a comment about transport initialization Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

2022-05-14

Network sync refactoring (part 3) (#11347) * Move `light.v1.proto` schema into new crate `sc-network-light` * Move `sc_network::light_client_requests` and submodule to `sc_network_light::light_client_requests` * Fix apparently outdated reference in documentation and visibility modifier * Fix rustdoc check * Update lock file

2022-05-03

Network sync refactoring (part 2) (#11322) * Move `api.v1.proto` schema into new crate `sc-network-sync` * Move `sc_network::protocol::sync::state` module into `sc_network_sync::state` * Move `sc_network::protocol::sync::blocks` module into `sc_network_sync::blocks` and some data structures from `sc_network::protocol::message` module into `sc_network_sync::message` * Move some data structures from `sc_network::config` and `sc_network::request_responses` into new `sc-network-common` crate * Move `sc_network::protocol::sync::warm` and `sc_network::warp_request_handler` modules into `sc_network_sync` * Move `client/network/sync/src/lib.rs` to `client/network/sync/src/lib_old.rs` to preserve history of changes of the file in the next commit * Move `client/network/src/protocol/sync.rs` on top of `client/network/sync/src/lib.rs` to preserve history of changes * Move `sc_network::protocol::sync` to `sc_network_sync` with submodules, move message data structures around accordingly * Move `sc_network::block_request_handler` to `sc_network_sync::block_request_handler` * Move `sc_network::state_request_handler` to `sc_network_sync::state_request_handler` * Add re-exports for compatibility reasons * Apply suggestions from code review Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

2021-12-05

Authentication of PeerIds in authority discovery records (#10317) * Consolidating test and production code * Signing/verifying authority discovery records with PeerId Unsigned records cannot be rejected yet, they just produce a warning in the log. * Upgrading to libp2p 0.40 * libp2p::identity and sp_core::crypto Ed25519 are compatible * Rejecting authority records unsigned by peer id can be configured * Fixes based on review comments * No command-line argument needed * info was still too much spam in the logs * Added tests for both strict and loose validation * Fixing based on review comments * Pierre preferred a signing method * Ooops, I need to slow down * Update bin/node/cli/src/service.rs * Reexport libp2p crypto used in sc-network * Added proto3 compatibility tests. And import noise. Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

2021-11-07

Fast sync child trie support. (#9239) * state machine proofs. * initial implementation * Remove todo. * Extend test and fix import. * fix no proof, with proof ko. * fix start at logic. * Restore response size. * Rework comments. * Add explicit ref * Use compact proof. * ref change * elaborato on empty change set condition. * KeyValueState renaming. * Do not add two time child trie with same root to sync reply. * rust format * Fix merge. * fix warnings and fmt * fmt * update protocol id to V2

2021-07-17

Storage chains sync (#9171) * Sync storage chains * Test * Apply suggestions from code review Co-authored-by: cheme <emericchevalier.pro@gmail.com> * Separate block body and indexed body * Update client/db/src/lib.rs Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com> Co-authored-by: cheme <emericchevalier.pro@gmail.com> Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com> Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

2021-06-22

Fast sync (#8884) * State sync * Importing state fixes * Bugfixes * Sync with proof * Status reporting * Unsafe sync mode * Sync test * Cleanup * Apply suggestions from code review Co-authored-by: cheme <emericchevalier.pro@gmail.com> Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * set_genesis_storage * Extract keys from range proof * Detect iter completion * Download and import bodies with fast sync * Replaced meta updates tuple with a struct * Fixed reverting finalized state * Reverted timeout * Typo * Doc * Doc * Fixed light client test * Fixed error handling * Tweaks * More UpdateMeta changes * Rename convert_transaction * Apply suggestions from code review Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com> * Code review suggestions * Fixed count handling Co-authored-by: cheme <emericchevalier.pro@gmail.com> Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

2021-04-28

Support sending and receiving multiple Justifications (#8266) * client/network: support sending multiple justifications * network: flag support for multiple justifications in protobuf request * Update client/network/src/protocol.rs Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * network: update comment on protobuf field Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

2021-02-03

Storage chains: serve transactions over IPFS/bitswap (#7963) * IPFS server for transactions * Style * Indent * Log message * CLI option * Apply suggestions from code review Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com> * Style * Style * Minor fixes Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

2020-11-23

grandpa: remove light-client specific block import pipeline (#7546) * grandpa: remove light-client specific block import * consensus, network: remove finality proofs

2020-08-12

client/authority-discovery: Introduce AuthorityDiscoveryService (#6760) * client/authority-discovery: Rename AuthorityDiscovery to XXXWorker * client/authority-discovery: Introduce AuthorityDiscoveryService Add a basic `AuthorityDiscoveryService` implementation which enables callers to get the addresses for a given `AuthorityId` from the local cache. * client/authority-discovery: Split into worker and service mod Move `Service` and `Worker` to their own Rust modules resulting in the following file structure. ├── build.rs ├── Cargo.toml └── src ├── error.rs ├── lib.rs ├── service.rs ├── tests.rs ├── worker │   ├── addr_cache.rs │   ├── schema │   │   └── dht.proto │   └── tests.rs └── worker.rs * client/authority-discovery: Cache PeerId -> AuthorityId mapping * client/authority-discovery: Update priority group on interval Instead of updating the authority discovery peerset priority group each time a new DHT value is found, update it regularly on an interval. This removes the need for deterministic random selection. Instead of trying to return a random stable set of `Multiaddr`s, the `AddrCache` now returns a random set on each call. * client/authority-discovery: Implement Service::get_authority_id * client/authority-discovery: Use HashMap instead of BTreeMap * client/authority-discovery: Rework priority group interval * client/authority-discovery: Fix comment * bin/node/cli: Update authority discovery constructor * client/authority-discovery: Fuse from_service receiver * client/authority-discovery: Remove Rng import * client/authority-discovery: Ignore Multiaddr without PeerId * client/authority-discovery/service: Add note on returned None * client/authority-discovery/addr_cache: Replace double clone with deref

2020-04-30

Move around stuff in sc_network (#5847)

2020-04-24

Use new block requests protocol (#5760) * Use new block requests protocol * Tweak comment

2020-04-22

Add a protocol that answers finality proofs (#5718) * Add a protocol that answers finality proofs * Fix documentation * Use Toggle

2020-04-20

Child trie api changes BREAKING (#4857) Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

2020-03-02

Remove request ID from the new protocol (#5049)

2020-02-17

Fix various spelling errors (#4940) * meaningfull -> meaningful * initialise -> initialize * tokans -> tokens * incentivise -> incentivize * lenght -> length * incentivisation -> incentivization * doesnt't -> doesn't * overwriten -> overwritten * lifecycle -> life cycle * lifecycle -> life cycle * usefull -> useful * noone -> no one * spaming -> spamming * defered -> deferred * hieght -> height * sumation -> summation * ingore -> ignore * registed -> registered * Auxialary -> Auxiliary * loggin -> logging * independance -> independence * trailling -> trailing * responsability -> responsibility * trunkated -> truncated * Weither -> Whether * informations -> information * Runtume -> Runtime * choosen -> chosen * delcataion -> declaration * Unchekced -> Unchecked * defintion -> definition * scrach -> scratch * imput -> input * transfered -> transferred * endownment -> endowment * Determinator -> Determiner * relevent -> relevant * emited -> emitted * acocunt -> account * proprotional -> proportional * instantiaion -> instantiation * commited -> committed * tombstonedead -> tombstone * uwnrap -> unwrap * acount -> account * specialised -> specialized * existant -> existent * requried -> required * Anull -> Annul * AUTHORITES -> AUTHORITIES * underyling -> underlying * recognisable -> recognizable * Capitalise -> Capitalize * reportfor -> report for * hearbeat -> heartbeat * onlineness -> being online * creater -> creator * Bytearray -> Byte array * Despoit -> Deposit * substratced -> subtracted * Curent -> Current * imbalanes -> imbalances * countfown -> countdown * inexisting -> inexistent * additionaly -> additionally * substracted -> subtracted * auxilary -> auxiliary * parital -> partial * in't -> isn't * compatability -> compatibility * infomation -> information * etected -> detected * extrinsiscs -> extrinsics * reprensentation -> representation * coonfiguration -> configuration * primtives -> primitives * miscelanious -> miscellaneous * VERISON -> VERSION * endcoded -> encoded * Genrates -> Generates * miliseconds -> milliseconds * occured -> occurred * trully -> truely * truely -> truly * conjuction -> conjunction * encouters -> encounters * customised -> customized * deterministicly -> deterministically * finalisation -> finalization * pluggable -> plugable * wakeup -> wake-up * interemdiate -> intermediate * intepreting -> interpreting * finalzied -> finalized * throgh -> through * extinsic -> extrinsic * convient -> convenient * allocater -> allocator * propagateable -> propagatable * succesfuly -> successfully * finalising -> finalizing * publically -> publicly * phrasee -> phrase * substration -> substractions * substractions -> subtractions * neccessarily -> necessarily * Inlucde -> Include * unefficient -> inefficient * thay -> they * funtion -> function * datastructures -> data structures * infromation -> information * propagatable -> propagable * ecountered -> encountered * recognise -> recognize * intergration -> integration * lastet -> latest * datatypes -> data types * datatype -> data type * Strongarming -> Strong Arming * avaible -> available * Commiting -> Committing * Retreiving -> Retrieving * shoud -> should * canonicaliziation -> canonicalization * comitted -> committed * clonable -> cloneable * Uknown -> Unknown * reponse -> response * arbitary -> arbitrary * Capapbilities -> Capabilities * responsbile -> responsible * initialisation -> initialization * cames -> came * intemediate -> intermediate * reqeust -> request * intance -> instance * explcitly -> explicitly * neighor -> neighbor * reolving -> resolving * untill -> until * Validte -> Validate * deserailize -> deserialize * literaly -> literally * preceeding -> preceding * abpve -> above * chcecked -> checked * numbet -> number * Unknow -> Unknown * halfs -> halves * gossup -> gossip * givent -> given * immediatelly -> immediately * slicable -> sliceable * conensus -> consensus * Mimicks -> Mimics * acccept -> accept * serialise -> serialize * exstrinsics -> extrinsics * panicks -> panics * maintaince -> maintenance * repeatidely -> repeatedly * anecstor -> ancestor * becasue -> because * processer -> processor * Prunning -> Pruning * insterested -> interested * unuseful -> not useful * yeided -> yielded * descendfing -> descending * corresponts -> corresponds * survivew -> survive * keps -> keeps * ligh -> light * prerequisities -> prerequisites * positiion -> position * depedency -> dependency * extrinisic -> extrinsic * atomicaly -> atomically * staticly -> statically * resul -> result * timestamb -> timestamp * Utilites -> Utilities * ammount -> amount * pocess -> process * exteral -> external * Update client/finality-grandpa/src/tests.rs * Update primitives/io/src/lib.rs Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update primitives/blockchain/src/lib.rs Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update frame/support/src/weights.rs Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update bin/node/cli/tests/common.rs Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update client/api/src/execution_extensions.rs Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update client/cli/src/params.rs Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update client/executor/common/src/sandbox.rs Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update client/api/src/execution_extensions.rs Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update client/finality-grandpa/src/communication/mod.rs Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update client/state-db/src/pruning.rs Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update frame/contracts/src/tests.rs Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com> * Update client/api/src/execution_extensions.rs * bump impl * timestamb -> timestamp Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

2020-02-12

network: Use "one shot" protocol handler. (#3520) * network: Use "one shot" protocol handler. Add two new `NetworkBehaviour`s, one handling remote block requests and another one to handle light client requests (both local and from remote). The change is motivated by the desire to use multiple substreams of a single connection for different protocols. To achieve this, libp2p's `OneShotHandler` is used as a protocol handler in each behaviour. It will open a fresh substream for the duration of the request and close it afterwards. For block requests, we currently only handle incoming requests from remote and tests are missing. For light client handling we support incoming requests from remote and also ported a substantial amount of functionality over from `light_dispatch.rs` (including several tests). However the result lacks in at least two aspects: (1) We require external updates w.r.t. the best block per peer and currently nothing updates this information. (2) We carry a lot of peer-related state around. Both aspects could be simplified by externalising peer selection and just requiring a specific peer ID where the request should be sent to. We still have to maintain some peer related state due to the way libp2p's swarm and network behaviour work (e.g. we must make sure to always issue `NetworkBehaviourAction::SendEvent`s to peers we are connected to, otherwise the actions die a silent death. Another change implemented here is the use of protocol buffers as the encoding for network messages. Certain individual fields of messages are still SCALE encoded. There has been some discussion about this in another PR (https://github.com/paritytech/substrate/pull/3452), so far without resolution. * Uncomment `Behaviour::light_client_request`. * Add license headers.

2019-11-14

Reorganising the repository - external renames and moves (#4074) * Adding first rough ouline of the repository structure * Remove old CI stuff * add title * formatting fixes * move node-exits job's script to scripts dir * Move docs into subdir * move to bin * move maintainence scripts, configs and helpers into its own dir * add .local to ignore * move core->client * start up 'test' area * move test client * move test runtime * make test move compile * Add dependencies rule enforcement. * Fix indexing. * Update docs to reflect latest changes * Moving /srml->/paint * update docs * move client/sr-* -> primitives/ * clean old readme * remove old broken code in rhd * update lock * Step 1. * starting to untangle client * Fix after merge. * start splitting out client interfaces * move children and blockchain interfaces * Move trie and state-machine to primitives. * Fix WASM builds. * fixing broken imports * more interface moves * move backend and light to interfaces * move CallExecutor * move cli off client * moving around more interfaces * re-add consensus crates into the mix * fix subkey path * relieve client from executor * starting to pull out client from grandpa * move is_decendent_of out of client * grandpa still depends on client directly * lemme tests pass * rename srml->paint * Make it compile. * rename interfaces->client-api * Move keyring to primitives. * fixup libp2p dep * fix broken use * allow dependency enforcement to fail * move fork-tree * Moving wasm-builder * make env * move build-script-utils * fixup broken crate depdencies and names * fix imports for authority discovery * fix typo * update cargo.lock * fixing imports * Fix paths and add missing crates * re-add missing crates

2019-09-06

core/authority-discovery: Enable authorities to discover each other (#3452) With the *authority-discovery* module an authoritative node makes itself discoverable and is able to discover other authorities. Once discovered, a node can directly connect to other authorities instead of multi-hop gossiping information. 1. **Making itself discoverable** 1. Retrieve its external addresses 2. Adds its network peer id to the addresses 3. Sign the above 4. Put the signature and the addresses on the libp2p Kademlia DHT 2. **Discovering other authorities** 1. Retrieve the current set of authorities 2. Start DHT queries for the ids of the authorities 3. Validate the signatures of the retrieved key value pairs 4. Add the retrieved external addresses as ~reserved~ priority nodes to the peerset * node/runtime: Add authority-discovery as session handler The srml/authority-discovery module implements the OneSessionHandler in order to keep its authority set in sync. This commit adds the module to the set of session handlers. * core/network: Make network worker return Dht events on poll Instead of network worker implement the Future trait, have it implement the Stream interface returning Dht events. For now these events are ignored in build_network_future but will be used by the core/authority-discovery module in subsequent commits. * *: Add scaffolding and integration for core/authority-discovery module * core/authority-discovery: Implement module logic itself