Proto commits in vmware/concord-bft

These 65 commits are when the Protocol Buffers files have changed:

2023-05-01

Introduce generic header to accomodate Eth and Non-Eth blocks (#2986) 1. Generic header would be added at V4 block adder layer to bring uniformity in block structure. 2. Block, Transactions and other relevant definitions would be moved from proto to CMF format. Co-authored-by: Mahaveer Darade <mdarade@vmware.com>

2023-04-27

Remove redundant proto file containing block and transaction (#2952) Athena also has this redundant proto file. All the relevant components would be referring to types defined in this single proto file. Co-authored-by: Mahaveer Darade <mdarade@vmware.com>

2023-04-07

Application specific data storage in privacy wallet service (#2977) * protobuf messages for application data * methods to set and get generic application data in FileBasedUserStorage for privacy service * align TestUTTClientApi * make FileBasedUserStorage a member of PrivacyWalletServiceImpl * add set/get app data request/response to PrivacyWallet protobuf messages * implementation of SetAppData and GetAppData messages handling in privacy service * unit test for getting not set app data value from privacy service * make set/get app data fields repeated * change field ordering in PrivacyWalletRequest proto message * remove duplicated test after rebase

The documentation is generated from this commit.

2023-03-23

Add signature field to response proto object (#2963)

2023-03-15

Clientservice and replica handling for primary-only logic for Filter APIs (#2955)

2023-03-14

send filter api request to the first validator node id

2023-03-09

Wallet service fixes (#2951) * move user implementation to store coins in map instead of a vector * Add ability to get a map of nullifier->value for existing normal coins

2023-02-26

add user_id parameter in GetStateResponse of PrivacyService (#2946)

2023-02-15

privacy wallet service flows (mint, budget, transfer, and burn) (#2929) * last executed sequence number is an application/wallet abstraction. This commit is to remove this notion from the User module * Implement mint, burn, transact, and claimCoins flows * Unitests for budget and mint * remove userPKI header file * Add unitests for privacy service flows * Remove streaming for stated transactions * Add getState request * fix proto file * fix proto file * fix proto file * Add number of output coins * catch exception to figure out error precisely, add debug logs around for development, cleanit up later * Address reviews * Address reviews * Fix tidy checks --------- Co-authored-by: Senthil Balakrishnan <balakrishnas@vmware.com>

2023-02-01

Wallet-service - configure & registration flows (#2924) In this PR we present two flows for the wallet service configure - initiate the wallet with the utt public configuration register - register a user. This flow is combined of two sub-flows: (a) get registration input, and (b) update registration s2 and signature In addition, some functionality regarding certificates and crypto was added, in order to complete the full flows in the frontend

2023-01-24

break files to granular level. The next push would comprise of collecting all common wallet code into a library

2023-01-18

draft make

2022-12-12

Moving execution service api to open source

2022-11-20

Privacy feature (#2859) * extend replica api to return some necessry objects needed for utt * Add INTERNAL_FLAG to message's flags * Add INTERNAL_FLAG to message's flags * Fix internal flags issues * fix compilation error * add pragma once * add some necessary libutt api * Fix an error with randomness when creating a coin by the replicas * add an option to create determisitc coin commitment for coins created by replicas * Separate the wallet in its own source file * The wallet now assumes only a single user. The user id is provided as a command line argument * WIP budget, mint, transfer, burn operations * wallet fixes * utt client api user now has more consistent error handling by throwing exceptions instead of returning a bool result. Allow updateMintTx and updateBurnTx to ignore the transaction if the owner is someone else * Add support for mint transactions originating from the wallet * Add number of outputs to utt::Transaction * TestUTTClientApi now relies on the fact that updateBurnTx and updateMintTx ignore mints/burns intended for a different user when synchronizing * Multiplex multiple users in the same wallet for initial testing purposes * Rename wallet command 'info' to 'show' * Add a way to pre-create budget tokens in the sample wallet * Fix creation of budget locally to not happen before registration * Check if the wallet exists when processing commands * Add comment about the workaround dealing with the wallet service not returning complete messages occasionaly when getting tx data * Add public balance to wallet-cli * Refactor wallet cli app to avoid excessive nesting when validating input. Add a simple debug output to inspect the coins of a user * Rework wallet-cli to use a bi-directional stream for all wallet requests * Fix pki was referenced as a local variable. Renaming * Initial split of the privacy-admin-cli application with just the deploy command from utt-wallet-cli * Transform the deploy command into a configure command in the wallet-cli. Instead of generating and deploying a config, the configure command will fetch an already deployed configuration by the admin-cli * Use the public config obtained from the privacy contract * Make wallet-cli single user by providing a user-id as an argument * Add full validation capabilites * Fix a bug where the new api transaction object exposes the input coins * Cleanup admin and wallet cli outputs. Demote some loginfo to logdbg statements in libutt. Fix wallet-cli reading transfer instead of burn response (didn't affect correctness) * Improve the new utt api testing framework, integrating it with gtest and remove old files * Fix clang-tidy checks * Add getter for partial signatures map from the complete signature message * complete rebase * Add debug logs to user in utt-client-api * Add ability to create budget from admin cli * Remove option to create budgets from wallet. This is done now by the admin. Remove requirement to get the full privacy config - this was only needed for temporary creation of budget tokens locally * Move utt new api to pImpl * Simple e2e test binary for utt privacy feature * Add randomness to rcm1 (rcm1 in created by the client, so it's deterministic) * Add public transfer command * Debug logs in e2e test for utt privacy feature * disable NTL_THREADS in libutt * Extract E2eTestSuite class in e2e privacy feature testing * Separate e2e privacy test scenario from test execution env - E2eTestScenario class extracted Co-authored-by: Evgeniy Dzhurov <edzhurov@vmware.com> Co-authored-by: cloudnoize <elerer@vmware.com> Co-authored-by: Maciej Grychowski <maciej.grychowski@onet.eu>

2022-11-17

Add public transfer command

Remove option to create budgets from wallet. This is done now by the admin. Remove requirement to get the full privacy config - this was only needed for temporary creation of budget tokens locally

Add ability to create budget from admin cli

Simple e2e test binary for utt privacy feature

Transform the deploy command into a configure command in the wallet-cli. Instead of generating and deploying a config, the configure command will fetch an already deployed configuration by the admin-cli

Add public balance to wallet-cli

Fix creation of budget locally to not happen before registration

Add number of outputs to utt::Transaction

Add support for mint transactions originating from the wallet

Rework wallet-cli to use a bi-directional stream for all wallet requests

Initial split of the privacy-admin-cli application with just the deploy command from utt-wallet-cli

WIP budget, mint, transfer, burn operations

2022-10-07

Copy gRPC health.proto to concord-bft This commit adds gRPC health.proto to concord-bft. Since, we don't intend to use the default implementation of gRPC health-check, copying the proto files is required.

Add support for deployment and registration of users through the sample wallet cli app (#2827) * WIP Sample wallet cli app * WIP wallet cli app * libutt::api curve points can now be de/serialized as byte vectors to be consistent with other data. In this case we need to pass S2 from the server to the client which is a curve point * Fix compilation after rebase onto master * Check if the UTT client api is initialized for each top-level function * Move test keys to a separate source file. Use vector<uint64_t> for s2 since the rest of the system uses this format - it needs to be made consistent with other opaque data types and use vector<uint8_t> eventually. WIP wallet-cli * Add a TestUserPKInfrastructure with a number of pre-generate RSA keys for testing * Remove unrelated changes

2022-09-01

Add skeleton of a utt wallet command-line application

2022-08-24

Add skeleton of a utt wallet command-line application

2022-08-23

Change subproject from wallet-app to wallet-cli. Add primary UTT interface to the wallet grpc api

2022-08-19

Add a test utt wallet app

2022-08-18

Test utt wallet app using gRPC

2022-04-29

Correct SubscribeRequest description in event.proto

2022-04-26

Added changes to address review comments and standardisation of error messages

2022-04-11

Add changes to create concord request to send the application request along with additional data from client.

2022-03-30

Revert "Added changes to fix compatibility issue for modules using raw request/response and typed_request/response" This reverts commit 2231ed4e1c44edf368b907fdf62557687810cc78.

2022-03-28

Added changes to fix compatibility issue for modules using raw request/response and typed_request/response 1> Rename raw request to old attribute name 2> Removed one of constraint for backward compatibility for referring to raw request type. 3> Replaced changes in code during request and response processing

2022-03-09

Update new Any request in request.proto file to handle both application request type during transition wq# Please enter the commit message for your changes. Lines starting

2022-02-14

Add support for block IDs in GetRecentSnapshotResponse Add support for both event group IDs and block IDs when returning the point in the blockchain's history at which the state snapshot was taken. For more details, see the description of the `taken_at` field in StateSnapshotService.GetRecentSnapshotResponse.

Add support for block IDs in GetRecentSnapshotResponse Add support for both event group IDs and block IDs when returning the point in the blockchain's history at which the state snapshot was taken. For more details, see the description of the `taken_at` field in StateSnapshotService.GetRecentSnapshotResponse.

2022-02-03

Add `ledger_time` to `GetRecentSnapshotResponse` The ledger time at which the snapshot was taken.

2022-01-24

Rename StateSnapshotService to ReplicaStateSnapshotService

2022-01-18

Merge pull request #2224 from robem/fix-events-doc Update events.proto doc

2022-01-14

Add state_snapshot gRPC interface b/w TRC-TRS This commit add state_snapshot.proto, that defines gRPC interface between TRC and TRS for state streaming.

Update events.proto doc EventGroups start at index 1. Zero is an invalid event group id. This change updates the documentation accordingly.

2022-01-11

Fix naming in the State Snapshot proto file * rename `StateSnapshot` to `StateSnapshotService` in order to conform to service naming conventions * fix a reference to the `last_received_key` field (`from_key` was wrongly used there)

2022-01-10

Return reply data from the client pool to the client service when operation succeeds (#2204) * Revert "Revert "bft_error_handling: Moving status code to OperationResult"" This reverts commit 5bc1ff4fbc1152dd2a533cb81e766ca61d734c21. * Return reply data from the client pool to the client service when operation succeeds.

2021-11-25

Add FAILED_PRECONDITION as error for StreamSnapshot Rationale is that we might return it if Concord Client cannot verify the state from the replicas (i.e. the signatures don't match). It can be used in different scenarios too.

2021-11-16

Add FAILED_PRECONDITION as error for StreamSnapshot Rationale is that we might return it if Concord Client cannot verify the state from the replicas (i.e. the signatures don't match). It can be used in different scenarios too.

2021-11-03

Introduce Concord Client's StateSnapshot service (#1950) The `StateSnapshot` service can be used to initialize a client application from a state snapshot visible to Concord Client. Details and explanations are given as comments inside the protobuf file. This interface is the output of work mostly done by @meiersi-da, @andrewjstone and @robem.

2021-10-26

EventGroup: Update tracing This change propagates the trace context from concord to the caller of concord client. Thereby, we leverage the existing legacy interface (bytes) for internal trace context transfer (via thin_replica.proto) but expose it in W3C HTTP header format within a "text map". Note, the functionality has not been end-to-end tested given that we still lack a tracer implementation (such as Jaeger). This will be introduced soon and should uncover any bugs.

2021-08-03

EventService: Event group start at 1 In the protobuf definition, we are stating that the first event group is 0 but in fact it is 1. In addition, we've been setting the "first" last verified state to the requested id and thereby starting at the requested id + 1. The protobuf documentation for the event service clearly states that the subscription request will return updates *starting* at the given ID.

2021-07-29

Clientservice: Move proto API Given that the API itself is a `vmware.concord.client` API, it should live under the `client` directory. Clientservice simply implements the API but is not in the picture when it comes to its definition.

Clientservice: Split services in two We have two different gRPC services. The recommendation is to split each of them into their own file and adjust the directory structure so that versioning becomes easier.

Clientservice: syntax, package, import, option As per Google's recommendation.

2021-07-21

ConcordClient: Legacy events in proto In order to support the events API that existed before concord client, we introduce Events in concord_client.proto as a possible value to be streamed to the caller. Essentially, events and event groups are different formats on the blockchain and therefore, the caller might retrieve both types. However, we're switching to the latter and hence, once the stream turned to EventGroups it won't go back to Events.

2021-07-20

ThinReplica: Add EventGroups in thin_replica.proto With this PR we're adding EventGroups to the thin replica protocol. EventGroups are documented in concord_client.proto. With this change in place, we can send EventGroups from TRS to the TRC.

2021-06-30

ConcordClient: Correct subscribe/unsubscribe behavior At the moment, our API is limited to one subscription only. The previous code didn't handle subscribe/unsubscribe properly which lead to various segfaults. This change introduces an exception which is thrown if a subscription is active already.

2021-06-29

Remove key_prefix from thin_replica.proto

2021-06-24

ClientService: Document event stream limitation At the moment, the thin replica client only supports a single subscription at a time. Given that concord client will be backed by the TRC this limitation will extend to the gRPC interface. Ofc, this is not intuitive for a gRPC service but we hope to correct this in the future.

2021-05-28

Copy TRS to concord-bft This commit copies TRS files to concord-bft. The purpose of this commit is to provide a reference for the reviewer to review the changes made to the files in question for their move to open-source.

Add concord_kvbc.proto to concord-bft This commit adds concord_storage.proto to kvbc/proto folder in concord-bft undre the name concord_kvbc.proto. The proto file defines a message type `ValueWithTrids` that stores with values with their corresponding ACLs, in this case, TRIDs (thin-replica IDs). On the basis of these TRIDs, values are filtered in kvbc_app_filter.cpp for TRS subscribers.

2021-05-12

Add thin_replica.proto to concord-bft

2021-05-03

ConcordClient: Service interface With this change, we publish the future interface for the Concord Client service. In short, Concord Client service is a wrapper around the Concord Client library which includes a pool of bft clients and a subscription mechanism. Concord Client and its components will be introduced and documented in future changes.