Proto commits in sonm-io/core

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

2019-08-01

feat: remote secure shell (#1895) This commit allows exposing a server on a host, which act as a remote secure shell, executing allowed commands within the security policy.

The documentation is generated from this commit.

feat: add bid and ask tags to dwh deal (#1896)

2019-07-19

Revert "feat: ablitily to extract bid and ask tags from DWH (#1886)" (#1893) This reverts commit 76bbd8cf0e9724bf7e22bd81e9e02091d7c8344f.

2019-07-17

feat: sys init capability for QoS service (#1891)

2019-07-02

feat: ablitily to extract bid and ask tags from DWH (#1886)

2019-05-21

feat: task OOM notification (#1879)

2019-02-19

aux: add package name to proto files (#1854)

2019-02-15

feat: remote GPU tuner (#1853)

2019-01-30

fix: flush IFB on demand when using remote service (#1844)

2019-01-29

feat: remote QoS (#1841) Changelog: exclude. Required for TE.

2019-01-25

feat: worker inspection (#1835) Currently, if something goes wrong on a Worker we're still able to gain access to the remote computer via SSH to find of what's going on there. In the future, due to obvious trusted platform limitations, we will lose those capabilities, since there will be no direct access to the machine. This leads us to a situation where we still need to collect debugging information, but more likely in a post-mortem way. This feature introduces the inspection service on the Worker side, which is capable to collect various runtime and history information and provide it to the subject that is explicitly allowed by Worker's admin. New methods include config, list of open files, network information, various Docker info and of course - logs. Now it is possible both for Worker's admin and allowed users to subscribe to Worker logs. Also, Worker's API now allows to add and to remove additional capabilities like inspection service access or SSH'ing into the host.

2019-01-23

refactor: drop devices storage (#1837)

2019-01-16

WIP

2018-12-19

feat: devices storage (#1738) Smart contract for storing worker devices in blockchain. This PR includes contract which stores transparent worker data as byte array, and node and cli interface for fetching that data. Worker updates it's devices during startup as json-serialized Hardware structure (including benchmarks), which can be fetched for each worker via cli interface.

2018-12-03

fix: proper RAM metrics collection (#1769)

2018-11-28

refactor: drop unused abi bindings (#1760)

2018-11-26

added method stub

feat: intoducing tensor server

2018-11-25

docs: add description for expose parameter and provide example (#1743)

2018-11-23

feat: experimental QUIC protocol support (#1724) Tread lightly. What is it now? Speak, fool. At last. No one orders me around! Introducing QUIC protocol support in most of the components. QUIC is an experimental transport layer network protocol, which we use as an alternative to TCP to achieve reliable communication between our components. The main reason to include this is to increase connection establishing stability between any 2 peers, which are located under private networks. QUIC is designed at top of UDP, which is more predictable and investigated for NAT penetration, unlike we do for TCP in a hacky way. Since QUIC allows to multiplex many connections into a single socket there is no need to penetrate NAT for the same client-server pair multiple times. After the first successful attempt, the punched hole will be reused. The Rendezvous server has been updated to support UDP hole punching using QUIC protocol. It additionally exposes its gRPC server on the same UDP port as for TCP, since it is possible to reuse ports in case of different network protocols used. The wire-level left the same - gRPC. Affected components: Worker, Node, Rendezvous. Both Node and Worker now expose its gRPC services on both TCP and UDP sockets, the port number is the same. At last, Windows users now can utilize our NAT punching feature, since multiplexed UDP communication does not require SO_REUSEPORT flag. So relaying now isn't the only single option for Windows. Note, that this feature is currently in an experimental stage. You can try it by specifying "SONM_ENABLE_QUIC=true" environment while starting Node server.

2018-11-21

feat: background metrics collector for worker (#1715) This PR Introduces hardware metrics of the worker, consists of the following parts: * MetricsHandler plugin bound to worker's instance, it's responsible for handling hardware-specific metrics providers (currently for GPUs only), updating the whole set of metrics by the timer, return last known state. * gRPC method for WorkerManagement API, can be used to retrieve metrics by the master or admin address.

2018-11-14

feat: worker geolocation (#1692) Now Workers are capable to detect the country ISO code where they are located and provide it in the status handler.

2018-11-13

feat: extended network metrics in Worker (#1691) The extended network metrics are now returned when requesting container/task stats. This includes traffic/packet minute-averaged rates. Required for determining the network pressure caused by a task.

2018-11-07

feat: return order IDs after price prediction (#1682)

2018-10-18

chore: migrate metrics protofile to proto3 (#1653) It is required for some gRPC generators, which doesn't support proto2. No wire breaking effects noticed.

feat: multiple pools (#1579) This PR introduces separate resource pools for spot resources, forward resources and committed (involved in the deal) resources, providing an ability to control worker more smoothly.

2018-10-17

feat: added BidTag to DWHDeal

2018-10-15

feat: blacklist purge method (#1639)

feat: able to stop multiple tasks (#1504) This commit adds a new method to the Node API which provides able to stop several tasks at once.

2018-10-05

aux: migrate to go.mod (#1597)

refactor: deprecate TaskManagement service (#1539) This PR deprecates TaskManagement service, providing raw proxy to worker service via interceptor

feat: price prediction for suppliers (#1594) Node service is now capable to obtain an approximate predicted price for devices configuration, i.e. implements price prediction for suppliers.

2018-10-04

change: single shot ask plans (#1541) This PR changes ask-plan behavior, now after the order is canceled, or a deal is closed ask plan is removed. This should work better with Optimus bot, as it does continuous optimization and also simplifies the code, allowing to make separate scheduling for forwarding deals

2018-10-02

feat: introducing GetOrdersByIDs method in DWH (#1570) This PR introduces a method to DWH API that allows you to get orders with required IDs.

2018-09-25

feat: dwh stats handle (#1540) The new GetStats() handle returns current DWH stats.

2018-09-24

wip

2018-09-13

feat: extended status for Worker (#1456) This commit extends status method with following data: - Master and admin addresses. - Are benchmarks passed? - Is the master address confirmed? We are forced to start gRPC server before benchmarks passed and master confirmed. Now, on worker's startup, only WorkerManagement service will be registered, and no NPP listener used. When all of the setup routines completed, we're closing gRPC server and re-create it with fully-registered services and NPP listener.

2018-09-07

feat: add push_on_stop flag to container desc (#1426) This PR introduces a push_on_stop flag to task specification that makes pushing to remote repository conditional.

2018-09-05

feat: NPP metrics in Node (#1392) This commit adds NPP metric collection on a client side, allowing to see how exactly connections are established, which kind of NAT punching tool was used and how much time it was consumed. Also there is a new service in Node - monitoring, where one can fetch collected metrics.

2018-08-30

chore: worker's code cleanup (#1367) - Removed several stale TODOs, which was introduced during previous large refactorings. - Method's docstrings fixed to match the golang's standards. - Import and variable names clashing avoided.

2018-08-29

chore: mark node's Balance method as deprecated (#1357)

feat: parallel order and deal purging (#1354) This PR extends order and deal API, providing an ability to remove several orders and deals at a time (including full purge). This is done on Node in a parallel manner reducing the time needed for operation.

2018-08-24

feat: price predictor service (#1254) This commit allows to optionally activate order price predictor service on Node. This service is capable of continuous training on current marketplace's state to be able to predict an order price from its benchmarks and netflags.

2018-08-22

feat: show public IPs in deal status (#1335)

2018-08-20

feat: info handle in Relay (#1328) This commit activates info handle in Relay server, allowing to view current server's state, similar how we do in Rendezvous.

2018-08-15

feat: dwh l2 (#1295)

2018-08-09

feat: allow to obtain balance by specified wallet (#1294) This commit extends Node API to be able to call blockchain's `balanceOf` with given eth address as parameter. On the CLI-side there is still a single method but with an optional extra parameter.

2018-08-06

refactor: update gRPC

2018-08-01

refactor: better task tag (#1241)

2018-07-27

feat: optional tag for task (#1236)

2018-07-26

feat: optionaly expand deal's orders in cli (#1238)

2018-07-23

feat: SNM token transfer for sidechain This commit allows to transfer tokens from the logged in account to another address in the Sidechain.

2018-07-19

feat: ping worker before deal (#1196) Now node trying to connect to the worker before opening deal (both OpenDeal and QuickBuy). If a worker is not responding to the status request - deal will not be opened and user will be noticed about worker unavailability. User can force deal opening with the `--force` flag passed to related CLI's sub-commands.

2018-07-17

feat: show Ethereum balance on live network (#1203) This commit extends the Ethereum API Client with `GetEthereumBalanceAt` method that allows to obtain Ethereum balance. Now `TokenAPI` returns an `Balance` struct which represents balances in both Eth and SNM tokens. Thus protobuf's API extended to be able to represent both values. And finally, CLI able to print balance also in Ethereum (for Masterchain).

2018-06-30

fix: validator logo -> kyc_price (dwh) (#1144)

feat: add ask plan timestamps (#1134) This PR adds creation timestamp orderPlaced timestamp and function to get time for which order has not been sold, which can be useful for Optimus.

2018-06-29

feat: GetProfiles userID like-search (DWH) (#1125)

2018-06-28

feat: ability to drop benchmark cache (#1115)

fix: proper check for worker availability for the deal (#1102) This commit adds a check that worker is available when the user making deal with `quick-buy` handle. Also, this commit remove a message that warns the user about the worker inaccessibility when showing deal list.

refactor: extend NPP identifier with protocol (#1104) This allows to specify some protocol you want to publish/resolve as the Rendezvous allows. It is required for further NAT traversal extensions. What is changed: not relay protocol allows to specify "protocol" field in order to match between different protocols for the same peer ID. In both Relay and Rendezvous meeting groups are now associated with complex string representation a-la URI. This is the BREAKING change. Both Rendezvous, Worker and Node must be updated. Relay, however, does not need to.

2018-06-27

feat: worker debug state (#1084) This PR adds handle to worker to retrieve useful debugging info, e.g. current resource scheduler state and salesman state.

2018-06-26

feat: uptime in Relay metrics (#1086) Now Relay will return its uptime while answering for its metrics handle.

2018-06-25

feat: dwh extended validators (#1063)

2018-06-21

feat: allowance handle for node and cli (#1050)

2018-06-20

feat: deposit-withdraw for node and cli (#986) This PR adds able to transfer tokens between Masterchain and SONM sidechain.

feat: AnyUserID field for DealsRequest in DWH (#1031) This introduces a new search field to DealsRequest: AnyUserID. If provided, supplierID, consumerID, and masterID fields are ignored. Any of the mentioned IDs must match the provided value.

aux: redeploy contracts (#998)

2018-06-19

feat: profile management for cli and node (#1002) This PR adds profile management support for cli and node: - The `profile status` command shows profile details; - The `profile remove-attr` commands removes attribute from current user's profile. - Profile list can be obtained via the node API, but not implemented in cli.

feat: order purge handle for cli and node (#997)

2018-06-15

fix: store certificate id in DWH (#1001)

feat: quick buy for cli and node (#990) This PR adds able to perform quick buy for given ASK order on market.

2018-06-14

feat: maintenance for worker (#960)

2018-06-13

feat: implemented GetBlacklistsContainingUser for DWH (#973) GetBlacklistsContainingUser() returns IDs of all users that blacklisted target user ID. This is a BREAKING change.

feat: dwh orders blacklist filters (#978)

refactor: fix and refactor netflags (#962)

2018-06-08

fix: profile stats & updates (#961)

2018-06-04

feat: fake GPU device (#927) This PR adds new GPU driver that allows to emulate GPU-device persistence for any testing purposes.

2018-05-30

feat: MasterID field for orders (#912)

2018-05-29

fix: mangle registry host inside reference (#871)

feat: change requests API (#793) Added: - Extend blockchain API with `CreateChangeRequest` and `CancelChangeRequest` methods. - Extend Node API and cli with change request management commands.

fix: CounterpartyID (DWH) (#867)

2018-05-28

feat: manually allow to expose ports (#850)

refactor: drop meaningless embedding of StatusMapReply (#840)

feat: country & identity level multiselect (DWH) (#848)

2018-05-25

refactor: start task is splitted to settings and dealID (#833)

feat: container restart policy (#830)

refactor: parse task config like a god This commit allows task configs to be loaded directly from YAML file without intermediate boilerplate parsing. This was real pain in ass. Snake case is also supported. This change allows to remove large chunk of code, opening a door for easier refactoring or/and new feature additions. Changed: - Removed top-level "task" container - no need for this. - Field "name" renamed to "image". - Registry "user" renamed to "username" to match Docker spec. - Registry "name" renamed to "server_address" to match Docker spec. - StartTaskRequest can now be validated. - Replace assertions with require in tests.

2018-05-24

feat: manage blacklists from Node (#766) Added: - `list` and `remove` methods into cli and node. `list` accepts an optional parameter to observe other user's blacklists. - Blockchain's BlacklistAPI implementation now waiting for transaction results.

feat: 'ask-plan purge' handle (#818)

2018-05-21

refact: rename all hub mentionings (#784)

refact: drop miner proto (#782)

refactor: dead code elimination (#781)

2018-05-18

feat: master-worker relationships (#775)

feat: add --blacklist flag for deal finish cmd (#764)

2018-05-17

chore: dwh refactoring (#739) This PR introduces vast changes to DWH code structure. * All DB-specific code is removed from DWH-server to sqlStorage, an implementation of the storage interface. * SQL-commands & DB-initialization code shared by various backends (Postgres and SQLite) are decomposed into sqlStorage. * Redundant locks/unlocks are removed (we expect the utilised databases to handle locks). numBenchmarks type is changed to uint64 throughout DWH. * A minor bug is fixed: if Order.DealID was not specified, profile stats were not updated while processing OrderUpdated event. * Tests were prettified (no 700-line tests anymore).

2018-05-15

chore: unify counterparty for bid and ask yamls (#760)

fix: show exposed ports when starting a task (#756) - portMap transfers as mapping between containerPort and endpoints, not string. - converting from Docker's `nat.PortMap` to protobuf performing only once - at response serializing. - Accidentally removed useless `minerID` field from TaskStatus message.

2018-05-11

Feat: variable benchmarks, cold start & fixes (DWH) (#705) * benchmarks number can grow up to 100 values, * `cold start` is introduced: indices are not built until a certain block is processed, * fixed counterparty filter (zero eth address), * fixed onOrderUpdated for inactive orders (this will fix profile stats) * fixed grpc return codes (added `NotFound`)

feat: added count to response (DWH) (#726)

2018-05-10

feat: extend deal info with tasks and resources (#712)