Proto commits in isc-projects/stork

These 59 commits are when the Protocol Buffers files have changed:

Commit:f359631
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#2396] XFR status in textual format

Commit:1738b59
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#2395] Documented XFR fields in agent.proto

Commit:9840be9
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#2395] Added gRPC endpoint to receive XFR

Commit:818f7a8
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#2394] Store interfaces instead of addrs

Commit:e218eed
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#2394] Renamed ipAddresses to MachineIPAddresses

Commit:b8eb500
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#2394] Include IP addrs in machine state

Commit:1fd6fc0
Author:William Leuschner
Committer:William Leuschner

[#2059] Fix indentation; shrink some repetitions

Commit:6d1df57
Author:William Leuschner
Committer:William Leuschner

[#2059] More backend test fixes

Commit:db859f6
Author:William Leuschner
Committer:William Leuschner

[#2059] Fix duped leases from same-agent daemons Due to a design oversight in the gRPC message format (which I would have avoided if I had looked at every other gRPC message a bit more closely), the Stork server polling each agent *once per daemon* caused duplicated leases in the database. Each agent was replying with the list of every lease from every daemon it knew of. The gRPC message now asks for a specific daemon by access point address/port, ensuring that the returned leases are attributed to the correct daemon and only returned once.

Commit:966822a
Author:William Leuschner
Committer:William Leuschner

[#2058] rename ip_version to family

Commit:506656a
Author:William Leuschner
Committer:William Leuschner

[#2057] Additional changes requested in review * Stop the MemfileSnooper by sending to the channel, not by closing it * Don't export error constants in the test module * Remove leftover debugging log calls * Rename leaseKey.ID to leaseKey.Identifier (as Stork uses ID for database primary keys) * Switch the gRPC API from returning to streaming, to simplify the implementation. * Added missing documentation comment on allowLeaseTracking() * Changed allowLeaseTrackingFlag to isLeaseTrackingAllowed, following Stork's conventions * Move the `status-get` response deserialization code into a separate file, rather than wedging it into the `config-get` file * Add JSON omit tag to IPVersion in lease.go * Use IPType from backend/util/cidr.go instead of making a substantially identical enum in lease.go

Commit:b8952bd
Author:William Leuschner
Committer:William Leuschner

[#2057] Add GetKeaLeases() to agent gRPC API The final implementation step: use the MemfileSnooper's GetSnapshot() function to obtain a snapshot of the current lease status, encode that snapshot into a packed format (to avoid having very large Protobufs, since the docs recommend against that), and then reply to the request with the snapshot.

Commit:09045ea
Author:William Leuschner
Committer:William Leuschner

[#2057] Add Lease Protobuf structure This new Protobuf structure stores a DHCPv4 or DHCPv6 lease. Storing both kinds of lease in one structure will make all of the code which handles them simpler, and avoid incurring performance costs associated with generic code in Go.

Commit:721bd34
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#2201] Force populating the zones

Commit:02407c6
Author:Slawek Figiel
Committer:Slawek Figiel

[#1835] Direct Kea API support

Commit:d98ac4e
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#1634] Code cleanup Removed unused field from the agent.proto and fixed typos in the comments

Commit:ae3341f
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#1634] Return BIND 9 config over a stream

Commit:41e5f13
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#1634] Define filtering by BIND 9 file type

Commit:3ca9ed4
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#1634] Get BIND 9 config over gRPC

Commit:a797826
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#1805] Agent gathers and sends RPZ flag

The documentation is generated from this commit.

Commit:84b27fb
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#1751] Detecting and monitoring PowerDNS server

Commit:6f216c1
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#1751] Fetch uptime from PowerDNS

Commit:7cec92d
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#1751] Populate the PowerDNS info in server

Commit:38d23b6
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#1714] Extend RRs in API defs

Commit:22e6af3
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#1714] Send AXFR to fetch the zone from DNS

Commit:21c457c
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#1784] Improved communication with stats The gRPC API and the BIND9 HTTP client are better suited to handle long lasting calls to the stats channel. The client avoids making calls to the root path of the stats channel, and thus avoid fetching zones. The gRPC call for ForwardToNamedStats now uses a set of parameters instead of URL to specify the stats channel host, port and request type. The request types are represented by an enum, so it is harder to make a mistake calling invalid endpoint.

Commit:f03d14c
Author:Marcin Siodelski
Committer:Piotrek Zadroga

[#1669] Comments in the api file

Commit:8091d98
Author:Marcin Siodelski
Committer:Piotrek Zadroga

[#1669] Populate zones from agents to server

Commit:577c539
Author:Marcin Siodelski
Committer:Piotrek Zadroga

[#1669] New gRPC call to fetch zones from agent

Commit:3cbe0db
Author:Slawek Figiel
Committer:Slawek Figiel

[#1624] Refactor to HTTP client

Commit:b685c08
Author:Andrei Pavel
Committer:Andrei Pavel

[#1566] Fix typos

Commit:4bded27
Author:Slawek Figiel
Committer:Slawek Figiel

[#1059] Remove existing agent compression

Commit:24a53b0
Author:Slawek Figiel
Committer:Slawek Figiel

[#945] Return credentials status from Agent

Commit:fab83a2
Author:Slawek Figiel
Committer:Slawek Figiel

[#709] Remove unecessary blank line

Commit:f0fc3a9
Author:Slawek Figiel
Committer:Slawek Figiel

[#709] Single point change in api

Commit:e1862f8
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#652] Bump up protoc versions

Commit:fa6364d
Author:Slawek Figiel
Committer:Slawek Figiel

[#347] Remove unused parameters

Commit:0db998d
Author:Slawek Figiel
Committer:Slawek Figiel

[#527] Fetch SSL status from Kea and use HTTPS

Commit:5af66c8
Author:Michal Nowikowski
Committer:Michal Nowikowski

[#481] agent server TLS - part 2 - added code for registering a machine in the server and performing key and certs exchange but ut us not used fully yet - added server-token and agent-token based agent authorizations but for now it is disabled - added API for presenting and regenerating server token but it is not used in UI yet - updated content of reference agent.env agent config file

Commit:d55f567
Author:Michal Nowikowski
Committer:Michal Nowikowski

[#398] fixed problem with adding kea with 4500 subnets

Commit:1bfdb00
Author:Michal Nowikowski
Committer:Michal Nowikowski

[#344] addressed review issues

Commit:ebaf081
Author:Marcin Siodelski
Committer:Michal Nowikowski

[#344] Fix tailing beyond file size

Commit:142ffbc
Author:Marcin Siodelski
Committer:Michal Nowikowski

[#344] Tailing log file Added agent function to tail the log file and the corresponding server side function.

Commit:6e0fab1
Author:Matthijs Mekking
Committer:Matthijs Mekking

[#64] Implement agentcomm fwd stats request Introduce ForwardToNamedStats that handles forwarding statistics request to named statistics-channel and returning the response to the server.

Commit:7100731
Author:Matthijs Mekking
Committer:Matthijs Mekking

[#64] Introduce agent.ForwardToNamedStats Introduce a Stork agent function to forward a statistics request to the named daemon statistics-channel. The statistics-channel access point is, just like with Kea Control Agent, represented by an URL. This reuses the CAClient. The CAClient has a Kea (Control Agent) specific name but should be renamed because it is now also used for the BIND 9 named statistics-channel.

Commit:a1292f3
Author:Matthijs Mekking
Committer:Matthijs Mekking

[#170] Change api/agent.proto Change the api to allow multiple access points per app.

Commit:d9088df
Author:Michal Nowikowski
Committer:Michal Nowikowski

[#162] fixed args naming in agent.proto in ForwardRndcCommand

Commit:b632d82
Author:Matthijs Mekking
Committer:Matthijs Mekking

[#130] Forward rndc command from server Redesign such that the server will issue the command to the agent and the agent will forward the command to rndc (that in its turn will contact named). This moves logic from the agent to the server, and will require new rpc Messages.

Commit:d67f0ff
Author:Matthijs Mekking
Committer:Matthijs Mekking

[#130] Add control key to app Add `CtrlKey` to App structure and model. This is a generic application parameter to interact with the application, similar to control address and control port. Kea in principle does not use the `CtrlKey` and will leave it empty. For BIND 9 it is optional, but common.

Commit:1b12d39
Author:Michal Nowikowski
Committer:Michal Nowikowski

[#136] added storing kea config in db Agent monitor has been streamlined. It detects only a presence of application. Now Stork server when learns that there is a new app then calls agent for extra information. In case of Kea app it invokes a series of ForwardToKeaOverHTTP agent calls which are passed to Kea CA. Server asks for versions, state and configuration of Kea daemons. In case of BIND 9 app server calls GetBind9State which gathers BIND 9 specific information e.g. named version. Getting Kea hooks information is reduced. Now the data is retrieved from db which is stored there in the phase described above.

Commit:9e05cf1
Author:Michal Nowikowski
Committer:Michal Nowikowski

[#114] fixes for golangci-lint tool Changes include: * automatic fixes by golangci-lint * manual fixes for golangci-lint * removed all globals * added CAClient to avoid httpClient11 global, this involves creating NewStorkAgent function, and replaces client.go * reorganized imports, now they are in the following order (separated by empty line): stdlib imports, 3rd party packages, our stork packages * all acronyms are using capital letters now (eg. URL instead of Url) * replaces app types from explicit strings like "kea" to consts like KeaAppType * machineToRestApi func does not return error anymore so error handling has been removed * split Serve function from restservice.go to reduce its complexity * reordered return values so error is always last

Commit:ba5d1a9
Author:Matthijs Mekking
Committer:Matthijs Mekking

[#106] add Bind9 application Add the Bind9 application to Stork agent, api, and backend. This is almost a copy of the kea application but changed it to be Bind9 specific. Add some unit tests to make rake happy, and ignore low test coverage for updateMachineFields[Bind9,Kea] for now.

Commit:6af3c36
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#110] Control address returned from agent

Commit:b44f0c0
Author:Marcin Siodelski
Committer:Marcin Siodelski

[#109] Implemented forwarding Kea command Added new GRPC command to the Agent, to forward the Kea command to the Kea instance over HTTP. The command name is ForwardToKeaOverHttp to highlight that it is intended for sending the commands over Kea's Control Agent (HTTP). In the future it is envisaged that we will also have ForwardToKeaOverUnix which will be for forwarding the command over the UNIX domain socket. This future command will lack the service list and the response will be slightly different, therefore it makes sense to split them into two commands.

Commit:0b09538
Author:Michal Nowikowski
Committer:Michal Nowikowski

[#24] renamed service to app

Commit:106e480
Author:Michal Nowikowski
Committer:Matthijs Mekking

[#24] added basic support for services - added detecting kea in agent - added storing service information in database - added in webui presenting services list and information about particular service

Commit:8b20c83
Author:Michal Nowikowski
Committer:Michal Nowikowski

[#23] added initial getting list of machines

Commit:951bceb
Author:Michal Nowikowski

[#70] improved gRPC API definition

Commit:2145d66
Author:Michal Nowikowski
Committer:Michal Nowikowski

[#26] prepared initial agent with gRPC and connected ReST api with agents part - server connects to agents using gRPC - added defining commandline arguments to agent and server - improved CI, switched to our own docker image for gitlab ci pipelines