package kudu.rpc

Get desktop application:
View/edit binary Protocol Buffers messages

An authentication type. This is modeled as a oneof in case any of these authentication types, or any authentication types in the future, need to add extra type-specific parameters during negotiation.

Used in: NegotiatePB

• oneof type

  • AuthenticationTypePB.Sasl sasl = 1

    The server and client mutually authenticate via SASL.

  • AuthenticationTypePB.Token token = 2

    The server authenticates the client via a signed token, and the client authenticates the server by verifying its certificate has been signed by a trusted CA. Token authentication requires the connection to be TLS encrypted.

  • AuthenticationTypePB.Certificate certificate = 3

    The server and client mutually authenticate by certificate. Certificate authentication requires the connection to be TLS encrypted.

Used in: AuthenticationTypePB

(message has no fields)

Used in: AuthenticationTypePB

(message has no fields)

Used in: AuthenticationTypePB

(message has no fields)

The connection context is sent as part of the connection establishment. It establishes the context for ALL RPC calls within the connection. This is sent on connection setup after the connection preamble is sent and SASL has been negotiated. No response is sent from the server to the client.

• optional UserInformationPB DEPRECATED_user_info = 2

  UserInfo beyond what is determined as part of security handshake at connection time (kerberos, tokens etc). DEPRECATED: No longer used in Kudu 1.1 and later. The 'real_user' should be taken from the SASL negotiation. Impersonation (effective user) was never supported, so we'll have to add that back at some point later.

• optional bytes encoded_nonce = 3

  If the server sends a nonce to the client during the SASL_SUCCESS negotiation step, the client is required to encode it with SASL integrity protection and return it in this field. The nonce protects the server against a Kerberos replay attack.

• optional bool include_traces = 1

• repeated RpcConnectionPB inbound_connections = 1

• repeated RpcConnectionPB outbound_connections = 2

Request and response for dumping previously sampled RPC calls.

(message has no fields)

• repeated RpczMethodPB methods = 1

Sent as response when is_error == true.

• required string message = 1

• optional ErrorStatusPB.RpcErrorCodePB code = 2

  TODO: Make code required?

  Specific error identifier.

• repeated uint32 unsupported_feature_flags = 3

  If the request is failed due to an unsupported feature flag, the particular flag(s) that were not supported will be sent back to the client.

These codes have all been inherited from Hadoop's RPC mechanism.

Used in: ErrorStatusPB

• FATAL_UNKNOWN = 10

• ERROR_APPLICATION = 1

  Non-fatal RPC errors. Connection should be left open for future RPC calls. ------------------------------------------------------------ The application generated an error status. See the message field for more details.

• ERROR_NO_SUCH_METHOD = 2

  The specified method was not valid.

• ERROR_NO_SUCH_SERVICE = 3

  The specified service was not valid.

• ERROR_SERVER_TOO_BUSY = 4

  The server is overloaded - the client should try again shortly.

• ERROR_INVALID_REQUEST = 5

  The request parameter was not parseable, was missing required fields, or the server does not support the required feature flags.

• ERROR_REQUEST_STALE = 6

  The server might have previously received this request but its response is no longer cached. It's unknown whether the request was executed or not.

• ERROR_UNAVAILABLE = 7

  The server is not able to complete the connection or request at this time. The client may try again later.

• ERROR_INVALID_AUTHORIZATION_TOKEN = 17

  The authorization token is invalid or expired. Unlike FATAL_INVALID_AUTHENTICATION_TOKEN, receipt of this code does not mean that a reconnection attempt should be made; just that the client should obtain a new authz token.

• FATAL_SERVER_SHUTTING_DOWN = 11

  FATAL_* errors indicate that the client should shut down the connection. ------------------------------------------------------------ The RPC server is already shutting down.

• FATAL_INVALID_RPC_HEADER = 12

  Fields of RpcHeader are invalid.

• FATAL_DESERIALIZING_REQUEST = 13

  Could not deserialize RPC request.

• FATAL_VERSION_MISMATCH = 14

  IPC Layer version mismatch.

• FATAL_UNAUTHORIZED = 15

  Auth failed.

• FATAL_INVALID_AUTHENTICATION_TOKEN = 16

  The authentication token is invalid or expired. The connection negotiation failed, and the client should obtain a new authn token and try to reconnect.

Message type passed back & forth for the SASL negotiation.

• repeated RpcFeatureFlag supported_features = 1

  When the client sends its NEGOTIATE step message, it sends its set of supported RPC system features. In the response to this message, the server sends back its own. This allows the two peers to agree on whether newer extensions of the RPC system may be used on this connection. We use a list of features rather than a simple version number to make it easier for the Java and C++ clients to implement features in different orders while still maintaining compatibility, as well as to simplify backporting of features out-of-order.

• required NegotiatePB.NegotiateStep step = 2

  The current negotiation step.

• optional bytes token = 3

  The SASL token, containing either the challenge during the SASL_CHALLENGE step, or the response during the SASL_RESPONSE step.

• optional bytes tls_handshake = 5

  During the TLS_HANDSHAKE step, contains the TLS handshake message.

• optional bytes channel_bindings = 6

  The tls-server-end-point channel bindings as specified in RFC 5929. Sent from the server to the client during the SASL_SUCCESS step when the Kerberos (GSSAPI) SASL mechanism is used with TLS, in order to bind the Kerberos authenticated channel to the TLS channel. The value is integrity protected through SASL. The client is responsible for validating that the value matches the expected value.

• optional bytes nonce = 9

  A random nonce sent from the server to the client during the SASL_SUCCESS step when the Kerberos (GSSAPI) SASL mechanism is used with TLS. The nonce must be sent back to the server, wrapped in SASL integrity protection, as part of the connection context.

• repeated NegotiatePB.SaslMechanism sasl_mechanisms = 4

  During the NEGOTIATE step, contains the supported SASL mechanisms. During the SASL_INITIATE step, contains the single chosen SASL mechanism.

• repeated AuthenticationTypePB authn_types = 7

  During the client to server NEGOTIATE step, contains the supported authentication types. During the server to client NEGOTIATE step, contains the chosen authentication type.

• optional security.SignedTokenPB authn_token = 8

  During the TOKEN_EXCHANGE step, contains the client's signed authentication token.

Used in: NegotiatePB

• UNKNOWN = 999

• NEGOTIATE = 1

• SASL_SUCCESS = 0

• SASL_INITIATE = 2

• SASL_CHALLENGE = 3

• SASL_RESPONSE = 4

• TLS_HANDSHAKE = 5

• TOKEN_EXCHANGE = 6

Used in: NegotiatePB

• required string mechanism = 2

  The SASL mechanism, i.e. 'PLAIN' or 'GSSAPI'.

Used in: RequestHeader

• required string service_name = 1

  Service name for the RPC layer. The client created a proxy with this service name. Example: kudu.rpc_test.CalculatorService

• required string method_name = 2

  Name of the RPC method.

The header for the RPC request frame.

Used in: RpcCallInProgressPB, RpczSamplePB

• required int32 call_id = 3

  A sequence number that uniquely identifies a call to a single remote server. This number is sent back in the Response and allows to match it to the original Request. Hadoop specifies a uint32 and casts it to a signed int. That is counterintuitive, so we use an int32 instead. Allowed values (inherited from Hadoop): 0 through INT32_MAX: Regular RPC call IDs. -2: Invalid call ID. -3: Connection context call ID. -33: SASL negotiation call ID. NOTE: these calls must be increasing but may have gaps.

• optional RemoteMethodPB remote_method = 6

  RPC method being invoked. Not used for "connection setup" calls.

• optional uint32 timeout_millis = 10

  Propagate the timeout as specified by the user. Note that, since there is some transit time between the client and server, if you wait exactly this amount of time and then respond, you are likely to cause a timeout on the client.

• repeated uint32 required_feature_flags = 11

  Feature flags that the service must support in order to properly interpret this request. The client can pass any set of flags, and if the server doesn't support any of them, then it will fail the request. NOTE: these are for evolving features at the level of the application, not the RPC framework. Hence, we have to use a generic int type rather than a particular enum. NOTE: the server will only interpret this field if it supports the APPLICATION_FEATURE_FLAGS flag.

• optional RequestIdPB request_id = 15

  The unique id of this request, if it's retriable and if the results are to be tracked. The request id is unique per logical request, i.e. retries of the same RPC must have the same request id. Note that this is different from 'call_id' in that a call_id is unique to a server while a request_id is unique to a logical request (i.e. the request_id remains the same when a request is retried on a different server). Optional for requests that are naturally idempotent or to maintain compatibility with older clients for requests that are not.

• repeated uint32 sidecar_offsets = 16

  Byte offsets for side cars in the main body of the request message. These offsets are counted AFTER the message header, i.e., offset 0 is the first byte after the bytes for this protobuf.

The Id of a retriable RPC, whose results should be tracked on the server (see result_tracker.h). This also includes some information that is useful for execution/garbage collection.

Used in: RequestHeader

• required string client_id = 1

  The (globally unique) id of the client performing this RPC.

• required int64 seq_no = 2

  The (per-client unique) sequence number of this RPC.

• required int64 first_incomplete_seq_no = 3

  The sequence number of the first RPC that has not been marked as completed by the client. Unset if there isn't an incomplete RPC.

• required int64 attempt_no = 4

  The number of times this RPC has been tried. Set to 1 in the first attempt.

• required int32 call_id = 1

• optional bool is_error = 2

  If this is set, then this is an error response and the response message will be of type ErrorStatusPB instead of the expected response type.

• repeated uint32 sidecar_offsets = 3

  Byte offsets for side cars in the main body of the response message. These offsets are counted AFTER the message header, i.e., offset 0 is the first byte after the bytes for this protobuf.

Used in: RpcConnectionPB

• required RequestHeader header = 1

• optional string trace_buffer = 2

• optional uint64 micros_elapsed = 3

• optional RpcCallInProgressPB.State state = 4

Used in: RpcCallInProgressPB

• UNKNOWN = 999

• ON_OUTBOUND_QUEUE = 1

  States for OutboundCall

• SENDING = 2

• SENT = 3

• TIMED_OUT = 4

• FINISHED_ERROR = 5

• FINISHED_SUCCESS = 6

• NEGOTIATION_TIMED_OUT = 7

• FINISHED_NEGOTIATION_ERROR = 8

• CANCELLED = 9

Debugging information about a currently-open RPC connection.

Used in: DumpConnectionsResponsePB

• required string remote_ip = 1

• required RpcConnectionPB.StateType state = 2

• optional string remote_user_credentials = 3

  TODO: swap out for separate fields

• repeated RpcCallInProgressPB calls_in_flight = 4

• optional int64 outbound_queue_size = 5

• optional SocketStatsPB socket_stats = 6

  Information on the actual TCP connection as reported by the kernel.

• optional TransportDetailsPB transport_details = 7

  Transport-specific details.

Used in: RpcConnectionPB

• UNKNOWN = 999

• NEGOTIATING = 0

  Connection is still being negotiated.

• OPEN = 1

  Connection is active.

Features supported by the RPC system itself. Note that this should be used to evolve the RPC _system_, not the semantics or compatibility of individual calls. For example, if we were to add a feature like call or response wire compression in the future, we could add a flag here to indicate that the client or server supports that feature. Optional features which may safely be ignored by the receiver do not need a feature flag, instead the optional field feature of ProtoBuf may be utilized.

Used in: NegotiatePB

• UNKNOWN = 0

• APPLICATION_FEATURE_FLAGS = 1

  The RPC system is required to support application feature flags in the request and response headers.

• TLS = 2

  The RPC system supports TLS protected connections. If both sides support this flag, the connection will automatically be wrapped in a TLS protected channel following a TLS handshake.

• TLS_AUTHENTICATION_ONLY = 3

  If both sides advertise TLS_AUTHENTICATION_ONLY, this means that they agree that, after handshaking TLS, they will *not* wrap the connection in a TLS-protected channel. Instead, they will use TLS only for its handshake-based authentication. This is currently used for loopback connections only, so that compute frameworks which schedule for locality don't pay encryption overhead.

A set of samples for a particular RPC method.

Used in: DumpRpczStoreResponsePB

• required string method_name = 1

• repeated RpczSamplePB samples = 2

A single sampled RPC call.

Used in: RpczMethodPB

• optional RequestHeader header = 1

  The original request header.

• optional string trace = 2

  The stringified request trace.

• optional int32 duration_ms = 3

  The number of millis that this call took to complete.

• repeated TraceMetricPB metrics = 4

  The metrics from the sampled trace.

The SocketStatsPB message is used to report on socket-level information for RPC-related TCP connections (Linux-only). Essentially, the message contains some metrics and counters from Linux-specific 'tcp_info' structure defined in /usr/include/linux/tcp.h. For more information on the TCP on Linux, see http://man7.org/linux/man-pages/man7/tcp.7.html

Used in: RpcConnectionPB

• optional uint32 rtt = 1

• optional uint32 rttvar = 2

• optional uint32 snd_cwnd = 3

• optional uint32 total_retrans = 4

• optional uint64 pacing_rate = 5

• optional uint64 max_pacing_rate = 6

• optional uint64 bytes_acked = 7

• optional uint64 bytes_received = 8

• optional uint32 segs_out = 9

• optional uint32 segs_in = 10

• optional uint64 send_queue_bytes = 11

• optional uint64 receive_queue_bytes = 12

• optional int64 send_bytes_per_sec = 13

  Calculated sender throughput.

A particular TraceMetric key/value pair from a sampled RPC.

Used in: RpczSamplePB

• optional string child_path = 1

  A '.'-separated path through the parent-child trace hierarchy.

• optional string key = 2

• optional int64 value = 3

Transport-related information for an RPC connection.

Used in: RpcConnectionPB

• optional TransportDetailsPB.TcpDetails tcp = 1

• optional TransportDetailsPB.TlsDetails tls = 2

TCP-specific details.

Used in: TransportDetailsPB

• optional int32 max_segment_size = 1

  Maximum segment size for the packets: this directly maps into the TCP_MAXSEG socket option.

TLS-specific details. NOTE: TLS/SSL doesn't map nicely into a single layer of the TCP/IP or the OSI model, but intuitively that's something related to the transport.

Used in: TransportDetailsPB

• optional string protocol = 1

  The name of the TLS protocol negotiated to protect the connection (e.g. TLSv1.3).

• optional string cipher_suite = 2

  Description of the TLS cipher suite used.

User Information proto. Included in ConnectionContextPB on connection setup.

Used in: ConnectionContextPB, rpc_test.WhoAmIResponsePB

• optional string effective_user = 1

• required string real_user = 2