Get desktop application:
View/edit binary Protocol Buffers messages
The service implemented by conformance test servers. This is implemented by the reference servers, used to test clients, and is expected to be implemented by test servers, since this is the service used by reference clients. Test servers must implement the service as described.
A bidirectional-streaming operation. The first request indicates the response headers, response messages, trailers, and an optional error to send back. The response data should be sent in the order indicated, and the server should wait between sending response messages as indicated. Response message data is specified as bytes and should be included in the data field of the ConformancePayload in each response. Servers should send responses indicated according to the rules of half duplex vs. full duplex streams. Once all responses are sent, the server should either return an error if specified or close the stream without error. Servers should immediately send response headers on the stream before sleeping for any specified response delay and/or sending the first message so that clients can be unblocked reading response headers. If a response definition is not specified OR is specified, but response data is empty, the server should skip sending anything on the stream. Stream headers and trailers should always be set on the stream if provided regardless of whether a response is sent or an error is thrown. If the full_duplex field is true: - the handler should read one request and then send back one response, and then alternate, reading another request and then sending back another response, etc. - if the server receives a request and has no responses to send, it should throw the error specified in the request. - the service should echo back all request properties in the first response including the last received request. Subsequent responses should only echo back the last received request. - if the response_delay_ms duration is specified, the server should wait the given duration after reading the request before sending the corresponding response. If the full_duplex field is false: - the handler should read all requests until the client is done sending. Once all requests are read, the server should then send back any responses specified in the response definition. - the server should echo back all request properties, including all request messages in the order they were received, in the first response. Subsequent responses should only include the message data in the data field. - if the response_delay_ms duration is specified, the server should wait that long in between sending each response message.
Tells the server how to reply; required in the first message in the stream. Should be ignored in subsequent messages.
Tells the server whether it should wait for each request before sending a response. If true, it indicates the server should effectively interleave the stream so messages are sent in request->response pairs. If false, then the response stream will be sent once all request messages are finished sending with the only delays between messages being the optional fixed milliseconds defined in the response definition. This field is only relevant in the first message in the stream and should be ignored in subsequent messages.
Additional data for subsequent messages in the stream. Also used to pad the request size to test large request messages.
The conformance payload to respond with
A client-streaming operation. The first request indicates the response headers and trailers and also indicates either a response message or an error to send back. Response message data is specified as bytes. The service should echo back request properties, including all request messages in the order they were received, in the ConformancePayload and then include the message data in the data field. If the input stream is empty, the server's response will include no data, only the request properties (headers, timeout). Servers should only read the response definition from the first message in the stream and should ignore any definition set in subsequent messages. Servers should allow the response definition to be unset in the request and if it is, set no response headers or trailers and return no response data. The returned payload should only contain the request info.
Tells the server how to reply once all client messages are complete. Required in the first message in the stream, but should be ignored in subsequent messages.
Additional data for subsequent messages in the stream. Also used to pad the request size to test large request messages.
The conformance payload to respond with
A unary endpoint denoted as having no side effects (i.e. idempotent). Implementations should use an HTTP GET when invoking this endpoint and leverage query parameters to send data.
The response definition which should be returned in the conformance payload
Additional data. Only used to pad the request size to test large request messages.
The conformance payload to respond with.
A server-streaming operation. The request indicates the response headers, response messages, trailers, and an optional error to send back. The response data should be sent in the order indicated, and the server should wait between sending response messages as indicated. Response message data is specified as bytes. The service should echo back request properties in the first ConformancePayload, and then include the message data in the data field. Subsequent messages after the first one should contain only the data field. Servers should immediately send response headers on the stream before sleeping for any specified response delay and/or sending the first message so that clients can be unblocked reading response headers. If a response definition is not specified OR is specified, but response data is empty, the server should skip sending anything on the stream. When there are no responses to send, servers should throw an error if one is provided and return without error if one is not. Stream headers and trailers should still be set on the stream if provided regardless of whether a response is sent or an error is thrown.
The response definition which should be returned in the conformance payload.
Additional data. Only used to pad the request size to test large request messages.
The conformance payload to respond with
A unary operation. The request indicates the response headers and trailers and also indicates either a response message or an error to send back. Response message data is specified as bytes. The service should echo back request properties in the ConformancePayload and then include the message data in the data field. If the response_delay_ms duration is specified, the server should wait the given duration after reading the request before sending the corresponding response. Servers should allow the response definition to be unset in the request and if it is, set no response headers or trailers and return no response data. The returned payload should only contain the request info.
The response definition which should be returned in the conformance payload
Additional data. Only used to pad the request size to test large request messages.
The conformance payload to respond with.
A unary endpoint that the server should not implement and should instead return an unimplemented error when invoked.
(message has no fields)
(message has no fields)
Describes one call the client should make. The client reads these from stdin and, for each one, invokes an RPC as directed and writes the results (in the form of a ClientCompatResponse message) to stdout.
Used in:
The name of the test that this request is performing. When writing test cases, this is a required field.
Test suite YAML definitions should NOT set values for these next nine fields (fields 2 - 10). They are automatically populated by the test runner. If a test is specific to one of these values, it should instead be indicated in the test suite itself (where it defines the required features and relevant values for these fields). The HTTP version to use for the test (i.e. HTTP/1.1, HTTP/2, HTTP/3).
The protocol to use for the test (i.e. Connect, gRPC, gRPC-web).
The codec to use for the test (i.e. JSON, proto/binary).
The compression to use for the test (i.e. brotli, gzip, identity).
The server host that this request will be sent to.
The server port that this request will be sent to.
If non-empty, the server is using TLS. The bytes are the server's PEM-encoded certificate, which the client should verify and trust.
If present, the client certificate credentials to use to authenticate with the server. This will only be present when server_tls_cert is non-empty.
If non-zero, indicates the maximum size in bytes for a message. If the server sends anything larger, the client should reject it.
The fully-qualified name of the service this test will interact with. If specified, method must also be specified. If not specified, defaults to "connectrpc.conformance.v1.ConformanceService".
The method on `service` that will be called. If specified, service must also be specified. If not specified, the test runner will auto-populate this field based on the stream_type.
The stream type of `method` (i.e. unary, client stream, server stream, full-duplex bidi stream, or half-duplex bidi stream). When writing test cases, this is a required field.
If protocol indicates Connect and stream type indicates Unary, this instructs the client to use a GET HTTP method when making the request.
Any request headers that should be sent as part of the request. These include only custom header metadata. Headers that are part of the relevant protocol (such as "content-type", etc) should not be stated here.
The actual request messages that will sent to the server. The type URL for all entries should be equal to the request type of the method. There must be exactly one for unary and server stream methods but can be zero or more for client and bidi stream methods. For client and bidi stream methods, all entries will have the same type URL.
The timeout, in milliseconds, for the request. This is equivalent to a deadline for the request. If unset, there will be no timeout.
Wait this many milliseconds before sending a request message. For client or bidi stream methods, this delay should be applied before each request sent.
If present, the client should cancel the RPC instead of allowing to complete normally.
The following field is only used by the reference client. If you are implementing a client under test, you may ignore it or respond with an error if the client receives a request where it is set. When this field is present, it defines the actual HTTP request that will be sent. The above group of fields must still be provided and valid so that the reference client knows how it should try to interpret the server's response.
Used in:
These fields determine the timing of cancellation. If none are present, the client should cancel immediately after all request messages are sent and the send side is closed (as if the after_close_send_ms field were present and zero).
When present, the client should cancel *instead of* closing the send side of the stream, after all requests have been sent. This applies only to client and bidi stream RPCs.
When present, the client should delay for this many milliseconds after closing the send side of the stream and then cancel. This applies to all types of RPCs. For unary and server stream RPCs, where the API usually does not allow explicitly closing the send side, the cancellation should be done immediately after invoking the RPC (which should implicitly send the one-and-only request and then close the send-side). For APIs where unary RPCs block until the response is received, there is no point after the request is sent but before a response is received to cancel. So the client must arrange for the RPC to be canceled asynchronously before invoking the blocking unary call.
When present, the client should cancel right after reading this number of response messages from the stream. When present, this will be greater than zero. This applies only to server and bidi stream RPCs.
The outcome of one ClientCompatRequest.
The test name that this response applies to.
These fields determine the outcome of the request. With regards to errors, any unexpected errors that prevent the client from issuing the RPC and following the instructions implied by the request can be reported as an error. These would be errors creating an RPC client from the request parameters or unsupported/illegal values in the request (e.g. a unary request that defines zero or multiple request messages). However, once the RPC is issued, any resulting error should instead be encoded in response.
The client is not able to fulfill the ClientCompatRequest. This may be due to a runtime error or an unexpected internal error such as the requested protocol not being supported. This is completely independent of the actual RPC invocation.
Used in:
A message describing the error that occurred. This string will be shown to users running conformance tests so it should include any relevant details that may help troubleshoot or remedy the error.
The result of a ClientCompatRequest, which may or may not be successful. The client will build this message and return it back to the test runner.
Used in: ,
All response headers read from the response.
Servers should echo back payloads that they received as part of the request. This field should contain all the payloads the server echoed back. Note that There will be zero-to-one for unary and client stream methods and zero-to-many for server and bidi stream methods.
The error received from the actual RPC invocation. Note this is not representative of a runtime error and should always be the proto equivalent of a Connect or gRPC error.
All response headers read from the response.
The number of messages that were present in the request but that could not be sent because an error occurred before finishing the upload.
The following field is only set by the reference client. It communicates the underlying HTTP status code of the server's response. If you are implementing a client-under-test, you should ignore this field and leave it unset.
This field is used only by the reference client, and it can be used to provide additional feedback about problems observed in the server response or in client processing of the response. If non-empty, the test case is considered failed even if the result above matches all expectations. If you are implementing a client-under-test, you should ignore this field and leave it unset.
Used in: ,
Used in: , , ,
not used; will be ignored
Used in: , , , ,
Config defines the configuration for running conformance tests. This enumerates all of the "flavors" of the test suite to run.
The features supported by the client or server under test. This is used to filter the set of test cases that are run. If absent, an empty message is used. See Features for more on how empty/absent fields are interpreted.
This can indicate additional permutations that are supported that might otherwise be excluded based on the above features.
This can indicates permutations that are not supported even though their support might be implied by the above features.
ConfigCase represents a single resolved configuration case. When tests are run, the Config and the supported features therein are used to compute all of the cases relevant to the implementation under test. These configuration cases are then used to select which test cases are applicable.
TODO: we could probably model some of the constraints on what is a valid vs. invalid config case using protovalidate rules
Used in:
If unspecified, indicates cases for all versions.
If unspecified, indicates cases for all protocols.
If unspecified, indicates cases for all codecs.
If unspecified, indicates cases for all compression algorithms.
If unspecified, indicates cases for all stream types.
If absent, indicates cases for plaintext (no TLS) but also for TLS if features indicate that TLS is supported.
If absent, indicates cases without client certs but also cases that use client certs if features indicate they are supported.
If absent, indicates cases that do not test message receive limits but also cases that do test message receive limits if features indicate they are supported.
Used in: , , , , ,
Any response data specified in the response definition to the server should be echoed back here.
Echoes back information about the request stream observed so far.
Used in:
The query params observed in the request URL.
Used in:
The server echos back the request headers it observed here.
The timeout observed that was included in the request. Other timeouts use a type of uint32, but we want to be lenient here to allow whatever value the RPC server observes, even if it's outside the range of uint32.
The server should echo back all requests received. For unary and server-streaming requests, this should always contain a single request For client-streaming and half-duplex bidi-streaming, this should contain all client requests in the order received and be present in each response. For full-duplex bidirectional-streaming, this should contain all requests in the order they were received since the last sent response.
If present, the request used the Connect protocol and a GET method. This captures other relevant information about the request. If a server implementation is unable to populate this (due to the server framework not exposing all of these details to application code), it may be an empty message. This implies that the server framework, at a minimum, at least expose to application code whether the request used GET vs. POST.
An error definition used for specifying a desired error response
Used in: , ,
The error code. For a list of Connect error codes see: https://connectrpc.com/docs/protocol#error-codes
If this value is absent in a test case response definition, the contents of the actual error message will not be checked. This is useful for certain kinds of error conditions where the exact message to be used is not specified, only the code.
Errors in Connect and gRPC protocols can have arbitrary messages attached to them, which are known as error details.
Features define the feature set that a client or server supports. They are used to determine the server configurations and test cases that will be run. They are defined in YAML files and are specified as part of the --conf flag to the test runner.
TODO: we could probably model some of the constraints on what are valid vs. invalid (i.e. conflicting/impossible) features using protovalidate rules
Used in:
Supported HTTP versions. If empty, HTTP 1.1 and HTTP/2 are assumed.
Supported protocols. If empty, all three are assumed: Connect, gRPC, and gRPC-Web.
Supported codecs. If empty, "proto" and "json" are assumed.
Supported compression algorithms. If empty, "identity" and "gzip" are assumed.
Supported stream types. If empty, all stream types are assumed. This is usually for clients, since some client environments may not be able to support certain kinds of streaming operations, especially bidirectional streams.
Whether H2C (unencrypted, non-TLS HTTP/2 over cleartext) is supported. If absent, true is assumed.
Whether TLS is supported. If absent, true is assumed.
Whether the client supports TLS certificates. If absent, false is assumed. This should not be set if supports_tls is false.
Whether trailers are supported. If absent, true is assumed. If false, implies that gRPC protocol is not allowed.
Whether half duplex bidi streams are supported over HTTP/1.1. If absent, false is assumed.
Whether Connect via GET is supported. If absent, true is assumed.
Whether a message receive limit is supported. If absent, true is assumed.
Used in: , , , ,
A tuple of name and values (ASCII) for a header or trailer entry.
Used in: , , , , , , , ,
Header/trailer name (key).
Header/trailer value. This is repeated to explicitly support headers and trailers where a key is repeated. In such a case, these values must be in the same order as which values appeared in the header or trailer block.
MessageContents represents a message in a request body.
Used in: , , ,
The message data can be defined in one of three ways.
Arbitrary bytes.
Arbitrary text.
An actual message. The message inside the Any will be serialized to the protobuf binary formats, and the resulting bytes will be the contents.
If specified and not identity, the above data will be compressed using the given algorithm.
Used in: , , , ,
TODO: Support add'l protocols: PROTOCOL_GRPC_WEB_TEXT = 4; PROTOCOL_REST_TRANSCODING = 5;
RawHTTPRequest models a raw HTTP request. This can be used to craft custom requests with odd properties (including certain kinds of malformed requests) to test edge cases in servers.
Used in:
The HTTP verb (i.e. GET , POST).
The URI to send the request to.
Any headers to set on the request.
These query params will be encoded and added to the uri before the request is sent.
This provides an easier way to define a complex binary query param than having to write literal base64-encoded bytes in raw_query_params.
The body is a single message.
The body is a stream, encoded using a five-byte prefix before each item in the stream.
Used in:
Query param name.
Query param value.
If true, the message contents will be base64-encoded and the resulting string used as the query parameter value.
RawHTTPResponse models a raw HTTP response. This can be used to craft custom responses with odd properties (including certain kinds of malformed responses) to test edge cases in clients.
Used in: ,
If status code is not specified, it will default to a 200 response code.
Headers to be set on the response.
The body is a single message.
The body is a stream, encoded using a five-byte prefix before each item in the stream.
Trailers to be set on the response.
Describes one configuration for an RPC server. The server is expected to expose the connectrpc.conformance.v1.ConformanceService RPC service. The configuration does not include a port. The process should pick an available port, which is typically done by using port zero (0) when creating a network listener so that the OS selects an available ephemeral port. These properties are read from stdin. Once the server is listening, details about the server, in the form of a ServerCompatResponse, are written to stdout. Each test process is expected to start only one RPC server. When testing multiple configurations, multiple test processes will be started, each with different properties.
Signals to the server that it must support at least this protocol. Note that it is fine to support others. For example if `PROTOCOL_CONNECT` is specified, the server _must_ support at least Connect, but _may_ also support gRPC or gRPC-web.
Signals to the server the minimum HTTP version to support. As with `protocol`, it is fine to support other versions. For example, if `HTTP_VERSION_2` is specified, the server _must_ support HTTP/2, but _may_ also support HTTP/1.1 or HTTP/3.
If true, generate a certificate that clients will be configured to trust when connecting and return it in the `pem_cert` field of the `ServerCompatResponse`. The certificate can be any TLS certificate where the subject matches the value sent back in the `host` field of the `ServerCompatResponse`. Self-signed certificates (and `localhost` as the subject) are allowed. If false, the server should not use TLS and instead use a plain-text/unencrypted socket.
If non-empty, the clients will use certificates to authenticate themselves. This value is a PEM-encoded cert that should be trusted by the server. When non-empty, the server should require that clients provide certificates and they should validate that the certificate presented is valid. This will always be empty if use_tls is false.
If non-zero, indicates the maximum size in bytes for a message. If the client sends anything larger, the server should reject it.
If use_tls is true, this provides details for a self-signed TLS cert that the server may use. The provided certificate is only good for loopback communication: it uses "localhost" and "127.0.0.1" as the IP and DNS names in the certificate's subject. If the server needs a different subject or the client is in an environment where configuring trust of a self-signed certificate is difficult or infeasible. If the server implementation chooses to use these credentials, it must echo back the certificate in the ServerCompatResponse and should also leave the host field empty or explicitly set to "127.0.0.1". If it chooses to use a different certificate and key, it must send back the corresponding certificate in the ServerCompatResponse.
The outcome of one ServerCompatRequest.
The host where the server is running. This should usually be `127.0.0.1`, unless your program actually starts a remote server to which the client should connect.
The port where the server is listening.
The TLS certificate, in PEM format, if `use_tls` was set to `true`. Clients will verify this certificate when connecting via TLS. If `use_tls` was set to `false`, this should always be empty.
StreamContents represents a sequence of messages in a request body.
Used in: ,
The messages in the stream.
Used in:
must be in the range 0 to 255.
if absent use actual length of payload
A definition of responses to be sent from a streaming endpoint. Can be used to define responses for server-streaming or bidi-streaming calls.
Used in: ,
Response headers to send
Response data to send
Wait this many milliseconds before sending each response message
Optional error to raise, but only after sending any response messages. In the event an immediate error is thrown before any responses are sent, (i.e. the equivalent of a trailers-only response), then servers should build a RequestInfo message with available information and append that to the error details.
Response trailers to send - together with the error if present
This field is only used by the reference server. If you are implementing a server under test, you can ignore this field or respond with an error if the server receives a request where it is set. For test definitions, this field should be used instead of the above fields.
Used in: , ,
TLSCreds represents credentials for TLS. It includes both a certificate and corresponding private key. Both are encoded in PEM format.
Used in: ,
Used in:
Defines the RPC that the client should invoke. The first eight fields are not fully specified. Instead the first field, test_name, must be present but is a prefix -- other characteristics that identify one permutation of the test case will be appended to this name. The next seven fields (http_version, protocol, codec, compression, host, port, and server_tls_cert) must not be present. They are all populated by the test harness based on the test environment (e.g. actual server and port to use) and characteristics of a single permutation.
To support extremely large messages, as well as very precisely-sized messages, without having to encode them fully or perfectly in YAML test cases, this value can be specified. When non-empty, this value should have no more entries than there are messages in the request stream. The first value is applied to the first request message, and so on. For each entry, if the size is present, it is used to expand the data field in the request (which is actually part of the response definition). The specified size is added to the current limit on message size that the server will accept. That sum is the size of the the serialized message that will be sent, and the data field will be padded as needed to reach that size.
Defines the expected response to the above RPC. The expected response for a test is auto-generated based on the request details. The conformance runner will determine what the response should be according to the values specified in the test suite and individual test cases. This value can also be specified explicitly in the test case YAML. However, this is typically only needed for exception test cases. If the expected response is mostly re-stating the response definition that appears in the requests, test cases should rely on the auto-generation if possible. Otherwise, specifying an expected response can make the test YAML overly verbose and harder to read, write, and maintain. If the test induces behavior that prevents the server from sending or client from receiving the full response definition, it will be necessary to define the expected response explicitly. Timeouts, cancellations, and exceeding message size limits are good examples of this. Specifying an expected response explicitly in test definitions will override the auto-generation of the test runner.
When expected_response indicates that an error is expected, in some cases, the actual error code returned may be flexible. In that case, this field provides other acceptable error codes, in addition to the one indicated in the expected_response. As long as the actual error's code matches any of these, the error is considered conformant, and the test case can pass.
Used in:
The size, in bytes, relative to the limit. For example, to expand to a size that is exactly equal to the limit, this should be set to zero. Any value greater than zero indicates that the request size will be that many bytes over the limit.
TestSuite represents a set of conformance test cases. This is also the schema used for the structure of a YAML test file. Each YAML file represents a test suite, which can contain numerous cases. Each test suite has various properties that indicate the kinds of features that are tested. Test suites may be skipped based on whether the client or server under test implements these features.
Test suite name. When writing test suites, this is a required field.
The mode (client or server) that this test suite applies to. This is used in conjunction with the `--mode` flag passed to the conformance runner binary. If the mode on the suite is set to client, the tests will only be run if `--mode client` is set on the command to the test runner. Likewise if mode is server. If this is unset, the test case will be run in both modes.
The actual test cases in the suite.
If non-empty, the protocols to which this suite applies. If empty, this suite applies to all protocols.
If non-empty, the HTTP versions to which this suite applies. If empty, this suite applies to all HTTP versions.
If non-empty, the codecs to which this suite applies. If empty, this suite applies to all codecs.
If non-empty, the compression encodings to which this suite applies. If empty, this suite applies to all encodings.
Indicates the Connect version validation behavior that this suite relies on.
If true, the cases in this suite rely on TLS and will only be run against TLS server configurations.
If true, the cases in this suite rely on the client using TLS certificates to authenticate with the server. (Should only be true if relies_on_tls is also true.)
If true, the cases in this suite rely on the Connect GET protocol.
If true, the cases in this suite rely on support for limiting the size of received messages. When true, mode should be set to indicate whether it is the client or the server that must support the limit.
Used in:
Used when the suite is agnostic to the server's validation behavior.
Used when the suite relies on the server validating the presence and correctness of the Connect version header or query param.
Used when the suite relies on the server ignore any Connect header or query param.
Used in:
Used when the test suite does not apply to a particular mode. Such tests are run, regardless of the current test mode, to verify both clients and servers under test.
Indicates tests that are intended to be used only for a client-under-test. These cases can induce very particular and/or aberrant responses from the reference server, to verify how the client reacts to such responses.
Indicates tests that are intended to be used only for a server-under-test. These cases can induce very particular and/or aberrant requests from the reference client, to verify how the server reacts to such requests.
A definition of a response to be sent from a single-response endpoint. Can be used to define a response for unary or client-streaming calls.
Used in: , ,
Response headers to send
Response data to send
Error to raise instead of response message Servers should build a RequestInfo and append it to the details of the requested error.
Response trailers to send - together with the error if present
Wait this many milliseconds before sending a response message
This field is only used by the reference server. If you are implementing a server under test, you can ignore this field or respond with an error if the server receives a request where it is set. For test definitions, this field should be used instead of the above fields.
Details about various values as observed on the wire. This message is used only by the reference client when reporting results and should not be populated by clients under test.
The HTTP status code of the response.
When processing an error from a Connect server, this should contain the actual JSON received on the wire.
Any HTTP trailers observed after the response body. These do NOT include trailers that conveyed via the body, as done in the gRPC-Web and Connect streaming protocols.
Any trailers that were transmitted in the final message of the response body for a gRPC-Web response. This could differ from the ClientResponseResult.response_trailers field since the RPC client library might canonicalize keys and it might choose to remove "grpc-status" et al from the set of metadata. This field will capture all of the entries and their exact on-the-wire spelling and formatting.