package tensorflow.serving

Get desktop application:
View/edit binary Protocol Buffers messages

A single class.

Used in: Classifications

• string label = 1

  Label or name of the class.

• float score = 2

  Score for this class (e.g., the probability the item belongs to this class). As per the proto3 default-value semantics, if the score is missing, it should be treated as 0.

Used in: ClassifyLog

• optional ModelSpec model_spec = 1

  Model Specification. If version is not specified, will use the latest (numerical) version.

• optional Input input = 2

  Input data.

Used in: ClassifyLog

• optional ModelSpec model_spec = 2

  Effective Model Specification used for classification.

• optional ClassificationResult result = 1

  Result of the classification.

Contains one result per input example, in the same order as the input in ClassificationRequest.

Used in: ClassificationResponse, InferenceResult

• repeated Classifications classifications = 1

List of classes for a single item (tensorflow.Example).

Used in: ClassificationResult

• repeated Class classes = 1

Used in: PredictionLog

• optional ClassificationRequest request = 1

• optional ClassificationResponse response = 2

Specifies one or more fully independent input Examples. See examples at: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/example/example.proto

Used in: Input

• repeated Example examples = 1

Specifies one or more independent input Examples, with a common context Example. The common use case for context is to cleanly and optimally specify some features that are common across multiple examples. See example below with a search query as the context and multiple restaurants to perform some inference on. context: { features: { feature: { key : "query" value: { bytes_list: { value: [ "pizza" ] } } } } } examples: { features: { feature: { key : "cuisine" value: { bytes_list: { value: [ "Pizzeria" ] } } } } } examples: { features: { feature: { key : "cuisine" value: { bytes_list: { value: [ "Taqueria" ] } } } } } Implementations of ExampleListWithContext merge the context Example into each of the Examples. Note that feature keys must not be duplicated between the Examples and context Example, or the behavior is undefined. See also: tensorflow/core/example/example.proto https://developers.google.com/protocol-buffers/docs/proto3#maps

Used in: Input

• repeated Example examples = 1

• optional Example context = 2

Inference result, matches the type of request or is an error.

Used in: MultiInferenceResponse

• optional ModelSpec model_spec = 1

• oneof result

  • ClassificationResult classification_result = 2

  • RegressionResult regression_result = 3

Inference request such as classification, regression, etc...

Used in: MultiInferenceRequest

• optional ModelSpec model_spec = 1

  Model Specification. If version is not specified, will use the latest (numerical) version. All ModelSpecs in a MultiInferenceRequest must access the same model name.

• string method_name = 2

  Signature's method_name. Should be one of the method names defined in third_party/tensorflow/python/saved_model/signature_constants.py. e.g. "tensorflow/serving/classify".

Used in: ClassificationRequest, MultiInferenceRequest, RegressionRequest

• oneof kind

  • ExampleList example_list = 1

  • ExampleListWithContext example_list_with_context = 2

Used in: LoggingConfig

• string type = 1

  Identifies the type of the LogCollector we will use to collect these logs.

• string filename_prefix = 2

  The prefix to use for the filenames of the logs.

Metadata logged along with the request logs.

Used in: PredictionLog

• optional ModelSpec model_spec = 1

• optional SamplingConfig sampling_config = 2

• repeated string saved_model_tags = 3

  List of tags used to load the relevant MetaGraphDef from SavedModel.

  TODO(b/33279154): Add more metadata as mentioned in the bug.

Configuration for logging query/responses.

• optional LogCollectorConfig log_collector_config = 1

• optional SamplingConfig sampling_config = 2

Metadata for an inference request such as the model name and version.

Used in: ClassificationRequest, ClassificationResponse, InferenceResult, InferenceTask, LogMetadata, PredictRequest, PredictResponse, RegressionRequest, RegressionResponse, SessionRunRequest, SessionRunResponse

• string name = 1

  Required servable name.

• oneof version_choice

  Optional choice of which version of the model to use. Expected to be left unset in the common case. Should be specified when there is a strong version consistency requirement (e.g. when the model signature changes across versions and requests need to be version-specific). When left unspecified, the system will serve the best available version. This is typically the latest version, though during version transitions, notably when serving on a fleet of instances, may be either the previous or new version.

  • google.protobuf.Int64Value version = 2

    Use this specific version number.

  • string version_label = 4

    Use the version associated with the given label.

• string signature_name = 3

  A named signature to evaluate. If unspecified, the default signature will be used.

Used in: PredictionLog

• optional MultiInferenceRequest request = 1

• optional MultiInferenceResponse response = 2

Inference request containing one or more requests.

Used in: MultiInferenceLog

• repeated InferenceTask tasks = 1

  Inference tasks.

• optional Input input = 2

  Input data.

Inference request containing one or more responses.

Used in: MultiInferenceLog

• repeated InferenceResult results = 1

  List of results; one for each InferenceTask in the request, returned in the same order as the request.

Used in: PredictionLog

• optional PredictRequest request = 1

• optional PredictResponse response = 2

PredictRequest specifies which TensorFlow model to run, as well as how inputs are mapped to tensors and how outputs are filtered before returning to user.

Used in: PredictLog, PredictStreamedLog

• optional ModelSpec model_spec = 1

  Model Specification. If version is not specified, will use the latest (numerical) version.

• map<string, TensorProto> inputs = 2

  Input tensors. Names of input tensor are alias names. The mapping from aliases to real input tensor names is stored in the SavedModel export as a prediction SignatureDef under the 'inputs' field.

• repeated string output_filter = 3

  Output filter. Names specified are alias names. The mapping from aliases to real output tensor names is stored in the SavedModel export as a prediction SignatureDef under the 'outputs' field. Only tensors specified here will be run/fetched and returned, with the exception that when none is specified, all tensors specified in the named signature will be run/fetched and returned.

• optional PredictStreamedOptions predict_streamed_options = 5

  Options for streaming requests to control how multiple requests/responses are handled within a single stream.

• optional bytes client_id = 6

  Client identifier to group requests belonging to a specific entity. Example entities can be product ids, service names, user ids etc. Servers can use this to optimize placement, caching and colocation. TODO(b/329897437): Migrate to client_id in RequestOptions.

• optional PredictRequest.RequestOptions request_options = 7

Options for PredictRequest.

Used in: PredictRequest

• optional bytes client_id = 1

  Client identifier to group requests belonging to a specific entity. Example entities can be product ids, service names, user ids etc. Servers can use this to optimize placement, caching and colocation.

• optional RequestOptions.DeterministicMode deterministic_mode = 2

Deterministic mode for the request. When specified, model servers will reduce numeric instability based on different mode selections.

Used in: RequestOptions

• DETERMINISTIC_MODE_UNSPECIFIED = 0

• FIXED_DECODER_SLOT = 1

  Only supported in disaggregated serving. When set, the request will be pinned to a fixed decoder slot index that's deterministic across processes.

Response for PredictRequest on successful run.

Used in: PredictLog, PredictStreamedLog

• optional ModelSpec model_spec = 2

  Effective Model Specification used to process PredictRequest.

• map<string, TensorProto> outputs = 1

  Output tensors.

Used in: PredictionLog

• repeated PredictRequest request = 1

• repeated PredictResponse response = 2

Options only used for streaming requests that control how inputs/ouputs are handled in the stream.

Request state used to handle splitting of requests. NONE is the default when the stream request is not split. SPLIT is used when multiple streamed requests are used to generate a logical request. END_SPLIT should be called for the last split of the logical request. NONE can not be interspersed with SPLIT before END_SPLIT is called. If another request is sent on the same stream after END_SPLIT, it can be any of the RequestState since a new logical request has started. If END_SPLIT is called on its own the behavior is the same as NONE. Some examples with a mix of request states and the logical request. Example 1 : SPLIT SPLIT END_SPLIT Will be treated as a single logical request. Example 2: NONE END_SPLIT NONE Will be treated as three logical requests (1. NONE 2. END_SPLIT, 3. NONE) Example 3: SPLIT SPLIT Invalid because END_SPLIT is never call. Example 4: SPLIT NONE SPLIT END_SPLIT Invalid because is interspersed with SPLIT. Example 5: SPLIT END_SPLIT SPLIT SPLIT END_SPLIT Will be treated as two logical requests (1. SPLIT, END_SPLIT 2. SPLIT, SPLIT, END_SPLIT)

Used in: PredictRequest

• PredictStreamedOptions.RequestState request_state = 1

  Request state used to handle segmentation of requests.

• map<string, int32> split_dimensions = 2

  Input tensors split dimensions. Defines the dimension used to split input tensors specified in PredictRequest.inputs. The dimension will be used for concatenation of multiple SPLIT requests. For input tensor in PredictRequest.inputs that are not contained in this map, the tensors from the first SPLIT request will be used. For example, with an original input tensor of [[1, 2, 3, 4], [5, 6, 7, 8]]. For a split dimension of 0 and two requests (SPLIT and END_SPLIT), the input tensors for request 1 should be [1, 2, 3, 4] and request 2 should be be [5, 6, 7, 8]. For a split dimension of 1 and two requests (SPLIT and END_SPLIT), the input tensors for request 1 should be [[1, 2], [5, 6]] and request 2 should be [[3, 4], [7, 8]].

Used in: PredictStreamedOptions

• NONE = 0

• SPLIT = 1

• END_SPLIT = 2

Logged model inference request.

• optional LogMetadata log_metadata = 1

• oneof log_type

  • ClassifyLog classify_log = 2

  • RegressLog regress_log = 3

  • PredictLog predict_log = 6

  • PredictStreamedLog predict_streamed_log = 7

  • MultiInferenceLog multi_inference_log = 4

  • SessionRunLog session_run_log = 5

Used in: PredictionLog

• optional RegressionRequest request = 1

• optional RegressionResponse response = 2

Regression result for a single item (tensorflow.Example).

Used in: RegressionResult

• float value = 1

Used in: RegressLog

• optional ModelSpec model_spec = 1

  Model Specification. If version is not specified, will use the latest (numerical) version.

• optional Input input = 2

  Input data.

Used in: RegressLog

• optional ModelSpec model_spec = 2

  Effective Model Specification used for regression.

• optional RegressionResult result = 1

Contains one result per input example, in the same order as the input in RegressionRequest.

Used in: InferenceResult, RegressionResponse

• repeated Regression regressions = 1

Used in: LogMetadata, LoggingConfig

• double sampling_rate = 1

  Requests will be logged uniformly at random with this probability. Valid range: [0, 1.0].

• int32 attributes = 2

  Bitwise OR of above attributes

Attributes of requests that can be optionally sampled. Note: Enabling more attributes will increase logging storage requirements.

• ATTR_DEFAULT = 0

• ATTR_REQUEST_ORIGIN = 1

• ATTR_REQUEST_CRITICALITY = 2

Used in: PredictionLog

• optional SessionRunRequest request = 1

• optional SessionRunResponse response = 2

Used in: SessionRunLog

• optional ModelSpec model_spec = 1

  Model Specification. If version is not specified, will use the latest (numerical) version.

• repeated NamedTensorProto feed = 2

  Tensors to be fed in the step. Each feed is a named tensor.

• repeated string fetch = 3

  Fetches. A list of tensor names. The caller expects a tensor to be returned for each fetch[i] (see RunResponse.tensor). The order of specified fetches does not change the execution order.

• repeated string target = 4

  Target Nodes. A list of node names. The named nodes will be run to but their outputs will not be fetched.

• bool tensor_name_is_alias = 6

  If true, treat names in feed/fetch/target as alias names than actual tensor names (that appear in the TF graph). Alias names are resolved to actual names using `SignatureDef` in SavedModel associated with the model.

Used in: SessionRunLog

• optional ModelSpec model_spec = 3

  Effective Model Specification used for session run.

• repeated NamedTensorProto tensor = 1

  NOTE: The order of the returned tensors may or may not match the fetch order specified in RunRequest.