package tensorflow.serving
Get desktop application:
View/edit binary Protocol Buffers messages
ModelService provides methods to query and update the state of the server, e.g. which models/versions are being served.
rpc GetModelStatus (, )
model_service.proto:17Gets status of model. If the ModelSpec in the request does not specify version, information about all versions of the model will be returned. If the ModelSpec in the request does specify a version, the status of only that version will be returned.
message
get_model_status.proto:12GetModelStatusRequest contains a ModelSpec indicating the model for which to get status.
optional model_spec = 1
Model Specification. If version is not specified, information about all versions of the model will be returned. If a version is specified, the status of only that version will be returned.
message
get_model_status.proto:64Response for ModelStatusRequest on successful run.
repeated model_version_status = 1
Version number and status information for applicable model version(s).
Reloads the set of served models. The new config supersedes the old one, so if a model is omitted from the new config it will be unloaded and no longer served.
message
model_management.proto:9optional config = 1
message
model_management.proto:13optional status = 1
open source marker; do not remove PredictionService provides access to machine-learned models loaded by model_servers.
rpc Classify (, )
prediction_service.proto:17Classify.
message
classification.proto:33optional model_spec = 1
Model Specification. If version is not specified, will use the latest (numerical) version.
optional input = 2
Input data.
message
classification.proto:42optional model_spec = 2
Effective Model Specification used for classification.
optional result = 1
Result of the classification.
GetModelMetadata - provides access to metadata for loaded models.
message
get_model_metadata.proto:15optional model_spec = 1
Model Specification indicating which model we are querying for metadata. If version is not specified, will use the latest (numerical) version.
repeated string metadata_field = 2
Metadata fields to get. Currently supported: "signature_def".
message
get_model_metadata.proto:23optional model_spec = 1
Model Specification indicating which model this metadata belongs to.
map<string, > metadata = 2
Map of metadata field name to metadata field. The options for metadata field name are listed in GetModelMetadataRequest. Currently supported: "signature_def".
MultiInference API for multi-headed models.
message
inference.proto:46Inference request containing one or more requests.
repeated tasks = 1
Inference tasks.
optional input = 2
Input data.
message
inference.proto:55Inference request containing one or more responses.
repeated results = 1
List of results; one for each InferenceTask in the request, returned in the same order as the request.
rpc Predict (, )
prediction_service.proto:23Predict -- provides access to loaded TensorFlow model.
message
predict.proto:12PredictRequest specifies which TensorFlow model to run, as well as how inputs are mapped to tensors and how outputs are filtered before returning to user.
optional model_spec = 1
Model Specification. If version is not specified, will use the latest (numerical) version.
map<string, > inputs = 2
Input tensors. Names of input tensor are alias names. The mapping from aliases to real input tensor names is stored in the SavedModel export as a prediction SignatureDef under the 'inputs' field.
repeated string output_filter = 3
Output filter. Names specified are alias names. The mapping from aliases to real output tensor names is stored in the SavedModel export as a prediction SignatureDef under the 'outputs' field. Only tensors specified here will be run/fetched and returned, with the exception that when none is specified, all tensors specified in the named signature will be run/fetched and returned.
message
predict.proto:34Response for PredictRequest on successful run.
optional model_spec = 2
Effective Model Specification used to process PredictRequest.
map<string, > outputs = 1
Output tensors.
rpc Regress (, )
prediction_service.proto:20Regress.
message
regression.proto:23optional model_spec = 1
Model Specification. If version is not specified, will use the latest (numerical) version.
optional input = 2
Input data.
message
regression.proto:32optional model_spec = 2
Effective Model Specification used for regression.
optional result = 1
message
classification.proto:11A single class.
Used in:
string label = 1
Label or name of the class.
float score = 2
Score for this class (e.g., the probability the item belongs to this class). As per the proto3 default-value semantics, if the score is missing, it should be treated as 0.
message
classification.proto:27Contains one result per input example, in the same order as the input in ClassificationRequest.
Used in: ,
repeated classifications = 1
message
classification.proto:21List of classes for a single item (tensorflow.Example).
Used in:
repeated classes = 1
message
input.proto:15Specifies one or more fully independent input Examples. See examples at: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/example/example.proto
Used in:
repeated examples = 1
message
input.proto:72Specifies one or more independent input Examples, with a common context Example. The common use case for context is to cleanly and optimally specify some features that are common across multiple examples. See example below with a search query as the context and multiple restaurants to perform some inference on. context: { features: { feature: { key : "query" value: { bytes_list: { value: [ "pizza" ] } } } } } examples: { features: { feature: { key : "cuisine" value: { bytes_list: { value: [ "Pizzeria" ] } } } } } examples: { features: { feature: { key : "cuisine" value: { bytes_list: { value: [ "Taqueria" ] } } } } } Implementations of ExampleListWithContext merge the context Example into each of the Examples. Note that feature keys must not be duplicated between the Examples and context Example, or the behavior is undefined. See also: tensorflow/core/example/example.proto https://developers.google.com/protocol-buffers/docs/proto3#maps
Used in:
repeated examples = 1
optional context = 2
Config proto for FileSystemStoragePathSource.
repeated servables = 5
The servables to monitor for new versions, and aspire.
string servable_name = 1
A single servable name/base_path pair to monitor. DEPRECATED: Use 'servables' instead. TODO(b/30898016): Stop using these fields, and ultimately remove them here.
string base_path = 2
int64 file_system_poll_wait_seconds = 3
How long to wait between file-system polling to look for children of 'base_path', in seconds. If set to zero, filesystem will be polled exactly once. If set to a negative value (for testing use only), polling will be entirely disabled.
bool fail_if_zero_versions_at_startup = 4
If true, then FileSystemStoragePathSource::Create() and ::UpdateConfig() fail if, for any configured servables, the file system doesn't currently contain at least one version under the base path. (Otherwise, it will emit a warning and keep pinging the file system to check for a version to appear later.) DEPRECATED: Use 'servable_versions_always_present' instead, which includes this behavior. TODO(b/30898016): Remove 2019-10-31 or later.
bool servable_versions_always_present = 6
If true, the servable is always expected to exist on the underlying filesystem. FileSystemStoragePathSource::Create() and ::UpdateConfig() will fail if, for any configured servables, the file system doesn't currently contain at least one version under the base path. In addition, if a polling loop find the base path empty, it will not unload existing servables.
A servable name and base path to look for versions of the servable.
Used in:
string servable_name = 1
The servable name to supply in aspired-versions callback calls. Child paths of 'base_path' are considered to be versions of this servable.
string base_path = 2
The path to monitor, i.e. look for child paths of the form base_path/123.
optional servable_version_policy = 4
The policy to determines the number of versions of the servable to be served at the same time.
A policy that dictates which version(s) of a servable should be served.
Used in: ,
oneof policy_choice
latest = 100
all = 101
specific = 102
Serve all versions found on disk.
Used in:
(message has no fields)
Serve the latest versions (i.e. the ones with the highest version numbers), among those found on disk. This is the default policy, with the default number of versions as 1.
Used in:
uint32 num_versions = 1
Number of latest versions to serve. (The default is 1.)
Serve a specific version (or set of versions). This policy is useful for rolling back to a specific version, or for canarying a specific version while still serving a separate stable version.
Used in:
repeated int64 versions = 1
The version numbers to serve.
message
inference.proto:36Inference result, matches the type of request or is an error.
Used in:
optional model_spec = 1
oneof result
classification_result = 2
regression_result = 3
message
inference.proto:23Inference request such as classification, regression, etc...
Used in:
optional model_spec = 1
Model Specification. If version is not specified, will use the latest (numerical) version. All ModelSpecs in a MultiInferenceRequest must access the same model name.
string method_name = 2
Signature's method_name. Should be one of the method names defined in third_party/tensorflow/python/saved_model/signature_constants.py. e.g. "tensorflow/serving/classify".
message
input.proto:77Used in: , ,
oneof kind
example_list = 1
example_list_with_context = 2
message
log_collector_config.proto:6Used in:
string type = 1
Identifies the type of the LogCollector we will use to collect these logs.
string filename_prefix = 2
The prefix to use for the filenames of the logs.
message
logging_config.proto:15Configuration for logging query/responses.
Used in:
optional log_collector_config = 1
optional sampling_config = 2
message
model_server_config.proto:19Common configuration for loading a model being served.
Used in:
string name = 1
Name of the model.
string base_path = 2
Base path to the model, excluding the version directory. E.g> for a model at /foo/bar/my_model/123, where 123 is the version, the base path is /foo/bar/my_model. (This can be changed once a model is in serving, *if* the underlying data remains the same. Otherwise there are no guarantees about whether the old or new data will be used for model versions currently loaded.)
ModelType model_type = 3
Type of model. TODO(b/31336131): DEPRECATED. Please use 'model_platform' instead.
string model_platform = 4
Type of model (e.g. "tensorflow"). (This cannot be changed once a model is in serving.)
optional model_version_policy = 7
Version policy for the model indicating which version(s) of the model to load and make available for serving simultaneously. The default option is to serve only the latest version of the model. (This can be changed once a model is in serving.)
map<string, int64> version_labels = 8
String labels to associate with versions of the model, allowing inference queries to refer to versions by label instead of number. Multiple labels can map to the same version, but not vice-versa. An envisioned use-case for these labels is canarying tentative versions. For example, one can assign labels "stable" and "canary" to two specific versions. Perhaps initially "stable" is assigned to version 0 and "canary" to version 1. Once version 1 passes canary, one can shift the "stable" label to refer to version 1 (at that point both labels map to the same version -- version 1 -- which is fine). Later once version 2 is ready to canary one can move the "canary" label to version 2. And so on.
optional logging_config = 6
Configures logging requests and responses, to the model. (This can be changed once a model is in serving.)
message
model_server_config.proto:71Static list of models to be loaded for serving.
Used in:
repeated config = 1
message
model_server_config.proto:76ModelServer config.
Used in:
oneof config
ModelServer takes either a static file-based model config list or an Any proto representing custom model config that is fetched dynamically at runtime (through network RPC, custom service, etc.).
model_config_list = 1
custom_model_config = 2
message
model.proto:9Metadata for an inference request such as the model name and version.
Used in: , , , , , , , , , ,
string name = 1
Required servable name.
oneof version_choice
Optional choice of which version of the model to use. Recommended to be left unset in the common case. Should be specified only when there is a strong version consistency requirement. When left unspecified, the system will serve the best available version. This is typically the latest version, though during version transitions, notably when serving on a fleet of instances, may be either the previous or new version.
version = 2
Use this specific version number.
string version_label = 4
Use the version associated with the given label.
string signature_name = 3
A named signature to evaluate. If unspecified, the default signature will be used.
The type of model. TODO(b/31336131): DEPRECATED.
Used in:
MODEL_TYPE_UNSPECIFIED = 0
TENSORFLOW = 1
OTHER = 2
message
get_model_status.proto:20Version number, state, and status for a single version of a model.
Used in:
int64 version = 1
Model version.
ModelVersionStatus.State state = 2
Model state.
optional status = 3
Model status.
States that map to ManagerState enum in tensorflow_serving/core/servable_state.h
Used in:
UNKNOWN = 0
Default value.
START = 10
The manager is tracking this servable, but has not initiated any action pertaining to it.
LOADING = 20
The manager has decided to load this servable. In particular, checks around resource availability and other aspects have passed, and the manager is about to invoke the loader's Load() method.
AVAILABLE = 30
The manager has successfully loaded this servable and made it available for serving (i.e. GetServableHandle(id) will succeed). To avoid races, this state is not reported until *after* the servable is made available.
UNLOADING = 40
The manager has decided to make this servable unavailable, and unload it. To avoid races, this state is reported *before* the servable is made unavailable.
END = 50
This servable has reached the end of its journey in the manager. Either it loaded and ultimately unloaded successfully, or it hit an error at some point in its lifecycle.
message
regression.proto:11Regression result for a single item (tensorflow.Example).
Used in:
float value = 1
message
regression.proto:17Contains one result per input example, in the same order as the input in RegressionRequest.
Used in: ,
repeated regressions = 1
message
logging_config.proto:8Used in:
double sampling_rate = 1
Requests will be logged uniformly at random with this probability. Valid range: [0, 1.0].
message
get_model_metadata.proto:11Message returned for "signature_def" field.
map<string, > signature_def = 1
message
status.proto:11Status that corresponds to Status in third_party/tensorflow/core/lib/core/status.h.
Used in: ,
error.Code error_code = 1
Error code.
string error_message = 2
Error message. Will only be set if an error was encountered.