package org.pytorch.serve.grpc.management
Get desktop application:
View/edit binary Protocol Buffers messages
Provides detailed information about the default version of a model.
message
torch_management.proto:12string model_name = 1
Name of model to describe.
required
string model_version = 2
Version of model to describe.
optional
rpc ListModels (, )
torch_management.proto:104List registered models in TorchServe.
message
torch_management.proto:19int32 limit = 1
Use this parameter to specify the maximum number of items to return. When this value is present, TorchServe does not return more than the specified number of items, but it might return fewer. This value is optional. If you include a value, it must be between 1 and 1000, inclusive. If you do not include a value, it defaults to 100.
optional
int32 next_page_token = 2
The token to retrieve the next set of results. TorchServe provides the token when the response from a previous call has more results than the maximum page size.
optional
Register a new model in TorchServe.
message
torch_management.proto:27int32 batch_size = 1
Inference batch size, default: 1.
optional
string handler = 2
Inference handler entry-point. This value will override handler in MANIFEST.json if present.
optional
int32 initial_workers = 3
Number of initial workers, default: 0.
optional
int32 max_batch_delay = 4
Maximum delay for batch aggregation, default: 100.
optional
string model_name = 5
Name of model. This value will override modelName in MANIFEST.json if present.
optional
int32 response_timeout = 6
Maximum time, in seconds, the TorchServe waits for a response from the model inference code, default: 120.
optional
string runtime = 7
Runtime for the model custom service code. This value will override runtime in MANIFEST.json if present.
optional
bool synchronous = 8
Decides whether creation of worker synchronous or not, default: false.
optional
string url = 9
Model archive download url, support local file or HTTP(s) protocol.
required
bool s3_sse_kms = 10
Decides whether S3 SSE KMS enabled or not, default: false.
optional
Configure number of workers for a default version of a model.This is a asynchronous call by default. Caller need to call describeModel to check if the model workers has been changed.
message
torch_management.proto:59string model_name = 1
Name of model to scale workers.
required
string model_version = 2
Model version.
optional
int32 max_worker = 3
Maximum number of worker processes.
optional
int32 min_worker = 4
Minimum number of worker processes.
optional
int32 number_gpu = 5
Number of GPU worker processes to create.
optional
bool synchronous = 6
Decides whether the call is synchronous or not, default: false.
optional
int32 timeout = 7
Waiting up to the specified wait time if necessary for a worker to complete all pending requests. Use 0 to terminate backend worker process immediately. Use -1 for wait infinitely.
optional
rpc SetDefault (, )
torch_management.proto:113Set default version of a model
message
torch_management.proto:83string model_name = 1
Name of model whose default version needs to be updated.
required
string model_version = 2
Version of model to be set as default version for the model
required
Unregister the default version of a model from TorchServe if it is the only version available.This is a asynchronous call by default. Caller can call listModels to confirm model is unregistered
message
torch_management.proto:91string model_name = 1
Name of model to unregister.
required
string model_version = 2
Name of model to unregister.
optional
message
torch_management.proto:7Used as response type in: ManagementAPIsService.DescribeModel, ManagementAPIsService.ListModels, ManagementAPIsService.RegisterModel, ManagementAPIsService.ScaleWorker, ManagementAPIsService.SetDefault, ManagementAPIsService.UnregisterModel
string msg = 1
Response string of different management API calls.