package com.autodeployai.serving.protobuf

Get desktop application:
View/edit binary Protocol Buffers messages

Provides access to AI models served by AI-Serving.

• rpc Validate (ValidateRequest, ModelInfo)

  ai-serving.proto:10

  message ValidateRequest

  ai-serving.proto:22

  Specifies model with its type to validate. Currently, both types "PMML" and "ONNX" are supported.

  • bytes model = 1

  • string type = 2

• rpc Deploy (DeployRequest, DeployResponse)

  ai-serving.proto:12

  message DeployRequest

  ai-serving.proto:29

  Specifies a servable name, and model with its type. Currently, both types "PMML" and "ONNX" are supported.

  • string name = 1

  • bytes model = 2

  • string type = 3

  message DeployResponse

  ai-serving.proto:35

  • optional ModelSpec model_spec = 1

    Specifies the deployed model specification: the specified servable name and the deployed version starts from 1.

• rpc Undeploy (UndeployRequest, UndeployResponse)

  ai-serving.proto:13

  message UndeployRequest

  ai-serving.proto:41

  • optional ModelSpec model_spec = 1

    Specifies which model to un-deploy.

  message UndeployResponse

  ai-serving.proto:46

  • optional ModelSpec model_spec = 1

    Specifies which model has been un-deployed.

• rpc Predict (PredictRequest, PredictResponse)

  ai-serving.proto:15

  message PredictRequest

  ai-serving.proto:151

  Request to predict

  • optional ModelSpec model_spec = 1

  • optional RecordSpec X = 2

    Input payload

  • repeated string filter = 3

    Output filters to specify which output fields need to be returned. If the list is empty, all outputs will be included.

  message PredictResponse

  ai-serving.proto:163

  Response for predicting request on successful run

  • optional ModelSpec model_spec = 1

  • optional RecordSpec result = 2

    Output result

• rpc GetModelMetadata (GetModelMetadataRequest, GetModelMetadataResponse)

  ai-serving.proto:17

  message GetModelMetadataRequest

  ai-serving.proto:51

  • optional ModelSpec model_spec = 1

    Specifies which model to get metadata.

  message GetModelMetadataResponse

  ai-serving.proto:56

  • optional ModelSpec model_spec = 1

    Specifies which model metadata is returned.

  • repeated ModelMetadata metadata = 2

    Model metadata.

Field info

Used in: ModelInfo

• string name = 1

  A unique name.

• string type = 2

  Field type, main two kinds: - scalar types for PMML models: float, double, integer, string and so on. - tensor, map, and list for ONNX models.

• string optype = 3

  Determines which operations are defined on the values: - categorical - ordinal - continuous

• repeated int64 shape = 4

  Field shape dimensions, mainly used for the tensor field, None for others.

• string values = 5

  A string describes valid values for this field.

Used in: RecordSpec, Value

• repeated Value values = 1

  Repeated field of dynamically typed values.

Used as response type in: DeploymentService.Validate

Used as field type in: ModelMetadata

• string type = 1

  Model type.

• string serialization = 2

  Model serialization type.

• string runtime = 3

  The runtime library to handle such model.

• repeated Field predictors = 4

  A list of predictors involved to predict this model.

• repeated Field targets = 5

  A list of targets.

• repeated Field outputs = 6

  A list of outputs could be produced by this model.

• repeated Field redundancies = 7

  A list of redundancy fields not picked up by this model.

• string algorithm = 8

  Model algorithm.

• string function_name = 9

  Mining function: regression, classification, clustering, or associationRules.

• string description = 10

  Model description.

• optional google.protobuf.Int32Value version = 11

  Model version.

• string format_version = 12

  The version of model serialization standard.

• string hash = 13

  The MD5 hash string of this model file.

• int64 size = 14

  The size of this model file in bytes

• optional google.protobuf.Timestamp created_at = 15

  Model creation timestamp.

• string app = 16

  The application that generated this model.

• string app_version = 17

  The version of the application.

• string copyright = 18

  Model copyright.

• string source = 19

  Original model source.

Model metadata with versions

Used in: GetModelMetadataResponse

• string id = 1

  Model ID

• string name = 2

  A unique model name

• optional google.protobuf.Timestamp created_at = 3

  Model creation timestamp.

• optional google.protobuf.Timestamp update_at = 4

  Model last updated timestamp.

• int32 latest_version = 5

  The latest version number.

• repeated ModelInfo versions = 6

  Model version(s).

Contains the model name and version

Used in: DeployResponse, GetModelMetadataRequest, GetModelMetadataResponse, PredictRequest, PredictResponse, UndeployRequest, UndeployResponse

• string name = 1

  Required servable name.

• optional google.protobuf.Int32Value version = 2

  Optional choice of which version of the model to use. The latest version is used when left unspecified

Used in: Value

• NULL_VALUE = 0

  Null value.

Used in: RecordSpec, Value

• map<string, Value> fields = 1

  Unordered map of dynamically typed values.

Takes more than one records, there are two formats supported: - `records` : list like [{column -> value}, … , {column -> value}] - `split` : dict like {columns -> [columns], data -> [values]}

Used in: PredictRequest, PredictResponse

• repeated Record records = 1

• repeated string columns = 2

• repeated ListValue data = 3

StringStringEntryProto follows the pattern for cross-proto-version maps. See https://developers.google.com/protocol-buffers/docs/proto3#maps

Used in: TensorProto

• string key = 1

• string value = 2

Tensors A serialized tensor value. Compatible with the onnx.TensorProto: https://github.com/onnx/onnx/blob/main/onnx/onnx.proto3

Used in: Value

• repeated int64 dims = 1

  The shape of the tensor.

• int32 data_type = 2

  The data type of the tensor. This field MUST have a valid TensorProto.DataType value

• optional TensorProto.Segment segment = 3

• repeated float float_data = 4

  For float and complex64 values Complex64 tensors are encoded as a single array of floats, with the real components appearing in odd numbered positions, and the corresponding imaginary component appearing in the subsequent even numbered position. (e.g., [1.0 + 2.0i, 3.0 + 4.0i] is encoded as [1.0, 2.0 ,3.0 ,4.0] When this field is present, the data_type field MUST be FLOAT or COMPLEX64.

• repeated int32 int32_data = 5

  For int32, uint8, int8, uint16, int16, uint4, int4, bool, float8 and float16 values float16 and float8 values must be bit-wise converted to an uint16_t prior to writing to the buffer. uint4 and int4 values must be packed to 4bitx2 prior to writing to the buffer, the first element is stored in the 4 LSB and the second element is stored in the 4 MSB. When this field is present, the data_type field MUST be INT32, INT16, INT8, INT4, UINT16, UINT8, UINT4, BOOL, FLOAT16, BFLOAT16, FLOAT8E4M3FN, FLOAT8E4M3FNUZ, FLOAT8E5M2, FLOAT8E5M2FNUZ

• repeated bytes string_data = 6

  For strings. Each element of string_data is a UTF-8 encoded Unicode string. No trailing null, no leading BOM. The protobuf "string" scalar type is not used to match ML community conventions. When this field is present, the data_type field MUST be STRING

• repeated int64 int64_data = 7

  For int64. When this field is present, the data_type field MUST be INT64

• string name = 8

  Optionally, a name for the tensor.

  namespace Value

• string doc_string = 12

  A human-readable documentation for this tensor. Markdown is allowed.

• bytes raw_data = 9

  Serializations can either use one of the fields above, or use this raw bytes field. The only exception is the string case, where one is required to store the content in the repeated bytes string_data field. When this raw_data field is used to store tensor value, elements MUST be stored in as fixed-width, little-endian order. Floating-point data types MUST be stored in IEEE 754 format. Complex64 elements must be written as two consecutive FLOAT values, real component first. Complex128 elements must be written as two consecutive DOUBLE values, real component first. Boolean type MUST be written one byte per tensor element (00000001 for true, 00000000 for false). uint4 and int4 values must be packed to 4bitx2, the first element is stored in the 4 LSB and the second element is stored in the 4 MSB. Note: the advantage of specific field rather than the raw_data field is that in some cases (e.g. int data), protobuf does a better packing via variable length storage, and may lead to smaller binary footprint. When this field is present, the data_type field MUST NOT be STRING or UNDEFINED

• repeated StringStringEntryProto external_data = 13

  Data can be stored inside the protobuf file using type-specific fields or raw_data. Alternatively, raw bytes data can be stored in an external file, using the external_data field. external_data stores key-value pairs describing data location. Recognized keys are: - "location" (required) - POSIX filesystem path relative to the directory where the ONNX protobuf model was stored - "offset" (optional) - position of byte at which stored data begins. Integer stored as string. Offset values SHOULD be multiples 4096 (page size) to enable mmap support. - "length" (optional) - number of bytes containing data. Integer stored as string. - "checksum" (optional) - SHA1 digest of file specified in under 'location' key.

• TensorProto.DataLocation data_location = 14

  If value not set, data is stored in raw_data (if set) otherwise in type-specified field.

• repeated double double_data = 10

  For double Complex128 tensors are encoded as a single array of doubles, with the real components appearing in odd numbered positions, and the corresponding imaginary component appearing in the subsequent even numbered position. (e.g., [1.0 + 2.0i, 3.0 + 4.0i] is encoded as [1.0, 2.0 ,3.0 ,4.0] When this field is present, the data_type field MUST be DOUBLE or COMPLEX128

• repeated uint64 uint64_data = 11

  For uint64 and uint32 values When this field is present, the data_type field MUST be UINT32 or UINT64

• repeated StringStringEntryProto metadata_props = 16

  Named metadata values; keys should be distinct.

Location of the data for this tensor. MUST be one of: - DEFAULT - data stored inside the protobuf message. Data is stored in raw_data (if set) otherwise in type-specified field. - EXTERNAL - data stored in an external location as described by external_data field.

Used in: TensorProto

• DEFAULT = 0

• EXTERNAL = 1

• UNDEFINED = 0

• FLOAT = 1

  Basic types.

  float

• UINT8 = 2

  uint8_t

• INT8 = 3

  int8_t

• UINT16 = 4

  uint16_t

• INT16 = 5

  int16_t

• INT32 = 6

  int32_t

• INT64 = 7

  int64_t

• STRING = 8

  string

• BOOL = 9

  bool

• FLOAT16 = 10

  IEEE754 half-precision floating-point format (16 bits wide). This format has 1 sign bit, 5 exponent bits, and 10 mantissa bits.

• DOUBLE = 11

• UINT32 = 12

• UINT64 = 13

• COMPLEX64 = 14

  complex with float32 real and imaginary components

• COMPLEX128 = 15

  complex with float64 real and imaginary components

• BFLOAT16 = 16

  Non-IEEE floating-point format based on IEEE754 single-precision floating-point number truncated to 16 bits. This format has 1 sign bit, 8 exponent bits, and 7 mantissa bits.

• FLOAT8E4M3FN = 17

  Non-IEEE floating-point format based on papers FP8 Formats for Deep Learning, https://arxiv.org/abs/2209.05433, 8-bit Numerical Formats For Deep Neural Networks, https://arxiv.org/pdf/2206.02915.pdf. Operators supported FP8 are Cast, CastLike, QuantizeLinear, DequantizeLinear. The computation usually happens inside a block quantize / dequantize fused by the runtime.

  float 8, mostly used for coefficients, supports nan, not inf

• FLOAT8E4M3FNUZ = 18

  float 8, mostly used for coefficients, supports nan, not inf, no negative zero

• FLOAT8E5M2 = 19

  follows IEEE 754, supports nan, inf, mostly used for gradients

• FLOAT8E5M2FNUZ = 20

  follows IEEE 754, supports nan, not inf, mostly used for gradients, no negative zero

• UINT4 = 21

  4-bit data-types

  Unsigned integer in range [0, 15]

• INT4 = 22

  Signed integer in range [-8, 7], using two's-complement representation

For very large tensors, we may want to store them in chunks, in which case the following fields will specify the segment that is stored in the current TensorProto.

Used in: TensorProto

• int64 begin = 1

• int64 end = 2

Extends `Value` of `Struct` with the support of TensorValue

Used in: ListValue, Record

• oneof kind

  The kind of value.

  • NullValue null_value = 1

    Represents a null value.

  • double number_value = 2

    Represents a double value.

  • string string_value = 3

    Represents a string value.

  • bool bool_value = 4

    Represents a boolean value.

  • Record record_value = 5

    Represents a structured value.

  • ListValue list_value = 6

    Represents a repeated `Value`.

  • TensorProto tensor_value = 7

    Represents a tensor `Value`.