package caffe2

Get desktop application:
View/edit binary Protocol Buffers messages

• required int64 max_batch_size = 1

• required int64 max_seq_size = 2

• required bool in_batch_broadcast = 3

• optional string onnxifi_blacklist_ops = 4

• optional int32 onnxifi_min_ops = 5

A named argument containing either singular float, integer and string values, or repeated float, int and string arrays.

Used in: NetDef, OperatorDef

• optional string name = 1

• optional float f = 2

• optional int64 i = 3

• optional bytes s = 4

• optional TensorProto t = 10

• optional NetDef n = 8

• repeated float floats = 5

• repeated int64 ints = 6

• repeated bytes strings = 7

• repeated TensorProto tensors = 11

• repeated NetDef nets = 9

• repeated QTensorProto qtensors = 12

Used to hold backend-specific options.

Used in: PartitionInfo

• required string backend_name = 1

  Name of the backend that the specified options apply to.

• repeated MapFieldEntry option = 2

  Flexible map for passing in the options.

Protobuf format for blobs that are not Tensors. We use a key to store the type of the blob. For example for a serialized DBProto, the type should be "DBReader" and the content should be a serialized DBProto object.

• optional string name = 1

• optional string type = 2

• optional TensorProto tensor = 3

• optional bytes content = 4

• optional QTensorProto qtensor = 5

• optional int32 content_num_chunks = 6

  If blob is not Tensor and is divided into chunks, content_num_chunks contains number of chunks, into which blob was divided.

• optional int32 content_chunk_id = 7

Protobuf format to serialize DBReader.

• optional string name = 1

  The name for the DB object in the workspace.

• optional string source = 2

  The source of the DB

• optional string db_type = 3

  The type of the DB

• optional string key = 4

  The current key of the DB if the DB supports seeking.

Device-specific options. We do not distinguish DeviceOption protos for different DeviceTypes, so currently all devices share the same DeviceOption proto. Fields that are specific to a device type is ignored if the type does not match. Note: if you add fields to the DeviceOption, make sure you add the corresponding changes to IsSameDevice() function in utils/proto_utils.{h,cc}.

Used in: NetDef, OperatorDef, TensorProto

• optional int32 device_type = 1

  [general] Options that need to be carried out before running the execution. optional DeviceType device_type = 1 [ default = CPU ];

  0 is CPU.

• optional int32 device_id = 2

  [general] Used together with device_type to identify the exact device

• optional uint32 random_seed = 3

  [general] The random seed to start the device random number generator with.

• optional string node_name = 4

  [general] What node this op should execute on. Used for net transformation purposes. Must be empty at execution time.

• optional int32 numa_node_id = 5

  [CPU and Linux specific] NUMA node id

• repeated string extra_info = 6

  [general] Extra information passed, not used at execution time currently.

DeviceType that Caffe2 currently supports. Note: if you add a device type, make sure you add the corresponding device line in the DeviceTypeName() function in caffe2/utils/proto_utils.cc and update c10/core/DeviceType.h

• PROTO_CPU = 0

  In default, we will use CPU.

• PROTO_CUDA = 1

  CUDA.

• PROTO_MKLDNN = 2

  Reserved for explicit MKLDNN

• PROTO_OPENGL = 3

  OpenGL

• PROTO_OPENCL = 4

  OpenCL

• PROTO_IDEEP = 5

  IDEEP.

• PROTO_HIP = 6

  AMD HIP

• PROTO_FPGA = 7

  FPGA

• PROTO_MSNPU = 8

  MSNPU

• PROTO_XLA = 9

  XLA / TPU

• PROTO_COMPILE_TIME_MAX_DEVICE_TYPES = 10

  Change the following number if you add more devices in the code.

• PROTO_ONLY_FOR_TEST = 20901

  This device type is only for test.

ExecutionStep is actually a sort-of-hacky way we simulate iteration right now.

Used in: PlanDef

• optional string name = 1

  ExecutionStep should either contain a set of substeps, or a set of network names to run in this execution step. They should NOT both be set at the same time.

• repeated ExecutionStep substep = 2

  An execution step could be recursive, in which it involves a set of substeps.

• repeated string network = 3

  Alternatively, an execution step could involve one or more networks. Note that you cannot have both substeps and networks. Choose one. Note that an execution step refers networks by their name. The actual network definition of the same name should be included in the network field of the plan. The reason is that a network object might hold internal states (think of a data layer), so we want to have the same network object that multiple steps could ask to run.

• optional int64 num_iter = 4

  Number of iterations to run this step. The substeps or the networks specified will be run sequentially, and one sequential run is considered one iteration. If this is not set, the number of iterations is assumed to be 1.

• optional string criteria_network = 5

  Criteria network specifies a single output (TensorCPU<bool>) of size (1), is run on every iteration by the executor, and execution terminates when the output[0] is `false`.

• optional string report_net = 7

  DEPRECATED. Use `run_every_ms`.

• optional int32 report_interval = 8

• optional int64 run_every_ms = 11

  If provided, execute this step at every time interval (in millisecs) while its sibiling execution steps execute in parallel. This step is guaranteed to run at least once after all non-interval siblings finished.

• optional bool concurrent_substeps = 6

  If false or not set, execute sub-steps serially. If true, execute all substeps concurrently, each one in a separate thread.

• optional string should_stop_blob = 9

  Name of a scalar boolean tensor. ES checks this blob AFTER every substeps/subnets. If specified, and the value is true, then ES will skip the rest and return immediately. This means that the report_net and the first step will always be called. Use cases: 1) the first substep stops the rest if data condition not met 2) the first substep decide which of the rest of the steps should be run. 3) external control ** It is the user's responsibility to not to put this blob in race conditions. ** For example when setting this blob in concurrent substeps

• optional bool only_once = 10

  if only_once is true, this step will only be executed once. this ONLY takes effect when using should_stop_blob

• optional bool create_workspace = 12

  Whether to create a child workspace for this step. If yes, the workflow and nets are re-created every time this step is run.

• optional int32 num_concurrent_instances = 13

  How many copies of the children execution steps to run concurrently.

ExternalDataProto stores the pointer to the content of TensorProto the content are stored in the raw format as little endian

Used in: TensorProto

• optional ExternalDataProto.SourceType source_type = 1

• optional string record_id = 2

  used together with type

• optional uint64 record_size = 5

  the size of the entire record (in bytes)

• optional int64 offset = 3

  the offset of the starting point, the content may be shared between multiple tensors

• repeated int64 strides = 4

  the strides of the content

type of the external storage type, can be the following:

Used in: ExternalDataProto

• INLINE_CONTAINER = 0

  the container defined in torch/csrc/jit/serialization.h is used, and record_id is the tag to help the runtime identify the data this type of storage is set as DEFAULT and recommended for external data storage

• SIMPLE_FILE = 1

  use external file to store the data, and record_id is the POSIX relative path to the file. this (simple) file is only for the data, and the data is stored as little endian in the file

MapFieldEntry follows the pattern for cross-proto-version maps. See https://developers.google.com/protocol-buffers/docs/proto3#maps

Used in: BackendOptions

• required string key = 1

• required string val = 2

Network definition.

Used in: Argument, PlanDef

• optional string name = 1

  the network's name

• repeated OperatorDef op = 2

  Operators that the network contains. Note: this is not named "operator" because that is a reserved word in C++.

• optional string type = 3

  The type of network that the net should be run with. This routes the network instantiation to different execution modes. The default mode, "simple", runs the operators in a sequential way as the original Caffe implementation does.

• optional int32 num_workers = 4

  the number of workers, if the operators in the network is to be carried out in parallel. Note: This is to be deprecated. Using the arg field with "num_workers" as key. Note 2: The old uses of this were never actually cleaned up

• optional DeviceOption device_option = 5

  The device option for the network. If a network has a specific device option and one of its operators does not have it set, we will copy over the device option to the operator. This allows us to basically avoid putting device options at every operator.

• repeated Argument arg = 6

• repeated string external_input = 7

  Two optional fields to declare external input and output of a net. If these two are set, when a net is created, we will sanity check for every op whether its input is declared (either as an external input, or as an intermediate blob created by one of the ops), and sanity check if all blobs in external_output are produced. In cases of memory optimization, declaring external_input and external_output also ensures that storage of these blobs are persistent: for any blob in external_input and external_output, after a network run finishes, their content are actually the right content. Any intermediate blobs' contents may be overwritten.

• repeated string external_output = 8

• repeated PartitionInfo partition_info = 9

  Partitioning info, indexed by partition names.

Operator Definition.

Used in: NetDef

• repeated string input = 1

  the name of the input blobs

• repeated string output = 2

  the name of output top blobs

• optional string name = 3

  the operator name. This is optional.

• optional string type = 4

  the operator type. This is needed to create the object from the operator registry.

• repeated Argument arg = 5

  arg is for the argument defined in operator schema

• optional DeviceOption device_option = 6

  The device option that the operator should run under.

• optional string engine = 7

  Optionally, one can specify an engine when there are multiple implementations available simultaneously for one device type. If one specifies an engine but that engine does not exist in the compiled Caffe2 binary, Caffe2 will fall back to the default engine of that device type.

• repeated string control_input = 8

  Additional 'fake' inputs used for expressing control dependencies in the operator graph. This can be used to ensure that an operator does not run until another operator is ready, for e.g. scheduling control. These are not passed as actual inputs to the Operator implementation, and are only used by the Net class for scheduling purposes.

• optional bool is_gradient_op = 9

  is_gradient_op argument is only used as a hint in shape inference and has no runtime significance

• optional string debug_info = 10

  debug information associated with the construction of the operator. This is an optional string with no assumed characteristics as operators can be constructed in any language.

• optional string domain = 11

  the domain of the operator to help runtime distinguish which operator library this OperatorDef refers to. For example, both caffe2 and aten has `Add` operator, with domain, we can easily decide which operator to execute. to support multiple operator libs, we use domain to distinguish which operator lib we refer to: - "caffe2" means this uses Caffe2 operator library - "aten" means this uses ATen operator library - "c10" is for the fused library - if the domain is missing or empty, we use "caffe2", this is for legacy models, new serializer should always export an OperatorDef with domain and op_version

• optional int64 op_version = 12

  each operator is has its own version number. operator version information each time, we change the API or semantics of the operator, we bump the version for the operator. the runtime system should check the op_version of each OperatorDef and decide it should reject or accept the model

Partition definition.

Used in: NetDef

• required string name = 1

  Name of the partition.

• repeated int32 device_id = 2

  A list of logic device ID, indicating which devices this partition can be executed on. If empty, it means the partition won't run on device but on host CPU instead.

• optional string extra_info = 3

  Extra debug info.

• repeated BackendOptions backend_options = 4

  Flexible map for passing options specific to a backend.

• optional string name = 1

  All the networks that are used in this execution. Note that networks should be ordered in the way they are executed, i.e. for a layer in a network, all its input blobs should already have been initialized by the layers or networks defined before it.

• repeated NetDef network = 2

  The networks that are going to be used in this plan.

• repeated ExecutionStep execution_step = 3

Used in: Argument, BlobProto

• repeated int64 dims = 1

• required int32 precision = 2

• required double scale = 3

• required double bias = 4

• required bool is_signed = 5

• repeated int32 data = 6

• optional string name = 7

• optional TensorProto.DataType data_type = 8

• repeated double scales = 9

  Multi-group quantization params

• repeated double biases = 10

• optional int32 axis = 11

  Multi-group quantization needed, indicates in which dimension we do the "group wise quantization"

• optional bool is_multiparam = 12

  It should be true if it is a multi-group quantization proto

TensorBoundShape is used to save bound shape inference result for a tensor. TensorBoundShape.shape is inferred shape for this tensor. TensorBoundShape.dimType contains dim_type for every dimension. eg: for dimension i, shape.dims[i] is the inferred shape and dim_type[i] is corresponding dim_type.

Used in: TensorBoundShapes

• optional TensorShape shape = 1

• repeated TensorBoundShape.DimType dim_type = 2

  dim_type.size() == shape.dims.size()

• optional string name = 3

Used in: TensorBoundShape

• UNKNOWN = 0

  unknown

• CONSTANT = 1

  constant

• BATCH = 2

  batch, corresponding dimension is batch_size

• BATCH_OF_FEATURE_MAX = 3

  batch_of_feature_max, corresponding shape is inferred_feature_length * batch_size

• BATCH_OF_FEATURE_MAX_DEFAULT = 4

  batch_of_feature_max_default corresponding shape is default_feature_length * batch_size

• FEATURE_MAX = 5

  feature_max, corresponding shape is inferred_feature_length

• FEATURE_MAX_DEFAULT = 6

  feature_max_default, corresponding shape is default_feature_length

• repeated TensorBoundShape shapes = 1

• optional int64 max_batch_size = 2

• optional int64 max_feature_len = 3

TensorProto stores serialized Tensor objects.

Used in: Argument, BlobProto, TensorProtos

• repeated int64 dims = 1

  The dimensions in the tensor.

• optional TensorProto.DataType data_type = 2

• optional TensorProto.StorageType storage_type = 12

• repeated float float_data = 3

  For float

• repeated int32 int32_data = 4

  For int32, uint8, int8, uint16, int16, bool, and float16 Note about float16: in storage we will basically convert float16 byte-wise to unsigned short and then store them in the int32_data field.

• optional bytes byte_data = 5

  For bytes

• repeated bytes string_data = 6

  For strings

• repeated double double_data = 9

  For double

• repeated int64 int64_data = 10

  For int64

• optional bytes raw_data = 13

  store the raw data, contents are serialized as little-endian

• optional ExternalDataProto external_data = 14

  store the pointer to the data

• optional string name = 7

  Optionally, a name for the tensor.

• optional DeviceOption device_detail = 8

  Optionally, a TensorProto can contain the details about the device that it was serialized from. This is useful in cases like snapshotting a whole workspace in a multi-GPU environment.

• optional TensorProto.Segment segment = 11

Data type

Used in: QTensorProto, TensorProto, TensorShape

• UNDEFINED = 0

• FLOAT = 1

  Basic types

  float

• INT32 = 2

  int

• BYTE = 3

  byte, when deserialized, is going to be restored as uint8

• STRING = 4

  string

• BOOL = 5

  Less-commonly used data types

  bool

• UINT8 = 6

  uint8_t

• INT8 = 7

  int8_t

• UINT16 = 8

  uint16_t

• INT16 = 9

  int16_t

• INT64 = 10

  int64_t

• FLOAT16 = 12

  at::Half

• DOUBLE = 13

  double

• ZERO_COLLISION_HASH = 14

  zero-collision hash state

When loading from chunks this is going to indicate where to put data in the full array. When not used full data have to be present

Used in: TensorProto

• required int64 begin = 1

• required int64 end = 2

data storage

Used in: TensorProto

• TYPED = 1

  the content is stored in typed field, for example, if the data_type is FLOAT, float_data is used to store the content.

• RAW = 2

  the content is serialized in field raw_data as little-endian

• EXTERNAL = 3

  the pointer to the content is stored in field external_data the content is serialized as little-endian

• NO_CONTENT = 4

  When StorageType is NO_CONTENT, we use TensorProto to store only type and shape information. Reuse TensorProto to store type and shape because we can just have one proto, not having another ValueInfoProto

TensorProtos stores multiple TensorProto objects in one single proto. This is useful for small tensors; For anything big, consider using a DB for storage.

• repeated TensorProto protos = 1

Used in: TensorBoundShape, TensorShapes

• repeated int64 dims = 1

• optional TensorProto.DataType data_type = 2

• repeated int32 unknown_dims = 3

• optional bool unknown_shape = 4

• optional string name = 5

• repeated TensorShape shapes = 1