package toco

Get desktop application:
View/edit binary Protocol Buffers messages

Supported I/O file formats. Some formats may be input-only or output-only.

Used in: TocoFlags

• FILE_FORMAT_UNKNOWN = 0

• TENSORFLOW_GRAPHDEF = 1

  GraphDef, third_party/tensorflow/core/framework/graph.proto

• TFLITE = 2

  Tensorflow's mobile inference model. third_party/tensorflow/contrib/tflite/schema.fbs

• GRAPHVIZ_DOT = 3

  GraphViz Export-only.

IODataType describes the numeric data types to be used by the output format. See input_type and inference_type below.

Used in: TocoFlags

• IO_DATA_TYPE_UNKNOWN = 0

• FLOAT = 1

  Float32, not quantized

• QUANTIZED_UINT8 = 2

  Uint8, quantized

• INT32 = 3

  Int32, not quantized

• INT64 = 4

  Int64, not quantized

• STRING = 5

  String, not quantized

Next ID to USE: 5.

Used in: ModelFlags

• optional string name = 1

  Name of the input arrays, i.e. the arrays from which input activations will be read.

• repeated int32 shape = 2

  Shape of the input. For many applications the dimensions are {batch, height, width, depth}. Often the batch is left "unspecified" by providing a value of -1. The last dimension is typically called 'depth' or 'channels'. For example, for an image model taking RGB images as input, this would have the value 3.

• optional float mean_value = 3

  mean_value and std_value parameters control the interpretation of raw input activation values (elements of the input array) as real numbers. The mapping is given by: real_value = (raw_input_value - mean_value) / std_value In particular, the defaults (mean_value=0, std_value=1) yield real_value = raw_input_value. Often, non-default values are used in image models. For example, an image model taking uint8 image channel values as its raw inputs, in [0, 255] range, may use mean_value=128, std_value=128 to map them into the interval [-1, 1). Note: this matches exactly the meaning of mean_value and std_value in (TensorFlow via LegacyFedInput).

• optional float std_value = 4

ModelFlags encodes properties of a model that, depending on the file format, may or may not be recorded in the model file. The purpose of representing these properties in ModelFlags is to allow passing them separately from the input model file, for instance as command-line parameters, so that we can offer a single uniform interface that can handle files from different input formats. For each of these properties, and each supported file format, we detail in comments below whether the property exists in the given file format. Obsolete flags that have been removed: optional int32 input_depth = 3; optional int32 input_width = 4; optional int32 input_height = 5; optional int32 batch = 6 [ default = 1]; optional float mean_value = 7; optional float std_value = 8 [default = 1.]; optional int32 input_dims = 11 [ default = 4]; repeated int32 input_shape = 13; Next ID to USE: 16.

• repeated InputArray input_arrays = 1

  Information about the input arrays, i.e. the arrays from which input activations will be read.

• repeated string output_arrays = 2

  Name of the output arrays, i.e. the arrays into which output activations will be written.

• optional bool variable_batch = 10

  If true, the model accepts an arbitrary batch size. Mutually exclusive with the 'batch' field: at most one of these two fields can be set.

• repeated ModelFlags.RnnState rnn_states = 12

• repeated ModelFlags.ModelCheck model_checks = 14

• optional bool drop_control_dependency = 15

  If true, ignore control dependency requirements in input TensorFlow GraphDef. Otherwise an error will be raised upon control dependency inputs.

Checks applied to the model, typically after toco's comprehensive graph transformations. Next ID to USE: 4.

Used in: ModelFlags

• optional string count_type = 1

  Use the name of a type of operator to check its counts. Use "Total" for overall operator counts. Use "Arrays" for overall array counts.

• optional int32 count_min = 2

  A count of zero is a meaningful check, so negative used to mean disable.

• optional int32 count_max = 3

  If count_max < count_min, then count_min is only allowed value.

Used in: ModelFlags

• optional string state_array = 1

• optional string back_edge_source_array = 2

• optional int32 size = 3

• optional bool manually_create = 4

  TODO(benoitjacob): manually_create is a temporary hack: due to discrepancies between the current toco dims tracking and TensorFlow shapes, for some models we need to manually create RNN state arrays with a specified shape. Maybe we should actually implement back-edges as operators of their own, which would remove the need for much special-casing, including here, we could probably consistently let PropagateFixedSizes handle state arrays.

TocoFlags encodes extra parameters that drive tooling operations, that are not normally encoded in model files and in general may not be thought of as properties of models, instead describing how models are to be processed in the context of the present tooling job. Next Id: 11

• optional FileFormat input_format = 1

  Input file format

• optional FileFormat output_format = 2

  Output file format

• repeated IODataType input_types = 9

  Numeric data types of the input arrays in the output format. This controls what input types the output file will be expecting. This is not a description of the input types of the input file. For example, the input file may have a float input placeholder, but we may want to generate a quantized TFLite file from it, or a float TFLite file taking a quantized input. The length of this list should match the length of the input_arrays list in ModelFlags.

• optional IODataType inference_type = 4

  Numeric data type of the internal activations array and output array. As a matter of implementation detail, most model parameter arrays (weights, etc) will tend to also use this data type. Not all will, though: for instance, bias vectors will typically get quantized as int32 when weights and activations get quantized as uint8.

• optional float default_ranges_min = 5

  default_ranges_min and default_ranges_max are helpers to experiment with quantization of models. Normally, quantization requires the input model to have (min, max) range information for every activations array. This is needed in order to know how to quantize arrays and still achieve satisfactory accuracy. However, in some circumstances one would just like to estimate the performance of quantized inference, without caring about accuracy. That is what default_ranges_min and default_ranges_max are for: when specified, they will be used as default (min, max) range boundaries for all activation arrays that lack (min, max) range information, thus allowing for quantization to proceed. It should be clear from the above explanation that these parameters are for experimentation purposes only and should not be used in production: they make it easy to quantize models, but the resulting quantized model will be inaccurate.

• optional float default_ranges_max = 6

• optional bool drop_fake_quant = 7

  Ignore and discard FakeQuant nodes. For instance, that can be used to generate plain float code without fake-quantization from a quantized graph.

• optional bool reorder_across_fake_quant = 8

  Normally, FakeQuant nodes must be strict boundaries for graph transformations, in order to ensure that quantized inference has the exact same arithmetic behavior as quantized training --- which is the whole point of quantized training and of FakeQuant nodes in the first place. However, that entails subtle requirements on where exactly FakeQuant nodes must be placed in the graph. Some quantized graphs have FakeQuant nodes at unexpected locations, that prevent graph transformations that are necessary in order to generate inference code for these graphs. Such graphs should be fixed, but as a temporary work-around, setting this reorder_across_fake_quant flag allows toco to perform necessary graph transformaitons on them, at the cost of no longer faithfully matching inference and training arithmetic.

• optional bool allow_custom_ops = 10

  If true, allow TOCO to create TF Lite Custom operators for all the unsupported Tensorflow ops.