package tflite.proto

Get desktop application:
View/edit binary Protocol Buffers messages

An error that occurred during benchmarking. Used with event type ERROR.

Used in: BenchmarkEvent

• optional BenchmarkStage stage = 1

  How far benchmarking got.

• optional int32 exit_code = 2

  Process exit code.

• optional int32 signal = 3

  Signal the process received.

• repeated ErrorCode error_code = 4

  Handled tflite error.

• optional int32 mini_benchmark_error_code = 5

  Mini-benchmark error code.

Top-level benchmarking event stored on-device. All events for a model are parsed to detect the status.

Used in: BestAccelerationDecision

• optional TFLiteSettings tflite_settings = 1

  Which settings were used for benchmarking.

• optional BenchmarkEventType event_type = 2

  Type of the event.

• optional BenchmarkResult result = 3

  Result of benchmark, used when type is END.

• optional BenchmarkError error = 4

  Error during benchmark, used when type is ERROR.

• optional int64 boottime_us = 5

  Start timestamps. These are used for 1. Checking whether a test was started but not completed within a given deadline. 2. Optionally, telemetry timestamps.

• optional int64 wallclock_us = 6

Which stage of benchmarking the event is for. There might be multiple events with the same type, if a benchmark is run multiple times.

Used in: BenchmarkEvent

• UNDEFINED_BENCHMARK_EVENT_TYPE = 0

• START = 1

  Benchmark start. A start without an end can be interpreted as a test that has crashed or hung.

• END = 2

  Benchmarking completion. A model was successfully loaded, acceleration configured and inference run without errors. There may still be an issue with correctness of results, or with performance.

• ERROR = 3

  Benchmark was not completed due to an error. The error may be a handled error (e.g., failure in a delegate), or a crash.

• LOGGED = 4

  Benchmark data has been sent for logging.

Represent a failure during the initialization of the mini-benchmark.

Used in: MiniBenchmarkEvent

• optional int32 initialization_status = 1

  Status code returned by the mini-benchmark initialization function.

A correctness metric from a benchmark, for example KL-divergence between known-good CPU output and on-device output. These are primarily used for telemetry and monitored server-side.

Used in: BenchmarkResult

• optional string name = 1

• repeated float values = 2

Outcome of a successfully complete benchmark run. This information is intended to both be used on-device to select best compute configuration as well as sent to server for monitoring. Used with event type END.

Used in: BenchmarkEvent

• repeated int64 initialization_time_us = 1

  Time to load model and apply acceleration. Initialization may get run multiple times to get information on variance.

• repeated int64 inference_time_us = 2

  Time to run inference (call Invoke()). Inference may get run multiple times to get information on variance.

• optional int32 max_memory_kb = 3

  Maximum memory used. Measures size of application heap (does not necessarily take into account driver-side allocation.

• optional bool ok = 4

  Whether the inference produced correct results (validation graph output 'ok' for all test inputs). Used on-device to disallow configurations that produce incorrect results (e.g., due to OpenCL driver bugs).

• repeated BenchmarkMetric metrics = 5

  Metrics that were used to determine the 'ok' status.

When during benchmark execution an error occurred.

Used in: BenchmarkError

• UNKNOWN = 0

• INITIALIZATION = 1

  During model loading or delegation.

• INFERENCE = 2

  During inference.

Where to store mini-benchmark state.

Used in: MinibenchmarkSettings

• optional string storage_file_path = 1

  Base path to the files used to to store benchmark results in. Two files will be generated: one with the given path and an extra file to store the best acceleration results events at path storage_file_path + ".best.fb". Must be specific to the model. Note on Android, this should be the code cache directory.

• optional string data_directory_path = 2

  Path to a directory for intermediate files (lock files, extracted binaries). Note on Android, this typically is the data cache directory (i.e. the one returned by `getCacheDir()`).

Represent the decision on the best acceleration from the mini-benchmark.

Used in: MiniBenchmarkEvent

• optional int32 number_of_source_events = 1

  Number of events used to take the decision. Using just the size instaed of the full list of events to save space.

• optional BenchmarkEvent min_latency_event = 2

  Event with min latency in the source ones.

• optional int64 min_inference_time_us = 3

  Min latency as read from min_latency_event.

Used in: TFLiteSettings

• optional int32 num_threads = 1

  Set to -1 to let the interpreter choose. Otherwise, must be > 0.

One possible acceleration configuration.

• optional ExecutionPreference preference = 1

  Which preference to use this accelerator for.

• optional TFLiteSettings tflite_settings = 2

  How to configure TFLite

• optional string model_namespace_for_statistics = 3

  Identifiers to use for instrumentation and telemetry.

• optional string model_identifier_for_statistics = 4

• optional MinibenchmarkSettings settings_to_test_locally = 5

  'Maybe' acceleration: use mini-benchmark to select settings.

Coral Dev Board / USB accelerator delegate settings. See https://github.com/google-coral/edgetpu/blob/master/libedgetpu/edgetpu_c.h

Used in: TFLiteSettings

• optional string device = 1

  The Edge Tpu device to be used. See https://github.com/google-coral/libcoral/blob/982426546dfa10128376d0c24fd8a8b161daac97/coral/tflite_utils.h#L131-L137

• optional CoralSettings.Performance performance = 2

  The desired performance level. This setting adjusts the internal clock rate to achieve different performance / power balance. Higher performance values improve speed, but increase power usage.

• optional bool usb_always_dfu = 3

  If true, always perform device firmware update (DFU) after reset. DFU is usually only necessary after power cycle.

• optional int32 usb_max_bulk_in_queue_length = 4

  The maximum bulk in queue length. Larger queue length may improve USB performance on the direction from device to host. When not specified (or zero), `usb_max_bulk_in_queue_length` will default to 32 according to the current EdgeTpu Coral implementation.

Used in: CoralSettings

• UNDEFINED = 0

• MAXIMUM = 1

• HIGH = 2

• MEDIUM = 3

• LOW = 4

TFLite accelerator to use.

Used in: ErrorCode, TFLiteSettings

• NONE = 0

• NNAPI = 1

• GPU = 2

• HEXAGON = 3

• XNNPACK = 4

• EDGETPU = 5

  The EdgeTpu in Pixel devices.

• EDGETPU_CORAL = 6

  The Coral EdgeTpu Dev Board / USB accelerator.

EdgeTPU device spec.

Used in: EdgeTpuSettings

• optional EdgeTpuDeviceSpec.PlatformType platform_type = 1

  Execution platform for the EdgeTPU device.

• optional int32 num_chips = 2

  Number of chips to use for the EdgeTPU device.

• repeated string device_paths = 3

  Paths to the EdgeTPU devices;

• optional int32 chip_family = 4

  Chip family used by the EdgeTpu device.

EdgeTPU platform types.

Used in: EdgeTpuDeviceSpec

• MMIO = 0

• REFERENCE = 1

• SIMULATOR = 2

• REMOTE_SIMULATOR = 3

Used in: EdgeTpuSettings

• optional EdgeTpuPowerState inactive_power_state = 1

  Inactive power states between inferences.

• optional int64 inactive_timeout_us = 2

  Inactive timeout in microseconds between inferences.

Generic definitions of EdgeTPU power states.

Used in: EdgeTpuInactivePowerConfig, EdgeTpuSettings

• UNDEFINED_POWERSTATE = 0

  Undefined power state.

• TPU_CORE_OFF = 1

  TPU core is off but control cluster is on.

• READY = 2

  A non-active low-power state that has much smaller transition time to active compared to off.

• ACTIVE_MIN_POWER = 3

  Minimum power active state.

• ACTIVE_VERY_LOW_POWER = 4

  Very low performance, very low power.

• ACTIVE_LOW_POWER = 5

  Low performance, low power.

• ACTIVE = 6

  The normal performance and power. This setting usually provides the optimal perf/power trade-off for the average use-case.

• OVER_DRIVE = 7

  Maximum performance level. Potentially higher power and thermal. This setting may not be allowed in production depending on the system.

EdgeTPU Delegate settings.

Used in: TFLiteSettings

• optional EdgeTpuPowerState inference_power_state = 1

  Target inference power state for running the model.

• repeated EdgeTpuInactivePowerConfig inactive_power_configs = 2

  Inactive power states between inferences.

• optional int32 inference_priority = 3

  Priority for the inference request.

• optional EdgeTpuDeviceSpec edgetpu_device_spec = 4

  Device spec for creating the EdgeTpu device.

• optional string model_token = 5

  A unique identifier of the input TfLite model.

• optional EdgeTpuSettings.FloatTruncationType float_truncation_type = 6

  Float truncation type for EdgeTPU.

Float truncation types for EdgeTPU.

Used in: EdgeTpuSettings

• UNSPECIFIED = 0

• NO_TRUNCATION = 1

• BFLOAT16 = 2

• HALF = 3

A handled error.

Used in: BenchmarkError

• optional Delegate source = 1

  Which delegate the error comes from (or NONE, if it comes from the tflite framework).

• optional int32 tflite_error = 2

  What the tflite level error is.

• optional int64 underlying_api_error = 3

  What the underlying error is (e.g., NNAPI or OpenGL error).

ExecutionPreference is used to match accelerators against the preferences of the current application or usecase. Some of the values here can appear both in the compatibility list and as input, some only as input. These are separate from NNAPIExecutionPreference - the compatibility list design doesn't assume a one-to-one mapping between which usecases compatibility list entries have been developed for and what settings are used for NNAPI.

Used in: ComputeSettings

• ANY = 0

  Match any selected preference. Allowlist (semantically - value is same as on input).

• LOW_LATENCY = 1

  Match low latency preference. Both compatibility list and input.

• LOW_POWER = 2

  Math low power preference. Both compatibility list and input.

• FORCE_CPU = 3

  Never accelerate. Can be used for input to compatibility list or for standalone Acceleration configuration.

Whether to automatically fallback to TFLite CPU path on delegation errors. Typically fallback is enabled in production use but disabled in tests and benchmarks to ensure they test the intended path.

Used in: NNAPISettings, TFLiteSettings

• optional bool allow_automatic_fallback_on_compilation_error = 7

  Whether to allow automatically falling back to TfLite CPU path on compilation failure. Default is not allowing automatic fallback. This is useful in naive production usecases where the caller would prefer for the model to run even if it's not accelerated. More advanced users will implement fallback themselves; e.g., by using a different model on CPU. Note that compilation errors may occur either at initial ModifyGraphWithDelegate() time, or when calling AllocateTensors() after resizing.

• optional bool allow_automatic_fallback_on_execution_error = 8

  Whether to allow automatically falling back to TfLite CPU path on execution error. Default is not allowing automatic fallback. Experimental, use with care (only when you have complete control over the client code). The caveat above for compilation error holds. Additionally, execution-time errors are harder to handle automatically as they require invalidating the TfLite interpreter which most client code has not been designed to deal with.

Which GPU backend to select. Default behaviour on Android is to try OpenCL and if it's not available fall back to OpenGL.

Used in: GPUSettings

• UNSET = 0

• OPENCL = 1

• OPENGL = 2

  Not yet supported. VULKAN = 3; METAL = 4;

GPU inference priorities define relative priorities given by the GPU delegate to different client needs. Corresponds to TfLiteGpuInferencePriority.

Used in: GPUSettings

• GPU_PRIORITY_AUTO = 0

• GPU_PRIORITY_MAX_PRECISION = 1

• GPU_PRIORITY_MIN_LATENCY = 2

• GPU_PRIORITY_MIN_MEMORY_USAGE = 3

GPU Delegate settings. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/gpu/delegate.h

Used in: TFLiteSettings

• optional bool is_precision_loss_allowed = 1

  Ignored if inference_priority1/2/3 are set.

• optional bool enable_quantized_inference = 2

• optional GPUBackend force_backend = 3

• optional GPUInferencePriority inference_priority1 = 4

  Ordered priorities provide better control over desired semantics, where priority(n) is more important than priority(n+1). Therefore, each time inference engine needs to make a decision, it uses ordered priorities to do so. Default values correspond to GPU_PRIORITY_AUTO. AUTO priority can only be used when higher priorities are fully specified. For example: VALID: priority1 = MIN_LATENCY, priority2 = AUTO, priority3 = AUTO VALID: priority1 = MIN_LATENCY, priority2 = MAX_PRECISION, priority3 = AUTO INVALID: priority1 = AUTO, priority2 = MIN_LATENCY, priority3 = AUTO INVALID: priority1 = MIN_LATENCY, priority2 = AUTO, priority3 = MAX_PRECISION Invalid priorities will result in error. For more information, see TfLiteGpuDelegateOptionsV2.

• optional GPUInferencePriority inference_priority2 = 5

• optional GPUInferencePriority inference_priority3 = 6

Hexagon Delegate settings. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/hexagon/hexagon_delegate.h

Used in: TFLiteSettings

• optional int32 debug_level = 1

• optional int32 powersave_level = 2

• optional bool print_graph_profile = 3

• optional bool print_graph_debug = 4

Events generated by the mini-benchmark before and after triggering the different configuration-specific benchmarks

• optional bool is_log_flushing_event = 1

  Not using oneof because of the way the generated cpp code. See comment above on TfLite settings for details.

• optional BestAccelerationDecision best_acceleration_decision = 2

• optional BenchmarkInitializationFailure initialization_failure = 3

How to run a minibenchmark.

Used in: ComputeSettings

• repeated TFLiteSettings settings_to_test = 1

  Which settings to test. This would typically be filled in from an allowlist.

• optional ModelFile model_file = 2

  How to access the model. This would typically be set dynamically, as it depends on the application folder and/or runtime state.

• optional BenchmarkStoragePaths storage_paths = 3

  Where to store state. This would typically be set dynamically, as it depends on the application folder.

How to access the model for mini-benchmark. Since mini-benchmark runs in a separate process, it can not access an in-memory model. It can read the model either from a file or from a file descriptor. The file descriptor typically comes from the Android asset manager. Users should set either filename, or all of fd, offset and length.

Used in: MinibenchmarkSettings

• optional string filename = 1

  Filename for reading model from.

• optional int64 fd = 2

  File descriptor to read model from.

• optional int64 offset = 3

  Offset for model in file descriptor.

• optional int64 length = 4

  Length of model in file descriptor.

Used in: NNAPISettings

• UNDEFINED = 0

  Undefined.

• NNAPI_LOW_POWER = 1

  Prefer executing in a way that minimizes battery drain.

• NNAPI_FAST_SINGLE_ANSWER = 2

  Prefer returning a single answer as fast as possible, even if this causes more power consumption.

• NNAPI_SUSTAINED_SPEED = 3

  Prefer maximizing the throughput of successive frames, for example when processing successive frames coming from the camera.

Used in: NNAPISettings

• NNAPI_PRIORITY_UNDEFINED = 0

• NNAPI_PRIORITY_LOW = 1

• NNAPI_PRIORITY_MEDIUM = 2

• NNAPI_PRIORITY_HIGH = 3

NNAPI delegate settings.

Used in: TFLiteSettings

• optional string accelerator_name = 1

  Which instance (NNAPI accelerator) to use. One driver may provide several accelerators (though a driver may also hide several back-ends behind one name, at the choice of the driver vendor). Note that driver introspection is only available in Android Q and later.

• optional string cache_directory = 2

  NNAPI model compilation caching settings to be passed to tflite::StatefulNnApiDelegate

• optional string model_token = 3

• optional NNAPIExecutionPreference execution_preference = 4

  NNAPI execution preference to pass. See https://developer.android.com/ndk/reference/group/neural-networks.html

• optional int32 no_of_nnapi_instances_to_cache = 5

  Number of instances to cache for the same model (for input size changes). This is mandatory for getting reasonable performance in that case.

• optional FallbackSettings fallback_settings = 6

  Deprecated; use the fallback_settings in TFLiteSettings. Whether to automatically fall back to TFLite CPU path.

• optional bool allow_nnapi_cpu_on_android_10_plus = 7

  Whether to allow use of NNAPI CPU (nnapi-reference accelerator) on Android 10+ when an accelerator name is not specified. The NNAPI CPU typically performs less well than the TfLite built-in kernels; but allowing allows a model to be partially accelerated which may be a win.

• optional NNAPIExecutionPriority execution_priority = 8

• optional bool allow_dynamic_dimensions = 9

  Whether to allow dynamic dimension sizes without re-compilation. A tensor of with dynamic dimension must have a valid dims_signature defined. Only supported in NNAPI 1.1 and newer versions. WARNING: Setting this flag to true may result in model being rejected by accelerator. This should only be enabled if the target device supports dynamic dimensions of the model. By default this is set to false.

• optional bool allow_fp16_precision_for_fp32 = 10

  Whether to allow the NNAPI accelerator to optionally use lower-precision float16 (16-bit floating point) arithmetic when doing calculations on float32 (32-bit floating point).

• optional bool use_burst_computation = 11

  Whether to use NNAPI Burst mode. Burst mode allows accelerators to efficiently manage resources, which would significantly reduce overhead especially if the same delegate instance is to be used for multiple inferences.

How to configure TFLite.

Used in: BenchmarkEvent, ComputeSettings, MinibenchmarkSettings

• optional Delegate delegate = 1

  Which delegate to use.

• optional NNAPISettings nnapi_settings = 2

  How to configure the chosen delegate. (In principle we would like to use 'oneof', but flatc turns that into an nested anonymous table rather than a union. See https://github.com/google/flatbuffers/issues/4628).

• optional GPUSettings gpu_settings = 3

• optional HexagonSettings hexagon_settings = 4

• optional XNNPackSettings xnnpack_settings = 5

• optional CPUSettings cpu_settings = 6

  How to configure CPU execution.

• optional int32 max_delegated_partitions = 7

  Shared delegation settings.

• optional EdgeTpuSettings edgetpu_settings = 8

  For configuring the EdgeTpuDelegate.

• optional CoralSettings coral_settings = 10

  For configuring the Coral EdgeTpu Delegate.

• optional FallbackSettings fallback_settings = 9

  Whether to automatically fall back to TFLite CPU path.

XNNPack Delegate settings. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/xnnpack/xnnpack_delegate.h

Used in: TFLiteSettings

• optional int32 num_threads = 1