package nvdsinferserver.config

Get desktop application:
View/edit binary Protocol Buffers messages

* Network backend Settings

Used in: InferenceConfig

• repeated InputLayer inputs = 1

  * input tensors settings, optional

• repeated OutputLayer outputs = 2

  * outputs tensor settings, optional

• oneof infer_framework

  * inference framework

  • TritonParams trt_is = 3

    * [deprecated] TRT-IS inference framework. Use triton instead of trt_is

  • TritonParams triton = 4

    * Triton inference framework

• MemoryType output_mem_type = 5

  * Output tensor memory type. Default: MEMORY_TYPE_DEFAULT, it is Triton preferred memory type.

• bool disable_warmup = 6

  * disable warmup

* Deepstream Classifciation settings

Used in: PostProcessParams

• float threshold = 1

  * classifciation threshold

• string custom_parse_classifier_func = 2

  * custom function for classification parsing

* Custom lib for preload

Used in: InferenceConfig

• string path = 1

  * Path point to the custom library

* Deepstream Detection settings

Used in: PostProcessParams

• int32 num_detected_classes = 1

  * Number of classes detected by a detector network.

• map<int32, DetectionParams.PerClassParams> per_class_params = 2

  * Per class detection parameters. key-value is for <class_id:class_parameter>

• string custom_parse_bbox_func = 3

  * Name of the custom bounding box function in the custom library.

• oneof clustering_policy

  * cluster methods for bbox, choose one only

  • DetectionParams.Nms nms = 4

    * non-maximum-suppression, reserved, not supported yet

  • DetectionParams.DbScan dbscan = 5

    * DbScan clustering parameters

  • DetectionParams.GroupRectangle group_rectangle = 6

    * grouping rectagules

  • DetectionParams.SimpleCluster simple_cluster = 7

    * simple threshold filter

* DBScan object clustering

Used in: DetectionParams

• float pre_threshold = 1

  * Bounding box detection threshold.

  float post_threshold = 2;

• float eps = 3

  * Epsilon to control merging of overlapping boxes

• int32 min_boxes = 4

  * Minimum boxes in a cluster to be considered an object

• float min_score = 5

  * Minimum score in a cluster for it to be considered as an object

* cluster method based on grouping rectangles

Used in: DetectionParams

• float confidence_threshold = 1

  * detection score less this threshold would be rejected

• int32 group_threshold = 2

  * how many bbox can be clustered together

• float eps = 3

  * Epsilon to control merging of overlapping boxes

* non-maximum-suppression cluster method

Used in: DetectionParams

• float confidence_threshold = 1

  * detection score less this threshold would be rejected

• float iou_threshold = 2

  * IOU threshold

• int32 topk = 3

  * top kth detection results to keep after nms. 0), keep all

* specific parameters controled per class

Used in: DetectionParams

• float pre_threshold = 1

  * pre-threshold used for filter out confidence less than the value

* simple cluster method for confidence filter

Used in: DetectionParams

• float threshold = 1

  * detection score less this threshold would be rejected

* extrac controls

Used in: InferenceConfig

• bool copy_input_to_host_buffers = 1

  * enable if need copy input tensor data for application output parsing, it's disabled by default

• int32 output_buffer_pool_size = 2

  * defined how many buffers allocated for output tensors in the pool. Optional, default is 2, the value can be in range [2, 10+]

• string custom_process_funcion = 3

  * custom function to create a specific processor IInferCustomProcessor. e.g. custom_process_funcion: CreateCustomProcessor

Used in: PreProcessParams

• FRAME_SCALING_HW_DEFAULT = 0

• FRAME_SCALING_HW_GPU = 1

• FRAME_SCALING_HW_VIC = 2

* Inference configuration

Used in: PluginControl

• uint32 unique_id = 1

  * unique id, larger than 0, required for multiple models inference

• repeated int32 gpu_ids = 2

  * gpu id settings. Optional. support single gpu only at this time default values [0]

• uint32 max_batch_size = 3

  * max batch size. Required, can be reset by plugin

• optional BackendParams backend = 4

  * inference backend parameters. required

• oneof preprocessing

  * preprocessing for tensors, required

  • PreProcessParams preprocess = 5

  • InputTensorFromMeta input_tensor_from_meta = 10

• oneof postprocessing

  * postprocessing for all tensor data, required

  • PostProcessParams postprocess = 6

• optional CustomLib custom_lib = 7

  * Custom libs for tensor output parsing or preload, optional

• oneof advanced

  * advanced controls as optional

  • ExtraControl extra = 8

    * extrac controls

• oneof lstm_control

  * LSTM controller

  • LstmParams lstm = 9

    * LSTM parameters

• bool clip_object_outside_roi = 11

  * Clip the object bounding-box which lies outside the roi specified by nvdspreprosess plugin.

* Network Input layer information

Used in: BackendParams

• string name = 1

  * input tensor name, optional

• repeated int32 dims = 2

  * fixed inference shape, only required when backend has wildcard shape

• TensorDataType data_type = 3

  * tensor data type, optional. default TENSOR_DT_NONE

* Input tensor is preprocessed

Used in: InferenceConfig

• bool is_first_dim_batch = 1

  * first dims is not a batch-size

* Network LSTM Parameters

Used in: InferenceConfig

• repeated LstmParams.LstmLoop loops = 1

* init constant value for lstm input tensors, usually zero or one

Used in: LstmLoop

• float value = 1

  * const value

* LSTM loop information

Used in: LstmParams

• string input = 1

  * input tensor name

• string output = 2

  * output tensor name

• oneof init_state

  * initialize input tensor for first frame

  • InitConst init_const = 3

    * init const value, default is zero

• bool keep_output = 4

  * enable if need keep lstm output tensor data for application output parsing, it's disabled by default

Used in: PreProcessParams

• MEDIA_FORMAT_NONE = 0

• IMAGE_FORMAT_RGB = 1

• IMAGE_FORMAT_BGR = 2

• IMAGE_FORMAT_GRAY = 3

* Tensor memory type

Used in: BackendParams

• MEMORY_TYPE_DEFAULT = 0

• MEMORY_TYPE_CPU = 1

• MEMORY_TYPE_GPU = 2

* Other Network settings, need application to do postprocessing

Used in: PostProcessParams

• string type_name = 1

  * reserved field

* Network Onput layer information

Used in: BackendParams

• string name = 1

  * output tensor name

• uint64 max_buffer_bytes = 2

  * set max buffer bytes for output tensor

* Plugin Control settings for input / inference / output

• optional InferenceConfig infer_config = 1

  * Low-level libnvds_infer_server inference configuration settings

• optional PluginControl.InputControl input_control = 2

  * Control plugin input buffers, object filter before inference

• optional PluginControl.OutputControl output_control = 3

  * Control plugin output meta data after inference

* Boudingbox filter

Used in: DetectClassFilter, InputObjectControl

• uint32 min_width = 1

  * Boudingbox minimum width

• uint32 min_height = 2

  * Boudingbox minimum height

• uint32 max_width = 3

  * Boudingbox maximum width

• uint32 max_height = 4

  * Boudingbox maximum height

* Color values for Red/Green/Blue/Alpha, all values are in range [0, 1]

Used in: DetectClassFilter

• float r = 1

  * Red color value

• float g = 2

  * Green color value

• float b = 3

  * Blue color value

• float a = 4

  * Alpha color value

* Detection of classes filter

Used in: OutputDetectionControl

• optional BBoxFilter bbox_filter = 1

  * Detection Bounding box filter

• uint32 roi_top_offset = 2

  * Offset of the RoI from the top of the frame. Only objects within the RoI are output

• uint32 roi_bottom_offset = 3

  * Offset of the RoI from the bottom of the frame. Only objects within the RoI are output

• optional Color border_color = 4

  * Specify border color for detection bounding boxes

• optional Color bg_color = 5

  * Specify background color for detection bounding boxes

* Plugin input data control policy

Used in: PluginControl

• ProcessMode process_mode = 1

  * Processing mode setting, optional

• int32 operate_on_gie_id = 2

  * Unique ID of the GIE on whose metadata (bounding boxes) this GIE is to operate on. It is used for secondary GIE only.

• repeated int32 operate_on_class_ids = 3

  * Class IDs of the parent GIE on which this GIE is to operate on. It is used for secondary GIE only.

• oneof inference_interval

  * Specifies the number of consecutive, batches to be skipped for inference. Default is 0.

  • uint32 interval = 4

    For primary inference

  • uint32 secondary_reinfer_interval = 7

    For secondary inferrence

• bool async_mode = 5

  * Enables inference on detected objects and asynchronous metadata attachments. Works only when tracker-id is valid. It's used for classifier with secondary GIE only.

• oneof object_filter

  * Input object filter policy

  • InputObjectControl object_control = 6

    * input object control settings

* Input objects control

Used in: InputControl

• optional BBoxFilter bbox_filter = 1

  * Input bounding box of objects filter

* Plugin output data control policy

Used in: PluginControl

• bool output_tensor_meta = 1

  Enable attaching inference output tensor metadata

• oneof postprocess_control

  Postprocessing control policy

  • OutputDetectionControl detect_control = 2

    Detection results filter

• oneof ClassifierMeta

  • string classifier_type = 3

    Classifier type of a particular nvinferserver component.

* Output detection results control

Used in: OutputControl

• optional DetectClassFilter default_filter = 1

  * Default detection classes filter

• map<uint32, DetectClassFilter> specific_class_filters = 2

  * specifies detection filters per class instead of default filter

* Processing Mode

Used in: InputControl

• PROCESS_MODE_DEFAULT = 0

  * Processing Default Mode

• PROCESS_MODE_FULL_FRAME = 1

  * Processing Full Frame Mode

• PROCESS_MODE_CLIP_OBJECTS = 2

  * Processing Object Clipping Mode

* Post-processing settings

Used in: InferenceConfig

• string labelfile_path = 1

  * label file path. It relative to config file path if value is not absoluate path

• oneof process_type

  * post-process can only have one of the following types

  • DetectionParams detection = 2

    * deepstream detection parameters

  • ClassificationParams classification = 3

    * deepstream classification parameters

  • SegmentationParams segmentation = 4

    * deepstream segmentation parameters

  • OtherNetworkParams other = 5

    * deepstream other postprocessing parameters

  • TritonClassifyParams trtis_classification = 6

    * [deprecated] TRT-IS classification parameters

  • TritonClassifyParams triton_classification = 7

    * Triton classification parameters, replacing trtis_classification

* preprocessing settings

Used in: InferenceConfig

• MediaFormat network_format = 1

  * Network input format

• TensorOrder tensor_order = 2

  * Network input tensor order

• string tensor_name = 3

  * preprocessing data set to network tensor name

• int32 maintain_aspect_ratio = 4

  * Indicating if aspect ratio should be maintained when scaling to network resolution. Right/bottom areas will be filled with black areas.

• FrameScalingHW frame_scaling_hw = 5

  * Compute hardware to use for scaling frames / objects.

• uint32 frame_scaling_filter = 6

  * Interpolation filter to use while scaling. Refer to NvBufSurfTransform_Inter for supported filter values.

• oneof preprocess_method

  * Preprocessing methods

  • PreProcessParams.ScaleNormalize normalize = 7

    * usual scaling normalization for images

• int32 symmetric_padding = 8

  * Indicating if symmetric padding should be used or not while scaling to network resolution. Bottom-right padding is used by default.

* Input data normalization settings

Used in: PreProcessParams

• float scale_factor = 1

  * Normalization factor to scale the input pixels with.

• repeated float channel_offsets = 2

  * Per channel offsets for mean subtraction. This is an alternative to the mean image file. The number of offsets in the array should be exactly equalto the number of input channels.

• string mean_file = 3

  * Path to the mean image file (PPM format). Resolution of the file should be equal to the network input resolution.

* Deepstream segmentation settings

Used in: PostProcessParams

• float threshold = 1

  * Segmentation threshold

• int32 num_segmentation_classes = 2

  * Number of classes detected by the segmentation network.

• string custom_parse_segmentation_func = 3

  * Custom function for parsing segmentation output

Used in: InputLayer

• TENSOR_DT_NONE = 0

• TENSOR_DT_FP32 = 1

• TENSOR_DT_FP16 = 2

• TENSOR_DT_INT8 = 3

• TENSOR_DT_INT16 = 4

• TENSOR_DT_INT32 = 5

• TENSOR_DT_UINT8 = 6

• TENSOR_DT_UINT16 = 7

• TENSOR_DT_UINT32 = 8

• TENSOR_DT_FP64 = 9

• TENSOR_DT_INT64 = 10

• TENSOR_DT_UINT64 = 11

• TENSOR_DT_STRING = 12

Used in: PreProcessParams

• TENSOR_ORDER_NONE = 0

• TENSOR_ORDER_LINEAR = 1

• TENSOR_ORDER_NHWC = 2

* Triton classifcation settings

Used in: PostProcessParams, TritonParams

• uint32 topk = 1

  * top k classification results

• float threshold = 2

  * classifciation threshold

• string tensor_name = 3

  * [optional] specify which output tensor is used for triton classification.

Used in: TritonParams

• string url = 1

• bool enable_cuda_buffer_sharing = 2

  * Enable sharing of input CUDA buffers with local Triton server. If enabled, the input CUDA buffers are shared with the Triton server to improve performance. This feature should be enabled only when the Triton server is on the same machine. Applicable for x86 dGPU platform, not supported on Jetson devices. By default disabled, CUDA buffers are copied to system memory while creating the inference request

* Triton models repo settings

Used in: TritonParams

• repeated string root = 1

  * root directory for all models All models should set same @a root value

• uint32 log_level = 2

  * log verbose level, the larger the more logs output (0): ERROR; (1): WARNING; (2): INFO (3+): VERBOSE Level

• bool strict_model_config = 3

  * enable strict model config true: config.pbtxt must exsit. false: Triton try to figure model's config file, it may cause failure on different input/output dims.

• float tf_gpu_memory_fraction = 4

  * tensorflow gpu memory fraction, default 0.0

• bool tf_disable_soft_placement = 5

  * tensorflow soft placement, allowed by default

• float min_compute_capacity = 6

  * minimum compute capacity, dGPU: default 6.0; Jetson: default 5.3.

• string backend_dir = 7

  * triton backends directory

• string model_control_mode = 8

  * triton model control mode, select from "none": load all models in 'root' repo at startup once. "explicit": load/unload models by 'TritonParams'. If value is empty, will use "explicit" by default.

• repeated TritonModelRepo.CudaDeviceMem cuda_device_memory = 9

  * Triton server reserved cuda memory size for each device. If device not added, will use Triton runtime's default memory size 256MB. If \a cuda_memory_pool_byte_size is set 0, plugin will not reserve cuda memory on that device.

• oneof pinned_mem

  * Triton server reserved pinned memory size during initialization

  • uint64 pinned_memory_pool_byte_size = 10

• repeated TritonModelRepo.BackendConfig backend_configs = 11

* Triton backend config settings

Used in: TritonModelRepo

• string backend = 1

  * backend name

• string setting = 2

  * backend setting name

• string value = 3

  * backend setting value

* Cuda Memory settings for GPU device

Used in: TritonModelRepo

• uint32 device = 1

  * GPU device id

• uint64 memory_pool_byte_size = 2

  * Cuda Memory Pool byte size

* Triton inference backend parameters

Used in: BackendParams

• string model_name = 1

  * trt-is model name

• int64 version = 2

  * model version, -1 is for latest version, required

• repeated TritonClassifyParams class_params = 3

  * Triton classifications, optional

• oneof server

  • TritonModelRepo model_repo = 4

    * trt-is server model repo, all models must have same @a model_repo

  • TritonGrpcParams grpc = 5