package google.cloud.videointelligence.v1p3beta1
Get desktop application:
View/edit binary Protocol Buffers messages
Service that implements streaming Google Cloud Video Intelligence API.
rpc StreamingAnnotateVideo (stream , stream )
video_intelligence_streaming.proto:342Performs video annotation with bidirectional streaming: emitting results while sending video/audio bytes. This method is only available via the gRPC API (not REST).
The top-level message sent by the client for the `StreamingAnnotateVideo` method. Multiple `StreamingAnnotateVideoRequest` messages are sent. The first message must only contain a `StreamingVideoConfig` message. All subsequent messages must only contain `input_content` data.
oneof streaming_request
*Required* The streaming request, which is either a streaming config or video content.
video_config = 1
Provides information to the annotator, specifing how to process the request. The first `AnnotateStreamingVideoRequest` message must only contain a `video_config` message.
bytes input_content = 2
The video data to be annotated. Chunks of video data are sequentially sent in `StreamingAnnotateVideoRequest` messages. Except the initial `StreamingAnnotateVideoRequest` message containing only `video_config`, all subsequent `AnnotateStreamingVideoRequest` messages must only contain `input_content` field.
`StreamingAnnotateVideoResponse` is the only message returned to the client by `StreamingAnnotateVideo`. A series of zero or more `StreamingAnnotateVideoResponse` messages are streamed back to the client.
optional error = 1
If set, returns a [google.rpc.Status][] message that specifies the error for the operation.
optional annotation_results = 2
Streaming annotation results.
string annotation_results_uri = 3
GCS URI that stores annotation results of one streaming session. It is a directory that can hold multiple files in JSON format. Example uri format: gs://bucket_id/object_id/cloud_project_name-session_id
Detected entity from video analysis.
Used in: ,
string entity_id = 1
Opaque entity ID. Some IDs may be available in [Google Knowledge Graph Search API](https://developers.google.com/knowledge-graph/).
string description = 2
Textual description, e.g. `Fixed-gear bicycle`.
string language_code = 3
Language code for `description` in BCP-47 format.
Explicit content annotation (based on per-frame visual signals only). If no explicit content has been detected in a frame, no annotations are present for that frame.
Used in:
repeated frames = 1
All video frames where explicit content was detected.
Video frame level annotation results for explicit content.
Used in:
optional time_offset = 1
Time-offset, relative to the beginning of the video, corresponding to the video frame for this location.
Likelihood pornography_likelihood = 2
Likelihood of the pornography content..
Label annotation.
Used in:
optional entity = 1
Detected entity.
repeated category_entities = 2
Common categories for the detected entity. E.g. when the label is `Terrier` the category is likely `dog`. And in some cases there might be more than one categories e.g. `Terrier` could also be a `pet`.
repeated segments = 3
All video segments where a label was detected.
repeated frames = 4
All video frames where a label was detected.
Video frame level annotation results for label detection.
Used in:
optional time_offset = 1
Time-offset, relative to the beginning of the video, corresponding to the video frame for this location.
float confidence = 2
Confidence that the label is accurate. Range: [0, 1].
Video segment level annotation results for label detection.
Used in:
optional segment = 1
Video segment where a label was detected.
float confidence = 2
Confidence that the label is accurate. Range: [0, 1].
Bucketized representation of likelihood.
Used in:
LIKELIHOOD_UNSPECIFIED = 0
Unspecified likelihood.
VERY_UNLIKELY = 1
Very unlikely.
UNLIKELY = 2
Unlikely.
POSSIBLE = 3
Possible.
LIKELY = 4
Likely.
VERY_LIKELY = 5
Very likely.
Normalized bounding box. The normalized vertex coordinates are relative to the original image. Range: [0, 1].
Used in:
float left = 1
Left X coordinate.
float top = 2
Top Y coordinate.
float right = 3
Right X coordinate.
float bottom = 4
Bottom Y coordinate.
Annotations corresponding to one tracked object.
Used in:
optional entity = 1
Entity to specify the object category that this track is labeled as.
float confidence = 4
Object category's labeling confidence of this track.
repeated frames = 2
Information corresponding to all frames where this object track appears. Non-streaming batch mode: it may be one or multiple ObjectTrackingFrame messages in frames. Streaming mode: it can only be one ObjectTrackingFrame message in frames.
oneof track_info
Different representation of tracking info in non-streaming batch and streaming modes.
segment = 3
Non-streaming batch mode ONLY. Each object track corresponds to one video segment where it appears.
int64 track_id = 5
Streaming mode ONLY. In streaming mode, we do not know the end time of a tracked object before it is completed. Hence, there is no VideoSegment info returned. Instead, we provide a unique identifiable integer track_id so that the customers can correlate the results of the ongoing ObjectTrackAnnotation of the same track_id over time.
Video frame level annotations for object detection and tracking. This field stores per frame location, time offset, and confidence.
Used in:
optional normalized_bounding_box = 1
The normalized bounding box location of this object track for the frame.
optional time_offset = 2
The timestamp of the frame in microseconds.
Config for STREAMING_AUTOML_CLASSIFICATION.
Used in:
string model_name = 1
Resource name of AutoML model. Format: `projects/{project_id}/locations/{location_id}/models/{model_id}`
Config for STREAMING_AUTOML_OBJECT_TRACKING.
Used in:
string model_name = 1
Resource name of AutoML model. Format: `projects/{project_id}/locations/{location_id}/models/{model_id}`
Config for EXPLICIT_CONTENT_DETECTION in streaming mode.
No customized config support.
Used in:
(message has no fields)
Streaming video annotation feature.
Used in:
STREAMING_FEATURE_UNSPECIFIED = 0
Unspecified.
STREAMING_LABEL_DETECTION = 1
Label detection. Detect objects, such as dog or flower.
STREAMING_SHOT_CHANGE_DETECTION = 2
Shot change detection.
STREAMING_EXPLICIT_CONTENT_DETECTION = 3
Explicit content detection.
STREAMING_OBJECT_TRACKING = 4
Object detection and tracking.
STREAMING_AUTOML_CLASSIFICATION = 21
Video classification based on AutoML model.
STREAMING_AUTOML_OBJECT_TRACKING = 22
Object detection and tracking based on AutoML model.
Config for LABEL_DETECTION in streaming mode.
Used in:
bool stationary_camera = 1
Whether the video has been captured from a stationary (i.e. non-moving) camera. When set to true, might improve detection accuracy for moving objects. Default: false.
Config for STREAMING_OBJECT_TRACKING.
No customized config support.
Used in:
(message has no fields)
Config for SHOT_CHANGE_DETECTION in streaming mode.
No customized config support.
Used in:
(message has no fields)
Config for streaming storage option.
Used in:
bool enable_storage_annotation_result = 1
Enable streaming storage. Default: false.
string annotation_result_storage_directory = 3
GCS URI to store all annotation results for one client. Client should specify this field as the top-level storage directory. Annotation results of different sessions will be put into different sub-directories denoted by project_name and session_id. All sub-directories will be auto generated by program and will be made accessible to client in response proto. URIs must be specified in the following format: `gs://bucket-id/object-id` `bucket-id` should be a valid GCS bucket created by client and bucket permission shall also be configured properly. `object-id` can be arbitrary string that make sense to client. Other URI formats will return error and cause GCS write failure.
Streaming annotation results corresponding to a portion of the video that is currently being processed.
Used in:
repeated shot_annotations = 1
Shot annotation results. Each shot is represented as a video segment.
repeated label_annotations = 2
Label annotation results.
optional explicit_annotation = 3
Explicit content detection results.
repeated object_annotations = 4
Object tracking results.
Provides information to the annotator that specifies how to process the request.
Used in:
StreamingFeature feature = 1
Requested annotation feature.
oneof streaming_config
Config for requested annotation feature.
shot_change_detection_config = 2
Config for SHOT_CHANGE_DETECTION.
label_detection_config = 3
Config for LABEL_DETECTION.
explicit_content_detection_config = 4
Config for STREAMING_EXPLICIT_CONTENT_DETECTION.
object_tracking_config = 5
Config for STREAMING_OBJECT_TRACKING.
automl_classification_config = 21
Config for STREAMING_AUTOML_CLASSIFICATION.
automl_object_tracking_config = 22
Config for STREAMING_AUTOML_OBJECT_TRACKING.
optional storage_config = 30
Streaming storage option. By default: storage is disabled.
Video segment.
Used in: , ,
optional start_time_offset = 1
Time-offset, relative to the beginning of the video, corresponding to the start of the segment (inclusive).
optional end_time_offset = 2
Time-offset, relative to the beginning of the video, corresponding to the end of the segment (inclusive).