package sax.server.vision
Get desktop application:
View/edit binary Protocol Buffers messages
service VisionService
vision.proto:201rpc Classify (, )
vision.proto:203Returns the score (e.g., log pplx) given the text.
message
vision.proto:23string model_key = 1
bytes image_bytes = 2
optional extra_inputs = 3
message
vision.proto:37repeated texts = 1
rpc TextToImage (, )
vision.proto:206Returns image generation results given the text.
message
vision.proto:41string model_key = 1
string text = 2
optional extra_inputs = 3
message
vision.proto:62repeated images = 2
rpc TextAndImageToImage (, )
vision.proto:209Returns image generation results given the text and image.
message
vision.proto:47string model_key = 1
string text = 2
bytes image_bytes = 4
optional extra_inputs = 3
message
vision.proto:66repeated images = 2
rpc Embed (, )
vision.proto:213Returns an image embedding given an image.
message
vision.proto:70string model_key = 1
bytes image_bytes = 2
optional extra_inputs = 3
message
vision.proto:76repeated double embedding = 1
rpc Detect (, )
vision.proto:217Returns bounding box, label, and the score of objects detected given an image.
message
vision.proto:122string model_key = 1
bytes image_bytes = 2
repeated string text = 4
For open-set detection models, one can pass specified sets here. Elements in `text` describe concepts in the image that should be detected; it is up to the detection model to interpret these texts. Some detection models may interpret the text to be object names, and the corresponding response.bounding_boxes.text must be one of the given text elements.
repeated boxes_of_interest = 5
Optionally accept box inputs to predict region class labels. The boxes are N regions of interest in the image. The .text and .score fields should be unset.
optional extra_inputs = 3
message
vision.proto:142repeated bounding_boxes = 1
A list of bounding boxes. The bounding boxes have no explicit order.
rpc ImageToText (, )
vision.proto:220Returns text generation results given image_bytes.
message
vision.proto:147string model_key = 1
bytes image_bytes = 2
string text = 3
Optional prefix text.
optional extra_inputs = 4
message
vision.proto:155repeated texts = 1
rpc ImageToImage (, )
vision.proto:223Returns image generation results given image_bytes.
message
vision.proto:159string model_key = 1
bytes image_bytes = 2
optional extra_inputs = 4
message
vision.proto:165repeated images = 2
rpc VideoToText (, )
vision.proto:226Returns text generation results given video.
message
vision.proto:169string model_key = 1
repeated bytes image_frames = 2
string text = 3
Optional prefix text.
optional extra_inputs = 4
message
vision.proto:177repeated texts = 1
rpc VideoToToken (, )
vision.proto:229Returns video tokens results given video (tokenization).
message
vision.proto:181string model_key = 1
repeated bytes image_frames = 2
Video composed of multiple image frames.
optional extra_inputs = 3
message
vision.proto:187repeated double tokens = 1
quantized or soft tokens.
rpc TokenToVideo (, )
vision.proto:232Returns video bytes results given video tokens (de-tokenization).
message
vision.proto:191string model_key = 1
repeated double tokens = 2
quantized or soft tokens.
optional extra_inputs = 3
message
vision.proto:197repeated bytes image_frames = 2
Video composed of multiple image frames.
message
vision.proto:91Used in: ,
double cx = 1
Coordinates are in pixel space. Upper left corner of the image represents (0.0, 0.0) Bottom right corner of the image represents (image_width, image_height)
double cy = 2
double w = 3
double h = 4
string text = 5
A label for the bounding box object.
double score = 6
A positive number which represents the ranking (the higher the better) of the bounding boxes. The semantic meaning of the score is left for each model to define.
optional mask = 7
When a mask is present, it only contains values inside the bounding box defined by cx,cy,w,h. I.e.: mask[i, j] corresponding the image pixel[cx - w/2 + i, cy - h / 2 + j], where cx,cy,w,h are given in the BoundingBox. Outside the bounding box the mask is always zero. This enables sending smaller mask sizes, since the mask size is only the bounding box size. E.g. MaskRCNN returns masks that are always 28x28 pixels, and they resized by third_party/cloud_tpu/models/detection/utils/mask_utils.py.
message
vision.proto:29Used in: , ,
string text = 1
The label of the classified object.
double score = 2
The score of the classified object.
message
vision.proto:80Used in:
bytes mask_values = 1
mask represents a C-order 2-D uint8 array. The array's dimension are given by [mask_height, mask_width]. mask[i, j] / 255 represents the probability of the pixel being in the segment (the range [0,1] was scaled to [0, 255] for compression).
int32 mask_height = 2
int32 mask_width = 3
message
vision.proto:54Used in: , ,
bytes image = 1
The generated image in byte array format. TODO(jianlijianli): decide on a image encoding format; Currently PNG.
double score = 2
The score for the generated image.