package waymo.open_dataset.keypoints

Get desktop application:
View/edit binary Protocol Buffers messages

A 2D keypoint on a specific camera.

Used in: CameraKeypoints

• optional KeypointType type = 1

• optional Keypoint2d keypoint_2d = 2

  Camera coordinates.

• optional Keypoint3d keypoint_3d = 3

  3D keypoint in camera coordinate frame.

All 2D keypoints for a camera label (object).

Used in: Label

• repeated CameraKeypoint keypoint = 1

Keypoint relative to a specific camera image.

Used in: CameraKeypoint

• optional Vector2d location_px = 1

  Camera image coordinates (in pixels, x=0, y=0 is top-left pixel).

• optional KeypointVisibility visibility = 2

  Visibility attributes determined based on camera image only.

Used in: CameraKeypoint, LaserKeypoint

• optional Vector3d location_m = 1

  A 3D coordinate in vehicle or camera frame (depending which message it is part of - LaserKeypoint or CameraKeypoint).

• optional KeypointVisibility visibility = 2

  Visibility attributes determined based on all available data (camera image and or lidar).

Used in: KeypointMetrics

• optional float value = 1

  Numeric value of the metric.

• optional string group_name = 2

  Group of keypoins used for evaluation of the metric. Refer to metrics/python/keypoint_metrics.py for the list of canonical groups of keypoints.

• repeated KeypointGroupMetric mpjpe = 1

  Mean Per Joint Position Error.

• repeated KeypointThresholdMetric oks = 2

  Precision at different thresholds for Object Keypoint Similarity (aka OKS) values. We use scales `DEFAULT_PER_TYPE_SCALES` from metrics/python/keypoint_metrics.py to compute OKS for different keypoint types.

• repeated KeypointThresholdMetric pck = 3

  Percentage of Correct Keypoints.

• repeated KeypointGroupMetric pem = 4

  Pose Estimation Metric

• repeated KeypointGroupMetric precision = 5

  Keypoint detection precision

• repeated KeypointGroupMetric recall = 6

  Keypoint detection recall

• repeated KeypointGroupMetric oks_ap = 7

  Average precision computed over all thresholds for Object Keypoint Similarity (aka OKS) values.

Used in: KeypointMetrics

• optional float value = 1

  Numeric value of the metric.

• optional string group_name = 2

  Group of keypoins used for evaluation of the metric. Refer to metrics/python/keypoint_metrics.py for the list of canonical groups of keypoints.

• optional float threshold = 3

All types of keypoints except (NOSE and HEAD_CENTER) are defined as the 3D location where corresponing bones meet - inside the body. We use person-centric coordinates in this task. For example, the person’s right shoulder will be located on the left side of the image for frontal views and on the right side of the image for back views. Similarly for the other body joints.

Used in: CameraKeypoint, LaserKeypoint

• KEYPOINT_TYPE_UNSPECIFIED = 0

• KEYPOINT_TYPE_NOSE = 1

  Tip of nose.

• KEYPOINT_TYPE_LEFT_SHOULDER = 5

• KEYPOINT_TYPE_LEFT_ELBOW = 6

• KEYPOINT_TYPE_LEFT_WRIST = 7

• KEYPOINT_TYPE_LEFT_HIP = 8

• KEYPOINT_TYPE_LEFT_KNEE = 9

• KEYPOINT_TYPE_LEFT_ANKLE = 10

• KEYPOINT_TYPE_RIGHT_SHOULDER = 13

• KEYPOINT_TYPE_RIGHT_ELBOW = 14

• KEYPOINT_TYPE_RIGHT_WRIST = 15

• KEYPOINT_TYPE_RIGHT_HIP = 16

• KEYPOINT_TYPE_RIGHT_KNEE = 17

• KEYPOINT_TYPE_RIGHT_ANKLE = 18

• KEYPOINT_TYPE_FOREHEAD = 19

  Center of the forehead area.

• KEYPOINT_TYPE_HEAD_CENTER = 20

  A point in the center of head - a point in the middle between two ears. The nose and head center together create an imaginary line in the direction that the person is looking (i.e. head orientation).

Attributes related to the keypoint's visibility.

Used in: Keypoint2d, Keypoint3d

• optional bool is_occluded = 1

  Is true, if the keypoint is occluded by any object, a body part or its location can be determined only with large uncertainty. Is false if the keypoint is clearly visible.

A 3D keypoint.

Used in: LaserKeypoints

• optional KeypointType type = 1

• optional Keypoint3d keypoint_3d = 2

  3D keypoint in vehicle coordinate frame.

All 3D keypoints for a laser label (object).

Used in: Label, PoseEstimation

• repeated LaserKeypoint keypoint = 1

Used in: PoseEstimationSubmission

• optional PoseEstimation.Key key = 1

• optional LaserKeypoints laser_keypoints = 2

• optional Box3d box = 3

Fields to identify the predicted keypoints.

Used in: PoseEstimation

• optional string context_name = 1

  Same as Frame.context.name.

• optional int64 frame_timestamp_micros = 2

  Same as Frame.timestamp_micros.

• optional string account_name = 1

  This must be set as the full email used to register at waymo.com/open.

• optional string unique_method_name = 2

  This name needs to be short, descriptive and unique. Only the latest result of the method from a user will show up on the leaderboard.

• repeated string authors = 3

  Author information.

• optional string affiliation = 4

• optional string description = 5

  A brief description of the method.

• optional string method_link = 6

  Link to paper or other link that describes the method.

• repeated PoseEstimation pose_estimations = 7

  Estimated pose for all objects in the validation or test subset of the dataset.