package waymo.open_dataset.keypoints

Get desktop application:
View/edit binary Protocol Buffers messages

A 2D keypoint on a specific camera.

Used in: CameraKeypoints

• optional KeypointType type = 1

• optional Keypoint2d keypoint_2d = 2

  Camera coordinates.

• optional Keypoint3d keypoint_3d = 3

  3D keypoint in camera coordinate frame.

All 2D keypoints for a camera label (object).

Used in: Label

• repeated CameraKeypoint keypoint = 1

Keypoint relative to a specific camera image.

Used in: CameraKeypoint

• optional Vector2d location_px = 1

  Camera image coordinates (in pixels, x=0, y=0 is top-left pixel).

• optional KeypointVisibility visibility = 2

  Visibility attributes determined based on camera image only.

Used in: CameraKeypoint, LaserKeypoint

• optional Vector3d location_m = 1

  A 3D coordinate in vehicle or camera frame (depending which message it is part of - LaserKeypoint or CameraKeypoint).

• optional KeypointVisibility visibility = 2

  Visibility attributes determined based on all available data (camera image and or lidar).

All types of keypoints except (NOSE and HEAD_CENTER) are defined as the 3D location where corresponing bones meet - inside the body. We use person-centric coordinates in this task. For example, the person’s right shoulder will be located on the left side of the image for frontal views and on the right side of the image for back views. Similarly for the other body joints.

Used in: CameraKeypoint, LaserKeypoint

• KEYPOINT_TYPE_UNSPECIFIED = 0

• KEYPOINT_TYPE_NOSE = 1

  Tip of nose.

• KEYPOINT_TYPE_LEFT_SHOULDER = 5

• KEYPOINT_TYPE_LEFT_ELBOW = 6

• KEYPOINT_TYPE_LEFT_WRIST = 7

• KEYPOINT_TYPE_LEFT_HIP = 8

• KEYPOINT_TYPE_LEFT_KNEE = 9

• KEYPOINT_TYPE_LEFT_ANKLE = 10

• KEYPOINT_TYPE_RIGHT_SHOULDER = 13

• KEYPOINT_TYPE_RIGHT_ELBOW = 14

• KEYPOINT_TYPE_RIGHT_WRIST = 15

• KEYPOINT_TYPE_RIGHT_HIP = 16

• KEYPOINT_TYPE_RIGHT_KNEE = 17

• KEYPOINT_TYPE_RIGHT_ANKLE = 18

• KEYPOINT_TYPE_FOREHEAD = 19

  Center of the forehead area.

• KEYPOINT_TYPE_HEAD_CENTER = 20

  A point in the center of head - a point in the middle between two ears. The nose and head center together create an imaginary line in the direction that the person is looking (i.e. head orientation).

Attributes related to the keypoint's visibility.

Used in: Keypoint2d, Keypoint3d

• optional bool is_occluded = 1

  Is true, if the keypoint is occluded by any object, a body part or its location can be determined only with large uncertainty. Is false if the keypoint is clearly visible.

A 3D keypoint.

Used in: LaserKeypoints

• optional KeypointType type = 1

• optional Keypoint3d keypoint_3d = 2

  3D keypoint in vehicle coordinate frame.

All 3D keypoints for a laser label (object).

Used in: Label

• repeated LaserKeypoint keypoint = 1