package yb.tablet

Get desktop application:
View/edit binary Protocol Buffers messages

Used as request type in: tserver.TabletServerAdminService.AlterSchema, tserver.TabletServerAdminService.BackfillDone

Used as field type in: cdc.CDCRecordPB, consensus.ReplicateMsg, master.UpdateConsumerOnProducerMetadataRequestPB, tserver.ExportedTabletMetadataChanges

• optional bytes dest_uuid = 5

  UUID of server this request is addressed to.

• required bytes tablet_id = 1

• optional SchemaPB schema = 2

  TODO: Replace with the table descriptor (Schema, Column IDs, Column Attributes)

• repeated IndexInfoPB indexes = 7

• optional uint32 schema_version = 3

• optional string new_table_name = 4

• optional fixed64 propagated_hybrid_time = 6

• optional TableInfoPB add_table = 8

• repeated TableInfoPB add_multiple_tables = 17

  For batching multiple adds.

• optional uint32 wal_retention_secs = 9

• optional bytes remove_table_id = 10

• optional bool mark_backfill_done = 11

  Note that we don't use the value of this and just check has_mark_backfill_done.

• optional bytes alter_table_id = 12

  Used for alter schema on a non-primary table (e.g. colocated user table).

• optional bytes backfill_done_table_id = 13

  Used to update schema of a colocated user table upon backfill completion.

• optional bytes transaction_id = 14

  Transactions will be aborted except the one with this id which triggered the AlterTable.

  16 byte uuid

• optional bool should_abort_active_txns = 16

  Used for AlterTable to determine if active transactions on the tablet needs to be aborted.

• optional bytes retention_requester_id = 18

  For CDCSDK Stream creation with consistent snapshot

• optional bool cdc_sdk_require_history_cutoff = 19

• optional bool insert_packed_schema = 20

  Will insert the new schema into historical packing schema with schema version [schema_version - 1], and then reinsert the current packing schema with [schema_version]. Schema version must be bumped by 2 before this call is made.

Used as request type in: tserver.TabletServerAdminService.CloneTablet

Used as field type in: consensus.ReplicateMsg

• optional bytes dest_uuid = 1

  UUID of server this request is addressed to.

• optional fixed64 propagated_hybrid_time = 2

• optional string tablet_id = 3

  Not called source_tablet_id because some templated functions use req->tablet_id().

• optional string target_tablet_id = 4

• optional bytes source_snapshot_id = 5

• optional bytes target_snapshot_id = 6

• optional string target_table_id = 7

• optional string target_namespace_name = 8

• optional uint32 clone_request_seq_no = 9

• optional string target_pg_table_id = 10

• optional IndexInfoPB target_index_info = 11

• optional SchemaPB target_schema = 12

• optional PartitionSchemaPB target_partition_schema = 13

• optional bool target_skip_table_tombstone_check = 14

• optional bool clone_from_active_rocksdb = 15

  If set, checkout the active rocksdb folder to the target snapshot. This is set for tablets that were hidden before the snapshot that covers the clone time was taken (we do not take snapshots after hiding a tablet).

• repeated TableInfoPB colocated_tables = 16

  The colocated tables' infos used to build the superblock of the parent tablet. The colocated tables' info cannot be copied from the source tablet becuase: 1- The colocated tables don't have the same table ids in both source and target databases. 2- The colocated tables on the target can have different schemas than the tables on the source tablet. i.e when cloning to a time before DDl operations happened on the source tablet.

Used in: KvStoreInfoPB, RaftGroupReplicaSuperBlockPB, SnapshotFilePB

• optional string name = 1

  Required. Sometimes includes path.

• optional uint64 size_bytes = 2

  Required.

• optional uint64 inode = 3

  Used to avoid copying same files over network, so we could hardlink them.

The enum of full compaction states. Full compaction states are sent in TabletReports and recorded in-memory on the master.

Used in: master.FullCompactionStatusPB, master.GetCompactionStatusResponsePB, master.TabletReplicaFullCompactionStatus

• IDLE = 0

  Indicates that no full compaction is undergoing nor queued.

• COMPACTING = 1

  Indicates that a full compaction is currently queued or ongoing.

• FULL_COMPACTION_STATE_UNKNOWN = 2

  Indicates that we are are waiting for the next heartbeat.

KV-store meta data. Single KV-store is backed by one or two RocksDB instances, depending on whether distributed transactions are enabled for the table. KV-store for sys catalog could contain multiple tables.

Used in: RaftGroupReplicaSuperBlockPB

• required bytes kv_store_id = 1

  KV-store id. For more convenient logs browsing and investigation, by default set to raft_group_id for the KV-store which was created first in the Raft group.

• optional string rocksdb_dir = 2

  The RocksDB directory for this KV-store.

• repeated FilePB rocksdb_files = 3

  List of RocksDB SST files.

• repeated SnapshotFilePB snapshot_files = 4

  List of Snapshot files.

• repeated TableInfoPB tables = 5

  List of tables sharing this KV-store. Primary table always goes first.

• optional bytes lower_bound_key = 6

  Optional inclusive lower bound and exclusive upper bound for keys served by this KV-store. See docdb::KeyBounds.

• optional bytes upper_bound_key = 7

• optional bool parent_data_compacted = 8

  Whether KV-store regular DB parent files have been compacted since creation. We don't split post-split tablets which have uncompacted parent data as of 2020-06-23, since detecting effective middle key and tablet size for such tablets is not yet implemented.

• repeated bytes snapshot_schedules = 9

• optional uint64 last_full_compaction_time = 10

  Uint64 representation of a HybridTime indicating the last time the tablet was fully compacted. Defaults to 0 (i.e. HybridTime::kMin).

• optional uint64 post_split_compaction_file_number_upper_bound = 11

  Indicates the upper bound for a set of files, which should be included into post-split compaction. In this case post-split compaction picks a subset of files whose file number is limited by post_split_compaction_file_number_upper_bound exclusively and whose total size is less or equal than FLAGS_post_split_compaction_input_size_threshold_bytes. The parameter is not taken into account if FLAGS_post_split_compaction_input_size_threshold_bytes is set to 0. The value of post_split_compaction_file_number_upper_bound is set to 0 when post-split compaction is done for all the files in the selected subset.

Used to present the maintenance manager's internal state.

• optional MaintenanceManagerStatusPB.MaintenanceOpPB best_op = 1

  The next operation that would run.

• repeated MaintenanceManagerStatusPB.MaintenanceOpPB registered_operations = 2

  List of all the operations.

• repeated MaintenanceManagerStatusPB.CompletedOpPB completed_operations = 3

  This list is not guaranteed to be in any particular order. Can contain the same operation multiple times.

Used in: MaintenanceManagerStatusPB

• required string name = 1

• required int32 duration_millis = 2

• required int32 secs_since_start = 3

  Number of seconds since this operation started.

Used in: MaintenanceManagerStatusPB

• required string name = 1

• required uint32 running = 2

  Number of times this operation is currently running.

• required bool runnable = 3

• required uint64 ram_anchored_bytes = 4

• required int64 logs_retained_bytes = 5

• required double perf_improvement = 6

The super-block keeps track of the Raft group.

Used in: tserver.BeginRemoteBootstrapSessionResponsePB, tserver.BeginRemoteSnapshotTransferSessionResponsePB

• required bytes primary_table_id = 1

  Table ID of the primary table in this Raft group. For now we can only have more than one tables in master Raft group and sys catalog table is the primary table in this case.

• required bytes raft_group_id = 2

• optional PartitionPB partition = 13

  The partition of the table.

• optional string OBSOLETE_table_name = 7

  The latest schema TODO: maybe this should be TableSchemaPB? Need to actually put those attributes into use throughout the code. Using the simpler one for now.

• optional TableType OBSOLETE_table_type = 15

• optional SchemaPB OBSOLETE_schema = 8

• optional uint32 OBSOLETE_schema_version = 9

• optional PartitionSchemaPB OBSOLETE_partition_schema = 14

  The partition schema of the table.

• optional TabletDataState tablet_data_state = 10

  The current state of the Raft group's data.

• optional string wal_dir = 18

  The write-ahead log directory for this Raft group.

• optional OpIdPB tombstone_last_logged_opid = 12

  For tablets that have been tombstoned, stores the last OpId stored in the WAL before tombstoning. Only relevant for TOMBSTONED tablets.

• optional KvStoreInfoPB kv_store = 24

  KV-store for this Raft group.

• optional string OBSOLETE_rocksdb_dir = 16

  The RocksDB directory for this tablet.

• repeated FilePB OBSOLETE_rocksdb_files = 17

  List of RocksDB SST files.

• repeated SnapshotFilePB OBSOLETE_snapshot_files = 20

  List of Snapshot files.

• repeated DeletedColumnPB OBSOLETE_deleted_cols = 19

  Deleted column IDs with timestamps so that memory can be cleaned up.

• repeated IndexInfoPB OBSOLETE_indexes = 21

  Secondary indexes of the table.

• optional IndexInfoPB OBSOLETE_index_info = 22

  For index table: information about this index.

• repeated TableInfoPB OBSOLETE_tables = 23

  Tables co-located in this tablet.

• optional bool colocated = 25

  True if the raft group is for a colocated tablet.

• optional int64 cdc_min_replicated_index = 26

  Minimum replicated op id index for cdc streams. This value is used to retain log files that contain any entries with indices larger than this one. By default max int64 to avoid retaining any log files unnecessarily.

• optional bool is_under_xcluster_replication = 27

  Is this tablet currently a consumer tablet for xcluster replication.

• optional bool hidden = 28

  Reject client queries (read and write) to this tablet like it does not present.

• optional fixed64 restoration_hybrid_time = 32

  Picked at master while enabling writes after restoration.

• optional OpIdPB split_op_id = 29

  SPLIT_OP ID designated for this tablet (so child tablets will have this unset until they've been split themselves).

• repeated bytes split_child_tablet_ids = 30

• repeated bytes active_restorations = 31

• optional OpIdPB cdc_sdk_min_checkpoint_op_id = 33

  Minimum checkpoint op id for cdcsdk streams. This value is used to retain intents with op id larger than this one.

• optional fixed64 cdc_sdk_safe_time = 34

• repeated StatefulServiceKind hosted_stateful_services = 35

  List of stateful services hosted by this tablet.

• optional bool is_under_cdc_sdk_replication = 36

  Is this tablet currently a being replicated with cdc.

• optional OpIdPB last_flushed_change_metadata_op_id = 37

  OpId of the last change metadata operation performed on this tablet that has been flushed to disk. This value is used to determine if a particular raft op should be replayed during local tablet bootstrap of the tablet.

• optional uint32 last_attempted_clone_seq_no = 38

  The highest seq_no clone op that this tablet has applied. The clone was not necessarily successful. The master is responsible for not sending multiple clone requests with different sequence_numbers until the previous one is complete or aborted on all tablets.

The enum of Raft group states. Raft group states are sent in TabletReports and kept in TabletPeer.

Used in: master.ReportedTabletPB, master.TabletLocationsPB.ReplicaPB, TabletStatusPB, tserver.ListTabletsForTabletServerResponsePB.Entry

• UNKNOWN = 999

• NOT_STARTED = 5

  Raft group has not yet started.

• BOOTSTRAPPING = 0

  Indicates the Raft group is bootstrapping, i.e. that the Raft group is not available for RPC.

• RUNNING = 1

  Once the configuration phase is over Peers are in RUNNING state. In this state Peers are available for client RPCs.

• FAILED = 2

  The Raft group failed to for some reason. TabletPeer::error() will return the reason for the failure.

• QUIESCING = 3

  The Raft group is shutting down, and will not accept further requests.

• SHUTDOWN = 4

  The Raft group has been stopped.

Used in: KvStoreInfoPB, RaftGroupReplicaSuperBlockPB

• optional FilePB file = 1

• optional bytes snapshot_id = 2

Used as request type in: tserver.TabletServerAdminService.SplitTablet

Used as field type in: cdc.CDCRecordPB, consensus.ReplicateMsg

• optional bytes dest_uuid = 1

  UUID of server this request is addressed to.

• optional fixed64 propagated_hybrid_time = 2

• required bytes tablet_id = 3

• required bytes new_tablet1_id = 4

• required bytes new_tablet2_id = 5

• required bytes split_partition_key = 6

• required bytes split_encoded_key = 7

• optional string split_parent_leader_uuid = 8

  Parent tablet leader UUID from SPLIT_OP term.

Table information.

Used in: ChangeMetadataRequestPB, CloneTabletRequestPB, KvStoreInfoPB, RaftGroupReplicaSuperBlockPB, tserver.AddTableToTabletRequestPB

• optional bytes table_id = 1

  Table id, name, and type.

• optional string namespace_name = 10

• optional string namespace_id = 13

• optional string table_name = 2

• optional TableType table_type = 3

• optional OpIdPB op_id = 17

  Id of operation that added this table to the tablet.

• optional fixed64 hybrid_time = 16

  Time when this table was added to the tablet.

• optional string pg_table_id = 14

  In case the table was rewritten, explicitly store the TableId containing the PG table OID (as the table's TableId no longer matches).

• optional SchemaPB schema = 4

  Table schema and version.

• optional uint32 schema_version = 5

• optional PartitionSchemaPB partition_schema = 6

  Table's partitioning schema.

• repeated IndexInfoPB indexes = 7

  Secondary indexes of the table.

• optional IndexInfoPB index_info = 8

  For index table: information about this index.

• repeated DeletedColumnPB deleted_cols = 9

  Deleted column IDs with timestamps so that memory can be cleaned up.

• repeated dockv.SchemaPackingPB old_schema_packings = 11

• optional uint32 wal_retention_secs = 12

  CDC WAL retention time in secs of the table.

• optional bool skip_table_tombstone_check = 15

  Whether we can skip table tombstone check for this table (only applies to colocated tables).

State flags indicating whether the tablet is in the middle of being copied and is therefore not possible to bring up, whether it has been deleted, or whether the data is in a usable state.

Used in: master.ReportedTabletPB, RaftGroupReplicaSuperBlockPB, TabletStatusPB, tserver.DeleteTabletRequestPB

• TABLET_DATA_UNKNOWN = 999

• TABLET_DATA_COPYING = 0

  The tablet is set to TABLET_DATA_COPYING state when in the middle of remote bootstrap while copying data files from a remote peer. If a tablet server crashes with a tablet in this state, the tablet must be deleted and the remote bootstrap process must be restarted for that tablet.

• TABLET_DATA_READY = 1

  Fresh empty tablets and successfully copied tablets are set to the TABLET_DATA_READY state.

• TABLET_DATA_DELETED = 2

  This tablet is in the process of being deleted. The tablet server should "roll forward" the deletion during boot, rather than trying to load the tablet.

• TABLET_DATA_TOMBSTONED = 3

  The tablet has been deleted, and now just consists of a "tombstone".

• TABLET_DATA_SPLIT_COMPLETED = 4

  This tablet split has been completed. In this state tablet could only be used as a remote bootstrap source for some time (follower_unavailable_considered_failed_sec) and then will be shutdown. TODO(tsplit): After https://github.com/yugabyte/yugabyte-db/issues/1461: - add integration test for remote bootstrap from source split tablet before/after follower_unavailable_considered_failed_sec.

• TABLET_DATA_INIT_STARTED = 5

  This tablet has been initialized as a subtablet of another tablet undergoing a split. Once this tablet is ready to assume responsibility for it's portion of the split keyspace, it will be moved to the TABLET_DATA_READY state.

Used in: tserver.GetTabletStatusResponsePB, tserver.ListTabletsResponsePB.StatusAndSchemaPB, tserver.PgTabletsMetadataResponsePB

• required string tablet_id = 1

• optional string namespace_name = 15

• required string table_name = 2

• required string table_id = 10

• optional RaftGroupStatePB state = 3

• optional TabletDataState tablet_data_state = 8

• required string last_status = 4

• optional bytes start_key = 5

  DEPRECATED.

• optional bytes end_key = 6

  DEPRECATED.

• optional int64 estimated_on_disk_size = 7

• optional int64 consensus_metadata_disk_size = 11

• optional int64 wal_files_disk_size = 12

• optional int64 sst_files_disk_size = 13

• optional int64 uncompressed_sst_files_disk_size = 14

• optional PartitionPB partition = 9

• optional TableType table_type = 16

• optional bool is_hidden = 17

  Is the tablet hidden.

• optional bool parent_data_compacted = 18

  See KvStoreInfoPB.

• repeated bytes colocated_table_ids = 19

  A list of all tables that are colocated on this tablet.

• optional string pgschema_name = 20

• repeated bytes vector_index_finished_backfills = 21

Used in: cdc.CDCRecordPB, consensus.ReplicateMsg, tserver.UpdateTransactionRequestPB

• optional bytes transaction_id = 1

• optional TransactionStatus status = 2

• repeated bytes tablets = 3

  tablets has different meaning, depending on status: PENDING - list of involved tablets, if FLAGS_disable_heartbeat_send_involved_tablets = false. COMMITTED - list of involved tablets APPLYING - single entry, status tablet of this transaction PROMOTING - single entry, new status tablet of this transaction APPLIED - single entry, tablet that applied this transaction Not used is other cases.

• repeated int64 tablet_batches = 5

  Relevant only in SEALED state. Number of batches that should be replicated at appropriate tablet to interpret transaction as committed.

• optional fixed64 commit_hybrid_time = 4

  Relevant only in APPLYING state.

• optional bool sealed = 6

  Relevant only in CLEANUP and APPLYING status, true when we perform request for a transaction that was previously sealed.

• optional SubtxnSetPB aborted = 7

  Set of subtransaction IDs which are aborted.

• optional fixed64 start_time = 10

  The time at which this transaction started at the client.

• optional string host_node_uuid = 11

  Stores the uuid of the node hosting the transaction at the query layer.

• optional AppStatusPB deadlock_reason = 12

  Status containing deadlock info if the transaction was aborted due to deadlock.

• optional uint64 pg_session_req_version = 14

  Fields 14, 15 are used for session level transactions alone (when session advisory locks feature is enabled). The current pg session request version is propagated on transaction heartbeats. If the coordinator has a higher version than the one received in the request, it errors with the status holding the active pg session request version, post which YBTransaction updates its state.

• optional TransactionMetadataPB background_transaction_meta = 15

  For detection deadlocks spanning session advisory locks and row-level locks, the status tablet needs to be made aware of the active regular docdb transaction of the ysql session, if any.

Used in: consensus.ReplicateMsg, tserver.TruncateRequestPB

(message has no fields)

TODO(proto3) reserved 2, 3, 5, 6, 8, 9, 10, 12, 13, 18;

Used in: consensus.ReplicateMsg

• optional bytes unused_tablet_id = 1

• optional docdb.KeyValueWriteBatchPB write_batch = 7

• optional fixed64 client_id1 = 14

  Use 2 int64 to represent UUID.

• optional fixed64 client_id2 = 15

• optional int64 request_id = 16

• optional int64 min_running_request_id = 17

• optional fixed64 external_hybrid_time = 19

• optional uint64 batch_idx = 20

• optional uint64 start_time_micros = 21