package yb

Get desktop application:
View/edit binary Protocol Buffers messages

Error status returned by any RPC method. Every RPC method which could generate an application-level error should have this (or a more complex error result) as an optional field in its response. This maps to yb::Status in C++.

Used in: PgsqlResponsePB, ServerEntryPB, cdc.CDCErrorPB, consensus.ConsensusErrorPB, consensus.PerOpErrorPB, master.DeleteUniverseReplicationResponsePB, master.GetTabletLocationsResponsePB.Error, master.IndexStatusPB, master.IsAlterXClusterReplicationDoneResponsePB, master.IsCreateXClusterReplicationDoneResponsePB, master.IsSetupNamespaceReplicationWithBootstrapDoneResponsePB, master.IsSetupUniverseReplicationDoneResponsePB, master.ListSnapshotRestorationsResponsePB, master.MasterErrorPB, master.ReportedTabletPB, master.SysXClusterOutboundReplicationGroupEntryPB.NamespaceInfoPB, master.SysXClusterOutboundReplicationGroupEntryPB.TargetUniverseInfoPB, master.YsqlMajorCatalogUpgradeInfoPB, stateful_service.GetEchoCountResponsePB, stateful_service.GetEchoResponsePB, stateful_service.IncreaseMutationCountersResponsePB, stateful_service.PgCronGetLastMinuteResponsePB, stateful_service.PgCronSetLastMinuteResponsePB, tablet.TransactionStatePB, tserver.AbortTransactionRequestPB, tserver.EndRemoteBootstrapSessionRequestPB, tserver.GetTransactionStatusResponsePB, tserver.PgAcquireAdvisoryLockResponsePB, tserver.PgAcquireObjectLockResponsePB, tserver.PgActiveSessionHistoryResponsePB, tserver.PgAlterDatabaseResponsePB, tserver.PgAlterTableResponsePB, tserver.PgBackfillIndexResponsePB, tserver.PgCancelTransactionResponsePB, tserver.PgCheckIfPitrActiveResponsePB, tserver.PgClearExportedTxnSnapshotsResponsePB, tserver.PgCreateDatabaseResponsePB, tserver.PgCreateReplicationSlotResponsePB, tserver.PgCreateSequencesDataTableResponsePB, tserver.PgCreateTableResponsePB, tserver.PgCreateTablegroupResponsePB, tserver.PgCronGetLastMinuteResponsePB, tserver.PgCronSetLastMinuteResponsePB, tserver.PgDeleteDBSequencesResponsePB, tserver.PgDeleteSequenceTupleResponsePB, tserver.PgDropDatabaseResponsePB, tserver.PgDropReplicationSlotResponsePB, tserver.PgDropTableResponsePB, tserver.PgDropTablegroupResponsePB, tserver.PgExportTxnSnapshotResponsePB, tserver.PgFetchDataResponsePB, tserver.PgFetchSequenceTupleResponsePB, tserver.PgFinishTransactionResponsePB, tserver.PgGetActiveTransactionListResponsePB, tserver.PgGetCatalogMasterVersionResponsePB, tserver.PgGetDatabaseInfoResponsePB, tserver.PgGetIndexBackfillProgressResponsePB, tserver.PgGetLockStatusResponsePB, tserver.PgGetNewObjectIdResponsePB, tserver.PgGetReplicationSlotResponsePB, tserver.PgGetTableDiskSizeResponsePB, tserver.PgGetTableKeyRangesResponsePB, tserver.PgGetTablePartitionListResponsePB, tserver.PgGetTserverCatalogMessageListsResponsePB, tserver.PgGetTserverCatalogVersionInfoResponsePB, tserver.PgGetXClusterRoleResponsePB, tserver.PgHeartbeatResponsePB, tserver.PgImportTxnSnapshotResponsePB, tserver.PgInsertSequenceTupleResponsePB, tserver.PgIsInitDbDoneResponsePB, tserver.PgIsObjectPartOfXReplResponsePB, tserver.PgListClonesResponsePB, tserver.PgListLiveTabletServersResponsePB, tserver.PgListReplicationSlotsResponsePB, tserver.PgOpenTableResponsePB, tserver.PgPerformResponsePB, tserver.PgPollVectorIndexReadyResponsePB, tserver.PgReadSequenceTupleResponsePB, tserver.PgReleaseAdvisoryLockResponsePB, tserver.PgReserveOidsResponsePB, tserver.PgRollbackToSubTransactionResponsePB, tserver.PgServersMetricsResponsePB, tserver.PgSetTserverCatalogMessageListResponsePB, tserver.PgTabletServerCountResponsePB, tserver.PgTabletsMetadataResponsePB, tserver.PgTruncateTableResponsePB, tserver.PgUpdateSequenceTupleResponsePB, tserver.PgValidatePlacementResponsePB, tserver.PgWaitForBackendsCatalogVersionResponsePB, tserver.PgYCQLStatementStatsResponsePB, tserver.RemoteBootstrapErrorPB, tserver.TabletServerErrorPB, tserver.WriteResponsePB.PerRowErrorPB

• required AppStatusPB.ErrorCode code = 1

• optional string message = 2

• oneof error_codes

  For backward compatibility error_codes would also be filled. But errors would have priority if both are set.

  • int32 posix_code = 4

  • int64 ql_error_code = 5

• optional string source_file = 6

• optional int32 source_line = 7

• optional bytes errors = 9

  Stores encoded error codes of Status. See Status for details.

Used in: AppStatusPB

• UNKNOWN_ERROR = 999

• OK = 0

• NOT_FOUND = 1

• CORRUPTION = 2

• NOT_SUPPORTED = 3

• INVALID_ARGUMENT = 4

• IO_ERROR = 5

• ALREADY_PRESENT = 6

• RUNTIME_ERROR = 7

• NETWORK_ERROR = 8

• ILLEGAL_STATE = 9

• NOT_AUTHORIZED = 10

• ABORTED = 11

• REMOTE_ERROR = 12

• SERVICE_UNAVAILABLE = 13

• TIMED_OUT = 14

• UNINITIALIZED = 15

• CONFIGURATION_ERROR = 16

• INCOMPLETE = 17

• END_OF_FILE = 18

• INVALID_COMMAND = 19

• QL_ERROR = 20

• INTERNAL_ERROR = 21

• EXPIRED = 22

• LEADER_NOT_READY_TO_SERVE = 23

• LEADER_HAS_NO_LEASE = 24

• TRY_AGAIN_CODE = 25

• BUSY = 26

• SHUTDOWN_IN_PROGRESS = 27

• MERGE_IN_PROGRESS = 28

• COMBINED_ERROR = 29

• SNAPSHOT_TOO_OLD = 30

• DEPRECATED_HOST_UNREACHABLE = 31

• CACHE_MISS_ERROR = 32

• TABLET_SPLIT = 33

• REPLICATION_SLOT_LIMIT_REACHED = 34

Some optional/additional fields that is collected in addition to the wait statement Depending on the wait-state we may choose to persist at most one of these into the yb_active_universe_history.wait_event_aux field.

Used in: WaitStateInfoPB

• optional string tablet_id = 1

• optional string table_id = 2

• optional string method = 3

Used in: QLReadRequestPB, QLWriteRequestPB, WaitStateInfoPB, master.AcquireObjectLocksGlobalRequestPB, master.ReleaseObjectLocksGlobalRequestPB, tserver.AbortTransactionRequestPB, tserver.AcquireObjectLockRequestPB, tserver.BeginRemoteBootstrapSessionRequestPB, tserver.FetchDataRequestPB, tserver.PgAcquireAdvisoryLockRequestPB, tserver.PgAcquireObjectLockRequestPB, tserver.PgAlterDatabaseRequestPB, tserver.PgAlterTableRequestPB, tserver.PgBackfillIndexRequestPB, tserver.PgCancelTransactionRequestPB, tserver.PgCheckIfPitrActiveRequestPB, tserver.PgClearExportedTxnSnapshotsRequestPB, tserver.PgCreateDatabaseRequestPB, tserver.PgCreateReplicationSlotRequestPB, tserver.PgCreateTableRequestPB, tserver.PgCreateTablegroupRequestPB, tserver.PgCronGetLastMinuteRequestPB, tserver.PgCronSetLastMinuteRequestPB, tserver.PgDeleteDBSequencesRequestPB, tserver.PgDeleteSequenceTupleRequestPB, tserver.PgDropDatabaseRequestPB, tserver.PgDropReplicationSlotRequestPB, tserver.PgDropTableRequestPB, tserver.PgDropTablegroupRequestPB, tserver.PgExportTxnSnapshotRequestPB, tserver.PgFetchSequenceTupleRequestPB, tserver.PgFinishTransactionRequestPB, tserver.PgGetActiveTransactionListRequestPB, tserver.PgGetIndexBackfillProgressRequestPB, tserver.PgGetLockStatusRequestPB, tserver.PgGetNewObjectIdRequestPB, tserver.PgGetReplicationSlotRequestPB, tserver.PgGetTableDiskSizeRequestPB, tserver.PgGetTableKeyRangesRequestPB, tserver.PgGetTablePartitionListRequestPB, tserver.PgGetTserverCatalogMessageListsRequestPB, tserver.PgGetTserverCatalogVersionInfoRequestPB, tserver.PgGetXClusterRoleRequestPB, tserver.PgImportTxnSnapshotRequestPB, tserver.PgInsertSequenceTupleRequestPB, tserver.PgIsObjectPartOfXReplRequestPB, tserver.PgListClonesRequestPB, tserver.PgListLiveTabletServersRequestPB, tserver.PgListReplicationSlotsRequestPB, tserver.PgOpenTableRequestPB, tserver.PgPerformOptionsPB, tserver.PgPollVectorIndexReadyRequestPB, tserver.PgReadSequenceTupleRequestPB, tserver.PgReleaseAdvisoryLockRequestPB, tserver.PgReserveOidsRequestPB, tserver.PgRollbackToSubTransactionRequestPB, tserver.PgServersMetricsRequestPB, tserver.PgSetTserverCatalogMessageListRequestPB, tserver.PgTabletServerCountRequestPB, tserver.PgTabletsMetadataRequestPB, tserver.PgTruncateTableRequestPB, tserver.PgUpdateSequenceTupleRequestPB, tserver.PgValidatePlacementRequestPB, tserver.PgWaitForBackendsCatalogVersionRequestPB, tserver.PgYCQLStatementStatsRequestPB, tserver.ReadRequestPB, tserver.ReleaseObjectLockRequestPB, tserver.UpdateTransactionRequestPB, tserver.WriteRequestPB

• optional bytes root_request_id = 1

  The root level request id. i.e. the request id corresponding to the call from the application to YSQL-Pg/YCQL if set, should be 16 bytes in length.

• optional HostPortPB client_host_port = 2

  The client which sent the request to YSQL-Pg/YCQL.

• optional uint32 addr_family = 7

  The address family of the client host

• optional bytes top_level_node_id = 3

  The uuid of the local tserver which received the request from the client. i.e. the local tserver running the YSQL-Pg backend/YCQL proxy. if set, should be 16 bytes in length.

• optional int64 rpc_request_id = 4

  The request id for the "current" rpc. This may be different than the root level request id. e.g. consider a write-rpc sent by the local TServer to the destination TServer.

• optional uint64 query_id = 5

  Query id as seen on pg_stat_statements to identify identical normalized queries.

• optional int32 pid = 9

  pid of the YSQL/YCQL backend which is executing the query. TServer pid for background activities

• optional uint32 database_id = 8

  PG database OID. This will be 0 for YCQL.

Used in: consensus.ReplicateMsg, master.GetAutoFlagsConfigResponsePB, master.TSHeartbeatResponsePB, master.ValidateAutoFlagsConfigRequestPB

• required uint32 config_version = 1

• repeated PromotedFlagsPerProcessPB promoted_flags = 2

• optional fixed64 config_apply_time = 3

  Physical time at which the new config should be applied.

Used in: BlockRecordPB

• required fixed64 id = 1

An element found in a container metadata file of the log-backed block storage implementation. Each one tracks the existence (creation) or non-existence (deletion) of a particular block. They are written sequentially, with subsequent messages taking precedence over earlier ones (e.g. "CREATE foo" followed by "DELETE foo" means that block foo does not exist).

• required BlockIdPB block_id = 1

  The unique identifier of the block.

• required BlockRecordType op_type = 2

  Whether this is a CREATE or a DELETE.

• required uint64 timestamp_us = 3

  The time at which this block record was created, expressed in terms of microseconds since the epoch.

• optional int64 offset = 4

  The offset of the block in the container data file. Required for CREATE.

• optional int64 length = 5

  The length of the block in the container data file. Required for CREATE.

The kind of record.

Used in: BlockRecordPB

• UNKNOWN = 0

• CREATE = 1

• DELETE = 2

Used in: CDCSDKStreamCreateOptionsPB

• DYNAMIC_TABLES_UNDEFINED = 0

• DYNAMIC_TABLES_ENABLED = 1

• DYNAMIC_TABLES_DISABLED = 2

CDC SDK Consistent Snapshot Options

Used in: cdc.CreateCDCStreamRequestPB, master.CDCSDKStreamEntryPB, master.CDCStreamInfoPB, master.CreateCDCStreamRequestPB

• NOEXPORT_SNAPSHOT = 1

  Client does not wish to consume snapshot

• USE_SNAPSHOT = 2

  Client wishes to consume snapshot from the source universe

• EXPORT_SNAPSHOT = 3

  Client wishes to export snapshot to be used in other session

This message should be used for specifying stream creation options for CDCSDK streams.

Used in: cdc.CreateCDCStreamRequestPB, master.CreateCDCStreamRequestPB

• optional CDCSDKDynamicTablesOption cdcsdk_dynamic_tables_option = 1

• optional ReplicationSlotLsnType lsn_type = 12

• optional ReplicationSlotOrderingMode ordering_mode = 13

For clarification of field meaning see comments of appropriate fields in YBTransaction::Impl

Used in: QLWriteRequestPB

• optional TransactionMetadataPB metadata = 1

• optional ReadHybridTimePB read_time = 2

• map<string, fixed64> local_limits = 3

For clarification of field meaning see comments of appropriate fields in YBTransaction::Impl

Used in: QLResponsePB

• repeated TransactionInvolvedTabletPB tablets = 1

• optional fixed64 restart_read_ht = 2

• map<string, fixed64> read_restarts = 3

• optional ReadHybridTimePB used_read_time = 4

Used in: ReplicationInfoPB, master.SetPreferredZonesRequestPB

• repeated CloudInfoPB zones = 1

Used in: CloudInfoListPB, PlacementBlockPB, ReplicationInfoPB, ServerRegistrationPB, consensus.RaftPeerPB, consensus.StartRemoteBootstrapRequestPB, master.GetTransactionStatusTabletsRequestPB, master.SetPreferredZonesRequestPB, master.StatefulServiceInfoPB, master.TSInfoPB, tserver.StartRemoteSnapshotTransferRequestPB

• optional string placement_cloud = 1

• optional string placement_region = 2

• optional string placement_zone = 3

Used in: SchemaPB

• oneof value

  Note: an absent value means NULL.

  • uint32 colocation_id = 1

    Colocated YSQL user tables use 4-byte ID.

  • bytes cotable_id = 2

    Colocated YSQL system tables use 16-byte UUID.

TODO: Differentiate between the schema attributes that are only relevant to the server (e.g., encoding and compression) and those that also matter to the client.

Used in: QLResponsePB, SchemaPB, master.AlterTableRequestPB.AddColumn

• optional uint32 id = 1

• required string name = 2

• required QLTypePB type = 3

• optional bool is_key = 4

• optional bool is_hash_key = 5

• optional bool is_nullable = 6

• optional bool is_static = 7

• optional bool is_counter = 8

• optional int32 order = 12

• optional uint32 sorting_type = 9

• optional uint32 pg_type_oid = 14

  Added pg_type_oid for decoding the column value for CDC in 2.13. If the cluster has been upgraded, the value will be default and not the expected value.

• repeated QLJsonOperationPB OBSOLETE_json_operations = 13

  The JSON attribute was mistakenly placed here, but it is not used. To be safe, it's left here for now. JSON attribute (for c->'a'->>'b' case).

• optional bool marked_for_deletion = 15

  If set, it means that this column is in process of being dropped by an ongoing DDL transaction. If that DDL transaction commits, this column will be dropped, otherwise this field will be unset again.

• optional QLValuePB missing_value = 16

Supplemental protobuf container header, after the main header (see pb_util.h for details).

• required google.protobuf.FileDescriptorSet protos = 1

  The protobuf schema for the messages expected in this container. This schema is complete, that is, it includes all of its dependencies (i.e. other schemas defined in .proto files imported by this schema's .proto file).

• required string pb_type = 2

  The PB message type expected in each data entry in this container. Must be fully qualified (i.e. yb.tablet.RaftGroupReplicaSuperBlockPB).

Used in: cdc.RowMessage

• optional string column_name = 1

• optional int64 column_type = 2

• oneof datum

  • int32 datum_int32 = 3

  • int64 datum_int64 = 4

  • float datum_float = 5

  • double datum_double = 6

  • bool datum_bool = 7

  • string datum_string = 8

  • bytes datum_bytes = 9

  • bool datum_missing = 10

• oneof cdc_column_type

  • QLTypePB cql_type = 11

  • int64 pg_type = 12

• oneof cdc_datum

  • PgDatumPB pg_value = 13

  • QLValuePB cql_value = 14

  • QLValuePB pg_ql_value = 15

    We send the datum as QLValuePB if the client is Walsender. The Walsender does the conversion from QLValuePB to the PG datum.

• optional int32 col_attr_num = 16

  This is applicable only for YSQL table columns.

Used in: ExplainPlanPB

• optional string delete_type = 1

• optional string scan_type = 2

  For delete, scan_type is currently either primary key lookup or range scan.

• optional string key_conditions = 3

• optional string filter = 4

• optional int32 output_width = 5

Used in: tablet.RaftGroupReplicaSuperBlockPB, tablet.TableInfoPB

• optional int32 column_id = 1

  Required. Column Id.

• optional fixed64 deleted_hybrid_time = 2

  Required. Hybrid Time when column was deleted on this tablet.

See ql_rocksdb_storage.cc for details on each method.

Used in: PgsqlSamplingStatePB

• SPLIT_INTERSECTING_BLOCKS = 0

• SPLIT_INTERSECTING_BLOCKS_V3 = 1

• COMBINE_INTERSECTING_BLOCKS = 2

• oneof plan

  • SelectPlanPB select_plan = 1

  • InsertPlanPB insert_plan = 2

  • UpdatePlanPB update_plan = 3

  • DeletePlanPB delete_plan = 4

TODO(get_table_key_ranges): deprecate after/if separate GetTabletKeyRanges RPC to the tablet leader is used.

Used in: PgsqlReadRequestPB

• optional bytes lower_bound_key = 1

• optional bytes upper_bound_key = 2

• optional uint64 range_size_bytes = 3

• optional uint32 max_key_length = 4

Captures the state of an Histogram.

Used in: HistogramSnapshotsListPB

• required string type = 1

• required string name = 2

• optional string description = 3

• optional string level = 20

• required string unit = 4

• optional string label = 19

• required uint64 max_trackable_value = 5

• required int32 num_significant_digits = 6

• required uint64 total_count = 7

• optional uint64 total_sum = 18

• required uint64 min = 8

• required double mean = 9

• required uint64 percentile_75 = 10

• required uint64 percentile_95 = 11

• required uint64 percentile_99 = 12

• required uint64 percentile_99_9 = 13

• required uint64 percentile_99_99 = 14

• required uint64 max = 15

• repeated uint64 values = 16

• repeated uint64 counts = 17

• repeated HistogramSnapshotPB histograms = 1

Used in: AshMetadataPB, QLReadRequestPB, ServerRegistrationPB, cdc.CDCTabletPB.ReplicaPB, cdc.GetChangesResponsePB, cdc.ProducerEntryPB, consensus.RaftPeerPB, consensus.StartRemoteBootstrapRequestPB, master.AddNamespaceToXClusterReplicationRequestPB, master.AlterUniverseReplicationRequestPB, master.BlacklistPB, master.CreateXClusterReplicationRequestPB, master.IsAlterXClusterReplicationDoneRequestPB, master.IsCreateXClusterReplicationDoneRequestPB, master.SetupNamespaceReplicationWithBootstrapRequestPB, master.SetupUniverseReplicationRequestPB, master.StatefulServiceInfoPB, master.SysUniverseReplicationBootstrapEntryPB, master.SysUniverseReplicationEntryPB, master.TSInfoPB, master.XClusterDeleteOutboundReplicationGroupRequestPB, master.XClusterRemoveNamespaceFromOutboundReplicationGroupRequestPB, server.ServerStatusPB, tserver.GetPgSocketDirResponsePB, tserver.StartRemoteSnapshotTransferRequestPB

• required string host = 1

• required uint32 port = 2

This message contains the metadata of a secondary index of a table. It maps the index::columns to the expressions of table::columns. Notes on where to find metadata of an INDEX. - Each INDEX is represented by IndexInfo and IndexTable. IndexInfoPB contains the definitions of the INDEX. IndexTable contains duplicate user-data for quick access. - The IndexInfoPB is stored in the Catalog::Table being indexed. - The TablePropertiesPB is kept in Catalog::IndexTable. Internally, Catalog::IndexTable is just the same as any Catalog::Table.

Used in: master.BackfillJobPB, master.CreateTableRequestPB, master.GetTableSchemaResponsePB, master.SysTablesEntryPB, tablet.ChangeMetadataRequestPB, tablet.CloneTabletRequestPB, tablet.RaftGroupReplicaSuperBlockPB, tablet.TableInfoPB, tserver.BackfillIndexRequestPB, tserver.CreateTabletRequestPB, tserver.TableMetadataPB, tserver.TabletSnapshotOpRequestPB

• optional bytes table_id = 1

  Index table id.

• optional bytes indexed_table_id = 8

  Indexed table id.

• optional uint32 version = 2

  Index table's schema version.

• optional bool is_local = 3

  Whether the index is a local index

• optional bool is_unique = 7

  Whether the index is a unique index

• optional IndexPermissions index_permissions = 12

  We should only have this in the elements of "repeated IndexInfoPB indexes" of the SysTablesEntryPB of the main table.

• repeated IndexInfoPB.IndexColumnPB columns = 4

  Indexed and covering columns.

• optional uint32 hash_column_count = 5

  Number of hash columns in the index.

• optional uint32 range_column_count = 6

  Number of range columns in the index.

• repeated uint32 indexed_hash_column_ids = 9

  Hash column ids in the indexed table.

• repeated uint32 indexed_range_column_ids = 10

  Range column ids in the indexed table.

• optional bool use_mangled_column_name = 11

  The mangled-name flag is kept on both IndexInfo and IndexTable as the same mangled-name is used in both IndexInfo and IndexTable columns.

  Newer index has mangled name.

• optional string backfill_error_message = 31

  When backfill fails with error, the first message that it failed with.

• optional bool is_backfill_deferred = 32

  Should backfill be deferred.

• optional IndexInfoPB.WherePredicateSpecPB where_predicate_spec = 33

  Where clause for partial index.

• optional uint64 num_rows_processed_by_backfill_job = 34

  When the backfill job on this index is cleared, this field is set to the number of indexed table rows processed by the backfill job. The (default) value is zero, otherwise. For partial indexes, this includes non-matching rows of the indexed table.

• optional PgVectorIdxOptionsPB vector_idx_options = 35

Index column mapping. "colexpr" is used to compute the value of this column in an INDEX. - When a table is indexed by expressions, we create internal/hidden columns to store the index value, and "value_expr" specifies the indexing expression. - As of 07/2019, only QLJsonColumnOperationsPB is allowed for "colexpr". - In the current index design & implementation, expression can only reference ONE column. Example: Example for scalar index TABLE (a, b, c) INDEX (c) -> INDEX is a table whose column 'c' is referencing TABLE(c) colexpr = ref to "c" column_id. Example for JSON index TABLE (a, b, j) INDEX (j->>'b') -> INDEX is a table whose column 'j->>b' is referencing to TABLE(j) colexpr = j->'b'

Used in: IndexInfoPB

• optional uint32 column_id = 1

  Column id in the index table.

• optional string column_name = 3

  Generated column name in the index table.

• optional uint32 indexed_column_id = 2

  Corresponding column id in indexed table.

• optional QLExpressionPB colexpr = 4

  Column value in INDEX.

Used in: IndexInfoPB

• repeated uint32 column_ids = 1

  Columns referenced in the where_expr

• optional QLExpressionPB where_expr = 2

Indexes when created or deleted go through a series of steps to add a) delete, b) write and c) read permissions one by one. Backfill is done before the READ_WRITE_AND_DELETE state. If the backfill succeeds, the read permission is granted. If not, the given permissions are removed one by one, and the index will be deleted when it is unused. If the backfill is successfully complete, the index will be in READ_WRITE_AND_DELETE state, and remain so until the user deletes it. If an Index is dropped, it will move from READ_WRITE_AND_DELETE state throu the WRITE_AND_DELETE_WHILE_REMOVING and DELETE_ONLY_WHILE_REMOVING states to INDEX_UNUSED.

Used in: IndexInfoPB

• INDEX_PERM_DELETE_ONLY = 0

• INDEX_PERM_WRITE_AND_DELETE = 2

• INDEX_PERM_DO_BACKFILL = 4

• INDEX_PERM_READ_WRITE_AND_DELETE = 6

  This is the "success" state, where the index is completely usable.

• INDEX_PERM_WRITE_AND_DELETE_WHILE_REMOVING = 8

  Used while removing an index -- either due to backfill failure, or due to a client requested "drop index".

• INDEX_PERM_DELETE_ONLY_WHILE_REMOVING = 10

• INDEX_PERM_INDEX_UNUSED = 12

• INDEX_PERM_NOT_USED = 14

  Used as a sentinel value.

Used in: ExplainPlanPB

• optional string insert_type = 1

• optional int32 output_width = 2

When any server initializes a new filesystem (eg a new node is created in the cluster), it creates this structure and stores it persistently.

• required bytes uuid = 1

  The UUID which is assigned when the instance is first formatted. This uniquely identifies the node in the cluster.

• required string format_stamp = 2

  Human-readable string indicating when and where the node was first initialized.

• optional bool initdb_done_set_after_sys_catalog_restore = 3

  For all newly created clusters after this field is set, initialize this variable to true. For existing clusters this will be false. This will be used on the sys catalog restore path to indicate whether we want to run initdb.

• optional TserverInstanceMetadataPB tserver_instance_metadata = 4

  Tserver specific metadata.

Used in: TransactionInvolvedTabletPB

• MISSING = 0

• EXIST = 1

• MAY_EXIST = 2

Used in: TransactionMetadataPB

• NON_TRANSACTIONAL = 0

• SNAPSHOT_ISOLATION = 1

• SERIALIZABLE_ISOLATION = 2

• READ_COMMITTED = 3

Used in: QLJsonOperationPB

• JSON_OBJECT = 0

  The -> operator applied to a column.

• JSON_TEXT = 1

  The ->> operator applied to a column.

Used in: TabletLockInfoPB.TransactionLockInfoPB, TabletLockInfoPB.WaiterInfoPB

• repeated bytes hash_cols = 1

  TODO(pglocks): Omit prefix locks, which are implied to exist if their fully-qualified forms are present.

• repeated bytes range_cols = 2

• optional int32 attnum = 3

• optional uint32 column_id = 4

• optional bool multiple_rows_locked = 5

• optional fixed64 wait_end_ht = 6

  If this is set, then the lock has been granted.

• optional uint32 subtransaction_id = 7

• optional bool is_explicit = 8

• repeated LockMode modes = 9

• optional bytes pg_table_id = 10

  Populated only when the lock belongs to a colocated table. When set, TabletLockInfoPB::pg_table_id should be ignored.

Used in: LockInfoPB

• WEAK_READ = 0

• WEAK_WRITE = 1

• STRONG_READ = 2

• STRONG_WRITE = 3

Uniquely identify a particular instance of a particular server in the cluster.

Used in: ServerEntryPB, consensus.GetNodeInstanceResponsePB, master.GetMasterRegistrationResponsePB, master.ListLiveTabletServersResponsePB.Entry, master.ListTabletServersResponsePB.Entry, master.RefreshYsqlLeaseRequestPB, master.TSHeartbeatResponsePB, master.TSInformationPB, master.TSToMasterCommonPB, server.ServerStatusPB

• required bytes permanent_uuid = 1

  Unique ID which is created when the server is first started up. This is stored persistently on disk.

• required int64 instance_seqno = 2

  Sequence number incremented on every start-up of the server. This makes it easy to detect when an instance has restarted (and thus can be assumed to have forgotten any soft state it had in memory). On a freshly initialized server, the first sequence number should be 0.

• optional uint64 start_time_us = 3

  The time point when tserver/master starts up, it is derived from Env::NowMicros()

An id for a generic state machine operation. Composed of the leaders' term plus the index of the operation in that term, e.g., the <index>th operation of the <term>th leader.

Used in: rocksdb.NewFilePB, rocksdb.VersionEditPB, PostApplyTransactionMetadataPB, cdc.CDCCheckpointPB, cdc.GetLatestEntryOpIdResponsePB, cdc.UpdateCdcReplicatedIndexRequestPB, consensus.ConsensusRequestPB, consensus.ConsensusStatusPB, consensus.GetLastOpIdResponsePB, consensus.OperationStatusPB, consensus.PerOpErrorPB, consensus.ReplicateMsg, consensus.ReplicatedRetryableRequestRangePB, consensus.RunLeaderElectionRequestPB, consensus.TabletBootstrapStatePB, docdb.ApplyTransactionStatePB, docdb.ConsensusFrontierPB, log.LogEntryBatchPB, tablet.RaftGroupReplicaSuperBlockPB, tablet.TableInfoPB, tserver.ChangeMetadataResponsePB, tserver.CreateTabletResponsePB, tserver.RegisterLogAnchorRequestPB, tserver.UpdateLogAnchorRequestPB

• required int64 term = 1

  The term of an operation or the leader's sequence id.

• required int64 index = 2

The possible order modes for clients. Clients specify these in new scan requests. Ordered scans are fault-tolerant, and can be retried elsewhere in the case of tablet server failure. However, ordered scans impose additional overhead since the tablet server needs to sort the result rows.

• UNKNOWN_ORDER_MODE = 0

• UNORDERED = 1

  This is the default order mode.

• ORDERED = 2

The serialized format of a YB table partition.

Used in: cdc.CDCRecordPB, master.CreateTableRequestPB, master.SysTabletsEntryPB, master.TabletLocationsPB, tablet.RaftGroupReplicaSuperBlockPB, tablet.TabletStatusPB, tserver.CreateTabletRequestPB

• repeated int32 hash_buckets = 1

  The hash buckets of the partition. The number of hash buckets must match the number of hash bucket components in the partition's schema.

• optional bytes partition_key_start = 2

  The encoded start partition key (inclusive).

• optional bytes partition_key_end = 3

  The encoded end partition key (exclusive).

The serialized format of a YB table partition schema.

Used in: master.CreateTableRequestPB, master.GetTableSchemaResponsePB, master.SysTablesEntryPB, tablet.CloneTabletRequestPB, tablet.RaftGroupReplicaSuperBlockPB, tablet.TableInfoPB, tserver.CreateTabletRequestPB, tserver.ListTabletsResponsePB.StatusAndSchemaPB

• repeated PartitionSchemaPB.HashBucketSchemaPB hash_bucket_schemas = 1

• optional PartitionSchemaPB.RangeSchemaPB range_schema = 2

• optional PartitionSchemaPB.HashSchema hash_schema = 3

A column identifier for partition schemas. In general, the name will be used when a client creates the table since column IDs are assigned by the master. All other uses of partition schemas will use the numeric column ID.

Used in: HashBucketSchemaPB, RangeSchemaPB

• oneof identifier

  • int32 id = 1

  • string name = 2

Used in: PartitionSchemaPB

• repeated ColumnIdentifierPB columns = 1

  Column identifiers of columns included in the hash. Every column must be a component of the primary key.

• required int32 num_buckets = 2

  Number of buckets into which columns will be hashed. Must be at least 2.

• optional uint32 seed = 3

  Seed value for hash calculation. Administrators may set a seed value on a per-table basis in order to randomize the mapping of rows to buckets. Setting a seed provides some amount of protection against denial of service attacks when the hash bucket columns contain user provided input.

• optional HashBucketSchemaPB.HashAlgorithm hash_algorithm = 4

  The hash algorithm to use for calculating the hash bucket.

Used in: HashBucketSchemaPB

• UNKNOWN = 0

• MURMUR_HASH_2 = 1

Used in: PartitionSchemaPB

• MULTI_COLUMN_HASH_SCHEMA = 1

• REDIS_HASH_SCHEMA = 2

• PGSQL_HASH_SCHEMA = 3

Used in: PartitionSchemaPB

• repeated ColumnIdentifierPB columns = 1

  Column identifiers of columns included in the range. All columns must be a component of the primary key.

• repeated RangeSplitPB splits = 3

  Split points for range tables.

• repeated bytes DEPRECATED_split_rows = 2

  Field "split_rows" is only available in beta version for SPLIT AT feature. This feature had not been used, so the release version for SPLIT AT does not need to be compatible with the beta version in the past releases.

Used in: RangeSchemaPB

• optional bytes column_bounds = 1

  Each range column will be assigned a lower bound.

A filesystem instance can contain multiple paths. One of these structures is persisted in each path when the filesystem instance is created.

• required PathSetPB path_set = 1

  Describes this path instance as well as all of the other path instances that, taken together, describe a complete set.

• required string block_manager_type = 2

  Textual representation of the block manager that formatted this path.

• required uint64 filesystem_block_size_bytes = 3

  Block size on the filesystem where this instance was created. If the instance (and its data) are ever copied to another location, the block size in that location must be the same.

Describes a collection of filesystem path instances and the membership of a particular instance in the collection. In a healthy filesystem (see below), a path instance can be referred to via its UUID's position in all_uuids instead of via the UUID itself. This is useful when there are many such references, as the position is much shorter than the UUID.

Used in: PathInstanceMetadataPB

• required bytes uuid = 1

  Globally unique identifier for this path instance.

• repeated bytes all_uuids = 2

  All UUIDs in this path instance set. In a healthy filesystem: 1. There exists an on-disk PathInstanceMetadataPB for each listed UUID, and 2. Every PathSetPB contains an identical copy of all_uuids.

The possible roles for peers.

Used in: ServerEntryPB, master.CheckMasterTabletHealthResponsePB, master.GetMasterRegistrationResponsePB, master.TabletLocationsPB.ReplicaPB, tserver.CheckTserverTabletHealthResponsePB.TabletHealthPB

• FOLLOWER = 0

  Indicates this node is a follower in the configuration, i.e. that it participates in majorities and accepts Consensus::Update() calls.

• LEADER = 1

  Indicates this node is the current leader of the configuration, i.e. that it participates in majorities and accepts Consensus::Append() calls.

• LEARNER = 2

  New peers joining a quorum will be in this role for both PRE_VOTER and PRE_OBSERVER while the tablet data is being remote bootstrapped. The peer does not participate in starting elections or majorities.

• NON_PARTICIPANT = 3

  Indicates that this node is not a participant of the configuration, i.e. does not accept Consensus::Update() or Consensus::Update() and cannot participate in elections or majorities. This is usually the role of a node that leaves the configuration.

• READ_REPLICA = 4

  This peer is a read (async) replica and gets informed of the quorum write activity and provides time-line consistent reads.

• UNKNOWN_ROLE = 7

Suffixed with PERMISSION, because Google does not allow same enum name CREATE

Used in: master.GrantRevokePermissionRequestPB, master.ResourcePermissionsPB

• ALTER_PERMISSION = 0

• CREATE_PERMISSION = 1

• DROP_PERMISSION = 2

• SELECT_PERMISSION = 3

• MODIFY_PERMISSION = 4

• AUTHORIZE_PERMISSION = 5

• DESCRIBE_PERMISSION = 6

• ALL_PERMISSION = 999999999

To ensure compatibility between release versions, the numeric values of these datatypes cannot be changed once the types are implemented and released. Make sure this is in sync with YbcPgDataType in ybc_pg_typedefs.h.

Used in: QLTypePB, dockv.ColumnPackingPB

• UNKNOWN_DATA = 999

• NULL_VALUE_TYPE = 0

• INT8 = 1

• INT16 = 2

• INT32 = 3

• INT64 = 4

• STRING = 5

• BOOL = 6

• FLOAT = 7

• DOUBLE = 8

• BINARY = 9

• TIMESTAMP = 10

• DECIMAL = 11

• VARINT = 12

• INET = 13

• LIST = 14

• MAP = 15

• SET = 16

• UUID = 17

• TIMEUUID = 18

• TUPLE = 19

  TUPLE is not yet fully implemented, but it is a CQL type.

• TYPEARGS = 20

• USER_DEFINED_TYPE = 21

• FROZEN = 22

• DATE = 23

• TIME = 24

• JSONB = 25

• UINT8 = 100

  All unsigned datatypes will be removed from QL because databases do not have these types.

• UINT16 = 101

• UINT32 = 102

• UINT64 = 103

• GIN_NULL = 104

• VECTOR = 105

• BSON = 106

Used in: DatumMessagePB

• oneof datum

  • int32 datum_int32 = 1

  • int64 datum_int64 = 2

  • float datum_float = 3

  • double datum_double = 4

  • bool datum_bool = 5

  • string datum_string = 6

  • bytes datum_bytes = 7

  • bool datum_missing = 8

Used in: PgVectorIdxOptionsPB

• optional uint32 m = 1

• optional uint32 m0 = 2

• optional uint32 ef_construction = 3

Used in: PgVectorReadOptionsPB

• required uint32 ef_search = 1

Available replica identity modes for use in CDC

Used in: TablePropertiesPB, master.CDCStreamInfoPB, master.SysCDCStreamEntryPB

• DEFAULT = 0

  Entire updated row as new image, only key as old image for DELETE The name DEFAULT is taken from PG, however it is not the default replica identity mode

• FULL = 1

  Both old and new images of the entire row

• NOTHING = 2

  No old image for any operation

• CHANGE = 3

  Only the changed columns as new image, no old image except DELETE This is the default replica identity mode in YSQL

Used in: tserver.GetLocalPgTxnSnapshotResponsePB, tserver.PgExportTxnSnapshotRequestPB, tserver.PgImportTxnSnapshotResponsePB

• optional uint32 db_oid = 1

• optional uint32 isolation_level = 2

• optional bool read_only = 3

Used in: PgVectorIdxOptionsPB

• INVALID_DIST = 0

• DIST_L2 = 1

• DIST_IP = 2

• DIST_COSINE = 3

Used in: IndexInfoPB, tserver.PgCreateTableRequestPB

• optional PgVectorDistanceType dist_type = 1

• optional PgVectorIndexType idx_type = 2

• optional uint32 dimensions = 3

• optional uint32 column_id = 4

• optional PgHnswIndexOptionsPB hnsw = 7

• optional string id = 8

  Permanent index id. It should be unique for tablet, but when tablet is cloned, index id in new tablet remains the same.

Used in: PgVectorIdxOptionsPB

• UNKNOWN_IDX = 0

• DUMMY = 1

• IVFFLAT = 2

• HNSW = 3

Used in: PgsqlReadRequestPB

• required QLValuePB vector = 1

• required int32 prefetch_size = 2

• optional uint64 num_top_vectors_to_remove = 4

• oneof index_params

  • PgHnswReadOptionsPB hnsw_options = 3

Used in: PgsqlLockRequestPB

• repeated QLExpressionPB lock_partition_column_values = 1

• repeated QLExpressionPB lock_range_column_values = 2

-------------------------------------------------------------------------------------------------- Expressions. -------------------------------------------------------------------------------------------------- Builtin call expression. There are 3 different calls. - Builtin operators such as '>', '<', '=', ... These operators can be executed anywhere. - Builtin functions such as Now(). These functions can be executed anywhere. - Server builtin functions. Only tablet servers can execute these functions. TODO(neil) Regular builtin operators. This message can be executed anywhere. - This is more efficient than builtin call as it avoids most overheads of calling builtin lib. - Merge the current condition operator execution with this. - To optimize certain operation (such as +), replace it builtin function with builtin op.

Used in: PgsqlExpressionPB

• optional int32 opcode = 1

• repeated PgsqlExpressionPB operands = 2

Given: master --chunk--> tserver --batch--> postgres --page--> tserver The backfill spec is updated and passed around for each batch and page.

• optional int64 limit = 1

  [batch] Minimum number of table rows to backfill in this batch (see issue #10649). Stays constant for each batch and all of its pages. -1 means no limit.

• optional int64 count = 2

  [page] Number of table rows backfilled so far in this batch. Updated every page.

• optional bytes next_row_key = 3

  [batch] The next row key to read for a backfill batch. Defined/used similar to how it is used in paging state. All the intermediate pages of the batch have this unset. An example is easier to understand: - tserver --batch--> postgres: next_row_key = FOO - postgres --page--> tserver: next_row_key = FOO - postgres <--page-- tserver: next_row_key = unset; PgsqlPagingStatePB.next_row_key = BAR - postgres --page--> tserver: next_row_key = unset; PgsqlPagingStatePB.next_row_key = BAR - postgres <--page-- tserver: next_row_key = unset; PgsqlPagingStatePB.next_row_key = BAZ - postgres --page--> tserver: next_row_key = unset; PgsqlPagingStatePB.next_row_key = BAZ - postgres <--page-- tserver: next_row_key = QUX; PgsqlPagingStatePB.next_row_key = QUX - tserver <--batch-- postgres: next_row_key = QUX

This message defines an argument in a batch request from PgGate to DocDB. Instead of sending many requests of different arguments, a batch request would send one request that contains an array of independent arguments. DocDB will iterate the array to execute. DML_request(arg) [n] ---> DML_request ( PgsqlBatchArgument args[n] )

Used in: PgsqlReadRequestPB, PgsqlWriteRequestPB

• optional int64 order = 1

  The order number of this argument in a batch. Currently, this is only used for debugging, but it might be needed in the future.

• optional PgsqlExpressionPB ybctid = 2

  This attribute is used when fetching by rowid (ybctid). PgGate fetches "base_ctid" from SecondaryIndex and use it to select Targets from UserTable. SELECT <Targets> FROM <UserTable> WHERE ybctid IN (SELECT base_ybctid FROM <SecondaryIndex> WHERE <key_condition>)

• optional uint32 hash_code = 3

  Hash codes.

• optional uint32 max_hash_code = 4

• repeated PgsqlExpressionPB partition_column_values = 5

  Partition values.

• repeated PgsqlExpressionPB range_column_values = 6

  Range values.

Represents a column referenced by a PGSQL request. - column_id is a DocDB identifier of the column and used to locate a value within a row - attno is a Postgres identifier, must be specified if column is referenced by a PG expression - typid, typmod are needed to convert DocDB values to Postgres format (Datum and isnull pair). They must be specified if used in an expression evaluated by PGgate. - collid is for future collations support

Used in: PgsqlReadRequestPB, PgsqlWriteRequestPB

• optional int32 column_id = 1

• optional int32 attno = 2

• optional int32 typid = 3

• optional int32 typmod = 4

• optional int32 collid = 5

-------------------------------------------------------------------------------------------------- Column messages. -------------------------------------------------------------------------------------------------- ColumnRefs is a list of columns to be read by DocDB before a PGSQL request can be executed. DEPRECATED, new code should use col_refs field of type PgsqlColRefPB instead

Used in: PgsqlReadRequestPB, PgsqlWriteRequestPB

• repeated int32 ids = 1

ColumnValue is a value to be assigned to a table column by DocDB while executing a PGSQL request. Currently, this is used for SET clause. SET column-of-given-id = expr

Used in: PgsqlWriteRequestPB

• optional int32 column_id = 1

• optional PgsqlExpressionPB expr = 2

A logical condition that evaluates to true/false. Used in the WHERE clause.

Used in: PgsqlExpressionPB

• optional QLOperator op = 1

• repeated PgsqlExpressionPB operands = 2

An expression in a WHERE condition. - Bind values would be given by client and grouped into a repeated field that can be accessed by their indexes. - Alias values would be computed by server and grouped into repeated field that can be accessed by their indexes. - Code generator write indexes as ref. Executor deref indexes to get actual values.

Used in: PgsqlBCallPB, PgsqlBatchArgumentPB, PgsqlColumnValuePB, PgsqlConditionPB, PgsqlReadRequestPB, PgsqlWriteRequestPB

• oneof expr

  • QLValuePB value = 1

  • int32 column_id = 2

  • int32 bind_id = 3

    Bind variable index.

  • int32 alias_id = 4

    Alias index.

  • PgsqlBCallPB bfcall = 5

    Regular builtin calls.

  • PgsqlBCallPB tscall = 6

    Tablet server builtin calls.

  • PgsqlBCallPB bocall = 7

    Builtin operator calls.

  • PgsqlConditionPB condition = 8

    Logical condition that evaluates to true/false.

  • QLTupleExpressionPB tuple = 9

    Tuple expression in LHS.

Used in: PgsqlWriteRequestPB

• optional uint32 fetch_count = 1

• optional int64 inc_by = 2

• optional int64 min_value = 3

• optional int64 max_value = 4

• optional bool cycle = 5

Used in: tserver.WriteRequestPB

• optional QLClient client = 1

  Client info

  required

• optional PgsqlAdvisoryLockPB lock_id = 2

  All locks if not specified

• optional PgsqlLockRequestPB.PgsqlAdvisoryLockMode lock_mode = 3

• optional bool wait = 4

• optional bool is_lock = 5

  Lock or Unlock

• optional bytes partition_key = 6

• optional uint32 hash_code = 7

Used in: PgsqlLockRequestPB

• PG_LOCK_SHARE = 1

• PG_LOCK_EXCLUSIVE = 2

Make sure this is in sync with YbcPgMetricsCaptureType in ybc_pg_typedefs.h.

Used in: PgsqlReadRequestPB

• PGSQL_METRICS_CAPTURE_NONE = 0

• PGSQL_METRICS_CAPTURE_ALL = 1

Paging state for continuing a read request. For a SELECT statement that returns many rows, the client may specify how many rows to return at most in each fetch. This paging state maintains the state for returning the next set of rows of the statement. This paging state is opaque to the client. When there should be more rows to return from the same tablet in the next fetch, "next_row_key" is populated in DocDB (PgsqlReadOperation) with the DocKey of the next row to read. We also embed a hybrid-time which is the clean snapshot time for read consistency. We also populate the "next_partition_key" for the next row, which is the hash code of the hash portion of the DocKey. This next partition key is needed by YBClient (Batcher) to locate the tablet to send the request to and it doesn't have access to the DocDB function to decode and extract from the DocKey. When we are done returning rows from the current tablet and the next fetch should continue in the next tablet (possible only for full-table query across tablets), "next_partition_key" is populated by the current tablet with its exclusive partition-end key, which is the start key of next tablet's partition. "next_row_key" is empty in this case which means we will start from the very beginning of the next tablet. (TODO: we need to return the clean snapshot time in this case also).

Used in: PgsqlReadRequestPB, PgsqlResponsePB

• optional bytes table_id = 1

  Table UUID to verify the same table still exists when continuing in the next fetch.

• optional bytes next_partition_key = 2

  Partition key to find the tablet server of the next row to read.

• optional bytes next_row_key = 3

  The row key (SubDocKey = [DocKey + HybridTimestamp]) of the next row to read.

• optional uint64 total_num_rows_read = 4

  Running total number of rows read across fetches so far. Needed to ensure we read up to the number of rows in the SELECT's LIMIT clause across fetches.

• optional uint64 next_partition_index = 5

  For selects with IN condition on the hash columns there are multiple partitions that need to be queried, one for each combination of allowed values for the hash columns. This holds the index of the next partition and is used to resume the read from the right place.

• optional ReadHybridTimePB read_time = 6

  Used read time.

• optional double distance = 7

  Paging state fields for geometric indexes.

• optional bytes main_key = 8

  Value of the next geometric index row that can be used as a key into the main RocksDB store.

Boundary value.

Used in: PgsqlReadRequestPB

• optional bytes key = 1

  Boundary partition key.

• optional bool is_inclusive = 2

  Indicate whether or not the boundary is inclusive.

PgsqlRSColDesc is the descriptor of a selected column in a ResultSet (RS), which can be any expression and not just table columns.

Used in: PgsqlRSRowDescPB

• optional string name = 1

• optional QLTypePB ql_type = 2

Descriptor of a row in a resultset (RS).

Used in: PgsqlReadRequestPB, PgsqlWriteRequestPB

• repeated PgsqlRSColDescPB rscol_descs = 1

YB_TODO(upgrade): Handle in pg15 upgrade path.

Used in: PgsqlSamplingStatePB

• required uint64 s0 = 1

• required uint64 s1 = 2

TODO(neil) The protocol for select needs to be changed accordingly when we introduce and cache execution plan in tablet server.

Used in: tserver.PgPerformOpPB, tserver.ReadRequestPB

• optional QLClient client = 1

  Client info

  required

• optional uint64 stmt_id = 2

  Statement info. There's only SELECT, so we don't need different request type.

  required

• optional bytes table_id = 19

  Table id.

  required

• optional uint32 schema_version = 3

  Table schema version

  required

• optional uint32 hash_code = 4

  Hash Code: A hash code is used to locate efficiently a group of data of the same hash code. - First, it is used to find tablet server. - Then, it is used again by DocDB to narrow the search within a tablet. - In general, hash_code should be set by PgGate, but for some reasons, when the field "ybctid_column_value" is used, hash_code is set by "common" lib.

• optional PgsqlExpressionPB ybctid_column_value = 20

• repeated PgsqlExpressionPB partition_column_values = 5

  Primary key. - Partition columns are used to compute the hash code, e.g. for h1 = 1 AND h2 = 2 partition_column_values would be [1, 2]. - Range columns combining with partition columns are used for indexing.

• repeated PgsqlExpressionPB range_column_values = 18

• optional PgsqlReadRequestPB index_request = 21

  For select using local secondary index: this request selects the ybbasectids to fetch the rows in place of the primary key above.

• optional PgsqlExpressionPB where_expr = 8

  Where clause condition DEPRECATED in favor of where_clauses

• optional PgsqlExpressionPB condition_expr = 22

  TODO(alex): This should be renamed to make it clear that it's ONLY related to range columns! Conditions for range columns.

• repeated PgsqlBatchArgumentPB batch_arguments = 24

  If this attribute is present, this request is a batch request.

• optional PgsqlRSRowDescPB rsrow_desc = 6

  ------------------------------------------------------------------------------------------------ Output Argument Specification (Tuple descriptor). For now, we sent rsrow descriptor from proxy to tablet server for every request. RSRow is just a selected row. We call it rsrow to distinguish a selected row from a row of a table in the database in our coding.

• repeated PgsqlExpressionPB targets = 7

  required.

• optional PgsqlColumnRefsPB column_refs = 9

  ------------------------------------------------------------------------------------------------ Database Arguments - To be read from the DataBase. Listing of all columns that this operation is referencing. TServers will need to read these columns when processing this read request. DEPRECATED in favor of col_refs;

• optional bool is_forward_scan = 10

  ------------------------------------------------------------------------------------------------ Query options.

• optional bool distinct = 11

  Reading distinct columns?

• optional uint64 prefix_length = 39

  Current only used on SELECT DISTINCT scan. If the value is greater than 0, use the specified prefix length to scan the table.

• optional bool is_aggregate = 12

  Flag for reading aggregate values.

• optional uint64 limit = 13

  Limit number of rows to return. For SELECT, this limit is the smaller of the page size (max (max number of rows to return per fetch) & the LIMIT clause if present in the SELECT statement.

• optional uint64 size_limit = 40

• optional PgsqlPagingStatePB paging_state = 14

  ------------------------------------------------------------------------------------------------ Paging state retrieved from the last response.

• optional bool return_paging_state = 15

  Return paging state when "limit" number of rows are returned? In case when "limit" is the page size, this is set for PgsqlResponsePB to return the paging state for the next fetch.

• optional uint32 max_hash_code = 17

  the upper limit for partition (hash) key when paging.

• optional uint64 ysql_catalog_version = 16

  The version of the ysql system catalog this query was prepared against. Used when per-db catalog version mode is disabled.

• optional uint64 ysql_db_catalog_version = 36

  The version of the ysql per-db system catalog this query was prepared against. Used when per-db catalog version mode is enabled.

• optional uint32 ysql_db_oid = 37

  The oid of the database of the Postgres session where this query was issued. Used when per-db catalog version mode is enabled.

• optional RowMarkType row_mark_type = 23

  Row mark as used by postgres for row locking.

• optional WaitPolicy wait_policy = 32

  Used only for explicit row-locking to identify 3 possibilities: WAIT_BLOCK, WAIT_SKIP, or WAIT_ERROR. See WaitPolicy for detailed information.

• optional PgsqlPartitionBound lower_bound = 27

  Scan partition boundary. NOTE - Boundaries indicate the scan range that are given by SQL statement. This should be set by PgGate when it processed query specification. - paging_state indicates the current position of a scan. This should be set ONLY by DocDB who scans the table. - Don't overload these fields for different purposes to avoid confusion.

• optional PgsqlPartitionBound upper_bound = 28

• optional bytes DEPRECATED_max_partition_key = 25

  Deprecated fields. Field "max_partition_key" is replaced by "lower_bound" and "upper_bound". This field is only available in beta version for SPLIT_AT feature. The release version for SPLIT AT does not need to be compatible with the beta version as SPLIT AT feature were not used.

• optional bool is_for_backfill = 29

  Does this request correspond to scanning the indexed table for backfill?

• optional PgsqlSamplingStatePB sampling_state = 30

  If present, the request is to collect random sample

• repeated PgsqlSampleBlockPB sample_blocks = 44

  Boundaries of blocks to be sampled at 2nd stage for getting sample of rows, empty during 1st stage.

• optional bytes backfill_spec = 31

  Instruction for BACKFILL operation from the tablet server. This is the serialized-string of "message PgsqlBackfillSpecPB" which will be used by the tablet server handling the PgsqlRead to terminate the scan and checkpoint the paging state as desired.

• repeated PgsqlExpressionPB where_clauses = 33

  Pushed down Postgres conditions. Like in Postgres, conditions are implicitly ANDed. Per scanned row, expressions are evaluated by PGgate one by one, and if any is evaluated to false the row is skipped.

• optional uint32 expression_serialization_version = 43

• repeated PgsqlColRefPB col_refs = 34

  Columns referenced by this write operation. Provides mapping between DocDB and Postgres identifiers, and provides Postgres type information to convert between DocDB and Postgres formats. One entry per column referenced.

• optional bytes partition_key = 35

  Used only in pg client.

• optional PgsqlMetricsCaptureType metrics_capture = 38

  What metric changes to return in response.

• optional GetTabletKeyRangesEmbeddedRequestPB get_tablet_key_ranges_request = 41

  TODO(get_table_key_ranges): deprecate this field after separate separate GetTabletKeyRanges RPC to the tablet leader is used.

• optional PgVectorReadOptionsPB vector_idx_options = 42

Used in: PgsqlRequestMetricsPB

• optional uint64 metric = 1

• optional uint64 value = 2

Used in: PgsqlRequestMetricsPB

• optional uint64 metric = 1

• optional int64 sum = 2

• optional uint64 count = 3

Used in: PgsqlRequestMetricsPB

• optional uint64 metric = 1

• optional int64 value = 2

Used in: PgsqlResponsePB

• repeated PgsqlRequestGaugeMetricPB gauge_metrics = 1

  Gauge metrics changes as a result of processing the request.

• repeated PgsqlRequestCounterMetricPB counter_metrics = 2

  Counter metrics changes as a result of processing the request.

• repeated PgsqlRequestEventMetricPB event_metrics = 3

  Event stats metrics changes as a result of processing the request.

• optional int64 scanned_table_rows = 4

  Count of table rows scanned as a result of processing the request.

• optional int64 scanned_index_rows = 5

  Count of secondary index rows scanned as a result of processing the request.

Response from tablet server for both read and write.

Used in: tserver.PgPerformResponsePB, tserver.ReadResponsePB, tserver.WriteResponsePB

• optional PgsqlResponsePB.RequestStatus status = 1

  Internal Status information

  required

• optional bool skipped = 6

  True if this operation was not actually applied, for instance update to the same value.

• optional string error_message = 2

  User readable error message associated with Internal & External Status DEPRECATED: please use error_status instead

• optional int32 rows_data_sidecar = 4

  Sidecar of rows data returned

• optional PgsqlPagingStatePB paging_state = 5

  Paging state for continuing the read in the next QLReadRequestPB fetch.

• optional int64 batch_arg_count = 10

  When client sends a request that has a batch of many arguments, server might process only a subset of the arguments. Attribute "batch_arg_count" is to indicate how many arguments have been processed by server. NOTE: This variable could have been inside "paging state", but due to rolling upgrade I have to place it here, separated from paging status for singular request.

• repeated int64 batch_orders = 12

• repeated uint64 vector_index_distances = 19

  For use with geometric indexes. The nth value here is the distance of the nth returned row from the query point/vector. The distance is encoded into binary representation and converted to int64. So if we have original_distance1 < original_distance2, then encoded_distance1 < encoded_distance2. We are widely use the similar approach for table primary key when it is converted to binary representation. Here we do the same with addendum that distance could be always encoded to 8 bytes or less.

• repeated uint64 vector_index_ends = 20

  The offset of end of encoded row for particular vector.

• optional int32 rows_affected_count = 7

  Number of rows affected by the operation. Currently only used for update and delete.

• optional uint64 pg_error_code = 8

  PostgreSQL error code encoded as in errcodes.h or yb_pg_errcodes.h. See https://www.postgresql.org/docs/11/errcodes-appendix.html DEPRECATED: please use error_status instead

• optional uint32 txn_error_code = 9

  Transaction error code, obtained by static_cast of TransactionErrorTag::Decode of Status::ErrorData(TransactionErrorTag::kCategory) DEPRECATED: please use error_status instead

• optional PgsqlSamplingStatePB sampling_state = 11

  Random sample state to hand over to the next block

• optional bytes backfill_spec = 13

  Instruction for BACKFILL operation from master/table server. This is the serialized-string of "message PgsqlBackfillSpecPB". If 'is_backfill_batch_done' == true, Postgres will return this 'backfill_spec' to tablet server that sent out the 'BACKFILL' request statement.

• optional bool is_backfill_batch_done = 14

• optional uint32 partition_list_version = 15

  Table's partition list version obtained from YBTable during response preparation. Table cache should be invalidated on partition list version mismatch detected.

• repeated AppStatusPB error_status = 16

  Encapsulates data from a Status instance Repeated field provides support for multiple Statuses per response, allowing to return not only detailed error, but also warnings and notices.

• optional PgsqlRequestMetricsPB metrics = 17

  Metrics changes as a result of processing the request.

Response status

Used in: PgsqlResponsePB

• PGSQL_STATUS_OK = 0

• PGSQL_STATUS_SCHEMA_VERSION_MISMATCH = 1

• PGSQL_STATUS_RUNTIME_ERROR = 2

• PGSQL_STATUS_USAGE_ERROR = 3

• PGSQL_STATUS_RESTART_REQUIRED_ERROR = 4

• PGSQL_STATUS_DUPLICATE_KEY_ERROR = 5

Used in: PgsqlReadRequestPB

• required bytes lower_bound = 1

• required bytes upper_bound = 2

Random sampling state

Used in: PgsqlReadRequestPB, PgsqlResponsePB

• optional int32 targrows = 1

  target number of rows to collect

• optional int32 numrows = 2

  number of rows collected so far across all blocks

• optional double samplerows = 3

  number of rows considered for sample across all blocks

• optional double DEPRECATED_estimated_total_rows = 8

  number of total rows estimated

• optional double rowstoskip = 4

  current number of rows to skip

• optional double rstate_w = 5

  reservoir state W

• optional PgsqlRandState rand_state = 7

  128 bits of sampler random state.

• optional uint64 num_blocks_collected = 11

  number of blocks collected so far across all tablets

• optional uint64 num_blocks_processed = 12

  number of blocks considered so far for sample across all tablets

• optional bool is_blocks_sampling_stage = 13

  Whether caller expects only sample blocks boundaries as a result of this request processing.

• optional YsqlSamplingAlgorithm sampling_algorithm = 9

• optional DocDbBlocksSamplingMethod docdb_blocks_sampling_method = 10

Instruction.

Used in: tserver.PgPerformOpPB, tserver.WriteRequestPB

• optional QLClient client = 1

  Client info

  required

• optional uint64 stmt_id = 2

  Statement info

  client request id.

• optional PgsqlWriteRequestPB.PgsqlStmtType stmt_type = 3

  required

• optional bytes table_id = 4

  Table id.

  required

• optional uint32 schema_version = 5

  Table schema version.

  required

• optional uint32 hash_code = 6

  ------------------------------------------------------------------------------------------------ Row Identifiers provides information for the row that the server INSERTs, DELETEs, or UPDATEs. - hash_code, if provided, helps locating the row. It is used by the proxy to find tablet server and by DocDB to find a set of matching rows. - partition_column_values are for partition columns, e.g. for h1 = 1 AND h2 = 2 partition_column_values would be [1, 2]. - range_column_values are for range columns. - column_values are for non-primary columns. They are used to further identify the row to be inserted, deleted, or updated. NOTE: Primary columns are defined when creating tables. These columns are different from those columns that are used as part of an INDEX.

• repeated PgsqlExpressionPB partition_column_values = 7

• repeated PgsqlExpressionPB range_column_values = 8

• optional PgsqlExpressionPB ybctid_column_value = 9

• repeated PgsqlColumnValuePB column_values = 10

  Not used with UPDATEs. Use column_new_values to UPDATE a value.

• repeated bytes packed_rows = 26

  Inserted rows in packed format. Even indexes are used to encode keys, odd indexes are used to encode values. partition_column_values, range_column_values, ybctid_column_value are ignored if packed rows are specified.

• repeated PgsqlBatchArgumentPB batch_arguments = 18

  If this attribute is present, this request is a batch request.

• repeated PgsqlColumnValuePB column_new_values = 11

  ------------------------------------------------------------------------------------------------ Column New Values. - Columns to be overwritten (UPDATE SET clause). It shouldn't contain primary-key columns.

• optional PgsqlRSRowDescPB rsrow_desc = 12

  ------------------------------------------------------------------------------------------------ Tuple descriptor for RETURNING clause. For now, we sent rsrow descriptor from proxy to tablet server for every request. RSRow is just a selected row. We call it rsrow to distinguish a selected row from a row of a table in the database in our coding.

• repeated PgsqlExpressionPB targets = 13

  required for a RETURNING clause.

• optional PgsqlExpressionPB where_expr = 14

  ------------------------------------------------------------------------------------------------ Where clause condition

• optional PgsqlColumnRefsPB column_refs = 15

  Listing of all columns that this write operation is referencing. TServers will need to read these columns when processing the write request. DEPRECATED in favor of col_refs;

• optional uint64 ysql_catalog_version = 16

  The version of the ysql system catalog this query was prepared against. Used when per-db catalog version mode is disabled.

• optional uint64 ysql_db_catalog_version = 23

  The version of the ysql per-db system catalog this query was prepared against. Used when per-db catalog version mode is enabled.

• optional uint32 ysql_db_oid = 24

  The oid of the database of the Postgres session where this query was issued. Used when per-db catalog version mode is enabled.

• optional bool is_ysql_catalog_change_using_protobuf = 17

  True only if this changes a system catalog table (or index).

• optional bool is_backfill = 19

  Does this request correspond to backfilling an index table?

• optional bool is_delete_persist_needed = 20

  For DELETE requests, do they need to be persisted in regular rocksdb? This is currently only used for DELETEs to the index when the index is getting created with index backfill enabled.

• repeated PgsqlColRefPB col_refs = 21

  Columns refereced by this write operation. Provides mapping between DocDB and Postgres identifiers, and provides Postgres type information to convert between DocDB and Postgres formats. One entry per column referenced.

• optional bytes partition_key = 22

  Used only in pg client.

• optional PgsqlFetchSequenceParamsPB fetch_sequence_params = 25

• optional bool force_catalog_modifications = 27

  During ysql major catalog upgrades, we block pg from updating the catalog data. Bypass this check when we know its safe. Valid cases are InitDB, pg_upgrade, and temp tables.

Statement types

Used in: PgsqlWriteRequestPB

• PGSQL_INSERT = 1

• PGSQL_UPDATE = 2

• PGSQL_DELETE = 3

• PGSQL_UPSERT = 4

• PGSQL_TRUNCATE_COLOCATED = 5

• PGSQL_FETCH_SEQUENCE = 6

This represents one instance of a placement constraint for a table. It is used to dictate what is the minimum number of expected replicas in a certain cloud/region/zone combo.

Used in: PlacementInfoPB

• optional CloudInfoPB cloud_info = 1

  The cloud, region and zone information for this placement block.

• optional int32 min_num_replicas = 2

  The minimum number of replicas that should always be up in this placement.

This keeps track of the set of PlacementBlockPBs defining the placement requirements for a certain table. This is used both in the on-disk storage in SysCatalog, as well as in the actual table creation calls and in the schema returned to client queries. This is tightly coupled with the overall num_replicas for a certain table, as we want to both be able to specify requirements, per placement block, but also for the overall RF factor of the table.

Used in: ReplicationInfoPB

• optional int32 num_replicas = 1

• repeated PlacementBlockPB placement_blocks = 2

• optional bytes placement_uuid = 3

• optional bytes transaction_id = 1

  16 byte uuid

• optional OpIdPB apply_op_id = 2

• optional fixed64 commit_ht = 3

• optional fixed64 log_ht = 4

Used in: PromotedFlagsPerProcessPB

• required uint32 promoted_version = 1

Used in: AutoFlagsConfigPB

• required string process_name = 1

  Ex: yb-master, yb-tserver.

• repeated string flags = 2

• repeated PromotedFlagInfoPB flag_infos = 3

  flag_infos is only populated after version 2.20. AutoFlags that were promoted before this version will be backfilled with default values.

Arbitrary protobuf that has one PB dependency.

Used in: ProtoContainerTest3PB

• required ProtoContainerTestPB record = 1

Arbitrary protobuf has two PB dependencies. dependency.

• required ProtoContainerTestPB record_one = 1

• required ProtoContainerTest2PB record_two = 2

Arbitrary protobuf to test writing a containerized protobuf.

Used in: ProtoContainerTest2PB, ProtoContainerTest3PB

• required string name = 1

• required int32 value = 2

• optional string note = 3

• repeated string str_list = 4

• repeated int32 int_list = 5

Builtin call expression. There are 3 different calls. - Builtin operators such as '>', '<', '=', ... These operators can be executed anywhere. - Builtin functions such as Now(). These functions can be executed anywhere. - Server builtin functions. Only tablet servers can execute these functions. TODO(neil) Regular builtin operators. This message can be executed anywhere. - This is more efficient than builtin call as it avoids most overheads of calling builtin lib. - Merge the current condition operator execution with this. - To optimize certain operation (such as +), replace it builtin function with builtin op.

Used in: QLExpressionPB

• optional int32 opcode = 1

• repeated QLExpressionPB operands = 2

Client type.

Used in: PgsqlLockRequestPB, PgsqlReadRequestPB, PgsqlWriteRequestPB, QLReadRequestPB, QLWriteRequestPB

• YQL_CLIENT_CQL = 1

• YQL_CLIENT_PGSQL = 2

• YQL_CLIENT_REDIS = 3

A column value, optionally with subscripts, e.g. m['x'] or l[2]['x']

Used in: QLWriteRequestPB

• optional int32 column_id = 1

• repeated QLExpressionPB subscript_args = 3

• repeated QLJsonOperationPB json_args = 4

• optional QLExpressionPB expr = 2

A logical condition that evaluates to true/false. Used in the WHERE clause.

Used in: QLExpressionPB

• optional QLOperator op = 1

• repeated QLExpressionPB operands = 2

An expression in a WHERE condition

Used in: IndexInfoPB.IndexColumnPB, IndexInfoPB.WherePredicateSpecPB, PgsqlAdvisoryLockPB, QLBCallPB, QLColumnValuePB, QLConditionPB, QLJsonOperationPB, QLReadRequestPB, QLSubscriptedColPB, QLTupleExpressionPB, QLWriteRequestPB

• oneof expr

  • QLValuePB value = 1

  • int32 column_id = 2

  • QLSubscriptedColPB subscripted_col = 3

  • int32 bind_id = 4

    Bind variable index.

  • QLConditionPB condition = 5

    This should be replaced with builtin operator.

  • QLBCallPB bfcall = 6

    Regular builtin calls.

  • QLBCallPB tscall = 7

    Tablet server builtin calls.

  • QLBCallPB bocall = 8

    Builtin operator calls.

  • QLJsonColumnOperationsPB json_column = 9

    Json column operators.

  • QLTupleExpressionPB tuple = 10

    Tuple expression in LHS.

Represents operations applied to a json column.

Used in: QLExpressionPB

• optional int32 column_id = 1

• repeated QLJsonOperationPB json_operations = 2

Used in: ColumnSchemaPB, QLColumnValuePB, QLJsonColumnOperationsPB

• required JsonOperatorPB json_operator = 1

• required QLExpressionPB operand = 2

Used in: QLValuePB

• repeated QLValuePB keys = 1

• repeated QLValuePB values = 2

Expression operators.

Used in: PgsqlConditionPB, QLConditionPB

• QL_OP_NOOP = 0

• QL_OP_NOT = 1

  Logic operators that take one operand.

• QL_OP_IS_TRUE = 2

• QL_OP_IS_FALSE = 3

• QL_OP_AND = 4

  Logic operators that take two or more operands.

• QL_OP_OR = 5

• QL_OP_IS_NULL = 6

  Relation operators that take one operand.

• QL_OP_IS_NOT_NULL = 7

• QL_OP_EQUAL = 8

  Relation operators that take two operands.

• QL_OP_LESS_THAN = 9

• QL_OP_LESS_THAN_EQUAL = 10

• QL_OP_GREATER_THAN = 11

• QL_OP_GREATER_THAN_EQUAL = 12

• QL_OP_NOT_EQUAL = 13

• QL_OP_LIKE = 14

• QL_OP_NOT_LIKE = 15

• QL_OP_IN = 16

• QL_OP_NOT_IN = 17

• QL_OP_CONTAINS = 22

• QL_OP_CONTAINS_KEY = 23

• QL_OP_BETWEEN = 18

  Relation operators that take three operands.

• QL_OP_NOT_BETWEEN = 19

• QL_OP_EXISTS = 20

  Operators that take no operand. For use in "if" clause only currently.

  IF EXISTS

• QL_OP_NOT_EXISTS = 21

  IF NOT EXISTS

Used in: QLReadRequestPB, QLResponsePB

• optional bytes table_id = 1

  Table UUID to verify the same table still exists when continuing in the next fetch.

• optional bytes next_partition_key = 2

  Partition key to find the tablet server of the next row to read.

• optional bytes next_row_key = 3

  The row key (SubDocKey = [DocKey + HybridTimestamp]) of the next row to read.

• optional uint64 total_num_rows_read = 4

  Running total number of rows read across fetches so far. Needed to ensure we read up to the number of rows in the SELECT's LIMIT clause across fetches.

• optional uint64 next_partition_index = 5

  For selects with IN condition on the hash columns there are multiple partitions that need to be queried, one for each combination of allowed values for the hash columns. This holds the index of the next partition and is used to resume the read from the right place.

• optional uint64 total_rows_skipped = 6

  The number of valid rows that we've skipped so far. This is needed to properly implement SELECT's OFFSET clause.

• optional uint64 original_request_id = 7

  Request id of the first request, that started paging.

• optional ReadHybridTimePB read_time = 8

  Used read time.

• optional QLSelectRowCounterPB row_counter = 9

  SELECT row counters. NOTE: - The select-row-counter is always applied to the entire SELECT statement. - The rest of the counters might be applied to just one batch. When indexed query returns data in batches, and these counters would be for just one specific batch. Example on nested query. SELECT <data> FROM <table> WHERE <keys> IN (SELECT <keys> FROM <index> WHERE hash IN <set>) - The <row_counter> is used for query "SELECT <data>". - The rest of the counters are for nested query "SELECT <keys>".

• optional uint32 schema_version = 10

  Schema version: - A query is prepared for a specific schema version. - A paging state of a query execution is for a specific schema version. - When schema version changes, we can either continue or raise error for the execution. This is controlled by the g-flag 'cql_check_table_schema_in_paging_state'. The flag is 'true' by default. It means: raise error in the case to prevent data reinterpretation on the driver side. Example: - Prepare queryX = "SELECT a, b FROM t" - Execute queryX - get page1. - Alter t column c; -> This alter shouldn't affect the query on columns a & b. - Execute queryX (continue) -> Should this be an error? Yes, raise error if '--cql_check_table_schema_in_paging_state=true'.

QLRSColDesc is the descriptor of a SELECT'ed column in a ResultSet, which can be any expression and not just table columns.

Used in: QLRSRowDescPB

• optional string name = 1

• optional QLTypePB ql_type = 2

Descriptor of a row in a resultset.

Used in: QLReadRequestPB

• repeated QLRSColDescPB rscol_descs = 1

TODO(neil) The protocol for select needs to be changed accordingly when we introduce and cache execution plan in tablet server.

Used in: tserver.ReadRequestPB

• optional QLClient client = 1

  Client info

  required

• optional uint64 request_id = 2

  client request id - for debug tracing purpose only

• optional uint32 schema_version = 3

  Table schema version

  required

• optional uint32 hash_code = 4

  Hashed key of row(s) to read - all fields required. The hashed column values must be in the same order as the column order in the table schema. If only a subset of hash columns are specified in the WHERE clause of the SELECT statement, "hashed_column_values" will be empty and we will do a full-table query across tablets.

• repeated QLExpressionPB hashed_column_values = 5

• optional bool is_forward_scan = 18

• optional QLExpressionPB where_expr = 6

  Where clause condition

• optional QLExpressionPB if_expr = 21

  If clause condition

• optional QLRSRowDescPB rsrow_desc = 17

  TODO(neil) Currently, we need only the datatypes of the rsrow descriptor. However, when we optimize our execution pipeline to bypass QL layer, we might need to send the name as part of the prepared protobuf so that server know how to form result set without the help from QL. For now, we sent rsrow descriptor from proxy to tablet server for every request. RSRow is just a selected row. We call it rsrow to distinguish a selected row from a row of a table in the database in our coding.

  required.

• repeated QLExpressionPB selected_exprs = 7

  required.

• optional bool distinct = 12

  Reading distinct columns?

• optional uint64 prefix_length = 23

  Current only used on SELECT DISTINCT scan. If the value is greater than 0, use the specified prefix length to scan the table.

• optional uint64 limit = 8

  Limit number of rows to return. For QL SELECT, this limit is the smaller of the page size (max (max number of rows to return per fetch) & the LIMIT clause if present in the SELECT statement.

• optional uint64 offset = 9

  The offset from which we should start returning rows. This is primarily used to support the QL OFFSET clause.

• optional QLPagingStatePB paging_state = 10

  Paging state retrieved from the last response.

• optional bool return_paging_state = 11

  Return paging state when "limit" number of rows are returned? In case when "limit" is the page size, this is set for QLResponsePB to return the paging state for the next fetch.

• optional HostPortPB remote_endpoint = 13

  The remote endpoint sending this request. This is filled in by the server and should not be set.

• optional string proxy_uuid = 20

  If this request comes from proxy, it should be filled with UUID of this proxy.

• optional uint32 max_hash_code = 14

  the upper limit for partition (hash) key scan ranges (inclusive)

• optional QLReferencedColumnsPB column_refs = 15

  Listing of all columns that this operation is referencing. TServers will need to read these columns when processing this read request.

• optional int64 query_id = 16

  Id used to track different queries.

• optional bool is_aggregate = 19

  Flag for reading aggregate values.

• optional bool is_compatible_with_previous_version = 22

• optional AshMetadataPB ash_metadata = 24

These columns must be read by DocDB before a read or write request can be executed.

Used in: QLReadRequestPB, QLWriteRequestPB

• repeated int32 ids = 1

• repeated int32 static_ids = 2

Used in: tserver.ReadResponsePB, tserver.WriteResponsePB

• optional QLResponsePB.QLStatus status = 1

  Status and error message

  required

• optional string error_message = 2

• repeated ColumnSchemaPB column_schemas = 3

  Schema of the rows returned if present (used by conditional DML (write) request only as of Jan 2017).

• optional int32 rows_data_sidecar = 4

  Sidecar of rows data returned

• optional QLPagingStatePB paging_state = 5

  Paging state for continuing the read in the next QLReadRequestPB fetch.

• optional ChildTransactionResultPB child_transaction_result = 6

  Result of child transaction.

• optional bool applied = 7

  For conditional DML: indicate if the DML is applied or not according to the conditions.

Response status

Used in: QLResponsePB

• YQL_STATUS_OK = 0

• YQL_STATUS_SCHEMA_VERSION_MISMATCH = 1

• YQL_STATUS_RUNTIME_ERROR = 2

• YQL_STATUS_USAGE_ERROR = 3

• YQL_STATUS_RESTART_REQUIRED_ERROR = 4

• YQL_STATUS_QUERY_ERROR = 5

Paging state for continuing a read request. For a SELECT statement that returns many rows, the client may specify how many rows to return at most in each fetch. This paging state maintains the state for returning the next set of rows of the statement. This paging state is opaque to the client. When there should be more rows to return from the same tablet in the next fetch, "next_row_key" is populated in DocDB (QLReadOperation) with the DocKey of the next row to read. We also embed a hybrid-time which is the clean snapshot time for read consistency. We also populate the "next_partition_key" for the next row, which is the hash code of the hash portion of the DocKey. This next partition key is needed by YBClient (Batcher) to locate the tablet to send the request to and it doesn't have access to the DocDB function to decode and extract from the DocKey. When we are done returning rows from the current tablet and the next fetch should continue in the next tablet (possible only for full-table query across tablets), "next_partition_key" is populated by the current tablet with its exclusive partition-end key, which is the start key of next tablet's partition. "next_row_key" is empty in this case which means we will start from the very beginning of the next tablet. (TODO: we need to return the clean snapshot time in this case also). RowCounter message. - These state variables are exchanged between users and CQL server when query result is paged. - DocDB does not use these.

Used in: QLPagingStatePB

• optional int64 select_limit = 1

  The following counters represent LIMIT and OFFSET clause in SELECT and their status. SELECT ... FROM table LIMIT xxx OFFSET yyy; - When a field value == -1, that field is not used. - When a field value >= 0, it is valid. - Example: If "select_limit == 0", users are querying zero rows.

• optional int64 select_offset = 2

• optional int64 read_count = 3

• optional int64 skip_count = 4

Sequence of values used to represent Lists, Sets and Tuples

Used in: QLValuePB

• repeated QLValuePB elems = 1

Reference to a subcolumn, e.g. m['x'] or l[2]['x']

Used in: QLExpressionPB

• optional int32 column_id = 1

• repeated QLExpressionPB subscript_args = 2

Tuple expression in LHS of IN operation (in future for =, >, < too).

Used in: PgsqlExpressionPB, QLExpressionPB

• repeated QLExpressionPB elems = 1

Used in: ColumnSchemaPB, DatumMessagePB, PgsqlRSColDescPB, QLRSColDescPB, cdc.CDCSDKColumnInfoPB, master.CreateUDTypeRequestPB, master.SysUDTypeEntryPB, master.UDTypeInfoPB

• optional PersistentDataType main = 1

• repeated QLTypePB params = 2

• optional QLTypePB.UDTypeInfo udtype_info = 3

Fields for user-defined types

Used in: QLTypePB

• optional string keyspace_name = 1

• optional string name = 2

• optional string id = 3

• repeated string field_names = 4

A QL value

Used in: ColumnSchemaPB, DatumMessagePB, PgVectorReadOptionsPB, PgsqlExpressionPB, QLExpressionPB, QLMapValuePB, QLSeqValuePB, cdc.KeyValuePairPB, tserver.PgCreateColumnPB, tserver.PgCreateSplitBoundaryPB

• oneof value

  Note: an absent value means NULL

  • int32 int8_value = 1

    Note: min int size in protobuf is int32

  • int32 int16_value = 2

  • int32 int32_value = 3

  • int64 int64_value = 4

  • float float_value = 5

  • double double_value = 6

  • bytes string_value = 7

    Sometimes YB sends non-UTF8 string in string field. For example, this can happen with a collated string: under "en_US.utf8" collation, string 'abc' is encoded as string_value: "\000\201\014\r\016\001\t\t\t\001\t\t\t\000abc" which is not a valid UTF8 string. That is why 'bytes' is used for string_value field.

  • bool bool_value = 8

  • int64 timestamp_value = 9

  • bytes binary_value = 10

  • bytes inetaddress_value = 11

    raw bytes for inet address in network byte order.

  • QLMapValuePB map_value = 12

  • QLSeqValuePB set_value = 13

  • QLSeqValuePB list_value = 14

  • bytes decimal_value = 15

  • bytes varint_value = 16

  • QLSeqValuePB frozen_value = 17

  • bytes uuid_value = 18

    raw bytes for uuid value.

  • bytes timeuuid_value = 19

    raw bytes for timeuuid value.

  • bytes jsonb_value = 20

  • uint32 date_value = 21

    Number of days where 2^31 corresponds to 1970-01-01 (see DateTime::DateFromString)

  • int64 time_value = 22

    Number of nano-seconds, from 0 to (24 * 60 * 60 * 1,000,000,000 - 1) (see DateTime::kMaxTime)

  • uint32 uint32_value = 23

  • uint64 uint64_value = 24

  • QLVirtualValuePB virtual_value = 25

    Represent various system values.

  • uint32 gin_null_value = 26

    Null category for GIN indexes.

  • QLSeqValuePB tuple_value = 27

  • bytes bson_value = 28

    raw bytes for bson value.

-------------------------------------------------------------------------------------------------- YQL support. The following section is common for all query language. -------------------------------------------------------------------------------------------------- Represent system internal values.

Used in: QLValuePB

• LIMIT_MAX = 1

• LIMIT_MIN = 2

• COUNTER = 3

• SS_FORWARD = 4

• SS_REVERSE = 5

• TOMBSTONE = 6

• NULL_LOW = 7

• ARRAY = 8

Used in: tserver.WriteRequestPB

• optional QLWriteRequestPB.QLStmtType type = 1

  Statement type

  required

• optional QLClient client = 2

  Client info

  required

• optional uint64 request_id = 3

  client request id - for debug tracing purpose only

• optional uint32 schema_version = 4

  Table schema version

  required

• optional uint32 hash_code = 5

  Primary key of the row to insert/update/delete - all fields required. The hashed and range column values must be in the same order as the column order in the table schema. Note: the hash_code is the hash of the hashed_column_values. Technically, this can be recomputed by tserver also, but since the client already calculates this to look up the correct tablet server, it is passed in.

• repeated QLExpressionPB hashed_column_values = 6

• repeated QLExpressionPB range_column_values = 7

• repeated QLColumnValuePB column_values = 8

  Column values to insert/update/delete - required Note: DELETE statement has no column value.

• optional QLExpressionPB where_expr = 14

  Where clause condition -- currently this is only allowed for deletes.

• optional QLExpressionPB if_expr = 9

  If clause condition

• optional bool else_error = 17

  Else error clause

• optional uint64 ttl = 10

  Time to live in milliseconds.

• optional QLReferencedColumnsPB column_refs = 11

  Listing of all columns that this write operation is referencing. TServers will need to read these columns when processing the write request.

• optional int64 query_id = 12

  Id used to track different queries.

• optional uint64 user_timestamp_usec = 13

  User-provided timestamp in microseconds.

• repeated bytes update_index_ids = 15

  Ids of indexes that need update.

• optional ChildTransactionDataPB child_transaction_data = 16

  Child transaction data for additional write requests necessary for index updates, etc.

• optional bool returns_status = 18

  Whether to return a status row reporting applied status and execution errors (if any).

• optional bool is_backfill = 19

  Does this request correspond to backfilling an index table?

• optional bool is_compatible_with_previous_version = 20

• optional AshMetadataPB ash_metadata = 21

Statement types

Used in: QLWriteRequestPB

• QL_STMT_INSERT = 1

• QL_STMT_UPDATE = 2

• QL_STMT_DELETE = 3

See ReadHybridTime for explation of this message.

Used in: ChildTransactionDataPB, ChildTransactionResultPB, PgsqlPagingStatePB, QLPagingStatePB, tserver.GetLocalPgTxnSnapshotResponsePB, tserver.PgPerformOptionsPB, tserver.PgPerformResponsePB, tserver.ReadRequestPB, tserver.ReadResponsePB, tserver.WriteRequestPB, tserver.WriteResponsePB

• optional fixed64 read_ht = 1

• optional fixed64 DEPRECATED_max_of_read_time_and_local_limit_ht = 2

• optional fixed64 global_limit_ht = 3

• optional fixed64 in_txn_limit_ht = 4

  When a transaction reads data, all committed writes of other transactions with a timestamp <= read_ht are visible. However, a transaction's reads should be able to see all provisional writes from the same transaction irrespective of read_ht, except some based on additional constraints for YSQL (as described in src/yb/yql/pggate/README). The query layer uses in_txn_limit_ht to meet these constraints. If in_txn_limit_ht is set, any operation will not be able to read provisional writes from the same transaction which have a write timestamp higher than the in_txn_limit_ht. src/yb/yql/pggate/README also explains how this works along with YSQL's buffering logic.

• optional fixed64 local_limit_ht = 5

SADD

Used in: RedisWriteRequestPB

• optional RedisWriteMode mode = 2

  Following options are for ZADD only.

• optional bool ch = 3

  Modify the return value from the number of new elements added, to the total number of elements changed.

APPEND

Used in: RedisWriteRequestPB

(message has no fields)

Used in: RedisResponsePB

• repeated bytes elements = 1

• optional bool encoded = 2

Used in: RedisReadRequestPB

• optional RedisCollectionGetRangeRequestPB.GetRangeRequestType request_type = 1

• optional bool with_scores = 2

  Used only with ZRANGEBYSCORE, ZREVRANGE.

Used in: RedisCollectionGetRangeRequestPB

• TSRANGEBYTIME = 1

• ZRANGEBYSCORE = 2

• ZREVRANGE = 3

• ZRANGE = 4

• TSREVRANGEBYTIME = 5

• UNKNOWN = 99

Used in: RedisKeyValuePB, RedisResponsePB

• REDIS_TYPE_NONE = 1

• REDIS_TYPE_STRING = 2

• REDIS_TYPE_LIST = 3

• REDIS_TYPE_SET = 4

• REDIS_TYPE_HASH = 5

• REDIS_TYPE_SORTEDSET = 6

• REDIS_TYPE_TIMESERIES = 7

DEL, HDEL

Used in: RedisWriteRequestPB

(message has no fields)

EXISTS, HEXISTS

Used in: RedisReadRequestPB

(message has no fields)

RENAME

Used in: RedisReadRequestPB

(message has no fields)

GETRANGE

Used in: RedisReadRequestPB

• optional int32 start = 2

  Required

• optional int32 end = 3

  Required

GET, HGET, MGET, HMGET, HGETALL, SMEMBERS HKEYS, HKEYS, HLEN

Used in: RedisReadRequestPB

• optional RedisGetRequestPB.GetRequestType request_type = 1

Used in: RedisGetRequestPB

• GET = 1

• MGET = 3

• HGET = 2

• HMGET = 4

• HGETALL = 5

• HKEYS = 7

• HVALS = 8

• HLEN = 9

• HEXISTS = 10

• HSTRLEN = 11

• SMEMBERS = 6

• SISMEMBER = 12

• SCARD = 13

• ZCARD = 15

• TSGET = 14

• TSCARD = 16

• ZSCORE = 17

• LLEN = 18

• UNKNOWN = 99

GETSET

Used in: RedisWriteRequestPB

(message has no fields)

TTL, PTTL

Used in: RedisReadRequestPB

• optional bool return_seconds = 1

  By default, the return value is in milliseconds.

INCR, INCRBY, INCRBYFLOAT, HINCRBY, HINCRBYFLOAT, ZINCRBY, DECR, DECRBY, ZADD with incr option

Used in: RedisWriteRequestPB

• optional int64 increment_int = 2

• optional double increment_float = 3

Used in: RedisIndexRangePB

• required int64 index = 1

Used in: RedisReadRequestPB

• optional RedisIndexBoundPB lower_bound = 1

• optional RedisIndexBoundPB upper_bound = 2

LINSERT

Used in: RedisWriteRequestPB

• optional RedisOrder order = 2

  Required

- Even if just a key is needed, or there are multiple values, this is used. - In case of referring to an entity within a container, the type of the outer_key and the subkey or index of the inner entity is specified. - String : Set the key and value only (setting STRING type is optional as it is understood). - List : Set the key, index, and value. - Set : Set the key, and value (possibly multiple depending on the command). - Hash : Set key, subkey, value. - SortedSet : Set key, subkey, value (value is interpreted as score). - Timeseries: Set key, subkey, value (timestamp_subkey in RedisKeyValueSubKeyPB is interpreted as timestamp). - Value is not present in case of an append, get, exists, etc. For multiple inserts into a container, the subkey and value fields have one or more items.

Used in: RedisReadRequestPB, RedisWriteRequestPB

• optional uint32 hash_code = 1

  Note: the hash_code is the hash of the 'key' below. Technically, this can be recomputed by tserver also, but since the client already calculates this to look up the correct tablet server, it is passed in.

• optional bytes key = 2

  Required

• optional RedisDataType type = 3

• repeated RedisKeyValueSubKeyPB subkey = 4

  It is assumed that all subkeys are distinct for HMSET and SADD. For collection range requests, we would have exactly two subkeys denoting the lower bound and upper bound for the range request.

• optional int32 index = 5

• repeated bytes value = 6

Used in: RedisKeyValuePB, RedisSubKeyBoundPB

• oneof subkey

  • bytes string_subkey = 1

  • int64 timestamp_subkey = 2

    Timestamp used in the redis timeseries datatype.

  • double double_subkey = 3

    Double used in redis sorted set datatype.

KEYS

Used in: RedisReadRequestPB

• optional string pattern = 1

• optional int32 threshold = 2

No operation.

Used in: RedisWriteRequestPB

(message has no fields)

This enum is used to specify the insertion position (Insert after or before index).

Used in: RedisInsertRequestPB

• REDIS_ORDER_BEFORE = 1

• REDIS_ORDER_AFTER = 2

LPOP, RPOP, SPOP; blocking versions BLPOP etc. currently not supported.

Used in: RedisWriteRequestPB

• optional RedisSide side = 2

• optional int32 count = 3

  Count is allowed only when popping from a set.

RPUSH, RPUSHX, LPUSH, LPUSHX

Used in: RedisWriteRequestPB

• optional RedisSide side = 2

• optional bool check_existence = 3

Used in: tserver.ReadRequestPB

• oneof request

  • RedisGetRequestPB get_request = 2

  • RedisStrLenRequestPB strlen_request = 3

  • RedisExistsRequestPB exists_request = 4

  • RedisGetRangeRequestPB get_range_request = 5

  • RedisCollectionGetRangeRequestPB get_collection_range_request = 9

  • RedisGetTtlRequestPB get_ttl_request = 11

  • RedisKeysRequestPB keys_request = 12

  • RedisGetForRenameRequestPB get_for_rename_request = 13

• optional RedisKeyValuePB key_value = 6

• optional RedisSubKeyRangePB subkey_range = 7

• optional RedisIndexRangePB index_range = 8

• optional int32 range_request_limit = 10

  The maximum number of entries to retrieve for a range request.

Used in: tserver.ReadResponsePB, tserver.WriteResponsePB

• optional RedisResponsePB.RedisStatusCode code = 1

• oneof response_data

  • int64 int_response = 2

  • bytes string_response = 3

  • RedisArrayPB array_response = 4

  • bytes status_response = 5

  • bytes encoded_response = 7

• optional bytes error_message = 6

• optional RedisDataType type = 8

Used in: RedisResponsePB

• OK = 1

• NIL = 2

  Nil is when value is not found, but isn't an error case, eg. GET key when key has no value.

• NOT_FOUND = 5

  Not found is an error when an existing value is needed for the command, eg. RENAME.

• UNKNOWN_COMMAND = 3

• WRONG_TYPE = 4

• SERVER_ERROR = 6

  This code is set in the client to mark commands for which no response was received. This happens when the batch RPC fails somewhere and the individual requests don't get processed.

• INDEX_OUT_OF_BOUNDS = 7

• PARSING_ERROR = 8

SETRANGE

Used in: RedisWriteRequestPB

• optional int32 offset = 2

  Required

SET, SETNX, SETXX, HSET, HSETNX, LSET, MSET, HMSET, MSETNX

Used in: RedisWriteRequestPB

• optional bool expect_ok_response = 1

• optional int64 ttl = 2

  Expiration time in milliseconds.

• optional RedisWriteMode mode = 3

• optional SortedSetOptionsPB sorted_set_options = 14

PERSIST, (P)EXPIRE, (P)EXPIREAT

Used in: RedisWriteRequestPB

• oneof expiration

  Expiration time in milliseconds.

  • int64 ttl = 1

    For (P)EXPIRE

  • int64 absolute_time = 2

    For (P)EXPIREAT

This enum is used to specify the side of a list (For LPOP or RPOP etc.).

Used in: RedisPopRequestPB, RedisPushRequestPB

• REDIS_SIDE_LEFT = 1

• REDIS_SIDE_RIGHT = 2

STRLEN, HSTRLEN

Used in: RedisReadRequestPB

(message has no fields)

Wrapper for a subkey which denotes an upper/lower bound for a range request.

Used in: RedisSubKeyRangePB

• optional RedisKeyValueSubKeyPB subkey_bound = 3

• optional RedisSubKeyBoundPB.InfinityType infinity_type = 4

• optional bool is_exclusive = 6

Used in: RedisSubKeyBoundPB

• POSITIVE = 1

• NEGATIVE = 2

Used in: RedisReadRequestPB

• optional RedisSubKeyBoundPB lower_bound = 1

• optional RedisSubKeyBoundPB upper_bound = 2

Used in: RedisAddRequestPB, RedisSetRequestPB

• REDIS_WRITEMODE_UPSERT = 1

• REDIS_WRITEMODE_INSERT = 2

• REDIS_WRITEMODE_UPDATE = 3

A single Redis request. Some user commands like MGET should be split into multiple of these.

Used in: tserver.WriteRequestPB

• oneof request

  • RedisSetRequestPB set_request = 2

    Only one of the following fields should be set.

  • RedisGetSetRequestPB getset_request = 3

  • RedisAppendRequestPB append_request = 4

  • RedisDelRequestPB del_request = 5

  • RedisSetRangeRequestPB set_range_request = 6

  • RedisIncrRequestPB incr_request = 7

  • RedisPushRequestPB push_request = 8

  • RedisInsertRequestPB insert_request = 9

  • RedisPopRequestPB pop_request = 10

  • RedisAddRequestPB add_request = 11

  • RedisSetTtlRequestPB set_ttl_request = 12

  • RedisNoOpRequestPB no_op_request = 14

• optional RedisKeyValuePB key_value = 13

Used in: master.ReplicationStatusErrorPB, master.XClusterConsumerStreamTabletStatusPB

• REPLICATION_UNKNOWN_ERROR = 0

• REPLICATION_MISSING_OP_ID = 1

  One or more WAL segments are missing an expected op id.

• REPLICATION_SCHEMA_MISMATCH = 2

  There is a schema mismatch between the producer and consumer tables.

• REPLICATION_MISSING_TABLE = 3

  There is a colocated table missing on the consumer side.

• REPLICATION_ERROR_UNINITIALIZED = 4

  Replication state has not yet been initialized.

• REPLICATION_OK = 5

  Replication is healthy.

• REPLICATION_AUTO_FLAG_CONFIG_VERSION_MISMATCH = 6

  The AutoFlags config has changed and the new version is not compatible, or has not yet been validated.

• REPLICATION_SOURCE_UNREACHABLE = 7

  Error connecting to the source universe.

• REPLICATION_SYSTEM_ERROR = 8

  There was a generic system error.

• REPLICATION_PAUSED = 9

  The replication stream has been paused by the user.

Higher level structure to keep track of all types of replicas configured. This will have, at a minimum, the information about the replicas that are supposed to be active members of the raft configs, but can also include extra information, such as read only replicas.

Used in: master.AlterTableRequestPB, master.CreateTableRequestPB, master.CreateTransactionStatusTableRequestPB, master.GetTableSchemaResponsePB, master.ListTabletServersRequestPB, master.SysClusterConfigEntryPB, master.SysTablesEntryPB, master.ValidateReplicationInfoRequestPB, tserver.PgValidatePlacementRequestPB

• optional PlacementInfoPB live_replicas = 1

• repeated PlacementInfoPB read_replicas = 2

• repeated CloudInfoPB affinitized_leaders = 3

• repeated CloudInfoListPB multi_affinitized_leaders = 4

The type of LSN type which will be used with the replication slot and stream metadata.

Used in: CDCSDKStreamCreateOptionsPB, master.CDCStreamInfoOptionsPB, master.SysCDCStreamEntryPB

• ReplicationSlotLsnType_UNSPECIFIED = 0

• ReplicationSlotLsnType_SEQUENCE = 1

• ReplicationSlotLsnType_HYBRID_TIME = 2

The type of ordering mode to be used while using logical replication.

Used in: CDCSDKStreamCreateOptionsPB, master.CDCStreamInfoOptionsPB, master.SysCDCStreamEntryPB

• ReplicationSlotOrderingMode_UNSPECIFIED = 0

• ReplicationSlotOrderingMode_ROW = 1

• ReplicationSlotOrderingMode_TRANSACTION = 2

Used for Cassandra Roles and Permissions

Used in: master.GrantRevokePermissionRequestPB, master.ResourcePermissionsPB

• ALL_KEYSPACES = 1

• KEYSPACE = 2

• TABLE = 3

• ALL_ROLES = 4

• ROLE = 5

This enum matches enum RowMarkType defined in src/include/nodes/plannodes.h. The exception is ROW_MARK_ABSENT, which signifies the absence of a row mark.

Used in: PgsqlReadRequestPB, docdb.KeyValueWriteBatchPB

• ROW_MARK_EXCLUSIVE = 0

  Obtain exclusive tuple lock.

• ROW_MARK_NOKEYEXCLUSIVE = 1

  Obtain no-key exclusive tuple lock.

• ROW_MARK_SHARE = 2

  Obtain shared tuple lock.

• ROW_MARK_KEYSHARE = 3

  Obtain keyshare tuple lock.

• ROW_MARK_REFERENCE = 4

  Not supported. Used for postgres compatibility.

• ROW_MARK_COPY = 5

  Not supported. Used for postgres compatibility.

• ROW_MARK_ABSENT = 15

  Obtain no tuple lock (this should never sent be on the wire). The value should be high for convenient comparisons with the other row lock types.

Used in: log.LogSegmentHeaderPB, master.CreateTableRequestPB, master.GetTableSchemaFromSysCatalogResponsePB, master.GetTableSchemaResponsePB, master.InsertHistoricalColocatedSchemaPackingRequestPB, master.InsertPackedSchemaForXClusterTargetRequestPB, master.SysTablesEntryPB, master.YsqlDdlTxnVerifierStatePB, tablet.ChangeMetadataRequestPB, tablet.CloneTabletRequestPB, tablet.RaftGroupReplicaSuperBlockPB, tablet.TableInfoPB, tserver.CreateTabletRequestPB, tserver.GetCompatibleSchemaVersionRequestPB, tserver.ListTabletsResponsePB.StatusAndSchemaPB, tserver.TableMetadataPB, tserver.TabletSnapshotOpRequestPB

• repeated ColumnSchemaPB columns = 1

• optional TablePropertiesPB table_properties = 2

• optional ColocatedTableIdentifierPB colocated_table_id = 3

• optional string pgschema_name = 4

  Added in 2.13, any upgraded cluster may not have this value so use it appropriately.

Used in: ExplainPlanPB

• optional string select_type = 1

• optional string aggregate = 2

  Overarching information like "Aggregate" or "Limit"

• optional string key_conditions = 3

• optional string filter = 4

• optional int32 output_width = 5

Used in: master.ListMastersResponsePB

• optional AppStatusPB error = 1

  If there is an error communicating with the server (or retrieving the server registration on the server itself), this field will be set to contain the error. All subsequent fields are optional, as they may not be set if an error is encountered communicating with the individual server.

• optional NodeInstancePB instance_id = 2

• optional ServerRegistrationPB registration = 3

• optional PeerRole role = 4

  If an error has occurred earlier in the RPC call, the role may be not be set.

RPC and HTTP addresses for each server, as well as cloud related information.

Used in: ServerEntryPB, consensus.GetNodeInstanceResponsePB, master.GetMasterRegistrationResponsePB, master.SysTabletServerEntryPB, master.TSRegistrationPB, tserver.BeginRemoteBootstrapSessionRequestPB

• repeated HostPortPB private_rpc_addresses = 1

• repeated HostPortPB broadcast_addresses = 5

• repeated HostPortPB http_addresses = 2

• optional uint32 pg_port = 6

• optional CloudInfoPB cloud_info = 3

• optional bytes placement_uuid = 4

  Placement uuid of the tserver's cluster.

Used in: RedisSetRequestPB

• optional SortedSetOptionsPB.UpdateOptions update_options = 4

• optional bool ch = 5

• optional bool incr = 6

Used in: SortedSetOptionsPB

• NX = 1

• XX = 2

• NONE = 3

Stateful services.

Used in: master.CreateTableRequestPB, master.GetStatefulServiceLocationRequestPB, master.SysTablesEntryPB, tablet.RaftGroupReplicaSuperBlockPB, tserver.CreateTabletRequestPB

• TEST_ECHO = 0

  Test service.

• PG_AUTO_ANALYZE = 1

• PG_CRON_LEADER = 2

Used in: docdb.KeyValueWriteBatchPB, tserver.ReadRequestPB

• optional uint32 subtransaction_id = 1

• optional SubtxnSetPB aborted = 2

Used in: SubTransactionMetadataPB, docdb.ApplyExternalTransactionPB, docdb.ApplyTransactionStatePB, tablet.TransactionStatePB, tserver.GetLockStatusRequestPB.TransactionInfoPB, tserver.GetOldTransactionsResponsePB.OldTransactionMetadataPB, tserver.GetTransactionStatusResponsePB, tserver.ProbeTransactionDeadlockRequestPB, tserver.UpdateTransactionWaitingForStatusRequestPB.BlockingTransaction

• repeated uint32 set = 1

  This is not a simple set representation, but rather the encoded output of a yb::UnsignedIntSet<uint32_t>. An ascending nonoverlapping series of integer ranges [a,b) [c,d) ... (a<=b<c<=d...) are encoded as taking a,b, c,d, ... then compressing it by storing the set of deltas starting from 0. This gives: a-0, b-a, c-b,c-d, ...

Used in: tserver.GetTserverCatalogMessageListsResponsePB, tserver.PgGetTserverCatalogMessageListsResponsePB, tserver.PgSetTserverCatalogMessageListRequestPB

• optional bytes message_list = 2

Used in: tserver.GetTserverCatalogVersionInfoResponsePB, tserver.PgGetTserverCatalogVersionInfoResponsePB

• optional uint32 db_oid = 1

• optional int32 shm_index = 2

This enum matches enum table locks defined in src/include/storage/lockdefs.h. Table level lock conflict matrix. Source: https://www.postgresql.org/docs/15/explicit-locking.html#LOCKING-TABLES

Used in: docdb.ObjectLockPB, master.SysObjectLockEntryPB.LockTypesPB

• NONE = 0

  Though NONE is listed as a lock type, it seems to be a flag conveying "don't get a lock". Despite that, if a lock request with NONE type makes it to DocDB, we return an error status.

• ACCESS_SHARE = 1

• ROW_SHARE = 2

• ROW_EXCLUSIVE = 3

• SHARE_UPDATE_EXCLUSIVE = 4

• SHARE = 5

• SHARE_ROW_EXCLUSIVE = 6

• EXCLUSIVE = 7

• ACCESS_EXCLUSIVE = 8

Used in: SchemaPB, master.AlterTableRequestPB

• optional uint64 default_time_to_live = 1

• optional bool contain_counters = 2

• optional bool is_transactional = 3

• optional YBConsistencyLevel consistency_level = 5

  For index table only: consistency with respect to the indexed table.

• optional bool use_mangled_column_name = 6

• optional int32 num_tablets = 7

  Save this number of tablets in table schema stored in master SysCatalog. It might not be the true number of tablets in the table at runtime due to tablet splitting. It is just the number of tablets stored in the schema (for example, when user creates the table). Note: If CreateTableRequestPB::num_tablets is not specified, but this field is specified in the field CreateTableRequestPB::schema, CatalogManager::CreateTable() will create a table with number of tablets in this field.

• optional bool is_ysql_catalog_table = 8

• optional bool retain_delete_markers = 9

• optional uint64 backfilling_timestamp = 10

• optional uint32 partitioning_version = 11

  Used to distinguish which algorithm should be used for partition key and bounds generation, value == 0 stands for the default buggy algorithm for range partitioning case, see #12189.

• optional PgReplicaIdentity ysql_replica_identity = 12

This must exactly match YBTableType in client.h. We have static_assert's in tablet-test.cc to verify this.

Used in: master.CreateTableRequestPB, master.DdlLogEntryPB, master.GetTableLocationsResponsePB, master.GetTableSchemaResponsePB, master.ListTablesResponsePB.TableInfo, master.SysTablesEntryPB, master.SysUniverseReplicationBootstrapEntryPB.TableDataPB, tablet.RaftGroupReplicaSuperBlockPB, tablet.TableInfoPB, tablet.TabletStatusPB, tserver.CreateTabletRequestPB

• YQL_TABLE_TYPE = 2

• REDIS_TABLE_TYPE = 3

• PGSQL_TABLE_TYPE = 4

• TRANSACTION_STATUS_TABLE_TYPE = 5

• DEFAULT_TABLE_TYPE = 2

Used in: tserver.GetLockStatusResponsePB, tserver.PgGetLockStatusResponsePB.NodeLockStatusResponsePB

• optional bytes pg_table_id = 1

  The TableId containing the PG table OID. Note: This will NOT match the DocDB table's uuid if the table was rewritten.

• optional bytes tablet_id = 2

• repeated TabletLockInfoPB.TransactionLockInfoPB transaction_locks = 3

• repeated TabletLockInfoPB.WaiterInfoPB single_shard_waiters = 4

  Awaiting lock info of single shard waiters.

• optional uint64 term = 5

  Leader term of the peer populating the response. Used to filter out duplicate responses for the same tablet and ensure we pick the most recent results.

• optional bool is_advisory_lock_tablet = 6

  Indicates whether the tablet is associated with an advisory lock table. Advisory lock table has YQL table type and require different handling compared to regular PGSQL table.

Lock info for a given transaction at this tablet.

Used in: TabletLockInfoPB

• optional bytes id = 1

• repeated LockInfoPB granted_locks = 2

• optional WaiterInfoPB waiting_locks = 3

• optional bool has_additional_granted_locks = 4

Used in: TabletLockInfoPB, TransactionLockInfoPB

• repeated LockInfoPB locks = 1

• optional fixed64 wait_start_ht = 2

• repeated bytes blocking_txn_ids = 3

• optional bool has_additional_waiting_locks = 4

• optional string string1 = 1

• optional TestStringOptionalPB record = 2

• optional string string2 = 3

• repeated string string1 = 1

• repeated TestStringRepeatedPB record = 2

• repeated string string2 = 3

• required string string1 = 1

• required TestStringRequiredPB record = 2

• required string string2 = 3

• FOO = 1

• BAR = 2

• BAZ = 3

Used in: TestObjectOptionalPB

• optional string text = 1

Used in: TestObjectRepeatedPB

• repeated string text = 1

Used in: TestObjectRequiredPB

• required string text = 1

Used in: ChildTransactionResultPB

• optional bytes tablet_id = 1

• optional InvolvedTabletMetadataState metadata_state = 2

• optional uint64 num_batches = 3

  Number of batches written to this tablet during transaction.

Used in: TransactionMetadataPB

• GLOBAL = 1

• LOCAL = 2

Used in: ChildTransactionDataPB, docdb.KeyValueWriteBatchPB, master.AlterTableRequestPB, master.CreateNamespaceRequestPB, master.CreateTableRequestPB, master.CreateTablegroupRequestPB, master.DeleteTableRequestPB, master.DeleteTablegroupRequestPB, master.IsYsqlDdlVerificationDoneRequestPB, master.SysNamespaceEntryPB, master.SysTablesEntryPB, tablet.TransactionStatePB, tserver.ReadRequestPB

• optional bytes transaction_id = 1

  16 byte uuid

• optional IsolationLevel isolation = 2

• optional string status_tablet = 3

• optional fixed64 priority = 4

• optional fixed64 start_hybrid_time = 5

  Stores the time at which this transaction was taken by pg_client_session from the transaction pool. Serves as a lower bound on the ht at which the first RPC which may write intents for this transaction would have been issued to a tserver.

• optional int64 pg_txn_start_us = 9

  Stores the time at which the transaction was initiated from postgres' perspective.

• optional fixed64 metadata_write_time = 6

  Stores time when metadata was written to provisional records RocksDB on a participating tablet. So it could be used for cleanup.

• optional TransactionLocality locality = 7

  Stores whether the transaction is local or global; treated as global if not present for backwards compatability.

Used in: tablet.TransactionStatePB, tserver.AbortTransactionResponsePB, tserver.GetTransactionStatusResponsePB

• CREATED = 1

• PENDING = 2

• COMMITTED = 4

• SEALED = 5

• APPLIED_IN_ALL_INVOLVED_TABLETS = 7

• ABORTED = 8

• PROMOTED = 9

• APPLYING = 20

  The APPLYING and PROMOTING statuses are used in Raft in transaction participant tablets but not in status tablets.

• PROMOTING = 24

• APPLIED_IN_ONE_OF_INVOLVED_TABLETS = 21

  All following entries are not used in RAFT, but as events between status tablet and involved tablets:

• IMMEDIATE_CLEANUP = 22

• GRACEFUL_CLEANUP = 23

Used in: InstanceMetadataPB

• optional string universe_uuid = 1

  The universe_uuid received from the master leader on the first heartbeat after creation or an upgrade. This is the identifier for the cluster this server belongs to. It is passed in every heartbeat request and compared with the value on master leader to ensure a match. Only set on the tserver.

Used in: tserver.GetMetricsResponsePB, tserver.PgServerMetricsInfoPB

• required string name = 1

• required string value = 2

• repeated uint32 set = 1

Used in: ExplainPlanPB

• optional string update_type = 1

• optional string scan_type = 2

  For update, scan_type is currently only primary key lookup.

• optional string key_conditions = 3

• optional int32 output_width = 4

Information about the build environment, configuration, etc.

Used in: master.SysTabletServerEntryPB, master.TSRegistrationPB, server.ServerStatusPB

• optional string git_hash = 1

• optional string build_hostname = 2

• optional string build_timestamp = 3

• optional string build_username = 4

• optional bool build_clean_repo = 5

• optional string build_id = 6

• optional string build_type = 7

• optional string version_number = 9

• optional string build_number = 10

• optional uint32 ysql_major_version = 11

This enum matches enum LockWaitPolicy defined in src/include/nodes/lockoptions.h and applies only to explicit row-level locking policies. NOTE: WAIT_ERROR implies no-wait semantics in Postgres i.e., if a conflicting lock is held, fail self's transaction. However, in YSQL, the WAIT_ERROR enum maps to the Fail-on-Conflict conflict management policy (where we may fail other transactions depending on priority, as described in conflict_resolution.h). It doesn't map to no-wait semantics because they aren't yet supported by docdb. But, note that NOWAIT clause is supported for READ COMMITTED isolation level when no other isolation's transactions are active. YSQL acheives that without relying on no-wait semantics from the tserver. This is done by leveraging the fact that the Fail-on-Conflict policy for read committed transactions are effectively no-wait semantics because all read committed transactions have the same kHighest priority. TODO(concurrency-control): When NOWAIT support is to be extended for REPEATABLE READ and SERIALIZABLE isolation level, we would require using WAIT_ERROR to rightly mean the no-wait policy and another enum to represent the Fail-on-Conflict policy. YSQL specifies WaitPolicy explicitly only for reads via PgsqlReadRequestPB that take explicit row-level locks.

Used in: PgsqlReadRequestPB, docdb.KeyValueWriteBatchPB

• WAIT_BLOCK = 0

  Maps to Wait-on-Conflict conflict management policy as defined in conflict_resolution.h

• WAIT_SKIP = 1

  Maps to Skip-on-Conflict conflict management policy as defined in conflict_resolution.h This is needed for the SKIP LOCKED clause

• WAIT_ERROR = 2

  Maps to Fail-on-Conflict conflict management policy as defined in conflict_resolution.h

Represents the wait-state tracked for a given incoming rpc/request.

Used in: rpc.RpcCallInProgressPB, tserver.WaitStatesPB

• optional AshMetadataPB metadata = 1

  The metadata is rarely modified.

• optional uint32 wait_state_code = 2

  Represents the state of the curreent rpc/request when it was polled.

• optional AshAuxDataPB aux_info = 3

  Some additional information that may be persisted as yb_active_universe_history.wait_event_aux

• optional string wait_state_code_as_string = 4

  A string representation of wait_state_code for easy human consumption.

Information for a namespace currently/previously under automatic mode xCluster replication.

Used in: master.SysXClusterConfigEntryPB, master.TSHeartbeatResponsePB.XClusterHeartbeatInfo

• optional XClusterNamespaceInfoPB.XClusterRole role = 1

Used in: XClusterNamespaceInfoPB

• UNSPECIFIED = 0

  This should never occur; not stored.

• UNAVAILABLE = 1

  Used to denote that we cannot determine the role; not stored.

• NOT_AUTOMATIC_MODE = 2

  No automatic mode replication is occurring for this namespace.

• AUTOMATIC_SOURCE = 3

• AUTOMATIC_TARGET = 4

Used in: master.GetUniverseReplicationInfoResponsePB

• XCLUSTER_NON_TRANSACTIONAL = 0

  YCQL and YSQL. Unidirectional and Bidirectional. Table level.

• XCLUSTER_YSQL_TRANSACTIONAL = 1

  YSQL, Unidirectional only. Table level.

• XCLUSTER_YSQL_DB_SCOPED = 2

  YSQL, Transactional and Unidirectional only. DB level.

Used in: TablePropertiesPB, tserver.ChecksumRequestPB, tserver.ReadRequestPB

• STRONG = 1

  This consistency level provides Linearizability guarantees and is the default for our system.

• CONSISTENT_PREFIX = 2

  Consistent-prefix consistency means that we always see consistent snapshot of the database in a well-defined order. If operations A, B and C take place, we will either see A, AB, or ABC. Note that reads might still go back in time since we might see ABC on one replica and AB on another.

• USER_ENFORCED = 3

  For cross-shard transactions only: user-enforced consistency level means it is the user's responsibility to enforce consistency across shards or tables/indexes.

Client type.

Used in: cdc.CreateCDCStreamRequestPB, master.BootstrapProducerRequestPB, master.CreateNamespaceRequestPB, master.DeleteNamespaceRequestPB, master.ListNamespacesRequestPB, master.NamespaceIdentifierPB, master.SysCloneStatePB, master.SysNamespaceEntryPB, master.SysUniverseReplicationBootstrapEntryPB.NamespaceDataPB

• YQL_DATABASE_UNKNOWN = 1

• YQL_DATABASE_CQL = 2

• YQL_DATABASE_PGSQL = 3

  Pgsql database

• YQL_DATABASE_REDIS = 4

See yb_sampling_algorithm flag description. Should be in sync with YbcSamplingAlgorithmEnum in GUC (ybc_util.h).

Used in: PgsqlSamplingStatePB

• FULL_TABLE_SCAN = 0

• BLOCK_BASED_SAMPLING = 1