package tipb

Get desktop application:
View/edit binary Protocol Buffers messages

TopSQLAgent is the persistent agent service for TopSQL records

• rpc ReportPlanMeta (stream PlanMeta, EmptyResponse)

  topsql_agent.proto:28

  ReportPlanMeta reports plan meta to the agent. The agent should deal with plan meta similarly to SQL meta.

• rpc ReportSQLMeta (stream SQLMeta, EmptyResponse)

  topsql_agent.proto:25

  ReportSQLMeta reports SQL meta to the agent. The agent should ensure that the SQL meta exists before sending the SQL CPU time records to the remote database.

• rpc ReportTopRURecords (stream TopRURecord, EmptyResponse)

  topsql_agent.proto:30

  ReportTopRURecords is called periodically to save the in-memory TopRU records.

• rpc ReportTopSQLRecords (stream TopSQLRecord, EmptyResponse)

  topsql_agent.proto:22

  ReportTopSQLRecords is called periodically (e.g. per minute) to save the in-memory TopSQL records

TiDB implements TopSQLPubSub service for clients to subscribe to TopSQL data.

• rpc Subscribe (TopSQLSubRequest, stream TopSQLSubResponse)

  topsql_agent.proto:105

  Clients subscribe to TopSQL data through this RPC, and TiDB periodically (e.g. per minute) publishes TopSQL data to clients via gRPC stream.

  message TopSQLSubRequest

  topsql_agent.proto:128

  • repeated CollectorType collectors = 1

    Semantics: - collectors empty => default enable TOPSQL - collectors non-empty => authoritative (only those enabled) Examples: - TOPSQL only: collectors=[TOPSQL] (or empty) - TOPRU only: collectors=[TOPRU] - both: collectors=[TOPSQL, TOPRU]

  • optional TopRUConfig topru = 2

    Only used when COLLECTOR_TYPE_TOPRU is present in collectors.

  message TopSQLSubResponse

  topsql_agent.proto:142

  • oneof resp_oneof

    • TopSQLRecord record = 1

    • SQLMeta sql_meta = 2

    • PlanMeta plan_meta = 3

    • TopRURecord ru_record = 4

ANN = Approximate Nearest Neighbor. For some queries, ANN index can be used.

Used in: ColumnarIndexInfo, PartitionTableScan, TableScan

• optional ANNQueryType query_type = 1

• optional VectorDistanceMetric distance_metric = 2

• optional uint32 top_k = 3

• optional string column_name = 4

  For debug purpose only. Currently only used in explain.

• optional int64 deprecated_column_id = 5

  deprecated field, we use column to get the id first, and this field will not be set again. Retain this field to be compatible with older versions of TiDB

• optional bytes ref_vec_f32 = 6

  The reference vector to calculate distance with, with each element is a Float32

• optional int64 index_id = 8

• optional double max_distance = 10

  Only for ANNQueryType==Where

• optional uint32 hnsw_ef_search = 20

  Only for HNSW indexes

• optional ColumnInfo column = 21

  Persists the original vector column's type information (including nullability) to ensure correct data handling. This field is always populated with the column's schema metadata, regardless of whether enable_distance_proj is enabled.

• optional bool enable_distance_proj = 22

  If enabled, the content of TableScan's vector output column (whose ID is column.id) will be removed and TableScan will read a distance column (whose id must be -2000) as replacement, TiFlash persistent layer does not need to really read the Vector data column when index has been built.

Used in: ANNQueryInfo

• InvalidQueryType = 0

• OrderBy = 1

  Where = 2; // Not supported for now

Used in: ExplainOperator

• oneof access_object

  • ScanAccessObject scan_object = 1

  • DynamicPartitionAccessObjects dynamic_partition_objects = 2

  • string other_object = 3

Used in: Expr

• CompleteMode = 0

• FinalMode = 1

• Partial1Mode = 2

• Partial2Mode = 3

• DedupMode = 4

Used in: Executor

• repeated Expr group_by = 1

  Group by clause.

• repeated RpnExpr rpn_group_by = 4

• repeated Expr agg_func = 2

  Aggregate functions.

• repeated RpnExpr rpn_agg_func = 5

• optional bool streamed = 3

  If it is a stream aggregation.

• optional Executor child = 6

• optional TiFlashPreAggMode pre_agg_mode = 7

Used in: AnalyzeColumnsReq

• repeated int64 column_offsets = 1

• repeated int64 prefix_lengths = 2

Used in: AnalyzeReq

• optional int64 bucket_size = 1

  bucket_size is the max histograms bucket size, we need this because when primary key is handle, the histogram will be directly built.

• optional int64 sample_size = 2

  sample_size is the max number of samples that will be collected.

• optional int64 sketch_size = 3

  sketch_size is the max sketch size.

• repeated ColumnInfo columns_info = 4

  columns_info is the info of all the columns that needs to be analyzed.

• optional int32 cmsketch_depth = 5

• optional int32 cmsketch_width = 6

• repeated int64 primary_column_ids = 7

• optional int32 version = 8

• repeated int64 primary_prefix_column_ids = 9

• repeated AnalyzeColumnGroup column_groups = 10

• optional double sample_rate = 11

  sample_rate is the sampling rate that how many samples will collected. There must be one non-zero value in sample_rate and sample_size.

• optional double ndv_rate = 12

  ndv_rate is the sampling rate that determines how many samples will be used to calculate the NDV.

Used in: AnalyzeMixedResp

• repeated SampleCollector collectors = 1

  collectors is the sample collectors for columns.

• optional Histogram pk_hist = 2

  pk_hist is the histogram for primary key when it is the handle.

• optional RowSampleCollector row_collector = 3

Used in: AnalyzeReq

• optional int64 bucket_size = 1

  bucket_size is the max histograms bucket size.

• optional int32 num_columns = 2

  num_columns is the number of columns in the index.

• optional int32 cmsketch_depth = 3

• optional int32 cmsketch_width = 4

• optional int64 sample_size = 5

• optional int64 sketch_size = 6

• optional int32 top_n_size = 7

• optional int32 version = 8

Used in: AnalyzeMixedResp

• optional Histogram hist = 1

• optional CMSketch cms = 2

• optional SampleCollector collector = 3

• optional AnalyzeColumnsResp columns_resp = 1

• optional AnalyzeIndexResp index_resp = 2

• optional AnalyzeType tp = 1

• optional uint64 start_ts_fallback = 2

  Deprecated. Start Ts has been moved to coprocessor.Request.

• optional uint64 flags = 3

• optional int64 time_zone_offset = 4

• optional AnalyzeIndexReq idx_req = 5

• optional AnalyzeColumnsReq col_req = 6

Used in: AnalyzeReq

• TypeIndex = 0

• TypeColumn = 1

• TypeCommonHandle = 2

• TypeSampleIndex = 3

• TypeMixed = 4

• TypeFullSampling = 5

Used in: Executor

• optional string query = 1

Bucket is an element of histogram.

Used in: Histogram

• optional int64 count = 1

• optional bytes lower_bound = 2

• optional bytes upper_bound = 3

• optional int64 repeats = 4

• optional int64 ndv = 5

ByItem type for group by and order by.

Used in: Expr, Limit, Sort, TopN, Window

• optional Expr expr = 1

• optional RpnExpr rpn_expr = 3

• optional bool desc = 2

Used in: AnalyzeIndexResp, SampleCollector

• repeated CMSketchRow rows = 1

• repeated CMSketchTopN top_n = 2

• optional uint64 default_value = 3

Used in: CMSketch

• repeated uint32 counters = 1

Used in: CMSketch

• optional bytes data = 1

• optional uint64 count = 2

Used in: Executor

• required uint32 cte_id = 1

• required uint32 cte_source_num = 2

• required uint32 cte_sink_num = 3

• optional Executor child = 4

• repeated FieldType field_types = 5

Used in: Executor

• required uint32 cte_id = 1

• required uint32 cte_source_num = 2

• required uint32 cte_sink_num = 3

• repeated FieldType field_types = 4

Used in: ChecksumRequest

• Crc64_Xor = 0

• optional uint64 start_ts_fallback = 1

  Deprecated. Start Ts has been moved to coprocessor.Request.

• optional ChecksumScanOn scan_on = 2

• optional ChecksumAlgorithm algorithm = 3

• optional ChecksumRewriteRule rule = 4

• optional uint64 checksum = 1

• optional uint64 total_kvs = 2

• optional uint64 total_bytes = 3

Used in: ChecksumRequest

• optional bytes old_prefix = 1

• optional bytes new_prefix = 2

Used in: ChecksumRequest

• Table = 0

• Index = 1

Chunk contains multiple rows data and rows meta.

Used in: IntermediateOutput, SelectResponse

• optional bytes rows_data = 3

  Data for all rows in the chunk.

• repeated RowMeta rows_meta = 4

  Meta data for every row.

Used in: DAGRequest

• optional Endian endian = 1

  Represents the endian.

Used in: TopSQLSubRequest

• COLLECTOR_TYPE_UNSPECIFIED = 0

  For compatibility when this variable is not present. Defaults to COLLECTOR_TYPE_TOPSQL.

• COLLECTOR_TYPE_TOPSQL = 1

• COLLECTOR_TYPE_TOPRU = 2

  future: COLLECTOR_TYPE_XXX = 3;

Used in: ANNQueryInfo, AnalyzeColumnsReq, FTSQueryInfo, IndexInfo, IndexScan, PartitionTableScan, SPFreshSearchRequest, TableInfo, TableScan

• optional int64 column_id = 1

• optional int32 tp = 2

  MySQL type.

• optional int32 collation = 3

• optional int32 columnLen = 4

• optional int32 decimal = 5

• optional int32 flag = 6

• repeated string elems = 7

• optional bytes default_val = 8

  Encoded datum.

• optional bool pk_handle = 21

  PK handle column value is row handle.

• optional bool array = 22

Used in: PartitionTableScan, TableScan

• optional ColumnarIndexType index_type = 1

• oneof index

  • ANNQueryInfo ann_query_info = 2

  • InvertedQueryInfo inverted_query_info = 3

  • FTSQueryInfo fts_query_info = 4

Used in: ColumnarIndexInfo

• TypeInvalid = 0

• TypeVector = 1

• TypeInverted = 2

• TypeFulltext = 3

Used in: ExecutorExecutionSummary

• optional uint64 regions = 1

• optional uint64 read_tasks = 3

• optional uint64 physical_tables = 4

• optional uint64 columns = 5

• optional uint64 user_read_bytes = 6

• optional uint64 mvcc_input_rows = 7

• optional uint64 mvcc_input_bytes = 8

• optional uint64 mvcc_output_rows = 9

• optional uint64 total_read_block_ms = 10

• optional uint64 total_serialize_block_ms = 11

• optional uint64 total_init_reader_ms = 12

• optional uint64 total_prefetch_ms = 13

• optional uint64 rough_check_total_packs = 14

• optional uint64 rough_check_selected_packs = 15

• optional uint64 rough_check_skipped_packs = 16

• optional uint64 rough_check_unknown_packs = 17

• optional uint64 remote_segments = 18

• optional uint64 total_segments = 19

• optional uint64 total_deserialize_block_ms = 20

• required bool has_null = 1

• optional bytes consts = 2

  consts represents all non-null const args in repeated Datum format.

Data compression mode

Used in: ExchangeSender

• NONE = 0

  no compression

• FAST = 1

  fast compression/decompression speed, compression ratio is lower than HC mode

• HIGH_COMPRESSION = 2

  high compression (HC) ratio mode

DAGRequest represents the request that will be handled with DAG mode.

• optional uint64 start_ts_fallback = 1

  Transaction start timestamp. Deprecated. Start Ts has been moved to coprocessor.Request.

• repeated Executor executors = 2

  It represents push down Executors and follows the order of depth-first search with post-order traversal. That is: left child first, then right child, then parent. For example, a DAG: A / B / \ C D / / \ E F G / H Its order should be: [H, E, C, F, G, D, B, A] In most cases, there is only one child for each parent, that makes executors simple array from the srouce to the out most executors, and the response only need to output the final rows. But when a executor has more than one children, for example, IndexLookUp, some intermedidate result is required to output. The field `intermediate_output_channels` describes it.

• optional int64 time_zone_offset = 3

  time zone offset in seconds

• optional uint64 flags = 4

  flags are used to store flags that change the execution mode, it contains: ignore_truncate = 1 truncate error should be ignore if set. truncate_as_warning = 1 << 1 when ignored_truncate is not set, return warning instead of error if this flag is set. ... add more when needed.

• repeated uint32 output_offsets = 5

  It represents which columns we should output.

• optional bool collect_range_counts = 6

  It represents whether we collect the detailed scan counts in each range.

• optional uint64 max_warning_count = 7

  It indicates the maximum number of warning, which is the number of messages that SHOW WARNINGS displays.

• optional EncodeType encode_type = 8

  It indicates the encode type of response.

• optional uint64 sql_mode = 9

  It indicates the sql_mode.

• optional string time_zone_name = 11

  supply offset is not enough since we have daylight saving time present in some regions

• optional bool collect_execution_summaries = 12

  It represents whether or not TiKV should collect execution summaries. Execution summaries will be collected into `execution_summaries` field in the response.

• optional uint64 max_allowed_packet = 13

  Represents the maximum size of one packet, any generated string, or any parameter sent as long data.

• optional ChunkMemoryLayout chunk_memory_layout = 14

  Represents the chunk memory layout.

• optional bool is_rpn_expr = 15

  Represents whether the expression use RPN form.

• optional UserIdentity user = 16

  UserIdentity uses to do privilege check. It is only used in TiDB cluster memory table.

• optional Executor root_executor = 17

  Represents tree struct based executors, if this field is set, should ignore the executors field, currently only used in TiFlash

• optional bool force_encode_type = 18

  Force using the encode type specified by encode_type, currently only used in TiFlash

• optional uint32 div_precision_increment = 19

  It indicates the number of digits by which to increase the scale of the result of division operations performed with the / operator.

• repeated IntermediateOutputChannel intermediate_output_channels = 20

  It inidcates the intermdidate result channels.

DynamicPartitionAccessObject represents the partitions accessed by the children of this operator.

Used in: DynamicPartitionAccessObjects

• string database = 1

• string table = 2

• bool all_partitions = 3

• repeated string partitions = 4

Used in: AccessObject

• repeated DynamicPartitionAccessObject objects = 1

Used as response type in: TopSQLAgent.ReportPlanMeta, TopSQLAgent.ReportSQLMeta, TopSQLAgent.ReportTopRURecords, TopSQLAgent.ReportTopSQLRecords

(message has no fields)

Used in: DAGRequest, IntermediateOutput, SelectResponse

• TypeDefault = 0

• TypeChunk = 1

• TypeCHBlock = 2

  TypeCHBlock is used by TiSpark and TiFlash, in this encode mode, TiFlash will encode the data using native ch block format

Used in: ExchangeSender

• repeated bytes encoded_tasks = 1

Used in: ChunkMemoryLayout

• LittleEndian = 0

• BigEndian = 1

Used in: TableScan

• Local = 0

• TiKV = 1

• TiFlash = 2

Used in: SelectResponse, StreamResponse

• optional int32 code = 1

• optional string msg = 2

• Unknown = 0

• TiKvCoprGetRequest = 1000

• TiKvCoprHandleRequest = 1001

• TiKvCoprScheduleTask = 1002

• TiKvCoprGetSnapshot = 1003

• TiKvCoprExecuteDagRunner = 1004

• TiKvCoprExecuteBatchDagRunner = 1005

ExchangeReceiver accept connection and receiver data from ExchangeSender.

Used in: Executor

• repeated bytes encoded_task_meta = 1

• repeated FieldType field_types = 2

• optional ExchangeType tp = 3

• repeated bytes original_cte_prdocuer_task_meta = 4

• repeated bool same_zone_flag = 5

ExchangeSender will build connection with ExchangeReceiver.

Used in: Executor

• optional ExchangeType tp = 1

• repeated bytes encoded_task_meta = 2

• repeated Expr partition_keys = 3

• optional Executor child = 4

• repeated FieldType types = 5

  partition keys' types

• repeated FieldType all_field_types = 6

• optional CompressionMode compression = 7

• repeated EncodedBytesSlice upstream_cte_task_meta = 8

• repeated bool same_zone_flag = 9

Used in: ExchangeReceiver, ExchangeSender

• PassThrough = 0

• Broadcast = 1

• Hash = 2

Used in: Executor

• TypeTableScan = 0

• TypeIndexScan = 1

• TypeSelection = 2

• TypeAggregation = 3

  TODO: Rename it to hash aggregation after support stream aggregation in TiKV.

• TypeTopN = 4

• TypeLimit = 5

• TypeStreamAgg = 6

• TypeJoin = 7

• TypeKill = 8

• TypeExchangeSender = 9

• TypeExchangeReceiver = 10

• TypeProjection = 11

• TypePartitionTableScan = 12

• TypeSort = 13

• TypeWindow = 14

• TypeExpand = 15

  Expand executor is used to expand underlying data sources to feed different grouping sets.

• TypeExpand2 = 16

  Expand2 executor is used to expand underlying data sources to feed different grouping sets.

• TypeBroadcastQuery = 17

• TypeCTESink = 18

• TypeCTESource = 19

• TypeIndexLookUp = 20

It represents a Executor.

Used in: Aggregation, CTESink, DAGRequest, ExchangeSender, Expand, Expand2, IndexLookUp, Join, Limit, Projection, Selection, Sort, TopN, Window

• optional ExecType tp = 1

• optional TableScan tbl_scan = 2

• optional IndexScan idx_scan = 3

• optional Selection selection = 4

• optional Aggregation aggregation = 5

• optional TopN topN = 6

• optional Limit limit = 7

• optional ExchangeReceiver exchange_receiver = 8

• optional Join join = 9

• optional string executor_id = 10

• optional Kill kill = 11

• optional ExchangeSender exchange_sender = 12

• optional Projection Projection = 13

• optional PartitionTableScan partition_table_scan = 14

• optional Sort sort = 15

• optional Window window = 16

• optional uint64 fine_grained_shuffle_stream_count = 17

• optional uint64 fine_grained_shuffle_batch_size = 18

• optional Expand expand = 19

• optional Expand2 expand2 = 20

• optional BroadcastQuery broadcast_query = 21

• optional CTESink cte_sink = 22

• optional CTESource cte_source = 23

• optional IndexLookUp index_lookup = 24

• optional uint32 parent_idx = 25

  It indicates the parent index of current executor. Not set indicates its parent is the next executor in `DAGRequest.executors`.

Used in: SelectResponse, TiFlashExecutionInfo

• optional uint64 time_processed_ns = 1

  Total time cost in this executor. Includes self time cost and children time cost.

• optional uint64 num_produced_rows = 2

  How many rows this executor produced totally.

• optional uint64 num_iterations = 3

  How many times executor's `next()` is called.

• optional string executor_id = 4

  Coresponding executor id

• optional uint64 concurrency = 5

  The execution concurrency for this executor

• oneof DetailInfo

  • TiFlashScanContext tiflash_scan_context = 6

  • ColumnarScanContext columnar_scan_context = 10

• optional bytes ru_consumption = 7

  Serialize kvproto resource_manager.Consumption to tell tidb the consumption info. For now it's only for tiflash. And it's the ru consumption of one MPPTask/cop/batchCop instead of one executor.

• optional TiFlashWaitSummary tiflash_wait_summary = 8

  Only for tiflash, records the wait info.

• optional TiFlashNetWorkSummary tiflash_network_summary = 9

  Only for tiflash, records network info.

Deprecated in the nearly feature usage

Used in: Executor

• repeated GroupingSet grouping_sets = 1

  for grouping sets like: expr[a,b] and expr[c]

• optional Executor child = 2

• optional uint64 version = 3

  expand version

Used in: Executor

• repeated ExprSlice proj_exprs = 1

  for grouping sets generated projection levels, like [a, b, c, 1#gen_col1, 2#gen_col2]

• repeated string generated_output_names = 2

  for output names for generated cols like grouping_id etc, like "gen_col1", "gen_col2" here

• optional Executor child = 3

• optional ExplainOperator main = 1

• repeated ExplainOperator ctes = 2

• bool with_runtime_stats = 3

  with_runtime_stats represents if runtime stats are available. If not available, the act_rows, *_exec_info, memory_bytes and disk_bytes should not be used.

• bool discarded_due_to_too_long = 4

  If discarded_due_to_too_long is true. The main and ctes fields should be empty and should not be used. This field can be changed to a enum or int if we need to represent more states in the future.

• repeated ExplainOperator subqueries = 5

Used in: ExplainData

• string name = 1

• repeated ExplainOperator children = 2

• repeated OperatorLabel labels = 3

• double cost = 4

  the cost of the current operator

• double est_rows = 5

• uint64 act_rows = 6

• TaskType task_type = 7

• StoreType store_type = 8

• repeated AccessObject access_objects = 9

  The XXXReader/XXXScan/MemTable/PointGet/BatchPointGet may use this

• string operator_info = 10

• string root_basic_exec_info = 11

• repeated string root_group_exec_info = 12

• string cop_exec_info = 13

• int64 memory_bytes = 14

  memory_bytes and disk_bytes are expected to be displayed as "N/A" when they are -1, this will be consistent with the result of EXPLAIN ANALYZE.

• int64 disk_bytes = 15

• string brief_name = 16

• string brief_operator_info = 17

Evaluators should implement evaluation functions for every expression type.

Used in: Aggregation, ByItem, ExchangeSender, ExprSlice, FTSQueryInfo, GroupingExpr, Join, Limit, PartitionTableScan, Projection, RpnExpr, RuntimeFilter, Selection, TableScan, TiCIVectorQueryInfo, Window, WindowFrameBound

• optional ExprType tp = 1

• optional bytes val = 2

• repeated Expr children = 3

• optional uint32 rpn_args_len = 6

• optional ScalarFuncSig sig = 4

• optional FieldType field_type = 5

• optional bool has_distinct = 7

• repeated ByItem order_by = 8

• optional AggFunctionMode aggFuncMode = 9

Used in: Expand2

• repeated Expr exprs = 1

Children count 0.

Used in: Expr

• Null = 0

  Values are encoded bytes.

• Int64 = 1

• Uint64 = 2

• Float32 = 3

• Float64 = 4

• String = 5

• Bytes = 6

• MysqlBit = 101

  Mysql specific types.

• MysqlDecimal = 102

• MysqlDuration = 103

• MysqlEnum = 104

• MysqlHex = 105

• MysqlSet = 106

• MysqlTime = 107

• MysqlJson = 108

• TiDBVectorFloat32 = 121

• ValueList = 151

  Encoded value list.

• ColumnRef = 201

  Column reference. value is int64 column ID.

• Count = 3001

  Aggregate functions.

• Sum = 3002

• Avg = 3003

• Min = 3004

• Max = 3005

• First = 3006

• GroupConcat = 3007

• Agg_BitAnd = 3008

• Agg_BitOr = 3009

• Agg_BitXor = 3010

• Std = 3011

• Stddev = 3012

• StddevPop = 3013

• StddevSamp = 3014

• VarPop = 3015

• VarSamp = 3016

• Variance = 3017

• JsonArrayAgg = 3018

• JsonObjectAgg = 3019

• ApproxCountDistinct = 3020

• SumInt = 3021

• MinCount = 3022

• MaxCount = 3023

• RowNumber = 4001

  Window functions

• Rank = 4002

• DenseRank = 4003

• CumeDist = 4004

• PercentRank = 4005

• Ntile = 4006

• Lead = 4007

• Lag = 4008

• FirstValue = 4009

• LastValue = 4010

• NthValue = 4011

• ScalarFunc = 10000

  Scalar Function

FMSketch is used to count distinct values for columns.

Used in: RowSampleCollector, SampleCollector

• optional uint64 mask = 1

• repeated uint64 hashset = 2

Used in: FTSBooleanNode

• FTSBooleanModifierNone = 0

• FTSBooleanModifierBoost = 1

• FTSBooleanModifierDeBoost = 2

• FTSBooleanModifierNegate = 3

Used in: FTSBooleanQuery

• optional FTSBooleanOccur occur = 1

• optional FTSBooleanModifier modifier = 2

• oneof node

  • FTSBooleanTerm term = 3

  • FTSBooleanQuery sub_expression = 4

Used in: FTSBooleanNode

• FTSBooleanOccurInvalid = 0

• FTSBooleanOccurShould = 1

• FTSBooleanOccurMust = 2

• FTSBooleanOccurMustNot = 3

Used in: FTSBooleanNode, FTSQueryInfo

• repeated FTSBooleanNode nodes = 1

Used in: FTSBooleanNode

• optional FTSBooleanTermType term_type = 1

• optional string text = 2

• optional uint32 phrase_distance = 3

Used in: FTSBooleanTerm

• FTSBooleanTermInvalid = 0

• FTSBooleanTermWord = 1

• FTSBooleanTermPhrase = 2

• FTSBooleanTermPrefix = 3

Used in: ColumnarIndexInfo, IndexScan

• optional FTSQueryType query_type = 1

• optional int64 index_id = 2

• repeated ColumnInfo columns = 3

  Currently only one column is supported.

• repeated string column_names = 4

  For debug purpose only. Currently only used in explain.

• optional string query_text = 5

• optional string query_tokenizer = 6

  Always the same as parser/model/index_full_text.go:FullTextParserType

• optional uint32 top_k = 10

  These fields support pushdown for order-by and limit scenarios.

• repeated int64 sort_column_ids = 11

• repeated bool sort_column_asc = 12

• optional ScalarFuncSig query_func = 20

  Distinguish match_word, match_expression, match_prefix, match_regexp

• repeated Expr match_expr = 21

  Pass through the inverted index conditions for the hybrid index.

• optional FTSBooleanQuery boolean_query = 22

  Pass through the FTS boolean query.

Used in: FTSQueryInfo

• FTSQueryTypeInvalid = 0

• FTSQueryTypeNoScore = 1

  Means no scoring is ever needed, encourages the engine to use a fast path.

• FTSQueryTypeWithScore = 2

Used in: CTESink, CTESource, ExchangeReceiver, ExchangeSender, Expr, Join

• optional int32 tp = 1

• optional uint32 flag = 2

• optional int32 flen = 3

• optional int32 decimal = 4

• optional int32 collate = 5

• optional string charset = 6

• repeated string elems = 7

• optional bool array = 8

Used in: GroupingSet

• repeated Expr grouping_expr = 1

  for grouping expressions like: expr[a,b]

• required GroupingMode mode = 1

• repeated GroupingMark grouping_marks = 2

  2 dimension here, out-most dimension is for grouping(a,b) = grouping(a) << 1 + grouping(b); we should maintain a slice of grouping mark.

  Contain the grouping's meta info

Used in: GroupingFunctionMetadata

• repeated uint64 grouping_nums = 1

Used in: GroupingFunctionMetadata

• ModeBitAnd = 1

  Do 'and' operation, e.g. x & y

• ModeNumericCmp = 2

  Compare two number

• ModeNumericSet = 3

  Find if number in the set

Used in: Expand

• repeated GroupingExpr grouping_exprs = 1

Used in: AnalyzeColumnsResp, AnalyzeIndexResp

• optional int64 ndv = 1

  ndv is the number of distinct values.

• repeated Bucket buckets = 2

  buckets represents all the buckets.

• required bool in_union = 1

Used in: ScanAccessObject

• string name = 1

• repeated string cols = 2

• bool is_clustered_index = 3

• optional int64 table_id = 1

• optional int64 index_id = 2

• repeated ColumnInfo columns = 3

• optional bool unique = 4

Used in: Executor

• repeated uint32 index_handle_offsets = 1

  It represents the index columns we should use to build the row handle.

• optional bool keep_order = 2

• repeated Executor children = 3

Used in: Executor

• optional int64 table_id = 1

• optional int64 index_id = 2

• repeated ColumnInfo columns = 3

• optional bool desc = 4

• optional bool unique = 5

  check whether it is a unique index.

• repeated int64 primary_column_ids = 6

• optional FTSQueryInfo fts_query_info = 7

  only used by TiFlash

• optional TiCIVectorQueryInfo tici_vector_query_info = 8

  only used by TiFlash, for TiCI hybrid vector queries

It is the data of a intermidiate output channel

Used in: SelectResponse

• optional EncodeType encode_type = 1

• repeated Chunk chunks = 2

IntermediateOutputChannel is the channel description for the intermediate ouput. The SelectResponse of a DAGRequest may output some intermediate data because not all rows can be processed in DAG. For example, the executor IndexLookUp scans the index records and look up the rows locally. If a related row of a index is not found locally, this index record should be ouput into the intermediate channel for the further processment in the TiDB side.

Used in: DAGRequest

• required uint32 executor_idx = 1

  executor_idx indicates which executor outputs this intermediate result.

• repeated uint32 output_offsets = 2

  It represents which columns we should output.

Used in: ColumnarIndexInfo

• optional int64 index_id = 1

• optional int64 column_id = 2

Used in: TopRUConfig

• ITEM_INTERVAL_UNSPECIFIED = 0

• ITEM_INTERVAL_15S = 15

• ITEM_INTERVAL_30S = 30

• ITEM_INTERVAL_60S = 60

Used in: Executor

• optional JoinType join_type = 1

• optional JoinExecType join_exec_type = 2

• repeated Executor children = 3

• optional int64 inner_idx = 4

  0 or 1

• repeated Expr left_join_keys = 5

• repeated Expr right_join_keys = 6

• repeated FieldType probe_types = 7

  used by TiFlash join when new collation is enabled.

• repeated FieldType build_types = 8

• repeated Expr left_conditions = 9

• repeated Expr right_conditions = 10

• repeated Expr other_conditions = 11

• repeated Expr other_eq_conditions_from_in = 12

• optional bool is_null_aware_semi_join = 13

• repeated RuntimeFilter runtime_filter_list = 14

  only used by TiFlash

• repeated bool is_null_eq = 15

  only used by TiFlash join key null-safe equal

Used in: Join

• TypeHashJoin = 0

Used in: Join

• TypeInnerJoin = 0

• TypeLeftOuterJoin = 1

• TypeRightOuterJoin = 2

• TypeSemiJoin = 3

• TypeAntiSemiJoin = 4

• TypeLeftOuterSemiJoin = 5

• TypeAntiLeftOuterSemiJoin = 6

• TypeFullOuterJoin = 7

KeyRange is the encoded index key range, low is closed, high is open. (low <= x < high)

Used in: TableScan

• optional bytes low = 1

• optional bytes high = 2

Used in: Executor

• optional uint64 connID = 1

• optional bool query = 2

  Query indicates whether terminate a single query on this connection or the whole connection. If Query is true, terminates the statement the connection is currently executing, but leaves the connection itself intact. If Query is false, terminates the connection associated with the given ConnectionID, after terminating any statement the connection is executing. See https://dev.mysql.com/doc/refman/8.0/en/kill.html.

Used in: Executor

• optional uint64 limit = 1

  Limit the result to be returned.

• optional Executor child = 2

• repeated ByItem partition_by = 3

  If partition_by is not empty, it means need to return limitN of each partition. Generally used in sqls like `where row_number() over (partition by ...) < X`

• repeated Expr truncate_key_expr = 4

Used in: ExplainOperator

• empty = 0

  empty is not expected to be used.

• buildSide = 1

• probeSide = 2

• seedPart = 3

• recursivePart = 4

Used in: Executor

• optional int64 table_id = 1

• repeated ColumnInfo columns = 2

• optional bool desc = 3

• repeated int64 primary_column_ids = 4

• repeated int64 primary_prefix_column_ids = 5

• repeated int64 partition_ids = 6

• optional bool is_fast_scan = 7

  fast_scan is a feature only provided by TiFlash (but not TiKV).

• repeated Expr pushed_down_filter_conditions = 8

  conditions that are pushed down to storage layer, only used by TiFlash.

• repeated RuntimeFilter runtime_filter_list = 9

  only used by TiFlash

• optional int32 max_wait_time_ms = 10

  only used by TiFlash

• optional ANNQueryInfo deprecated_ann_query = 11

  only used by TiFlash

• repeated ColumnarIndexInfo used_columnar_indexes = 12

  only used by TiFlash

Used as request type in: TopSQLAgent.ReportPlanMeta

Used as field type in: TopSQLSubResponse

• bytes plan_digest = 1

• string normalized_plan = 2

  Plan text with sensitive fields trimmed. Producers should limit the size to less than 4KiB. Consider use `encoded_normalized_plan` if the size exceeds.

• string encoded_normalized_plan = 3

  If `normalized_plan` is unacceptably large, set `encoded_normalized_plan` instead. The textual normalized plan is expected to get by following steps: 1. decode from base64 2. decode from snappy 3. decode from github.com/pingcap/tidb/util/plancodec.DecodeNormalizedPlan

• bytes keyspace_name = 4

Used in: Executor

• repeated Expr exprs = 1

  Projection expressions.

• repeated RpnExpr rpn_exprs = 2

• optional Executor child = 3

Used for range frame's comparison when finding frame's boundary

Used in: WindowFrameBound

• Int = 0

• Float = 1

• Decimal = 2

• DateTime = 3

• Duration = 4

• optional bytes sql_digest = 1

• optional bytes plan_digest = 2

• optional ResourceGroupTagLabel label = 3

  Use to label the handling kv type of the request. This is for TiKV resource_metering to collect execution information by the key label.

• optional int64 table_id = 4

• optional bytes keyspace_name = 5

Used in: ResourceGroupTag

• ResourceGroupTagLabelUnknown = 0

• ResourceGroupTagLabelRow = 1

• ResourceGroupTagLabelIndex = 2

values are all in text format.

Used in: SelectResponse

• optional bytes handle = 1

• optional bytes data = 2

RowMeta contains row handle and length of a row.

Used in: Chunk

• optional int64 handle = 1

• optional int64 length = 2

Used in: RowSampleCollector

• repeated bytes row = 1

• optional int64 weight = 2

Used in: AnalyzeColumnsResp

• repeated RowSample samples = 1

• repeated int64 null_counts = 2

• optional int64 count = 3

• repeated FMSketch fm_sketch = 4

• repeated int64 total_size = 5

• repeated FMSketch singleton_sketch = 6

  singleton_sketch estimates the number of values that appear only once in the sample.

• optional int64 sketch_sample_count = 7

  sketch_sample_count is the number of samples processed when building fm_sketch and singleton_sketch.

Expression organized in RPN form. https://en.wikipedia.org/wiki/Reverse_Polish_notation

Used in: Aggregation, ByItem, Projection, Selection

• repeated Expr exprs = 1

  All children fields in exprs should be empty.

Used in: Join, PartitionTableScan, TableScan

• optional int32 id = 1

• repeated Expr source_expr_list = 2

• repeated Expr target_expr_list = 3

• optional string source_executor_id = 4

• optional string target_executor_id = 5

• optional RuntimeFilterType rf_type = 6

• optional RuntimeFilterMode rf_mode = 7

Used in: RuntimeFilter

• LOCAL = 0

• GLOBAL = 1

Used in: RuntimeFilter

• IN = 0

• MIN_MAX = 1

• BLOOM_FILTER = 2

• uint32 keyspace_id = 1

• int64 table_id = 2

• int64 index_id = 3

• uint64 start_ts = 4

• uint64 top_k = 5

• bytes query_vector_f32_le = 6

  query_vector_f32_le is f32 little-endian bytes, without the dims prefix.

• repeated ColumnInfo required_columns = 7

• double oversample_factor = 8

  Optional tuning parameters.

• uint32 base_beam_size = 9

  If zero, the server uses its default base beam size.

• bool read_only = 10

  read_only disables background fixups (split/merge) triggered by search observations. When enabled, the server must not enqueue fixups and should avoid starting any background fixup workers.

• bool need_distance = 11

  When true, the server populates SPFreshSearchRow.distance in each result row.

• repeated SPFreshSearchRow rows = 1

• optional SPFreshSearchStats stats = 2

Used in: SPFreshSearchResponse

• bytes handle_bytes = 1

• float distance = 2

• repeated bytes column_values = 3

Used in: SPFreshSearchResponse

• uint64 partitions_scanned = 1

• uint64 vectors_scanned = 2

• uint64 table_lookup_keys = 3

• uint64 table_lookup_bytes = 4

• uint64 permit_micros = 5

  Time costs of each major stage in microseconds.

• uint64 config_micros = 6

• uint64 index_open_micros = 7

• uint64 search_micros = 8

• uint64 table_lookup_micros = 9

• uint64 tikv_client_rpc_count = 10

  Aggregated tikv-client RPC stats (best-effort) for this /spfresh_search request.

• uint64 tikv_client_rpc_micros = 11

• bool read_only = 12

• double oversample_factor = 13

  Effective oversample factor used by the server for this request.

• uint64 partition_cache_hits = 14

  Process-local partition cache stats for this request.

• uint64 partition_cache_misses = 15

• uint32 keyspace_id = 1

• int64 table_id = 2

• int64 index_id = 3

Used as request type in: TopSQLAgent.ReportSQLMeta

Used as field type in: TopSQLSubResponse

• bytes sql_digest = 1

• string normalized_sql = 2

  SQL text with sensitive fields trimmed. Producers should limit the size to less than 4KiB. Truncation can be chosen to reduce size.

• bool is_internal_sql = 3

  If true, this sql and plan is internally generated by tidb itself, not user.

• bytes keyspace_name = 4

SampleCollector is used for collect samples and calculate the count and ndv of an column.

Used in: AnalyzeColumnsResp, AnalyzeIndexResp

• repeated bytes samples = 1

• optional int64 null_count = 2

• optional int64 count = 3

• optional FMSketch fm_sketch = 4

• optional CMSketch cm_sketch = 5

• optional int64 total_size = 6

Used in: Expr, FTSQueryInfo

• Unspecified = 0

• CastIntAsInt = 1

  Casting

• CastIntAsReal = 2

• CastIntAsString = 3

• CastIntAsDecimal = 4

• CastIntAsTime = 5

• CastIntAsDuration = 6

• CastIntAsJson = 7

• CastRealAsInt = 10

• CastRealAsReal = 11

• CastRealAsString = 12

• CastRealAsDecimal = 13

• CastRealAsTime = 14

• CastRealAsDuration = 15

• CastRealAsJson = 16

• CastDecimalAsInt = 20

• CastDecimalAsReal = 21

• CastDecimalAsString = 22

• CastDecimalAsDecimal = 23

• CastDecimalAsTime = 24

• CastDecimalAsDuration = 25

• CastDecimalAsJson = 26

• CastStringAsInt = 30

• CastStringAsReal = 31

• CastStringAsString = 32

• CastStringAsDecimal = 33

• CastStringAsTime = 34

• CastStringAsDuration = 35

• CastStringAsJson = 36

• CastTimeAsInt = 40

• CastTimeAsReal = 41

• CastTimeAsString = 42

• CastTimeAsDecimal = 43

• CastTimeAsTime = 44

• CastTimeAsDuration = 45

• CastTimeAsJson = 46

• CastDurationAsInt = 50

• CastDurationAsReal = 51

• CastDurationAsString = 52

• CastDurationAsDecimal = 53

• CastDurationAsTime = 54

• CastDurationAsDuration = 55

• CastDurationAsJson = 56

• CastJsonAsInt = 60

• CastJsonAsReal = 61

• CastJsonAsString = 62

• CastJsonAsDecimal = 63

• CastJsonAsTime = 64

• CastJsonAsDuration = 65

• CastJsonAsJson = 66

• CoalesceInt = 4201

  compare

• CoalesceReal = 4202

• CoalesceDecimal = 4203

• CoalesceString = 4204

• CoalesceTime = 4205

• CoalesceDuration = 4206

• CoalesceJson = 4207

  unimplemented in tidb

• LTInt = 100

• LTReal = 101

• LTDecimal = 102

• LTString = 103

• LTTime = 104

• LTDuration = 105

• LTJson = 106

• LEInt = 110

• LEReal = 111

• LEDecimal = 112

• LEString = 113

• LETime = 114

• LEDuration = 115

• LEJson = 116

• GTInt = 120

• GTReal = 121

• GTDecimal = 122

• GTString = 123

• GTTime = 124

• GTDuration = 125

• GTJson = 126

• GreatestInt = 4215

• GreatestReal = 4216

• GreatestDecimal = 4217

• GreatestString = 4218

• GreatestTime = 4219

• LeastInt = 4220

• LeastReal = 4221

• LeastDecimal = 4222

• LeastString = 4223

• LeastTime = 4224

• IntervalInt = 4225

• IntervalReal = 4226

• GEInt = 130

• GEReal = 131

• GEDecimal = 132

• GEString = 133

• GETime = 134

• GEDuration = 135

• GEJson = 136

• EQInt = 140

• EQReal = 141

• EQDecimal = 142

• EQString = 143

• EQTime = 144

• EQDuration = 145

• EQJson = 146

• NEInt = 150

• NEReal = 151

• NEDecimal = 152

• NEString = 153

• NETime = 154

• NEDuration = 155

• NEJson = 156

• NullEQInt = 160

• NullEQReal = 161

• NullEQDecimal = 162

• NullEQString = 163

• NullEQTime = 164

• NullEQDuration = 165

• NullEQJson = 166

• PlusReal = 200

  arithmetic

• PlusDecimal = 201

• PlusInt = 203

• MinusReal = 204

• MinusDecimal = 205

• MinusInt = 207

• MultiplyReal = 208

• MultiplyDecimal = 209

• MultiplyInt = 210

• DivideReal = 211

• DivideDecimal = 212

• IntDivideInt = 213

• IntDivideDecimal = 214

• ModReal = 215

• ModDecimal = 216

• ModInt = 217

• MultiplyIntUnsigned = 218

• PlusIntUnsignedUnsigned = 219

• PlusIntUnsignedSigned = 220

• PlusIntSignedUnsigned = 221

• PlusIntSignedSigned = 222

• ModIntUnsignedUnsigned = 223

• ModIntUnsignedSigned = 224

• ModIntSignedUnsigned = 225

• ModIntSignedSigned = 226

• MinusIntUnsignedUnsigned = 227

• MinusIntUnsignedSigned = 228

• MinusIntSignedUnsigned = 229

• MinusIntSignedSigned = 230

• MinusIntForcedUnsignedUnsigned = 231

• MinusIntForcedUnsignedSigned = 232

• MinusIntForcedSignedUnsigned = 233

• IntDivideIntUnsignedUnsigned = 234

• IntDivideIntUnsignedSigned = 235

• IntDivideIntSignedSigned = 236

• IntDivideIntSignedUnsigned = 237

• AbsInt = 2101

  math

• AbsUInt = 2102

• AbsReal = 2103

• AbsDecimal = 2104

• CeilIntToDec = 2105

• CeilIntToInt = 2106

• CeilDecToInt = 2107

• CeilDecToDec = 2108

• CeilReal = 2109

• FloorIntToDec = 2110

• FloorIntToInt = 2111

• FloorDecToInt = 2112

• FloorDecToDec = 2113

• FloorReal = 2114

• RoundReal = 2121

• RoundInt = 2122

• RoundDec = 2123

• RoundWithFracReal = 2124

• RoundWithFracInt = 2125

• RoundWithFracDec = 2126

• Log1Arg = 2131

• Log2Args = 2132

• Log2 = 2133

• Log10 = 2134

• Rand = 2135

• RandWithSeedFirstGen = 2136

• Pow = 2137

• Conv = 2138

• CRC32 = 2139

• Sign = 2140

• Sqrt = 2141

• Acos = 2142

• Asin = 2143

• Atan1Arg = 2144

• Atan2Args = 2145

• Cos = 2146

• Cot = 2147

• Degrees = 2148

• Exp = 2149

• PI = 2150

• Radians = 2151

• Sin = 2152

• Tan = 2153

• TruncateInt = 2154

• TruncateReal = 2155

• TruncateDecimal = 2156

• TruncateUint = 2157

• LogicalAnd = 3101

  op

• LogicalOr = 3102

• LogicalXor = 3103

• UnaryNotInt = 3104

• UnaryNotDecimal = 3105

• UnaryNotReal = 3106

• UnaryNotJSON = 3107

• UnaryMinusInt = 3108

• UnaryMinusReal = 3109

• UnaryMinusDecimal = 3110

• DecimalIsNull = 3111

• DurationIsNull = 3112

• RealIsNull = 3113

• StringIsNull = 3114

• TimeIsNull = 3115

• IntIsNull = 3116

• JsonIsNull = 3117

  unimplemented in tidb

• BitAndSig = 3118

• BitOrSig = 3119

• BitXorSig = 3120

• BitNegSig = 3121

• IntIsTrue = 3122

• RealIsTrue = 3123

• DecimalIsTrue = 3124

• IntIsFalse = 3125

• RealIsFalse = 3126

• DecimalIsFalse = 3127

• LeftShift = 3129

• RightShift = 3130

• IntIsTrueWithNull = 3142

• RealIsTrueWithNull = 3143

• DecimalIsTrueWithNull = 3144

• IntIsFalseWithNull = 3145

• RealIsFalseWithNull = 3146

• DecimalIsFalseWithNull = 3147

• BitCount = 3128

  other

• GetParamString = 3131

• GetVar = 3132

• RowSig = 3133

• SetVar = 3134

• ValuesDecimal = 3135

• ValuesDuration = 3136

• ValuesInt = 3137

• ValuesJSON = 3138

• ValuesReal = 3139

• ValuesString = 3140

• ValuesTime = 3141

• InInt = 4001

• InReal = 4002

• InDecimal = 4003

• InString = 4004

• InTime = 4005

• InDuration = 4006

• InJson = 4007

• IfNullInt = 4101

  control

• IfNullReal = 4102

• IfNullDecimal = 4103

• IfNullString = 4104

• IfNullTime = 4105

• IfNullDuration = 4106

• IfInt = 4107

• IfReal = 4108

• IfDecimal = 4109

• IfString = 4110

• IfTime = 4111

• IfDuration = 4112

• IfNullJson = 4113

• IfJson = 4114

• CaseWhenInt = 4208

• CaseWhenReal = 4209

• CaseWhenDecimal = 4210

• CaseWhenString = 4211

• CaseWhenTime = 4212

• CaseWhenDuration = 4213

• CaseWhenJson = 4214

  unimplemented in tidb

• AesDecrypt = 4501

  encryption

• AesEncrypt = 4502

• Compress = 4503

• MD5 = 4504

• Password = 4505

• RandomBytes = 4506

• SHA1 = 4507

• SHA2 = 4508

• Uncompress = 4509

• UncompressedLength = 4510

• AesDecryptIV = 4511

• AesEncryptIV = 4512

• Encode = 4513

• Decode = 4514

• SM3 = 4515

• Database = 4521

  info

• FoundRows = 4522

• CurrentUser = 4523

• User = 4524

• ConnectionID = 4525

• LastInsertID = 4526

• LastInsertIDWithID = 4527

• Version = 4528

• TiDBVersion = 4529

• RowCount = 4530

• Sleep = 4551

  miscellaneous

• Lock = 4552

• ReleaseLock = 4553

• DecimalAnyValue = 4554

• DurationAnyValue = 4555

• IntAnyValue = 4556

• JSONAnyValue = 4557

• RealAnyValue = 4558

• StringAnyValue = 4559

• TimeAnyValue = 4560

• InetAton = 4561

• InetNtoa = 4562

• Inet6Aton = 4563

• Inet6Ntoa = 4564

• IsIPv4 = 4565

• IsIPv4Compat = 4566

• IsIPv4Mapped = 4567

• IsIPv6 = 4568

• UUID = 4569

• VitessHash = 4570

• IsUUID = 4571

• TiDBShard = 4572

• GroupingSig = 4573

• UUIDv4 = 4574

• UUIDv7 = 4575

• UUIDVersion = 4576

• UUIDTimestamp = 4577

• IlikeSig = 4309

  like

• LikeSig = 4310

• RegexpSig = 4311

• RegexpUTF8Sig = 4312

• RegexpLikeSig = 4313

• RegexpLikeUTF8Sig = 4314

• RegexpInStrSig = 4315

• RegexpInStrUTF8Sig = 4316

• RegexpReplaceSig = 4317

• RegexpReplaceUTF8Sig = 4318

• RegexpSubstrSig = 4319

• RegexpSubstrUTF8Sig = 4320

• JsonExtractSig = 5001

  json

• JsonUnquoteSig = 5002

• JsonTypeSig = 5003

• JsonSetSig = 5004

• JsonInsertSig = 5005

• JsonReplaceSig = 5006

• JsonRemoveSig = 5007

• JsonMergeSig = 5008

• JsonObjectSig = 5009

• JsonArraySig = 5010

• JsonValidJsonSig = 5011

• JsonContainsSig = 5012

• JsonArrayAppendSig = 5013

• JsonArrayInsertSig = 5014

• JsonMergePatchSig = 5015

• JsonMergePreserveSig = 5016

• JsonContainsPathSig = 5017

• JsonPrettySig = 5018

• JsonQuoteSig = 5019

• JsonSearchSig = 5020

• JsonStorageSizeSig = 5021

• JsonDepthSig = 5022

• JsonKeysSig = 5023

• JsonLengthSig = 5024

• JsonKeys2ArgsSig = 5025

• JsonValidStringSig = 5026

• JsonValidOthersSig = 5027

• JsonStorageFreeSig = 5028

• JsonMemberOfSig = 5029

• VecAsTextSig = 5110

  vector

• VecFromTextSig = 5111

• VecDimsSig = 5112

• VecL1DistanceSig = 5113

• VecL2DistanceSig = 5114

• VecNegativeInnerProductSig = 5115

• VecCosineDistanceSig = 5116

• VecL2NormSig = 5117

• CastStringAsVectorFloat32 = 5180

• CastVectorFloat32AsInt = 5181

• CastVectorFloat32AsReal = 5182

• CastVectorFloat32AsString = 5183

• CastVectorFloat32AsDecimal = 5184

• CastVectorFloat32AsTime = 5185

• CastVectorFloat32AsDuration = 5186

• CastVectorFloat32AsJson = 5187

• CastVectorFloat32AsVectorFloat32 = 5188

• PlusVectorFloat32 = 5141

• MinusVectorFloat32 = 5142

• MultiplyVectorFloat32 = 5143

• VectorFloat32AnyValue = 5144

• CoalesceVectorFloat32 = 5145

• LTVectorFloat32 = 5146

• LEVectorFloat32 = 5147

• GTVectorFloat32 = 5148

• GreatestVectorFloat32 = 5149

• LeastVectorFloat32 = 5150

• GEVectorFloat32 = 5151

• EQVectorFloat32 = 5152

• NEVectorFloat32 = 5153

• NullEQVectorFloat32 = 5154

• VectorFloat32IsNull = 5155

• VectorFloat32IsTrue = 5156

• VectorFloat32IsFalse = 5157

• VectorFloat32IsTrueWithNull = 5158

• VectorFloat32IsFalseWithNull = 5159

• IfNullVectorFloat32 = 5160

• IfVectorFloat32 = 5161

• CaseWhenVectorFloat32 = 5162

• FTSMatchWord = 5201

  fts

• FTSMatchExpression = 5202

• FTSMatchPrefix = 5203

• FTSMatchRegexp = 5204

• FTSMatchPhrase = 5205

• DateFormatSig = 6001

  time

• DateLiteral = 6002

• DateDiff = 6003

• NullTimeDiff = 6004

• TimeStringTimeDiff = 6005

• DurationStringTimeDiff = 6006

• DurationDurationTimeDiff = 6007

• StringTimeTimeDiff = 6008

• StringDurationTimeDiff = 6009

• StringStringTimeDiff = 6010

• TimeTimeTimeDiff = 6011

• Date = 6012

• Hour = 6013

• Minute = 6014

• Second = 6015

• MicroSecond = 6016

• Month = 6017

• MonthName = 6018

• NowWithArg = 6019

• NowWithoutArg = 6020

• DayName = 6021

• DayOfMonth = 6022

• DayOfWeek = 6023

• DayOfYear = 6024

• WeekWithMode = 6025

• WeekWithoutMode = 6026

• WeekDay = 6027

• WeekOfYear = 6028

• Year = 6029

• YearWeekWithMode = 6030

• YearWeekWithoutMode = 6031

• GetFormat = 6032

• SysDateWithFsp = 6033

• SysDateWithoutFsp = 6034

• CurrentDate = 6035

• CurrentTime0Arg = 6036

• CurrentTime1Arg = 6037

• Time = 6038

• TimeLiteral = 6039

• UTCDate = 6040

• UTCTimestampWithArg = 6041

• UTCTimestampWithoutArg = 6042

• AddDatetimeAndDuration = 6043

• AddDatetimeAndString = 6044

• AddTimeDateTimeNull = 6045

• AddStringAndDuration = 6046

• AddStringAndString = 6047

• AddTimeStringNull = 6048

• AddDurationAndDuration = 6049

• AddDurationAndString = 6050

• AddTimeDurationNull = 6051

• AddDateAndDuration = 6052

• AddDateAndString = 6053

• SubDatetimeAndDuration = 6054

• SubDatetimeAndString = 6055

• SubTimeDateTimeNull = 6056

• SubStringAndDuration = 6057

• SubStringAndString = 6058

• SubTimeStringNull = 6059

• SubDurationAndDuration = 6060

• SubDurationAndString = 6061

• SubTimeDurationNull = 6062

• SubDateAndDuration = 6063

• SubDateAndString = 6064

• UnixTimestampCurrent = 6065

• UnixTimestampInt = 6066

• UnixTimestampDec = 6067

• ConvertTz = 6068

• MakeDate = 6069

• MakeTime = 6070

• PeriodAdd = 6071

• PeriodDiff = 6072

• Quarter = 6073

• SecToTime = 6074

• TimeToSec = 6075

• TimestampAdd = 6076

• ToDays = 6077

• ToSeconds = 6078

• UTCTimeWithArg = 6079

• UTCTimeWithoutArg = 6080

• Timestamp1Arg = 6081

• Timestamp2Args = 6082

• TimestampLiteral = 6083

• LastDay = 6084

• StrToDateDate = 6085

• StrToDateDatetime = 6086

• StrToDateDuration = 6087

• FromUnixTime1Arg = 6088

• FromUnixTime2Arg = 6089

• ExtractDatetime = 6090

• ExtractDuration = 6091

• AddDateStringString = 6092

• AddDateStringInt = 6093

• AddDateStringDecimal = 6094

• AddDateIntString = 6095

• AddDateIntInt = 6096

• AddDateDatetimeString = 6097

• AddDateDatetimeInt = 6098

• SubDateStringString = 6099

• SubDateStringInt = 6100

• SubDateStringDecimal = 6101

• SubDateIntString = 6102

• SubDateIntInt = 6103

• SubDateDatetimeString = 6104

• SubDateDatetimeInt = 6105

• FromDays = 6106

• TimeFormat = 6107

• TimestampDiff = 6108

• SubDateStringReal = 6109

• SubDateIntReal = 6110

• SubDateIntDecimal = 6111

• SubDateDatetimeReal = 6112

• SubDateDatetimeDecimal = 6113

• SubDateDurationString = 6114

• SubDateDurationInt = 6115

• SubDateDurationReal = 6116

• SubDateDurationDecimal = 6117

• AddDateStringReal = 6118

• AddDateIntReal = 6119

• AddDateIntDecimal = 6120

• AddDateDatetimeReal = 6121

• AddDateDatetimeDecimal = 6122

• AddDateDurationString = 6123

• AddDateDurationInt = 6124

• AddDateDurationReal = 6125

• AddDateDurationDecimal = 6126

• ExtractDatetimeFromString = 6127

• AddDateRealString = 6128

• AddDateRealInt = 6129

• AddDateRealReal = 6130

• AddDateRealDecimal = 6131

• AddDateDecimalString = 6132

• AddDateDecimalInt = 6133

• AddDateDecimalReal = 6134

• AddDateDecimalDecimal = 6135

• SubDateRealString = 6136

• SubDateRealInt = 6137

• SubDateRealReal = 6138

• SubDateRealDecimal = 6139

• SubDateDecimalString = 6140

• SubDateDecimalInt = 6141

• SubDateDecimalReal = 6142

• SubDateDecimalDecimal = 6143

• AddDateDurationStringDatetime = 6144

• AddDateDurationIntDatetime = 6145

• AddDateDurationRealDatetime = 6146

• AddDateDurationDecimalDatetime = 6147

• SubDateDurationStringDatetime = 6148

• SubDateDurationIntDatetime = 6149

• SubDateDurationRealDatetime = 6150

• SubDateDurationDecimalDatetime = 6151

• BitLength = 7001

  String functions

• Bin = 7002

• ASCII = 7003

• Char = 7004

• CharLengthUTF8 = 7005

• Concat = 7006

• ConcatWS = 7007

• Convert = 7008

• Elt = 7009

• ExportSet3Arg = 7010

• ExportSet4Arg = 7011

• ExportSet5Arg = 7012

• FieldInt = 7013

• FieldReal = 7014

• FieldString = 7015

• FindInSet = 7016

• Format = 7017

• FormatWithLocale = 7018

• FromBase64 = 7019

• HexIntArg = 7020

• HexStrArg = 7021

• InsertUTF8 = 7022

• Insert = 7023

• InstrUTF8 = 7024

• Instr = 7025

• LTrim = 7026

• LeftUTF8 = 7027

• Left = 7028

• Length = 7029

• Locate2ArgsUTF8 = 7030

• Locate3ArgsUTF8 = 7031

• Locate2Args = 7032

• Locate3Args = 7033

• Lower = 7034

• LpadUTF8 = 7035

• Lpad = 7036

• MakeSet = 7037

• OctInt = 7038

• OctString = 7039

• Ord = 7040

• Quote = 7041

• RTrim = 7042

• Repeat = 7043

• Replace = 7044

• ReverseUTF8 = 7045

• Reverse = 7046

• RightUTF8 = 7047

• Right = 7048

• RpadUTF8 = 7049

• Rpad = 7050

• Space = 7051

• Strcmp = 7052

• Substring2ArgsUTF8 = 7053

• Substring3ArgsUTF8 = 7054

• Substring2Args = 7055

• Substring3Args = 7056

• SubstringIndex = 7057

• ToBase64 = 7058

• Trim1Arg = 7059

• Trim2Args = 7060

• Trim3Args = 7061

• UnHex = 7062

• UpperUTF8 = 7063

• Upper = 7064

• CharLength = 7065

• LowerUTF8 = 7066

• GreatestDuration = 7067

• GreatestCmpStringAsTime = 7068

• LeastDuration = 7069

• LeastCmpStringAsTime = 7070

• ToBinary = 7071

• FromBinary = 7072

• GreatestCmpStringAsDate = 7073

• LeastCmpStringAsDate = 7074

• GreatestDate = 7075

  Greatest,Least functions will return Date Type value when all the parameters are Date types

• LeastDate = 7076

ScanAccessObject represents the access to a single table. It may contain multiple indexes and multiple partitions.

Used in: AccessObject

• string database = 1

• string table = 2

• repeated IndexAccess indexes = 3

• repeated string partitions = 4

Response for SelectRequest.

• optional Error error = 1

• repeated Row rows = 2

  Result rows.

• repeated Chunk chunks = 3

  Use multiple chunks to reduce memory allocation and avoid allocating large contiguous memory.

• repeated Error warnings = 4

• repeated int64 output_counts = 5

• optional int64 warning_count = 6

• repeated ExecutorExecutionSummary execution_summaries = 8

  The execution summary of each executor, in the order in request.

• optional EncodeType encode_type = 9

  It indicates the encode type of response.

• repeated int64 ndvs = 10

  ndvs collects the number of distinct value information per range. It will be used to serve as execution feedback information. Helping us improve the table's statistics information.

• repeated IntermediateOutput intermediate_outputs = 11

  It contains all the intermedidate outputs.

Used in: Executor

• repeated Expr conditions = 1

  Where conditions.

• repeated RpnExpr rpn_conditions = 2

• optional Executor child = 3

Used in: Executor

• repeated ByItem byItems = 1

• optional bool isPartialSort = 2

• optional Executor child = 3

Used in: ExplainOperator

• unspecified = 0

• tidb = 1

• tikv = 2

• tiflash = 3

• optional Error error = 1

• optional bytes data = 3

  Data for all rows

• repeated Error warnings = 4

• repeated int64 output_counts = 5

  output row count for each executor

• optional int64 warning_count = 6

• repeated int64 ndvs = 7

• optional int64 table_id = 1

• repeated ColumnInfo columns = 2

Used in: Executor

• optional int64 table_id = 1

• repeated ColumnInfo columns = 2

• optional bool desc = 3

• repeated int64 primary_column_ids = 4

• optional EngineType next_read_engine = 5

  which engine we should in next step, only used by tiflash

• repeated KeyRange ranges = 6

  For global read in join, we must point out the key ranges when we don't have the region info.

• repeated int64 primary_prefix_column_ids = 7

• optional bool keep_order = 8

• optional bool is_fast_scan = 9

  fast_scan is a feature only provided by TiFlash (but not TiKV).

• repeated Expr pushed_down_filter_conditions = 10

  conditions that are pushed down to storage layer, only used by TiFlash.

• repeated RuntimeFilter runtime_filter_list = 11

  only used by TiFlash

• optional int32 max_wait_time_ms = 12

  only used by TiFlash

• optional ANNQueryInfo deprecated_ann_query = 13

  only used by TiFlash

• repeated ColumnarIndexInfo used_columnar_indexes = 14

  only used by TiFlash

Used in: ExplainOperator

• unknown = 0

• root = 1

• cop = 2

• batchCop = 3

• mpp = 4

TiCIVectorQueryInfo carries a TiCI vector top-k query. Used when a hybrid index with a vector component is chosen by the planner. The primary operator is ANN top-k; an optional filter expression can be pushed down so that TiCI can apply it before or during the ANN search.

Used in: IndexScan

• optional int64 index_id = 1

  The index that contains the vector component.

• optional int64 column_id = 2

  Column ID of the vector column being searched.

• optional VectorDistanceMetric distance_metric = 3

  Distance metric to use for the search.

• optional uint32 top_k = 4

  Number of nearest neighbors to return.

• optional bytes query_vector = 5

  The query vector, serialized as a sequence of little-endian float32 values.

• optional uint32 dimension = 6

  Dimensionality of the vector. Must match the index definition.

• repeated Expr filter_expr = 7

  Optional pushed-down filter expression (inverted/scalar predicates). When present, TiCI applies filtering in conjunction with ANN search.

• optional string column_name = 8

  Column name, for debug/explain purposes only.

• repeated ExecutorExecutionSummary execution_summaries = 1

  The execution summary of each executor, no order limitation.

Used in: ExecutorExecutionSummary

• optional uint64 inner_zone_send_bytes = 1

• optional uint64 inner_zone_receive_bytes = 2

• optional uint64 inter_zone_send_bytes = 3

• optional uint64 inter_zone_receive_bytes = 4

Used in: Aggregation

• ForcePreAgg = 0

• Auto = 1

• ForceStreaming = 2

Used in: TiFlashScanContext

• optional string instance_id = 1

• optional uint64 region_num = 2

Used in: ExecutorExecutionSummary

• optional uint64 dmfile_scanned_packs = 1

• optional uint64 dmfile_skipped_packs = 2

• optional uint64 dmfile_data_scanned_rows = 3

• optional uint64 dmfile_data_skipped_rows = 4

• optional uint64 total_dmfile_rs_load_ms = 5

• optional uint64 total_dmfile_read_ms = 6

• optional uint64 total_build_snapshot_ms = 7

• optional uint64 local_regions = 8

• optional uint64 remote_regions = 9

• optional uint64 user_read_bytes = 10

• optional uint64 total_learner_read_ms = 11

• optional uint64 disagg_read_cache_hit_bytes = 12

• optional uint64 disagg_read_cache_miss_bytes = 13

• optional uint64 total_dmfile_rs_check_ms = 14

• optional uint64 stale_read_regions = 15

• optional uint64 segments = 16

• optional uint64 read_tasks = 17

• optional uint64 delta_rows = 18

• optional uint64 delta_bytes = 19

• optional uint64 mvcc_input_rows = 20

• optional uint64 mvcc_input_bytes = 21

• optional uint64 mvcc_output_rows = 22

• optional uint64 lm_skip_rows = 23

• optional uint64 total_build_inputstream_ms = 24

• optional uint64 total_build_bitmap_ms = 25

• optional uint64 min_local_stream_ms = 26

• optional uint64 max_local_stream_ms = 27

• optional uint64 min_remote_stream_ms = 28

• optional uint64 max_remote_stream_ms = 29

• repeated TiFlashRegionNumOfInstance regions_of_instance = 30

• optional uint64 dmfile_mvcc_scanned_rows = 31

• optional uint64 dmfile_mvcc_skipped_rows = 32

• optional uint64 dmfile_lm_filter_scanned_rows = 33

• optional uint64 dmfile_lm_filter_skipped_rows = 34

• optional uint64 vector_idx_load_from_s3 = 100

  Index file not available in disk

• optional uint64 vector_idx_load_from_disk = 101

  Index file available in disk and not cached in memory

• optional uint64 vector_idx_load_from_cache = 102

  Index file available in disk and cached in memory

• optional uint64 vector_idx_load_time_ms = 103

• optional uint64 vector_idx_search_time_ms = 104

• optional uint64 vector_idx_search_visited_nodes = 105

• optional uint64 vector_idx_search_discarded_nodes = 106

• optional uint64 vector_idx_read_vec_time_ms = 107

• optional uint64 vector_idx_read_others_time_ms = 108

• optional uint32 inverted_idx_load_from_s3 = 120

• optional uint32 inverted_idx_load_from_disk = 121

• optional uint32 inverted_idx_load_from_cache = 122

• optional uint64 inverted_idx_load_time_ms = 123

• optional uint64 inverted_idx_search_time_ms = 124

• optional uint32 inverted_idx_search_skipped_packs = 125

• optional uint64 inverted_idx_indexed_rows = 126

• optional uint64 inverted_idx_search_selected_rows = 127

• optional uint32 fts_n_from_inmemory_noindex = 150

• optional uint32 fts_n_from_tiny_index = 151

• optional uint32 fts_n_from_tiny_noindex = 152

• optional uint32 fts_n_from_dmf_index = 153

• optional uint32 fts_n_from_dmf_noindex = 154

• optional uint64 fts_rows_from_inmemory_noindex = 155

• optional uint64 fts_rows_from_tiny_index = 156

• optional uint64 fts_rows_from_tiny_noindex = 157

• optional uint64 fts_rows_from_dmf_index = 158

• optional uint64 fts_rows_from_dmf_noindex = 159

• optional uint64 fts_idx_load_total_ms = 160

• optional uint32 fts_idx_load_from_cache = 161

• optional uint32 fts_idx_load_from_column_file = 162

• optional uint32 fts_idx_load_from_stable_s3 = 163

• optional uint32 fts_idx_load_from_stable_disk = 164

• optional uint32 fts_idx_search_n = 165

• optional uint64 fts_idx_search_total_ms = 166

• optional uint64 fts_idx_dm_search_rows = 167

• optional uint64 fts_idx_dm_total_read_fts_ms = 168

• optional uint64 fts_idx_dm_total_read_others_ms = 169

• optional uint64 fts_idx_tiny_search_rows = 170

• optional uint64 fts_idx_tiny_total_read_fts_ms = 171

• optional uint64 fts_idx_tiny_total_read_others_ms = 172

• optional uint64 fts_brute_total_read_ms = 173

• optional uint64 fts_brute_total_search_ms = 174

• optional uint32 tici_vec_query_count = 200

• optional uint64 tici_vec_search_total_ms = 201

• optional uint64 tici_vec_filter_total_ms = 202

• optional uint64 tici_vec_candidate_count = 203

• optional uint64 tici_vec_result_count = 204

• optional uint64 tici_vec_shard_count = 205

Used in: ExecutorExecutionSummary

• optional uint64 minTSO_wait_ns = 1

  Time waited for minTSO satified

• optional uint64 pipeline_queue_wait_ns = 2

  Time waited in task queue.

• optional uint64 pipeline_breaker_wait_ns = 3

  Time waited for dependant pipeline finished.

Used in: Executor

• repeated ByItem order_by = 1

  Order by clause.

• optional uint64 limit = 2

• optional Executor child = 3

• repeated ByItem partition_by = 4

  If partition_by is not empty, it means need to return topN of each partition. Generally used in sqls like `where row_number() over (partition by ... order by ...) < X`

TopRU-only options

Used in: TopSQLSubRequest

• ItemInterval item_interval_seconds = 1

  allowed: 15/30/60; server validates and applies default if 0

TopRURecord represents RU statistics for a single (user, sql_digest, plan_digest) combination.

Used as request type in: TopSQLAgent.ReportTopRURecords

Used as field type in: TopSQLSubResponse

• bytes keyspace_name = 1

• string user = 2

• bytes sql_digest = 3

• bytes plan_digest = 4

• repeated TopRURecordItem items = 5

TopRURecordItem represents statistics within a single time bucket.

Used in: TopRURecord

• uint64 timestamp_sec = 1

  timestamp in second

• double total_ru = 2

  cumulative RU consumption (RRU + WRU)

• uint64 exec_count = 3

  execution count

• uint64 exec_duration = 4

  cumulative execution time (nanoseconds)

Used as request type in: TopSQLAgent.ReportTopSQLRecords

Used as field type in: TopSQLSubResponse

• bytes sql_digest = 1

• bytes plan_digest = 2

• repeated TopSQLRecordItem items = 3

• bytes keyspace_name = 4

Used in: TopSQLRecord

• uint64 timestamp_sec = 1

  timestamp in second

• uint32 cpu_time_ms = 2

  this value can be greater than 1000 when counting concurrent running SQL queries

• uint64 stmt_exec_count = 3

• map<string, uint64> stmt_kv_exec_count = 4

  target => count

• uint64 stmt_duration_sum_ns = 5

• uint64 stmt_duration_count = 6

• uint64 stmt_network_in_bytes = 7

  traffic from client

• uint64 stmt_network_out_bytes = 8

  traffic to client

Used in: DAGRequest

• optional string user_name = 1

• optional string user_host = 2

Note: The name of the enum is intentionally aligned with tidb/parser/index_vector.go.

Used in: ANNQueryInfo, TiCIVectorQueryInfo

• INVALID_DISTANCE_METRIC = 0

• L1 = 1

• L2 = 2

• COSINE = 3

• INNER_PRODUCT = 4

Note: The name of the enum is intentionally aligned with tidb/parser/index_vector.go.

• INVALID_INDEX_KIND = 0

• HNSW = 1

Used in: Executor

• repeated Expr func_desc = 1

• repeated ByItem partition_by = 2

• repeated ByItem order_by = 3

• optional WindowFrame frame = 4

• optional Executor child = 5

Used in: WindowFrameBound

• Following = 0

• Preceding = 1

• CurrentRow = 2

Used in: Window

• optional WindowFrameType type = 1

• optional WindowFrameBound start = 2

• optional WindowFrameBound end = 3

Used in: WindowFrame

• optional WindowBoundType type = 1

• optional bool unbounded = 2

• optional uint64 offset = 3

  only use for `rows` frame type

• repeated Expr calcFuncs = 4

  Deprecated

• optional Expr frame_range = 5

  only use for `range` frame type

• optional RangeCmpDataType cmp_data_type = 6

  only use for `range` frame type

Used in: WindowFrame

• Rows = 0

• Ranges = 1

• Groups = 2