Proto commits in pingcap/kvproto

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

Commit:e8e4e3c
Author:Dongpo Liu
Committer:GitHub

coprocessor: add batch task merge and serial execution controls (#1497) (#1592) Add explicit controls for batched coprocessor task results and execution: - Request.allow_batch_task_data_merge: the client declares it can consume a response whose batched tasks' results are merged into the single result carried by the response's data (e.g. the per-region sampling results of a batched analyze request). - StoreBatchTaskResponse.data_merged_into_response: the store acknowledges a merged task; the batch response carries no data of its own but still acknowledges the task and carries its execution details. - Request.execute_batch_tasks_serially: the client asks the store to run the primary task and all batched tasks one at a time, limiting within-request concurrency without guaranteeing execution order. A store that does not support merging keeps responding per task, and a client that does not set the merge request field keeps receiving per-task responses, so either side can upgrade first. Stores that do not support serial execution ignore the new request field and retain existing behavior. Signed-off-by: 0xPoe <poe.liu@pm.me>

Commit:14a9fc6
Author:JmPotato
Committer:GitHub

resource_manager: add replayable per-second RU observations (#1539) (#1588) * resource_manager: add replayable per-second RU observations * resource_manager: encode per-second RU timelines sparsely Add end_unix_sec as the coverage watermark and offset to each bucket so clients only send non-zero seconds. Seconds not listed are implicit zeros, which keeps idle groups publishing while cutting the payload of a large batch by an order of magnitude. * resource_manager: clarify the per-second RU report contract State that a nil ru_by_second means the request carries no timeline, including when the client leaves it out to fit the RPC size limit; that only the unlisted seconds inside [start_unix_sec, end_unix_sec) are zero; that an empty or reversed interval reports nothing; and that offsets must be strictly increasing and inside the interval, or the whole report is dropped. Comments only. * resource_manager: name the full deduplication key of RU reports The receiver deduplicates replayed seconds per client, keyspace, resource group and second, not per client and second alone. State that, and that only foreground consumption carries per-second reports. Comments only. * resource_manager: say that a report is acknowledged as a whole The receiver ignores seconds outside the window it can still summarize, yet a successful response acknowledges them with the rest of the report. State it in the contract, so an implementation reading only the proto knows that such seconds are not resent. --------- (cherry picked from commit 9192d38c8b671062fcbda39b39d71cb9ab9cede4) Signed-off-by: JmPotato <github@ipotato.me>

Commit:08fbbc3
Author:Ryan Leung
Committer:GitHub

pdpb: add headers to global config requests (#1571) * pdpb: add headers to global config requests Signed-off-by: Ryan Leung <rleungx@gmail.com> * pdpb: remove global config header compatibility test Signed-off-by: Ryan Leung <rleungx@gmail.com> --------- Signed-off-by: Ryan Leung <rleungx@gmail.com>

The documentation is generated from this commit.

Commit:1bf153f
Author:ris
Committer:GitHub

brpb: add S3 SSE-KMS encryption context (#1552) (#1583) (cherry picked from commit f8ad079c8e9812ed5900b0c12b5f1d97171464c4) Signed-off-by: ris <79858083+RidRisR@users.noreply.github.com>

Commit:9192d38
Author:JmPotato
Committer:GitHub

resource_manager: add replayable per-second RU observations (#1539) * resource_manager: add replayable per-second RU observations Signed-off-by: JmPotato <github@ipotato.me> * resource_manager: encode per-second RU timelines sparsely Add end_unix_sec as the coverage watermark and offset to each bucket so clients only send non-zero seconds. Seconds not listed are implicit zeros, which keeps idle groups publishing while cutting the payload of a large batch by an order of magnitude. Signed-off-by: JmPotato <github@ipotato.me> * resource_manager: clarify the per-second RU report contract State that a nil ru_by_second means the request carries no timeline, including when the client leaves it out to fit the RPC size limit; that only the unlisted seconds inside [start_unix_sec, end_unix_sec) are zero; that an empty or reversed interval reports nothing; and that offsets must be strictly increasing and inside the interval, or the whole report is dropped. Comments only. Signed-off-by: JmPotato <github@ipotato.me> * resource_manager: name the full deduplication key of RU reports The receiver deduplicates replayed seconds per client, keyspace, resource group and second, not per client and second alone. State that, and that only foreground consumption carries per-second reports. Comments only. Signed-off-by: JmPotato <github@ipotato.me> * resource_manager: say that a report is acknowledged as a whole The receiver ignores seconds outside the window it can still summarize, yet a successful response acknowledges them with the rest of the report. State it in the contract, so an implementation reading only the proto knows that such seconds are not resent. Signed-off-by: JmPotato <github@ipotato.me> --------- Signed-off-by: JmPotato <github@ipotato.me>

Commit:2c5f843
Author:ris
Committer:GitHub

brpb: add S3 SSE-KMS encryption context (#1552) (#1564) (cherry picked from commit f8ad079c8e9812ed5900b0c12b5f1d97171464c4) Signed-off-by: ris <79858083+RidRisR@users.noreply.github.com>

Commit:35c8d02
Author:lucasliang
Committer:GitHub

brpb: cherry-pick PiTR restore name routes (#1530) and ID map source name (#1549) to release-8.5 (#1560) * brpb: support PiTR restore name routes (#1530) This is a cherry-pick of #1530 to release-8.5. Add the protobuf metadata required by BR and PiTR to restore data to a different schema or table name. - Add `BackupMeta.pitr_id_map_route_fingerprint` to persist the canonical restore-route fingerprint with a PiTR ID map. - Extend `PitrTableMap` with the effective downstream database identity (`downstream_db_id` and `downstream_db_name`). - Add `has_foreign_keys`, `is_view` and `foreign_key_references` metadata so BR/PiTR can apply the restore-rename dependency safety checks. - Add `PitrDBMap.schema_routed` so a retry that loads the PiTR ID map keeps replaying the target schema's DBInfo even when all of its tables are routed elsewhere. - Bump `BackupSchemaVersion` from 1 to 2 because the backup metadata semantics have changed. - Regenerate the Go protobuf bindings and `scripts/proto.lock`. The new fields are additive. Zero/empty values preserve the behavior of backups and ID maps produced before restore-rename support, and older readers can ignore the unknown fields. Conflict resolution for release-8.5: upstream #1530 also carried the regenerated `pkg/pdpb/pdpb.pb.go` produced by a proto-format sync with master, which is unrelated to this change. That file is not part of this cherry-pick. `pkg/brpb/brpb.pb.go` and `scripts/proto.lock` were regenerated on release-8.5 with `make go` (protolock + protoc-gen-gofast) so they match this branch's formatting and protoc version. (cherry picked from commit 17b3dcbfa876807d1d4c360552f837691bdb159b) Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> * brpb: persist the source schema name in the PiTR ID map (#1549) This is a cherry-pick of #1549 to release-8.5. `PitrDBMap.name` (field 1) is overloaded: once a schema-level rename rule is applied it holds the effective downstream schema name, so the upstream source name is no longer recoverable from the persisted ID map. BR's PiTR restore persists this map (`mysql.tidb_pitr_id_map` or the checkpoint storage) and reads it back to keep the upstream->downstream identity stable, which is the only source of truth for a task that rebuilds its mapping from a log interval alone (log-only restore / split job / checkpoint resume). Add `PitrDBMap.source_name` (field 7) so the upstream source schema name is persisted separately and name routing can always be re-resolved from the upstream identity. Table-level source names do not need a new field: they are already recoverable from the log table history's latest location, and `PitrTableMap` is unchanged. Regenerate the Go protobuf bindings and `scripts/proto.lock`. Purely additive: new field number 7 (fields 1-6 already used), no renumbering, no wire-format change. An older reader ignores the field; an older writer leaves it empty and the consumer falls back to `name`. Conflict resolution for release-8.5: `PitrDBMap` is indented with 4 spaces on this branch while upstream #1549 was based on the 2-space proto-format sync with master, so the hunk was re-applied by hand in this branch's style and `pkg/brpb/brpb.pb.go` / `scripts/proto.lock` were regenerated with `make go` (protolock + protoc-gen-gofast) instead of taking the upstream generated files. (cherry picked from commit 4141ecadac8f689b06700ebbfbafab94b14266c7) Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> --------- Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>

Commit:f8ad079
Author:ris
Committer:GitHub

brpb: add S3 SSE-KMS encryption context (#1552) Signed-off-by: ris <79858083+RidRisR@users.noreply.github.com>

Commit:2856107
Author:Artem Danilov
Committer:GitHub

copr: add max_keys_read coprocessor Request (#1447) (#1550) (cherry picked from commit bb322278e9e549252dcbb6acc73071afd9908095) Signed-off-by: artem_danilov <artem.danilov@airbnb.com> Co-authored-by: tiancaiamao <tiancaiamao@gmail.com>

Commit:4141eca
Author:lucasliang
Committer:GitHub

brpb: persist the source schema name in the PiTR ID map (#1549) PitrDBMap.name holds the effective downstream schema name once a schema-level route is applied, so a later log-only restore or a checkpoint resume can no longer recover the upstream source name. Add a dedicated source_name field so name routing can always be re-resolved from the upstream identity. Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>

Commit:f4a9633
Author:xufei
Committer:GitHub

Port region split, PITR, and Top SQL protocol changes to materialized view (#1551) * pd: support disable-auto-split region (#1351) (#1386) (#1464) * pd: support disable-auto-split region (#1351) * pd: support disable-auto-split region * add comment --------- * generate go code --------- Signed-off-by: HunDunDM <hundundm@gmail.com> Signed-off-by: lhy1024 <admin@liudos.us> Co-authored-by: 混沌DM <hundundm@gmail.com> * cherry-pick compacted based PITR restore (#1492) * metaedit: add new field to record meta files that can be skipped (#1338) * update metaedit field name (#1339) * metaedit: add new field to record meta files that can be skipped * metaedit: rename field name * brpb with bytes (#1353) * brpb with bytes Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * brpb with bytes Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * brpb with bytes Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * brpb with bytes Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * brpb with bytes Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * add scripts to check Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * update rust-protobuf version Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * update rust-protobuf version Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * revert cargo version Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * test Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * test Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * revert debug Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * commit some suggestions Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * fix typos Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> --------- Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * save stats in migration (#1481) Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * br: Add batch download rpc for compact (#1360) * fix * trigger tests --------- Co-authored-by: Jianjun Liao <36503113+Leavrth@users.noreply.github.com> * add batch download latest mvcc rpc function (#1456) Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> --------- Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> Co-authored-by: 3pointer <luancheng@pingcap.com> Co-authored-by: ris <79858083+RidRisR@users.noreply.github.com> * resource_usage_agent: add RocksDB block read count to Top SQL resource dimensions (#1498) (#1537) * This is an automated cherry-pick of #1498 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> * resource_usage_agent: resolve release-8.5-20251208-v8.5.4 cherry-pick conflicts - proto: keep this branch's 4-space style, add rocksdb_block_read_count = 9 - pb.go: regenerated with the pinned toolchain (protoc-gen-gofast v1.3.2, goimports v0.6.0 per go.mod) so the XXX_* style and fileDescriptor bytes stay consistent with this branch Signed-off-by: jiong-nba <jiongnba@gmail.com> --------- Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> Signed-off-by: jiong-nba <jiongnba@gmail.com> Co-authored-by: Jiong <jiongnba@gmail.com> --------- Signed-off-by: HunDunDM <hundundm@gmail.com> Signed-off-by: lhy1024 <admin@liudos.us> Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> Signed-off-by: jiong-nba <jiongnba@gmail.com> Co-authored-by: lhy1024 <liuhanyang@pingcap.com> Co-authored-by: 混沌DM <hundundm@gmail.com> Co-authored-by: Jianjun Liao <36503113+Leavrth@users.noreply.github.com> Co-authored-by: 3pointer <luancheng@pingcap.com> Co-authored-by: ris <79858083+RidRisR@users.noreply.github.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io> Co-authored-by: Jiong <jiongnba@gmail.com>

Commit:17b3dcb
Author:lucasliang
Committer:GitHub

brpb: support PiTR restore name routes (#1530) * brpb: support PiTR restore name routes Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> * Update proto. Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> * Persist ForeignKeyReferences. Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> * brpb: add PitrDBMap.schema_routed Persist whether an explicit schema-level rename rule matched a source schema, so a retry that loads the PiTR ID map keeps replaying the target schema's DBInfo even when all of its tables are routed elsewhere. Signed-off-by: lucasliang <nkcs_lykx@hotmail.com> --------- Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>

Commit:d3340ae
Author:mittalrishabh
Committer:GitHub

kvrpcpb: report noisy resource groups in HealthFeedback (#1543) TiKV already knows which resource groups it blames for its own overload -- `ResourceGroupManager` keeps the set in `noisy_groups` and refreshes it on the control tick -- but the only way that verdict reaches a client today is the `|noisy_tenant` suffix on a `ServerIsBusy` reason, which arrives only when a request from that group is actually rejected. Every group has to be rejected once to be discovered, and the client has no way to learn that a group stopped being noisy. Carry the set on `HealthFeedback` instead, which is already attached to batch responses on a timer, so a client learns the whole set from any response. `noisy_groups` is a message rather than a bare repeated field so that a store which does not report the set at all stays distinguishable from one reporting an empty set: the former leaves the field unset, the latter sends an empty `names`, and only the latter licenses a client to clear what it knew. Signed-off-by: rishabh mittal <mittalrishabh@gmail.com> Co-authored-by: Jinpeng Zhang <zhangjinpeng@pingcap.com>

Commit:121875f
Author:mittalrishabh
Committer:GitHub

kvrpcpb: report noisy resource groups in HealthFeedback (#1544) TiKV already knows which resource groups it blames for its own overload -- `ResourceGroupManager` keeps the set in `noisy_groups` and refreshes it on the control tick -- but the only way that verdict reaches a client today is the `|noisy_tenant` suffix on a `ServerIsBusy` reason, which arrives only when a request from that group is actually rejected. Every group has to be rejected once to be discovered, and the client has no way to learn that a group stopped being noisy. Carry the set on `HealthFeedback` instead, which is already attached to batch responses on a timer, so a client learns the whole set from any response. `noisy_groups` is a message rather than a bare repeated field so that a store which does not report the set at all stays distinguishable from one reporting an empty set: the former leaves the field unset, the latter sends an empty `names`, and only the latter licenses a client to clear what it knew. (cherry picked from commit 15db7826d07d932bd9379dccb13b03e71335b6d3) Signed-off-by: rishabh mittal <mittalrishabh@gmail.com>

Commit:25d8b5a
Author:crazycs520

feat(kvproto): add scan and lock request

Commit:d80b6c5
Author:zyguan
Committer:GitHub

*: add txn protocol version declaration (#1534) Introduce a protocol version registry and the request declaration that lets a client tell the server which transaction protocol semantics it implements, so that requests from callers which cannot handle a feature can be rejected consistently instead of being served with best effort. - kvrpcpb: add the TxnProtocolVersion enum and Context.txn_protocol_version (tag 39). Version numbers accumulate monotonically: 0 is legacy, 1 means the client handles the structured incompatibility error, and 2 covers the shared lock protocol. A missing field and an explicit 0 both mean legacy. Also extend RequestOrigin with TiCDC, BR and TiFlash and document that the origin is client-provided auditing metadata, not an authorization decision. - errorpb: add IncompatibleRequest (reason, message, provided/min/max compatible version) and Error.incompatible_request (tag 23). The bounds are inclusive and the two error fields stay independent message fields. A rejection of a request that declared no version also sets ServerIsBusy.reason to the fixed "txn_protocol_incompatible" fallback for legacy clients; a declared non-zero version gets no fallback. - metapb: add TxnProtocolVersionRange and Store.txn_protocol_version_range (tag 14). The message field keeps its presence so that an absent range stays distinguishable from an explicit [0, 0]. Signed-off-by: zyguan <zhongyangguan@gmail.com>

Commit:b3733f7
Author:Ti Chi Robot
Committer:GitHub

resource_usage_agent: add RocksDB block read count to Top SQL resource dimensions (#1498) (#1527) * resource_usage_agent: add RocksDB block read count Signed-off-by: jiong-nba <jiongnba@gmail.com> * resource_usage_agent: document block read count semantics Signed-off-by: jiong-nba <jiongnba@gmail.com> --------- Signed-off-by: jiong-nba <jiongnba@gmail.com> Co-authored-by: jiong-nba <jiongnba@gmail.com>

Commit:8d2e3a2
Author:crazycs520

feat: add RaftGroup Store 2PC RPCs (#166) Reviewed-on: https://git.pingcap.net/pingkai/kvproto/pulls/166

Commit:0c9a611
Author:Ti Chi Robot
Committer:GitHub

resource_usage_agent: add RocksDB block read count to Top SQL resource dimensions (#1498) (#1537) * This is an automated cherry-pick of #1498 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> * resource_usage_agent: resolve release-8.5-20251208-v8.5.4 cherry-pick conflicts - proto: keep this branch's 4-space style, add rocksdb_block_read_count = 9 - pb.go: regenerated with the pinned toolchain (protoc-gen-gofast v1.3.2, goimports v0.6.0 per go.mod) so the XXX_* style and fileDescriptor bytes stay consistent with this branch Signed-off-by: jiong-nba <jiongnba@gmail.com> --------- Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> Signed-off-by: jiong-nba <jiongnba@gmail.com> Co-authored-by: Jiong <jiongnba@gmail.com>

Commit:5ccd130
Author:wlwilliamx

replication_modepb: add standby import mode flag (#160) Reviewed-on: https://git.pingcap.net/pingkai/kvproto/pulls/160 Reviewed-by: crazycs520 <chenshuang@pingcap.cn>

Commit:ab27878
Author:Ti Chi Robot
Committer:GitHub

resource_usage_agent: add RocksDB block read count to Top SQL resource dimensions (#1498) (#1524) * This is an automated cherry-pick of #1498 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> * resource_usage_agent: resolve release-8.5 cherry-pick conflicts Signed-off-by: jiong-nba <jiongnba@gmail.com> --------- Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> Signed-off-by: jiong-nba <jiongnba@gmail.com> Co-authored-by: Jiong <jiongnba@gmail.com>

Commit:f350952
Author:wshwsh12
Committer:3pointer

fmt

Commit:07fe5eb
Author:wshwsh12
Committer:3pointer

update tici

Commit:cb158a9
Author:wshwsh12
Committer:3pointer

proto

Commit:65b4e27
Author:Wenxuan
Committer:GitHub

feat(pdpb): add WatchGCStates API (#1528) Add a server-streaming API for observing effective GC state changes. Represent updates as ordered upsert and removal events so clients can maintain a materialized per-keyspace view and reload it after reconnecting. Keep GC barriers internal to PD. The stream omits barrier details and barrier-only updates while still reporting changes to the resulting effective safe points. Signed-off-by: Wenxuan Zhang <wenxuangm@gmail.com> Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

Commit:107095f
Author:Connor
Committer:GitHub

pdpb: report IA row storage size (#1518) * pdpb: report IA row storage size Signed-off-by: Connor1996 <zbk602423539@gmail.com> * pdpb: report Standard row storage size Signed-off-by: Connor1996 <zbk602423539@gmail.com> * pdpb: remove redundant Standard row storage size Signed-off-by: Connor1996 <zbk602423539@gmail.com> --------- Signed-off-by: Connor1996 <zbk602423539@gmail.com>

Commit:e98c296
Author:cfzjywxk
Committer:GitHub

kvrpcpb: add request origin to context (#1525) Backport the request-origin metadata added by #1479 to release-nextgen-202603. Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

Commit:63682db
Author:Jiong
Committer:jiong-nba

resource_usage_agent: add RocksDB block read count to Top SQL resource dimensions (#1498) * resource_usage_agent: add RocksDB block read count Signed-off-by: jiong-nba <jiongnba@gmail.com> * resource_usage_agent: document block read count semantics Signed-off-by: jiong-nba <jiongnba@gmail.com> --------- Signed-off-by: jiong-nba <jiongnba@gmail.com> Co-authored-by: ShuNing <nolouch@gmail.com> (cherry picked from commit af1ad1bef5b9dd611af5282711f47dbc31a42530)

Commit:af5291b
Author:wlwilliamx

replication_mode: add replication identity fields (#148) Reviewed-on: https://git.pingcap.net/pingkai/kvproto/pulls/148 Reviewed-by: crazycs520 <chenshuang@pingcap.cn>

Commit:623e58e
Author:Dongpo Liu
Committer:GitHub

coprocessor: add batch task merge and serial execution controls (#1497) Add explicit controls for batched coprocessor task results and execution: - Request.allow_batch_task_data_merge: the client declares it can consume a response whose batched tasks' results are merged into the single result carried by the response's data (e.g. the per-region sampling results of a batched analyze request). - StoreBatchTaskResponse.data_merged_into_response: the store acknowledges a merged task; the batch response carries no data of its own but still acknowledges the task and carries its execution details. - Request.execute_batch_tasks_serially: the client asks the store to run the primary task and all batched tasks one at a time, limiting within-request concurrency without guaranteeing execution order. A store that does not support merging keeps responding per task, and a client that does not set the merge request field keeps receiving per-task responses, so either side can upgrade first. Stores that do not support serial execution ignore the new request field and retain existing behavior. Signed-off-by: 0xPoe <poe.liu@pm.me>

Commit:9327469
Author:Wenxuan
Committer:GitHub

feat(kvrpcpb): add typed shared-lock upgrade errors (#1495) * feat(kvrpcpb): add lock upgrade conflict error Add the typed LockUpgradeConflict message to kvrpcpb and expose it on KeyError as field 13. Regenerate the corresponding Go bindings and update proto.lock so downstream repos can depend on a stable machine-readable upgrade-conflict protocol shape. * kvrpcpb: add shared lock lost error Signed-off-by: Wenxuan Zhang <wenxuangm@gmail.com> * kvrpcpb: document shared lock loss semantics Signed-off-by: Wenxuan Zhang <wenxuangm@gmail.com> --------- Signed-off-by: Wenxuan Zhang <wenxuangm@gmail.com>

Commit:af1ad1b
Author:Jiong
Committer:GitHub

resource_usage_agent: add RocksDB block read count to Top SQL resource dimensions (#1498) * resource_usage_agent: add RocksDB block read count Signed-off-by: jiong-nba <jiongnba@gmail.com> * resource_usage_agent: document block read count semantics Signed-off-by: jiong-nba <jiongnba@gmail.com> --------- Signed-off-by: jiong-nba <jiongnba@gmail.com> Co-authored-by: ShuNing <nolouch@gmail.com>

Commit:543bb79
Author:Wenxuan
Committer:GitHub

pdpb: expose global barriers in GetGCState (#1516) * pdpb: expose global barriers in GetGCState Signed-off-by: Wenxuan Zhang <wenxuangm@gmail.com> * pdpb: explain global barrier include flag Signed-off-by: Wenxuan Zhang <wenxuangm@gmail.com> * pdpb: sync generated field comment Signed-off-by: Wenxuan Zhang <wenxuangm@gmail.com> --------- Signed-off-by: Wenxuan Zhang <wenxuangm@gmail.com>

Commit:fb4b0fd
Author:jiong-nba

resource_usage_agent: document block read count semantics Signed-off-by: jiong-nba <jiongnba@gmail.com>

Commit:db3cf69
Author:jiong-nba

resource_usage_agent: document block read count semantics Signed-off-by: jiong-nba <jiongnba@gmail.com>

Commit:93fe539
Author:Ti Chi Robot
Committer:GitHub

keyspacepb: add load keyspace by ID RPC (#1509) Signed-off-by: Ryan Leung <rleungx@gmail.com> Co-authored-by: Ryan Leung <rleungx@gmail.com>

Commit:07aa8c6
Author:Jianjun Liao
Committer:GitHub

set download speed limit with ttl (#1430) (#1513) Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>

Commit:dc53c46
Author:Ti Chi Robot
Committer:GitHub

kvrpcpb: add read-pool task details (#1502) (#1508) * This is an automated cherry-pick of #1502 Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> * update --------- Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> Co-authored-by: Chao Wang <cclcwangchao@hotmail.com>

Commit:059694a
Author:disksing
Committer:GitHub

keyspacepb: split namespace service (#1505) Signed-off-by: disksing <i@disksing.com>

Commit:7022f46
Author:disksing
Committer:GitHub

add API V3 keyspace identity protos (#1458) * api: add API V3 keyspace identity protos Signed-off-by: disksing <i@disksing.com> * api: update proto lock Signed-off-by: disksing <i@disksing.com> * api: keep TSO group response focused Signed-off-by: disksing <i@disksing.com> * api-v3: normalize keyspace identity fields Signed-off-by: disksing <i@disksing.com> * api: remove namespace from keyspace lookup Signed-off-by: disksing <i@disksing.com> * Use oneof for keyspace identity fields Signed-off-by: disksing <i@disksing.com> * Clarify keyspace pagination comment Signed-off-by: disksing <i@disksing.com> * Adjust DFS stat keyspace scope fields Signed-off-by: disksing <i@disksing.com> * Remove GC safe point response keyspace identity Signed-off-by: disksing <i@disksing.com> * Remove service safe point response keyspace identity Signed-off-by: disksing <i@disksing.com> * Merge keyspace group lookup by identity Signed-off-by: disksing <i@disksing.com> * Add namespace metadata APIs Signed-off-by: disksing <i@disksing.com> * Make keyspace meta identity backward compatible * Keep AutoIDRequest keyspace fields compatible * Use oneof for remaining keyspace identities Signed-off-by: disksing <i@disksing.com> * keyspacepb: support V3 identity in load-by-ID request Signed-off-by: disksing <i@disksing.com> * externalworkloadpb: support V3 keyspace identity Signed-off-by: disksing <i@disksing.com> * autoid: identify keyspace in rebase requests Signed-off-by: disksing <i@disksing.com> * resource-manager: support V3 keyspace identity Signed-off-by: disksing <i@disksing.com> * externalworkloadpb: use oneof for keyspace identity Signed-off-by: disksing <i@disksing.com> * pdpb: remove unused TSO keyspace identity Signed-off-by: disksing <i@disksing.com> * Apply suggestion from @sunxiaoguang --------- Signed-off-by: disksing <i@disksing.com> Co-authored-by: disksing <disksing@macmini.local> Co-authored-by: Xiaoguang Sun <sunxiaoguang@users.noreply.github.com>

Commit:64fac48
Author:disksing

externalworkloadpb: use oneof for keyspace identity Signed-off-by: disksing <i@disksing.com>

Commit:57fb9c0
Author:disksing

resource-manager: support V3 keyspace identity Signed-off-by: disksing <i@disksing.com>

Commit:e30d0e9
Author:disksing

autoid: identify keyspace in rebase requests Signed-off-by: disksing <i@disksing.com>

Commit:ec90f51
Author:disksing

externalworkloadpb: support V3 keyspace identity Signed-off-by: disksing <i@disksing.com>

Commit:bd6c2b1
Author:disksing

keyspacepb: support V3 identity in load-by-ID request Signed-off-by: disksing <i@disksing.com>

Commit:b952ba8
Author:disksing

Merge remote-tracking branch 'origin/master' into task4/apiv3-identity-oneof # Conflicts: # pkg/kvrpcpb/kvrpcpb.pb.go

Commit:683dad8
Author:Chao Wang
Committer:GitHub

kvrpcpb: add read-pool task details (#1502) * kvrpcpb: add read_pool_task_details to trace read pool metrics:w * address comment

Commit:fd32127
Author:disksing

Use oneof for remaining keyspace identities Signed-off-by: disksing <i@disksing.com>

Commit:3631780
Author:JmPotato
Committer:GitHub

coprocessor: backport paging limits to release-nextgen-202603 (#1501) * copr: add max_keys_read coprocessor Request (#1447) Signed-off-by: artem_danilov <artem.danilov@airbnb.com> (cherry picked from commit bb322278e9e549252dcbb6acc73071afd9908095) Signed-off-by: JmPotato <github@ipotato.me> * proto: add paging_size_bytes to coprocessor.Request (#1448) Add a new uint64 field `paging_size_bytes` (field 17) to the coprocessor Request message. When set, TiKV will stop scanning when accumulated scanned bytes reach this limit, enabling byte-budget paging for finer-grained Resource Control. Note: rebased onto upstream master where field 16 is now taken by max_keys_read (#1450), so paging_size_bytes was bumped from field 16 to field 17. Signed-off-by: JmPotato <github@ipotato.me> (cherry picked from commit a15c348f8713e5c492b6dfb41224c464b5824979) Signed-off-by: JmPotato <github@ipotato.me> --------- Signed-off-by: artem_danilov <artem.danilov@airbnb.com> Signed-off-by: JmPotato <github@ipotato.me> Co-authored-by: Artem Danilov <tema.danilov@gmail.com>

Commit:2260772
Author:hongyunyan
Committer:GitHub

cdcpb: add CDC scan priority (#1493) * cdcpb: add scan priority to change data request * cdcpb: remove changefeed id from scan request Signed-off-by: hongyunyan <649330952@qq.com> * cdcpb: regenerate go binding for scan priority Signed-off-by: hongyunyan <649330952@qq.com> --------- Signed-off-by: hongyunyan <649330952@qq.com>

Commit:91eb3e4
Author:Jianjun Liao
Committer:GitHub

save stats in migration (#1481) (#1499) Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com>

Commit:78af7ed
Author:jiong

resource_usage_agent: add RocksDB block read count Signed-off-by: jiong <jiong@jiongdeMacBook-Pro.local>

Commit:5fa7b0c
Author:jiong-nba

resource_usage_agent: add RocksDB block read count Signed-off-by: jiong-nba <jiongnba@gmail.com>

Commit:7761cfb
Author:hongyunyan
Committer:hongyunyan

cdcpb: remove changefeed id from scan request Signed-off-by: hongyunyan <649330952@qq.com>

Commit:c3fb31a
Author:disksing

Keep AutoIDRequest keyspace fields compatible

Commit:57eabd0
Author:disksing

Make keyspace meta identity backward compatible

Commit:c57bf39
Author:disksing

Merge remote-tracking branch 'origin/master' into apiv3 # Conflicts: # pkg/keyspacepb/keyspacepb.pb.go # proto/keyspacepb.proto # scripts/proto.lock

Commit:1855247
Author:hongyunyan

cdcpb: merge latest master for scan priority Signed-off-by: hongyunyan <649330952@qq.com>

Commit:e800f9e
Author:lhy1024
Committer:GitHub

schedulingpb: add split reason to ask batch split request (#1459) (#1475) Signed-off-by: lhy1024 <19542290+lhy1024@users.noreply.github.com>

Commit:b4ca3f1
Author:Shenghui Wu
Committer:disksing

coprocessor: add TiCI estimate count RPC (#1457) * coprocessor: add TiCI estimate count rpc * Update proto/tikvpb.proto Co-authored-by: cfzjywxk <cfzjywxk@gmail.com> * tikvpb: fix TiFlash comment casing --------- Co-authored-by: cfzjywxk <cfzjywxk@gmail.com> Signed-off-by: disksing <i@disksing.com>

Commit:a0891f1
Author:Chao Wang
Committer:disksing

add fields to get commit timestamp in Get or BatchGet (#1371) (#1382) Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> Signed-off-by: disksing <i@disksing.com>

Commit:1a93aa8
Author:王超
Committer:disksing

kvrpcpb: add `UndeterminedResult` to `KeyError` (#1310) * kvrpcpb: add `UndeterminedResult` to `KeyError` Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * txn: move `UndeterminedResult` to region error Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> --------- Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> Signed-off-by: disksing <i@disksing.com>

Commit:75ecdbe
Author:Chao Wang
Committer:disksing

*: support async resolve lock in ResolveLockRequest (#1462) Signed-off-by: disksing <i@disksing.com>

Commit:27a4b91
Author:Bisheng Huang
Committer:disksing

tikvpb: add batch command transport timestamps (#1446) * tikvpb: add batch command transport timestamps Add client_send_time_ns to BatchCommandsRequest and tikv_send_time_ns to BatchCommandsResponse so callers can track when a batch request leaves the client and when TiKV finishes preparing the response. Signed-off-by: Bisheng Huang <hbisheng@gmail.com> * proto: update proto lock for transport timestamps Refresh scripts/proto.lock for the new batch command transport timestamp fields so the compatibility metadata matches the proto schema. Signed-off-by: Bisheng Huang <hbisheng@gmail.com> --------- Signed-off-by: Bisheng Huang <hbisheng@gmail.com> Signed-off-by: disksing <i@disksing.com>

Commit:edbd86e
Author:you06
Committer:disksing

Add `Op::SharedLock` and `Op::SharedPessimisticLock` for acquiring shared lock in pessimistic transacitons. (#1374) * reserve Op for shared lock Signed-off-by: you06 <you1474600@gmail.com> * distinguish between shared prewrite locks and shared pessimistic locks (#1377) * let LockInfo carry the full txn info when it's shared Signed-off-by: zyguan <zhongyangguan@gmail.com> * move the todo comment to the proper place Signed-off-by: zyguan <zhongyangguan@gmail.com> * distinguish between shared prewrite locks and shared pessimistic locks Signed-off-by: zyguan <zhongyangguan@gmail.com> --------- Signed-off-by: zyguan <zhongyangguan@gmail.com> * update kvproto Signed-off-by: you06 <you1474600@gmail.com> * add embed shared_lock_infos Signed-off-by: you06 <you1474600@gmail.com> --------- Signed-off-by: you06 <you1474600@gmail.com> Signed-off-by: zyguan <zhongyangguan@gmail.com> Co-authored-by: zyguan <zhongyangguan@gmail.com> Signed-off-by: disksing <i@disksing.com>

Commit:97976be
Author:Ziqian Qin
Committer:disksing

kvrpcpb: add request origin to context (#1479) * kvrpcpb: add request origin to context Signed-off-by: Ziqian Qin <eke@fastmail.com> * make Signed-off-by: Ziqian Qin <eke@fastmail.com> --------- Signed-off-by: Ziqian Qin <eke@fastmail.com> Signed-off-by: disksing <i@disksing.com>

Commit:cb308be
Author:王超
Committer:disksing

kvrpcpb: Add more fields in CommitRequest/Response to give more lock info (#1308) * kvrpcpb: Add more fields in CommitRequest/Response to give more lock info Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * Add `mvcc_info` to `TxnLockNotFound` * Add debug info Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * Add CommitRole Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * Update proto/kvrpcpb.proto --------- Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> Co-authored-by: cfzjywxk <cfzjywxk@gmail.com> Signed-off-by: disksing <i@disksing.com>

Commit:2aab4d2
Author:glorv
Committer:disksing

resource_control: add cross az traffic bytes in ru consumption (#1325) * add cross AZ traffic in resource consumption Signed-off-by: glorv <glorvs@163.com> * add network detail in exec detail Signed-off-by: glorv <glorvs@163.com> * use distinct read and write network traffic bytes Signed-off-by: glorv <glorvs@163.com> --------- Signed-off-by: glorv <glorvs@163.com> Signed-off-by: disksing <i@disksing.com>

Commit:f92cd61
Author:MyonKeminta
Committer:disksing

Support excluding GC barriers for GetGCState and GetAllKeyspacesGCState (#1461) * Support excluding GC barriers for GetGCState and GetAllKeyspacesGCState Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> * Add exclude global gc barriers parameter Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> --------- Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com> Signed-off-by: disksing <i@disksing.com>

Commit:1ac2946
Author:tiancaiamao
Committer:disksing

add global gc barrier API for pd (#1315) * proto/pdpb: add global gc barrier API for pd Signed-off-by: tiancaiamao <tiancaiamao@gmail.com> * address comment Signed-off-by: tiancaiamao <tiancaiamao@gmail.com> * update GetAllKeyspacesGCStatesResponse Signed-off-by: tiancaiamao <tiancaiamao@gmail.com> --------- Signed-off-by: tiancaiamao <tiancaiamao@gmail.com> Signed-off-by: disksing <i@disksing.com>

Commit:68d70b4
Author:JmPotato
Committer:disksing

feat(resource_manager): add keyspace support (#1311) * feat(resource_manager): add keyspace support Signed-off-by: JmPotato <github@ipotato.me> * Use KeyspaceIDValue wrapper Signed-off-by: JmPotato <github@ipotato.me> --------- Signed-off-by: JmPotato <github@ipotato.me> Signed-off-by: disksing <i@disksing.com>

Commit:d429ce8
Author:MyonKeminta
Committer:disksing

Add new GC API (#1288) * Add new GC API Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> * Update definition of keyspace specifier in requests to avoid potential misuses Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> * Update to the latest design * Update protocol Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> * Fix missing field Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> --------- Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com> Signed-off-by: disksing <i@disksing.com>

Commit:0069a1d
Author:disksing
Committer:disksing

Merge keyspace group lookup by identity Signed-off-by: disksing <i@disksing.com>

Commit:c86cd42
Author:Ryan Leung
Committer:disksing

keyspacepb: add load keyspace by ID RPC (#1491) Signed-off-by: disksing <i@disksing.com>

Commit:8a1f60f
Author:crazycs520

init support log replication (#112) Co-authored-by: crazycs520 <crazycs520@gmail.com> Reviewed-on: https://git.pingcap.net/pingkai/kvproto/pulls/112 Reviewed-by: jett <jett@pingcap.cn>

Commit:67de11b
Author:Ziqian Qin
Committer:GitHub

coprocessor: add bucket version to store batch task (#1488) (cherry picked from commit 46e67d198f6dca0bcf1a189cef9f5b6799c41abb) Signed-off-by: Ziqian Qin <eke@fastmail.com>

Commit:a8b285a
Author:hongyunyan

cdcpb: add scan priority for cse base

Commit:f3bb7de
Author:hongyunyan

cdcpb: add scan priority to change data request

Commit:b41e863
Author:Ryan Leung
Committer:GitHub

keyspacepb: add load keyspace by ID RPC (#1491)

Commit:21b5208
Author:Jianjun Liao
Committer:GitHub

cherry-pick compacted based PITR restore (#1492) * metaedit: add new field to record meta files that can be skipped (#1338) * update metaedit field name (#1339) * metaedit: add new field to record meta files that can be skipped * metaedit: rename field name * brpb with bytes (#1353) * brpb with bytes Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * brpb with bytes Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * brpb with bytes Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * brpb with bytes Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * brpb with bytes Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * add scripts to check Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * update rust-protobuf version Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * update rust-protobuf version Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * revert cargo version Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * test Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * test Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * revert debug Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * commit some suggestions Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * fix typos Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> --------- Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * save stats in migration (#1481) Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * br: Add batch download rpc for compact (#1360) * fix * trigger tests --------- Co-authored-by: Jianjun Liao <36503113+Leavrth@users.noreply.github.com> * add batch download latest mvcc rpc function (#1456) Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> --------- Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> Co-authored-by: 3pointer <luancheng@pingcap.com> Co-authored-by: ris <79858083+RidRisR@users.noreply.github.com>

Commit:f8bd26d
Author:tongjian
Committer:GitHub

Merge branch 'master' into pd-keyspace-load-by-id

Commit:5ec421b
Author:Ryan Leung

keyspacepb: add load keyspace by ID RPC Signed-off-by: Ryan Leung <rleungx@gmail.com>

Commit:c081917
Author:Lloyd
Committer:crazycs520

*: support scan changes rpc (#108) Co-authored-by: Lloyd-Pottiger <yan1579196623@gmail.com> Reviewed-on: https://git.pingcap.net/pingkai/kvproto/pulls/108 Reviewed-by: crazycs520 <chenshuang@pingcap.cn> Co-authored-by: Lloyd <qiuyang.yan@pingcap.cn> Co-committed-by: Lloyd <qiuyang.yan@pingcap.cn>

Commit:5c7ae11
Author:crazycs520

resolve merge conflicts with release-nextgen-202603

Commit:d592403
Author:crazycs520

Merge branch 'release-nextgen-202603' into release-pk-9.0-ultra-merge-v71901-without-v856-dev-cse

Commit:9046e96
Author:crazycs520

Merge branch 'release-7.1.9-0-without-v8.5.6' into release-pk-9.0-ultra-merge-v71901-without-v856-dev

Commit:b4e6138
Author:disksing

Add namespace metadata APIs Signed-off-by: disksing <i@disksing.com>

Commit:3566665
Author:disksing

Add namespace metadata APIs Signed-off-by: disksing <i@disksing.com>

Commit:a15c348
Author:JmPotato
Committer:GitHub

proto: add paging_size_bytes to coprocessor.Request (#1448) Add a new uint64 field `paging_size_bytes` (field 17) to the coprocessor Request message. When set, TiKV will stop scanning when accumulated scanned bytes reach this limit, enabling byte-budget paging for finer-grained Resource Control. Note: rebased onto upstream master where field 16 is now taken by max_keys_read (#1450), so paging_size_bytes was bumped from field 16 to field 17. Signed-off-by: JmPotato <github@ipotato.me>

Commit:177e124
Author:disksing

Merge keyspace group lookup by identity Signed-off-by: disksing <i@disksing.com>

Commit:e58f3b1
Author:disksing

Merge keyspace group lookup by identity Signed-off-by: disksing <i@disksing.com>

Commit:021d3ca
Author:disksing

Remove service safe point response keyspace identity Signed-off-by: disksing <i@disksing.com>

Commit:91be314
Author:disksing

Remove service safe point response keyspace identity Signed-off-by: disksing <i@disksing.com>

Commit:cabe761
Author:disksing

Remove GC safe point response keyspace identity Signed-off-by: disksing <i@disksing.com>

Commit:ac39bd6
Author:disksing

Remove GC safe point response keyspace identity Signed-off-by: disksing <i@disksing.com>

Commit:214a58f
Author:disksing

Adjust DFS stat keyspace scope fields Signed-off-by: disksing <i@disksing.com>

Commit:7f5d4da
Author:disksing

Clarify keyspace pagination comment Signed-off-by: disksing <i@disksing.com>

Commit:52f4995
Author:disksing

Clarify keyspace pagination comment Signed-off-by: disksing <i@disksing.com>

Commit:7615a31
Author:disksing

Use oneof for keyspace identity fields Signed-off-by: disksing <i@disksing.com>

Commit:58ce560
Author:disksing

Use oneof for keyspace identity fields Signed-off-by: disksing <i@disksing.com>