Proto commits in wesql/wescale

These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)

2024-11-02

Feat/declarative ddl (#560) * feat: register enable_declarative_ddl in fs; add dynamic update handler Signed-off-by: newborn22 <953950914@qq.com> * feat: add enable_declarative_ddl to sysvar Signed-off-by: newborn22 <953950914@qq.com> * fix: global setting will not be applied to current session immediately Signed-off-by: newborn22 <953950914@qq.com> * feat: add direct declarative ddl primitive Signed-off-by: newborn22 <953950914@qq.com> * fix: remove redundant buildVarCharRow and buildVarCharFields * fix: add interface to vcusor imp Signed-off-by: newborn22 <953950914@qq.com> * fix: build direct declarative plan * feat: add primitive to execute path * fix: fix bug * fix: don't cache dynameic info * fix: declarative ddl path * fix: fix bugs about temporary table Signed-off-by: newborn22 <953950914@qq.com> * fix: remove --declarative and related testcase and documents * fix: fix testcase Signed-off-by: newborn22 <953950914@qq.com> * fix: add e2e test * fix: add diff hints pflag * fix: add testcase * fix: add pflag * fix: add parameter handler * fix: fix testcase * fix: fix testcase * fix: fix bugs * fix: fix based on cr * fix: fix testcase * fix: add doc * fix: add tutorial doc Signed-off-by: newborn22 <953950914@qq.com> * fix: fix title Signed-off-by: newborn22 <953950914@qq.com> * fix: doc * fix: doc Signed-off-by: newborn22 <953950914@qq.com> * fix: fix workflow file Signed-off-by: newborn22 <953950914@qq.com> --------- Signed-off-by: newborn22 <953950914@qq.com> (cherry picked from commit debb8655438d1f65ac9398a91e874279be51cf86)

The documentation is generated from this commit.

Feat/declarative ddl (#560) * feat: register enable_declarative_ddl in fs; add dynamic update handler Signed-off-by: newborn22 <953950914@qq.com> * feat: add enable_declarative_ddl to sysvar Signed-off-by: newborn22 <953950914@qq.com> * fix: global setting will not be applied to current session immediately Signed-off-by: newborn22 <953950914@qq.com> * feat: add direct declarative ddl primitive Signed-off-by: newborn22 <953950914@qq.com> * fix: remove redundant buildVarCharRow and buildVarCharFields * fix: add interface to vcusor imp Signed-off-by: newborn22 <953950914@qq.com> * fix: build direct declarative plan * feat: add primitive to execute path * fix: fix bug * fix: don't cache dynameic info * fix: declarative ddl path * fix: fix bugs about temporary table Signed-off-by: newborn22 <953950914@qq.com> * fix: remove --declarative and related testcase and documents * fix: fix testcase Signed-off-by: newborn22 <953950914@qq.com> * fix: add e2e test * fix: add diff hints pflag * fix: add testcase * fix: add pflag * fix: add parameter handler * fix: fix testcase * fix: fix testcase * fix: fix bugs * fix: fix based on cr * fix: fix testcase * fix: add doc * fix: add tutorial doc Signed-off-by: newborn22 <953950914@qq.com> * fix: fix title Signed-off-by: newborn22 <953950914@qq.com> * fix: doc * fix: doc Signed-off-by: newborn22 <953950914@qq.com> * fix: fix workflow file Signed-off-by: newborn22 <953950914@qq.com> --------- Signed-off-by: newborn22 <953950914@qq.com>

The documentation is generated from this commit.

2024-09-28

Test Table Level RAW (#541) * add: latest_gtid_for_table.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * deal: safe_session.go with LatestGTIDForTable Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * deal: scatter_conn.go with LatestGTIDForTable Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: e2e test Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: tools version latest->0.24.0 Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: tools version latest->0.24.0 Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * feat: update gtid Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * update: query.proto Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * update: .pb.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: go.mod go.sum Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * feat: get gtid map Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * update: vtgate.proto Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * add: getReadAfterWriteGtid() Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * update: init opts.TableReadAfterWriteGtidMap Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: gtid rollback issue Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: init latestGTIDForTable in safeSession Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: add latest_gtid_for_table in ReadAfterWrite Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: scatter_conn.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: vtgate.proto Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: vtgate.proto Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: vtgate.proto Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: query_executor.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: query_executor.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: query_executor.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: query_executor.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: query_executor.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: query_executor.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: scattor_conn.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: scattor_conn.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: scattor_conn.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: safe_session.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: safe_session.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: safe_session.go Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: getRAWGtid Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * feat: support set table_level Signed-off-by: Terry Gao <gaoxuanhit@qq.com> * fix: getRAWGtid Signed-off-by: Terry Gao <gaoxuanhit@qq.com> --------- Signed-off-by: Terry Gao <gaoxuanhit@qq.com>

2024-04-02

feat: add show tablets_plans cmd (#469) * feat: add show tablets_plans cmd Signed-off-by: newborn22 <953950914@qq.com> * feat: cmd can use like and where clause to filter tablets Signed-off-by: newborn22 <953950914@qq.com> * feat: add common query rpc Signed-off-by: newborn22 <953950914@qq.com> * fix: fix like bugs; return error for unsupported where clause; add mtr test --------- Signed-off-by: newborn22 <953950914@qq.com>

2024-02-05

Refactor/show cmd in non transactional dml (#439) * feat: gen affected rows and dealing batch id of job table when user querying Signed-off-by: newborn22 <953950914@qq.com> * feat: delete affected rows and dealing batch id fileds in job table Signed-off-by: newborn22 <953950914@qq.com> * fix: fix bug in HandleDMLJobRequest in plan_execute.go Signed-off-by: newborn22 <953950914@qq.com> --------- Signed-off-by: newborn22 <953950914@qq.com>

2024-01-29

fix: throttle when submition (#435) fix: throttle when submition; fix bug in getRunningPeriodTime; fix bug in throttle job Signed-off-by: newborn22 <953950914@qq.com>

2024-01-19

Feature/support large dml split (#384) * feat: demo, delete without pause and resume, rebase main(74fe562b1ab99ff75bba7c1faaf4c7ea73017409) Signed-off-by: newborn22 <953950914@qq.com> * feat:demo, delete, submit display, pause and resume Signed-off-by: newborn22 <953950914@qq.com> * feat: demo, update,delete,pause,resume,submit display Signed-off-by: newborn22 <953950914@qq.com> * feat: support crash recover when job status is running or paused Signed-off-by: newborn22 <953950914@qq.com> * feat: support postpone and cancel Signed-off-by: newborn22 <953950914@qq.com> * feat: support throttle Signed-off-by: newborn22 <953950914@qq.com> * feat: ugly draft of batch job runner, single pk with int type Signed-off-by: newborn22 <953950914@qq.com> * feat: optimize pause, resume, recover; support table entry gc; all based on single table, single int pk Signed-off-by: newborn22 <953950914@qq.com> * feat: enrich batch process display Signed-off-by: newborn22 <953950914@qq.com> * feat: support single pk with mysql types which can convert into int, float and string Signed-off-by: newborn22 <953950914@qq.com> * feat: support multi PK with various types Signed-off-by: newborn22 <953950914@qq.com> * fix: rewrite code of batch generating process to make it more readable Signed-off-by: newborn22 <953950914@qq.com> * fix: rename function Signed-off-by: newborn22 <953950914@qq.com> * feat: support dynamic batch split Signed-off-by: newborn22 <953950914@qq.com> * fix: fix some bugs in nextBatchID generation; batchSize is min(threshold,userBatchSize) Signed-off-by: newborn22 <953950914@qq.com> * feat: support show dml_job [DETAILS] Signed-off-by: newborn22 <953950914@qq.com> * feat: add time period code in jobsheduler and jobrunner Signed-off-by: newborn22 <953950914@qq.com> * feat: support time period Signed-off-by: newborn22 <953950914@qq.com> * fix: fix some bugs and time period can work correctly Signed-off-by: newborn22 <953950914@qq.com> * fix: put split batch codes into funcion splitBatchIntoTwo Signed-off-by: newborn22 <953950914@qq.com> * fix: will return error if DML won't affect any rows Signed-off-by: newborn22 <953950914@qq.com> * feat: add notify mechanism to jobScheduler Signed-off-by: newborn22 <953950914@qq.com> * feat: batchSize = min(userBatchSize, threshold/table_index_count*ratioOfThreshold) Signed-off-by: newborn22 <953950914@qq.com> * fix: move constant var about sql to sqls.go Signed-off-by: newborn22 <953950914@qq.com> * fix: modify some cols in big_dml_jobs_table Signed-off-by: newborn22 <953950914@qq.com> * feat: add batch begin and end cols on batch table Signed-off-by: newborn22 <953950914@qq.com> * fix: modify dealing batch id format from '1+' to '1-1' Signed-off-by: newborn22 <953950914@qq.com> * feat: dml job details is shown by batchID order Signed-off-by: newborn22 <953950914@qq.com> * feat: add fail policy Signed-off-by: newborn22 <953950914@qq.com> * fix: modify the way to get batchID to execute, otherwise failpolicy skip will not work Signed-off-by: newborn22 <953950914@qq.com> * fix: clear some old todos; inhibit float type used as split col; inhibit sql without where clause and with limit or order by clause; Signed-off-by: newborn22 <953950914@qq.com> * fix: modify system table name; add StripComments func; call setResetQuery after calling setQuery; replace sleep with ticker Signed-off-by: newborn22 <953950914@qq.com> * fix: gen batchSQL by modify ast node instead of string operations when creating batch table Signed-off-by: newborn22 <953950914@qq.com> * fix: gen batchSQL and batchCountSQL when splitting sql by modify sql ast node instead of string operations Signed-off-by: newborn22 <953950914@qq.com> * fix: replace string literal with const var; fix error due to wrong rebase operation Signed-off-by: newborn22 <953950914@qq.com> * fix: fix wrong operion when rebase main Signed-off-by: newborn22 <953950914@qq.com> * fix: refactor genBatchSQL and genBatchCountSQL Signed-off-by: newborn22 <953950914@qq.com> * fix: refactor splitBatchIntoTwo, add related testcases Signed-off-by: newborn22 <953950914@qq.com> * fix: solve split brain problem; fix bug in genCountSQL Signed-off-by: newborn22 <953950914@qq.com> * fix:add failpoints; divide code into files; batchID start from x-2; declare unsupported PK type; fix failpolicy bugs Signed-off-by: newborn22 <953950914@qq.com> * fix: split large functions into smaller ones Signed-off-by: newborn22 <953950914@qq.com> * fix: fix testcase Signed-off-by: newborn22 <953950914@qq.com> * feat: refactor DML JOB controller * feat: register flag in fs; table gc Signed-off-by: newborn22 <953950914@qq.com> * fix: merge jobMonitor and JobSchduler as JobManager Signed-off-by: newborn22 <953950914@qq.com> * feat: add time zone in feature 'running period time' Signed-off-by: newborn22 <953950914@qq.com> * test: add interation tests, from parseDML to splitBatch Signed-off-by: newborn22 <953950914@qq.com> * fix: complete time period feature Signed-off-by: newborn22 <953950914@qq.com> * test: e2e test single pk int passed; add github ci Signed-off-by: newborn22 <953950914@qq.com> * fix: refacor e2e test Signed-off-by: newborn22 <953950914@qq.com> * test: enrich e2e test Signed-off-by: newborn22 <953950914@qq.com> * fix: delete runJobController func Signed-off-by: newborn22 <953950914@qq.com> * fix: let jobcontroller CI run on PR Signed-off-by: newborn22 <953950914@qq.com> * fix: refactor e2e test to isolate every testcase; modify time format written to mysql from RFC3339 to Datetime Signed-off-by: newborn22 <953950914@qq.com> --------- Signed-off-by: newborn22 <953950914@qq.com> Co-authored-by: earayu <earayu@163.com>

2024-01-12

feat: add reload user and reload privileges (#229) * feat: add reload user and reload privileges * fix: reserve USERS as vitess reserved keyword * feat: completed reload user * feat: add reload privileges * fix: query protobuf Signed-off-by: <919179287@qq.com> * fix: regenerate parser Signed-off-by: <919179287@qq.com> * test: fix e2e test Signed-off-by: <919179287@qq.com> * feat: add rows affected when reload users Signed-off-by: <919179287@qq.com> * docs: update Authentication doc * fix: fix magic string bug Signed-off-by: <919179287@qq.com> * feat: fix error message --------- Signed-off-by: <919179287@qq.com> Co-authored-by: lyq10085 <2894975338@qq.com> Co-authored-by: earayu <earayu@163.com>

2024-01-10

feat: add reload privileges

2023-12-08

Feature/branch skip copy phase (#378) * feat: add Branch command in vtctld Signed-off-by: <919179287@qq.com> * test: add Branch example Signed-off-by: <919179287@qq.com> * feat: add start and stop to Branch command Signed-off-by: <919179287@qq.com> * feat: add PrepareMergeBack and StartMergeBack on Branch Signed-off-by: <919179287@qq.com> * feat: add need_merge_back column in branch_table_rules Signed-off-by: <919179287@qq.com> * feat: add cleanup for branch Signed-off-by: <919179287@qq.com> * feat: add uuid for merge_ddl Signed-off-by: <919179287@qq.com> * feat: add default_filter_rule and support RAND func Signed-off-by: <919179287@qq.com> * test: fix unit test TestMigrateTables Signed-off-by: <919179287@qq.com> * test: fix TestSchemaDiff unit test Signed-off-by: <919179287@qq.com> * feat: add skip_copy_phase and gofakeit function Signed-off-by: <919179287@qq.com> * feat: set skip_copy_phase default values to true; fix magic string 'zone1'; push transfrom and filter funtion into source end but all fucntion; support fake function; Signed-off-by: <919179287@qq.com> * feat: cleanup will stop and delete vreplication by VReplicationExec Signed-off-by: <919179287@qq.com> * feat: remove comment when start branch Signed-off-by: <919179287@qq.com> * feat: add branch schemadiff command Signed-off-by: <919179287@qq.com> * feat: change branch prepare script Signed-off-by: earayu <earayu@163.com> * feat: change branch prepare script Signed-off-by: earayu <earayu@163.com> * test: add clearcomment cmd Signed-off-by: <919179287@qq.com> * fix: fix empty tablesettings Signed-off-by: <919179287@qq.com> --------- Signed-off-by: <919179287@qq.com> Signed-off-by: earayu <earayu@163.com> Co-authored-by: earayu <earayu@163.com>

fix: fix empty tablesettings Signed-off-by: <919179287@qq.com>

2023-12-04

feat: add skip_copy_phase and gofakeit function Signed-off-by: <919179287@qq.com>

feat: add default_filter_rule and support RAND func Signed-off-by: <919179287@qq.com>

feat: add need_merge_back column in branch_table_rules Signed-off-by: <919179287@qq.com>

feat: add uuid for merge_ddl Signed-off-by: <919179287@qq.com>

feat: add start and stop to Branch command Signed-off-by: <919179287@qq.com>

feat: add Branch command in vtctld Signed-off-by: <919179287@qq.com>

2023-11-29

Feature/postpone the read write split decision (#374) * fix: centralize the code for tablet type decision into the resolve function. Signed-off-by: newborn22 <953950914@qq.com> * fix: deal with some nil ptr case Signed-off-by: newborn22 <953950914@qq.com> * fix: fix some testcase Signed-off-by: newborn22 <953950914@qq.com> * fix: fix some testcase Signed-off-by: newborn22 <953950914@qq.com> * fix: move ratio to options, delete tablet type assignment in newVcursorImpl and getPlan Signed-off-by: newborn22 <953950914@qq.com> * fix: add ResolveTabletType to another code path Signed-off-by: newborn22 <953950914@qq.com> * fix: reconstruct code Signed-off-by: newborn22 <953950914@qq.com> * fix: Improved code encapsulation, the ResolveTabletType function centrally addresses the tablet type decision problem Signed-off-by: newborn22 <953950914@qq.com> * fix: fix test case Signed-off-by: newborn22 <953950914@qq.com> * fix: move parser2 outside Signed-off-by: newborn22 <953950914@qq.com> * feat: enrich the content of display sql execution vttablet Signed-off-by: newborn22 <953950914@qq.com> * fix: chang display set cmd, and fix some test case Signed-off-by: newborn22 <953950914@qq.com> * feat: enrich the content of display sql execution vttablet, and lb between replic and rdonly Signed-off-by: newborn22 <953950914@qq.com> * fix: fix bug about loadBalanceBetweenReplicAndRdonly flag Signed-off-by: newborn22 <953950914@qq.com> * test: add e2e test in wesql cluster tests * fix: fix bug in GetAllHealthyTabletStatus, add mutex Signed-off-by: newborn22 <953950914@qq.com> * doc: add docs about ddl pause and routing display * fix: modify codes base on review, excpet code involves rdonly txn * fix: delete resolveoptions.shouldForceRouteToRdonly, rd only txn can be routed to replic or rdonly --------- Signed-off-by: newborn22 <953950914@qq.com>

2023-11-28

Add Branch Functionality to WeSQL (#372) * feat: add Branch command in vtctld Signed-off-by: <919179287@qq.com> * test: add Branch example Signed-off-by: <919179287@qq.com> * feat: add start and stop to Branch command Signed-off-by: <919179287@qq.com> * feat: add PrepareMergeBack and StartMergeBack on Branch Signed-off-by: <919179287@qq.com> * feat: add need_merge_back column in branch_table_rules Signed-off-by: <919179287@qq.com> * feat: add cleanup for branch Signed-off-by: <919179287@qq.com> * feat: add uuid for merge_ddl Signed-off-by: <919179287@qq.com> * feat: add default_filter_rule and support RAND func Signed-off-by: <919179287@qq.com> * test: fix unit test TestMigrateTables Signed-off-by: <919179287@qq.com> * test: fix TestSchemaDiff unit test Signed-off-by: <919179287@qq.com> --------- Signed-off-by: <919179287@qq.com>

2023-11-21

feat: add display of sql execution node type (#369) * feat: add display of execution node type Signed-off-by: newborn22 <953950914@qq.com> * feat: add basic usage comment for show failpoints Signed-off-by: newborn22 <953950914@qq.com> * fix: delete some stash files Signed-off-by: newborn22 <953950914@qq.com> * fix: delete stash file Signed-off-by: newborn22 <953950914@qq.com> * fix: use system vars to implement swithes of enable_display_sql_execution_vttablet_type and enable_display_sql_execution_vttablet_type * fix: correct ReadWriteSplitForReadOnlyTxnUserInput system var name Signed-off-by: newborn22 <953950914@qq.com> --------- Signed-off-by: newborn22 <953950914@qq.com>

2023-11-15

Feature/intercept dml without where (#358) * feat: add interception of DELETE and UPDATE SQL statements without a WHERE condition Signed-off-by: newborn22 <953950914@qq.com> * feat: the switch is implemented as system vars now so users can use set session and set global statement Signed-off-by: newborn22 <953950914@qq.com> * fix: add func in fake_vursor_test.go in order to git commit Signed-off-by: newborn22 <953950914@qq.com> --------- Signed-off-by: newborn22 <953950914@qq.com>

2023-11-13

Feature/fix get schema (#359) * fix: GetSchema Signed-off-by: earayu <earayu@163.com> * fix: GetSchema Signed-off-by: earayu <earayu@163.com> * fix: GetSchema Signed-off-by: earayu <earayu@163.com> * fix: testcase Signed-off-by: earayu <earayu@163.com> --------- Signed-off-by: earayu <earayu@163.com>

2023-11-02

feat: Clean up OnlineDDL Jobs for dropped schema (#348) * feat: Clean up OnlineDDL Jobs for dropped schema Signed-off-by: earayu <earayu@163.com> * feat: Clean up OnlineDDL Jobs for dropped schema Signed-off-by: earayu <earayu@163.com> * feat: Clean up OnlineDDL Jobs for dropped schema Signed-off-by: earayu <earayu@163.com> * feat: Clean up OnlineDDL Jobs for dropped schema Signed-off-by: earayu <earayu@163.com> --------- Signed-off-by: earayu <earayu@163.com>

2023-10-13

fix: add tableSchema for vstreamer (#323) * fix: add tableSchema for vstreamer * fix: add vstreamer tableSchema

fix: add tableSchema for vstreamer

2023-10-10

feat: fix vstreamer dbname Signed-off-by: earayu <earayu@163.com>

2023-10-09

feat: read only transaction (#303) * feat: read only tx with an param Signed-off-by: newborn22 <953950914@qq.com> * test: add unit test in read_write_splitter_test.go Signed-off-by: newborn22 <953950914@qq.com> * fix: hasCreatedTempTables and hasAdvisoryLock have higher priority over read only transaction and should be routed to PRIMARY node Signed-off-by: newborn22 <953950914@qq.com> * fix: Added handling for the situation when read only and consistent snapshot both are enabled. And adjusted the order of the code to make it more readable Signed-off-by: newborn22 <953950914@qq.com> * fix: fix some test code based on last commit Signed-off-by: newborn22 <953950914@qq.com> * feat: refactor suggestTabletType to make it more readble Signed-off-by: newborn22 <953950914@qq.com> * feat: delete some unused code Signed-off-by: newborn22 <953950914@qq.com> * fix: set the defaultReadWriteSplittingPolicy to disable and EnableReadWriteSplitForReadOnlyTxn to false Signed-off-by: newborn22 <953950914@qq.com> * fix: fix the Test_suggestTabletType_read_only_transaction testcase about read only transaction begin sql Signed-off-by: newborn22 <953950914@qq.com> --------- Signed-off-by: newborn22 <953950914@qq.com>

2023-09-11

New load balance policy (LEAST_MYSQL_CONNECTED_CONNECTIONS, LEAST_MYSQL_RUNNING_CONNECTIONS, LEAST_TABLET_INUSE_CONNECTIONS) (#230) * chore: RFC proto Signed-off-by: lyq10085 <2894975338@qq.com> * chore: record pr in RFC Signed-off-by: lyq10085 <2894975338@qq.com> * feat: least_global_busy_connetions setup Signed-off-by: lyq10085 <2894975338@qq.com> * feat: add threadtracking functionality to repltracker.poller * feat: complete least_mysql_connected_connections and least_mysql_running_connections * fix: fix TestHealthStreamerBroadcast * feat: least_tablet_inuse_connections proto * chore: add more load balancer testcase --------- Signed-off-by: lyq10085 <2894975338@qq.com>

2023-09-08

Feature/support set failpoint (#254) * feat: add failPointTable to manage failpoint injection Signed-off-by: <919179287@qq.com> * feat: add set @put_failpoint and @remove_failpoint Signed-off-by: <919179287@qq.com> * feat: add show failpoint Signed-off-by: <919179287@qq.com> * fix: fix show failpoint Signed-off-by: <919179287@qq.com> * test: add FailPoint testcase * feat: refactor failpointkey Signed-off-by: earayu <earayu@163.com> * fix: modify show failpoint to show failpoints * fix: modify failpoint_key fullname * feat: fix set UDV Signed-off-by: earayu <earayu@163.com> * fix: fix remove_failpoint Signed-off-by: <919179287@qq.com> --------- Signed-off-by: <919179287@qq.com> Signed-off-by: earayu <earayu@163.com> Co-authored-by: earayu <earayu@163.com>

2023-08-24

New load balance policy (LEAST_MYSQL_CONNECTED_CONNECTIONS, LEAST_MYSQL_RUNNING_CONNECTIONS, LEAST_TABLET_INUSE_CONNECTIONS) (#230) * chore: RFC proto Signed-off-by: lyq10085 <2894975338@qq.com> * chore: record pr in RFC Signed-off-by: lyq10085 <2894975338@qq.com> * feat: least_global_busy_connetions setup Signed-off-by: lyq10085 <2894975338@qq.com> * feat: add threadtracking functionality to repltracker.poller * feat: complete least_mysql_connected_connections and least_mysql_running_connections * fix: fix TestHealthStreamerBroadcast * feat: least_tablet_inuse_connections proto * chore: add more load balancer testcase --------- Signed-off-by: lyq10085 <2894975338@qq.com>

2023-08-09

fix: Permission Auth Error (#226) * fix: Permission Auth Error Signed-off-by: <919179287@qq.com> * fix: fixed opts is null Signed-off-by: <919179287@qq.com> * fix: fix opt is nil Signed-off-by: <919179287@qq.com> * fix: Refet to a global const Signed-off-by: <919179287@qq.com> --------- Signed-off-by: <919179287@qq.com>

Feature/rewrite tablename (#218) * feat: rewrite tablename to reduce use statement in conn

2023-07-20

Feature/read write splitting ratio sys vars (#213) * Feature/read write splitting ratio sys vars

Feature/read write splitting ratio sys vars * feat: add read write splitting ratio sysVars

2023-07-13

Feature/mysqlbased authentication (#169) * perf:Compress mysql56GtidSet Signed-off-by: <919179287@qq.com> * test: add testcase for gtid intersection Signed-off-by: earayu <earayu@163.com> * fix:fixed the bug of the TestMysql56GTIDSetIntersect6 Signed-off-by: <919179287@qq.com> fix:check lint Signed-off-by: <919179287@qq.com> test:add Intersec Test Signed-off-by: <919179287@qq.com> test: add TrimSets testcases Signed-off-by: <919179287@qq.com> merge: rebase vitess-16.0-dev Signed-off-by: <919179287@qq.com> merge: vitess-release-16.0-dev and perf/Compressgtid Signed-off-by: <919179287@qq.com> test: add enetoend test CompressGtid Signed-off-by: <919179287@qq.com> test: remove MergeTest Signed-off-by: <919179287@qq.com> chore: add Intersect and TrimGTIDSet code comment Signed-off-by: <919179287@qq.com> fix: fixed show lastseengtid can't work Signed-off-by: <919179287@qq.com> fix: fixed hidden '_vt' database Signed-off-by: <919179287@qq.com> fix: Resolved the Mariadb issue with Intersect Signed-off-by: <919179287@qq.com> * feat: rebase dev Signed-off-by: earayu <earayu@163.com> * fix: deprecate CompressGtidSets When the lengths of intervals exceeds special length Signed-off-by: <919179287@qq.com> * docs: add 20230609_CompressGtidSets Signed-off-by: <919179287@qq.com> * fix: fixed the Example in docs Signed-off-by: <919179287@qq.com> * fix: authServer is able to validate different authMethod Signed-off-by: <919179287@qq.com> * fix: store into cache by passing full authentication Signed-off-by: <919179287@qq.com> * refactor: add annotation Signed-off-by: <919179287@qq.com> * fix: rebase bug Signed-off-by: <919179287@qq.com> * feat: Implement validation for ordering by host and support for '%' and '_' wildcard matching Signed-off-by: <919179287@qq.com> * feat: add ExecuteInternal rpc and pull privilege from table mysql.user Signed-off-by: <919179287@qq.com> * fix: add endtoend testcase tableacl_test Signed-off-by: <919179287@qq.com> * fix: GetPlan modify from '_vt' to target.keyspace Signed-off-by: <919179287@qq.com> * feat: add parm table-acl-config-mode Signed-off-by: <919179287@qq.com> * fix: use 'username@host' instead of 'username' as the key for authentication Signed-off-by: <919179287@qq.com> * test: add TestMysqlbasedACL Signed-off-by: <919179287@qq.com> * feat: pull priv info from mysql.table_priv Signed-off-by: <919179287@qq.com> * fix: multi acl Authorization Signed-off-by: <919179287@qq.com> * feat: pull privileges from mysql.db and fix planBuilder testcase Signed-off-by: <919179287@qq.com> * fix: fixed tableacl_test config Signed-off-by: <919179287@qq.com> * test: add database-level and table-level testcase Signed-off-by: <919179287@qq.com> * test: add Authorized unit test and TestReaderOnly,TestWriterOnly endtoend test Signed-off-by: <919179287@qq.com> --------- Signed-off-by: <919179287@qq.com> Signed-off-by: earayu <earayu@163.com> Co-authored-by: earayu <earayu@163.com>

2023-05-24

Implement Load Balancer for Read Write Splitter (#66) Signed-off-by: earayu <earayu@163.com>

2023-05-11

feat: refactor schema tracker (#52) Signed-off-by: earayu <earayu@163.com>

2023-05-09

feat: Read After Write (#51) Signed-off-by: earayu <earayu@163.com>

2023-05-04

feat: support read write seperation (#2) Signed-off-by: earayu <earayu@163.com>

2023-02-23

GetSchema rpc to streaming api (#12447) * GetSchema rpc to streaming api Signed-off-by: Harshit Gangal <harshit@planetscale.com> * addressed review comment Signed-off-by: Harshit Gangal <harshit@planetscale.com> --------- Signed-off-by: Harshit Gangal <harshit@planetscale.com>

2023-02-21

Schema RPC to fetch table/view definition (#12375) * feat: proto changes to add new rpc Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: vttablet changes in queryservice to implement the new rpc Signed-off-by: Harshit Gangal <harshit@planetscale.com> * added test on tabletserver Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: use the new schema rpc in vtgate to get updated view definition Signed-off-by: Harshit Gangal <harshit@planetscale.com> * remove error from getschemaresponse and change the enum name Signed-off-by: Harshit Gangal <harshit@planetscale.com> * updated proto files for vtadmin Signed-off-by: Harshit Gangal <harshit@planetscale.com> * fix e2e test Signed-off-by: Harshit Gangal <harshit@planetscale.com> --------- Signed-off-by: Harshit Gangal <harshit@planetscale.com>

2023-01-20

VReplication: Defer Secondary Index Creation (#11700) * Strip secondary keys during schema copy Signed-off-by: Matt Lord <mattalord@gmail.com> * Save ALTER to run when copy finishes Signed-off-by: Matt Lord <mattalord@gmail.com> * Move work to vcopier Signed-off-by: Matt Lord <mattalord@gmail.com> * go mod tidy Signed-off-by: Matt Lord <mattalord@gmail.com> * Re-apply stashed secondary keys Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor fixes and cleanup Signed-off-by: Matt Lord <mattalord@gmail.com> * Add Reshard support Signed-off-by: Matt Lord <mattalord@gmail.com> * Support multiple SQL actions Signed-off-by: Matt Lord <mattalord@gmail.com> * Rename action->>"$.action" to action->>"$.task" Signed-off-by: Matt Lord <mattalord@gmail.com> * Fix unit tests Signed-off-by: Matt Lord <mattalord@gmail.com> * Add new unit test Signed-off-by: Matt Lord <mattalord@gmail.com> * Improve the unit test Signed-off-by: Matt Lord <mattalord@gmail.com> * Add vtctl --defer-secondary-keys client flag Signed-off-by: Matt Lord <mattalord@gmail.com> * Fix binlog unit tests Signed-off-by: Matt Lord <mattalord@gmail.com> * Fix wrangler unit tests Signed-off-by: Matt Lord <mattalord@gmail.com> * Move workflow type check to a function Signed-off-by: Matt Lord <mattalord@gmail.com> * Update error in unit test and add OnlineDDL Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor unit test improvements Signed-off-by: Matt Lord <mattalord@gmail.com> * Improve name of table used Signed-off-by: Matt Lord <mattalord@gmail.com> * Improvements after self review Signed-off-by: Matt Lord <mattalord@gmail.com> * Ensure ALTERs are killed when engine shuts down Signed-off-by: Matt Lord <mattalord@gmail.com> * Fixup & deflake the unit test Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor improvements after self review Signed-off-by: Matt Lord <mattalord@gmail.com> * Auto handle more edge cases Signed-off-by: Matt Lord <mattalord@gmail.com> * More unit test improvements Signed-off-by: Matt Lord <mattalord@gmail.com> * Address review comments Signed-off-by: Matt Lord <mattalord@gmail.com> * Correct new error handling in killAction Signed-off-by: Matt Lord <mattalord@gmail.com> * Improve getTableSecondaryKeys function Signed-off-by: Matt Lord <mattalord@gmail.com> * Nitty comment improvements Signed-off-by: Matt Lord <mattalord@gmail.com> * Address review comments Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor improvements to log message Signed-off-by: Matt Lord <mattalord@gmail.com> * Fix unit tests Signed-off-by: Matt Lord <mattalord@gmail.com> * Add defer_secondary_keys to new unit tests added via main merge Signed-off-by: Matt Lord <mattalord@gmail.com> * More unit test fixes Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor improvements to the log messages Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor changes and a fix after self review Signed-off-by: Matt Lord <mattalord@gmail.com> * Ignore --defer-secondary-keys for Reshard merges Signed-off-by: Matt Lord <mattalord@gmail.com> * Nitty improvement to comment Signed-off-by: Matt Lord <mattalord@gmail.com> * Add full support for shard merges Signed-off-by: Matt Lord <mattalord@gmail.com> * Move newClientConnection() to vreplicator Signed-off-by: Matt Lord <mattalord@gmail.com> * Add unit test for new behavior Other fixes and minor improvements Signed-off-by: Matt Lord <mattalord@gmail.com> * Use the engine context rather than vreplicator context Otherwise the ALTER gets KILLed when the copy phase ends Signed-off-by: Matt Lord <mattalord@gmail.com> * Improve testing and other minor tweaks Signed-off-by: Matt Lord <mattalord@gmail.com> * Add post copy action cleanup to workflow cancel Signed-off-by: Matt Lord <mattalord@gmail.com> * More unit test fixes for post copy action cleanup Signed-off-by: Matt Lord <mattalord@gmail.com> Signed-off-by: Matt Lord <mattalord@gmail.com>

2023-01-19

notify view change to vtgate (#12115) * notify view change to vtgate Signed-off-by: Harshit Gangal <harshit@planetscale.com> * test: added unit test Signed-off-by: Harshit Gangal <harshit@planetscale.com> * fix: fix e2e test query Signed-off-by: Harshit Gangal <harshit@planetscale.com> Signed-off-by: Harshit Gangal <harshit@planetscale.com>

2023-01-17

OnlineDDL: support integer-to-enum conversion in `vitess` migrations (#12098) * message Rule: convert_int_to_enum Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * OnlineDDL/vreplication: handle conversion of integral type to enum type Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * admin web proto Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * minor naming and comment changes Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

2023-01-13

Remove Dead Legacy Workflow Manager Code (#12085) * Remove dead Workflow mgmt code This was pre-Legacy sharding, which was in turn pre-VReplication. Signed-off-by: Matt Lord <mattalord@gmail.com> * Remove related proto Signed-off-by: Matt Lord <mattalord@gmail.com> * Get rid of related flag usage Signed-off-by: Matt Lord <mattalord@gmail.com> * Add release notes line and run go mod tidy Signed-off-by: Matt Lord <mattalord@gmail.com> * Add missing command to removed list in release notes Signed-off-by: Matt Lord <mattalord@gmail.com> Signed-off-by: Matt Lord <mattalord@gmail.com>

2023-01-08

VDiff2: Properly Apply --only_pks Report Flag (#12025)

2022-12-08

optimize joins, redirect dml for reference tables (#11875) * go/vt/vtgate, proto: add "source" field to vschema * Add "source" field to vschema reference tables. * Model "source" field in go/vt/vtgate/vindexes/vschema.go. * Treat multiple tables in different keyspaces as unambiguous when all of those tables are references to or by each other. Signed-off-by: Max Englander <max@planetscale.com> * go/vt/vtgate/planbuilder: plan reference joins Given: * Two unsharded keyspace `k1` and sharded keyspace `k2`. * Source table `k1.r` and reference table `k2.r`. * Sharded table `k2.x`. The previous commit adds support to globally route `r` to `k1.r`. This means that `SELECT x JOIN r` will join `k1.r` to `x`, but what we want is to join `k2.r`. This commit propagates `vindexes.Table.Source` info from the `vindexes.VSchema` into `operators.Route`, and uses that information when planning joins. Signed-off-by: Max Englander <max@planetscale.com> * go/vt/vtgate/vindexes: test reference source constraints Signed-off-by: Max Englander <max@planetscale.com> * go/vt/vtgate/vindexes: FindTable unit tests for reference sources Signed-off-by: Max Englander <max@planetscale.com> * go/vt/vtgate/vindexes: build keyspace allow unresolved references Signed-off-by: Max Englander <max@planetscale.com> * go/vt/vtgate: executor unit test reference tables Signed-off-by: Max Englander <max@planetscale.com> * go/vt/vtgate: executor unit test reference dml Signed-off-by: Max Englander <max@planetscale.com> * go/vt/vtgate/vindexes: avoid bi-directional references (causes infinite loop) Signed-off-by: Max Englander <max@planetscale.com> * go/test/endtoend/vtgate: test reference tables Signed-off-by: Max Englander <max@planetscale.com> * tidy up Signed-off-by: Max Englander <max@planetscale.com> * address pr comments round 1 Signed-off-by: Max Englander <max@planetscale.com> * address pr feedback round 2: dont use replicas in tests when not needed Signed-off-by: Max Englander <max@planetscale.com> * address pr feedback round 2: use vterrors in vschema.go Signed-off-by: Max Englander <max@planetscale.com> * fix tests Signed-off-by: Max Englander <max@planetscale.com> * address pr feedback round 3: fix e2e test Signed-off-by: Max Englander <max@planetscale.com> * address pr feedback round 3: optimally route dml Signed-off-by: Max Englander <max@planetscale.com> * address pr feedback round 3: fix e2e & plan tests (primitive.GetTablesUsed) Signed-off-by: Max Englander <max@planetscale.com> * forgot to commit these :facepalm: Signed-off-by: Max Englander <max@planetscale.com> * move TablesUsed from engine => planbuilder Signed-off-by: Max Englander <max@planetscale.com> * rm engine.(any).GetTablesUsed Signed-off-by: Max Englander <max@planetscale.com> * rm engine.(any).MergedWith; revert engine.(*Route).TableNames => engine.(*Route).TableName Signed-off-by: Max Englander <max@planetscale.com> * tidy up Signed-off-by: Max Englander <max@planetscale.com> * address pr feedback round 3: redir gen4 insert to source Signed-off-by: Max Englander <max@planetscale.com> * address pr feedback round 4: remove unused (vindexFunc).TableName Signed-off-by: Max Englander <max@planetscale.com> Signed-off-by: Max Englander <max@planetscale.com>

2022-11-30

VSCopy: Send COPY_COMPLETED events when the copy operation is done (#11740) * VSCopy: Demonstrate to fail a test case on which the vstream API sends new events showing copy completed Signed-off-by: yoheimuta <yoheimuta@gmail.com> * VSCopy: Send new events when the copy operation is done Signed-off-by: yoheimuta <yoheimuta@gmail.com> * VSCopy: Fix typo Signed-off-by: yoheimuta <yoheimuta@gmail.com> * Initialize new map for the 'vstream * from' vtgate sql interface. Make vtadmin web protos Signed-off-by: Rohit Nayak <rohit@planetscale.com> * VSCopy: Make TestVStreamCopyBasic fail fast to avoid the end2end timeout out Signed-off-by: yoheimuta <yoheimuta@gmail.com> * VSCopy: stop sharing the 't1' table among multiple test cases running concurrently Signed-off-by: yoheimuta <yoheimuta@gmail.com> * VSCopy: refactor the function signature to be clearer Signed-off-by: yoheimuta <yoheimuta@gmail.com> * VSCopy: refactor the VEvents sorter to be simpler Signed-off-by: yoheimuta <yoheimuta@gmail.com> * VSCopy: refactor to stop the sorter from including a fully copied event Signed-off-by: yoheimuta <yoheimuta@gmail.com> Signed-off-by: yoheimuta <yoheimuta@gmail.com> Signed-off-by: Rohit Nayak <rohit@planetscale.com> Co-authored-by: Rohit Nayak <rohit@planetscale.com>

2022-11-29

Incremental logical backup and point in time recovery (#11097) * Unexplode Backup() function, pass BackupRequest as argument Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * use tabletmanagerdata.BackupRequest Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * make proto Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * removed duplicate tabletmanagerdata imports Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * tabletmanagerdatapb Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * vschemapb Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * require.NoError Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * proto: BackupRequest.incremental_from_pos Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * pass IncrementalFromPos Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * populate incremental_from_pos Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Storing ServerUUID and TabletAlias as part of backup MANIFESTO Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * use IncrementalFromPos in BackupParams, populate Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * executeIncrementalBackup Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * add unit tests for GTID 'Contains' Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Add binlog related functions in MySQLDaemon Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * More functionality in incremental backup Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * add backupBinlogDir as a valid backup directory Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * include FromPosition in the backup manifest. Find binlog files to backup Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * complete incremental backup Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * clarify the difference between user's requested position, and the FromPosition in the manifest. Add 'Incremental' (bool) to the manifest Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * make vtadmin_web_proto_types Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Add Keyspace, Shard to backup manifest Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * for good order, keyspace comes first Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * take into account purged GTIDs. Fix value of 'incrementalBackupToGTID' Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * endtoend tests for incremental backup. No restore validation as yet. Tests do not have a GitHub workflow yet. Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Adding CI shard: 'backup_pitr' Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * cleanup Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * backup_pitr tested via mysql80 Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * insert data with hint Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * refactor Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * FindPITRPath: find a shortest path to recover a GTID position, base on one full backup and zero or more inremental backups Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * more validation Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * more test cases Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * RestoreFromBackupRequest: RestoreToPos Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * vtctl Restore: '--restore_to_pos' Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * make vtadmin_web_proto_types Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * RestoreFromBackupRequest: RestoreToPos Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Unexplode: RestoreFromBackup() receives req *tabletmanagerdatapb.RestoreFromBackupRequest Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Unexplode: RestoreFromBackup() receives req *tabletmanagerdatapb.RestoreFromBackupRequest Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * make vtadmin_web_proto_types Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * populate restoreParams.RestoreToPos Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * simplifying the logic of finding relevant backup Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * fix switch/break logic Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * towards a restore path Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * golang version Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * fix workflows ubuntu version Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * skip nil manifests Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * FindBackupToRestore() returns a RestorePath, which is an ordered sequence of backup manifests/handles Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * linter suggestion Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * fix backup time comparison logic Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * vtctl Restore supports --dry_run flag Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * flag --incremental-from-pos accepts the value 'auto', which takes the next incremental backup from last good backup Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * make vtadmin_web_proto_types Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * endtoend: validate --incremental_from_pos=auto Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * towards applying binary logs: extracting onto temporary directory Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * apply binary log file Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * do not restore replication at end of PITR Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * take dryrun into consideration Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * testing restore to pos Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * testing restore to pos: wait for replication, avoid bogus writes Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * validating PITR path when binary logs are missing history Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * full backup manifest now includes 'PurgedPosition', which is necessary to build a restore path. Now evaluated in IsValidIncrementalBakcup Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * more recovery paths tests Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * restrucutre tests Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * log restore path Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * generate CI workflows Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * code comments Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * code comments Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * code comments Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * CI 57 and 80 Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * flags test Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * copyright year Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * go version Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * removed legacy mysql80 test Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * PITR: stop search for possible resotre path with the first valid path, even if it's not the optimal Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * support incrementally union-izing of previous-GTIDs when iterating binary logs Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * removed local metadata info Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * merged main, regenerated workflows Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * go mod tidy Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * rename conflicting variable Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * refactor: const value Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * dry run restore now returns with 0 exit code, no error Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * release notes Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

2022-11-23

Add support for transaction access mode (#11704) * feat: added transaction access mode to proto Signed-off-by: Harshit Gangal <harshit@planetscale.com> * refactor: tx_pool create transaction code Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: use tx access mode in vttablet and set that when starting a transaction Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: reset transaction access mode after commit/rollback/release Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: added parsing support for start transaction with characteristics Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: set the transaction characteristics in execute option on start transaction statement Signed-off-by: Harshit Gangal <harshit@planetscale.com> * refactor: little tx_pool refactor Signed-off-by: Harshit Gangal <harshit@planetscale.com> * test: added e2e test for tx access modes Signed-off-by: Harshit Gangal <harshit@planetscale.com> * added release notes Signed-off-by: Harshit Gangal <harshit@planetscale.com> * addressed review comments Signed-off-by: Harshit Gangal <harshit@planetscale.com> * generated protobuf files updated Signed-off-by: Harshit Gangal <harshit@planetscale.com> Signed-off-by: Harshit Gangal <harshit@planetscale.com>

2022-11-20

Dynamic tablet throttler config: enable/disable, set metrics query/threshold (#11604) * Support syntax: ALTER VITESS_MIGRATION {DISABLE|ENABLE|THRESHOLD='...') Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * vtgate supports ALTER VITESS_THROTTLER syntax Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * initial support for AlterThrottler in query executor Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * towards enable/disable in throttler Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * no cleanup ticker for caches, single initialization Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * init metrics query and threshold (temporary) Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * replace if-else with switch Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * parser: support 'show vitess_throttler status' command Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * vtgate: support 'show vitess_throttler status' command Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * implement SHOW VITESS_THROTTLER STATUS Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * ugly quick hack for reducing flakiness Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * undo hack Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * IsEnabled() now implies IsOpen Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * THRESHOLD is decimal/integer Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * experimental, accessing topo for config Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * make proto Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * store throttler config is SrvKeyspace in all cells Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * apply throttler config Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * AlterThrottler does not get handled by QueryExecutor Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * no schema, removed file Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * unsupport ALTER VITESS_THROTTLER syntax Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * repltracker enables heartbeats if ondemand is positive Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * throttler does not requests heartbeats when disabled Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * endtoend test to cover throttler config by topo Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * make generate_ci_workflows Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * graceful sleep Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * Enable throttler in Open() when not reading from topo (maintain backwards compatibility) Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * improve test logic with WaitGroup Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * adapt test logic to new on-demand behavior. Add more test cases Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * removed AlterThrottler from vtgate Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * removed AlterThrottler from ast Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * update test Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * test custom config Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * validate return from custom query to default query Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * updated workflow Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * update endtoend cli docs Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * update endtoend cli docs: --throttle_config_via_topo Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * vtadmin web proto Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * s/throttle_config_via_topo/throttler_config_via_topo Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * do not listen on KeyspaceSrv if not configured to read from topo Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * reverse --throttle_threshold default increase Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * dashes: --throttler-config-via-topo Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * throttler check: flag for skipping heartbeat request (avoids renewing lease) Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * wait-based status checks Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * wait-based TestLag Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * reorganizing, cleanup Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * remove excessive sleep Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * vtctl subflags use dashes Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * validate mutually exclusive flags Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * fix cli test Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * empty commit to kick CI Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * disable active reparents on tests Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * do not return list of cells. Instead, just return exit code Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * improve error message Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * typo Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * copyright year Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * vtctldclient: support UpdateThrottlerConfig Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * vt admin proto Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * refactored throttler code to its own file Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * simplify: options are of UpdateThrottlerConfigRequest type Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * added keyspace Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * implementing server-side UpdateThrottlerConfig Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * validating that vtctldclient works Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * copyright year Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * safer Open() flow that does not block execution Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * expected CLI Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * vt admin proto Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * CustomQuerySet: identify existence of flag Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * vt admin proto Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * update ci workflow Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * go mod tidy Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * check 'Changed' after parsing Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * more verbose message; do give throttler timt to update config Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * metricsQuery is atomic; report metrics query in SHOW VITESS_THROTTLER STATUS Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * cobra comment Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * goimport Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * fix CLI docs Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * release notes Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

2022-11-17

VTAdmin: display workflow type in workflows list (#11685) * Add workflow type and subtype to workflow output. Display workflow type on Workflows page Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Run prettier Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Remove type/subtype from stream proto since we only use it at the global Workflow level Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Use RowNamedValues for getting workflow streams Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Rollback local example hacks added for testing Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Update vtadmin protos Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Update protos Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Address review comments Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Fix lint issue Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Fix lint issue Signed-off-by: Rohit Nayak <rohit@planetscale.com> Signed-off-by: Rohit Nayak <rohit@planetscale.com>

2022-11-10

Allow users to control VReplication DDL handling (#11532) * Allow users to control VReplication DDL handling Signed-off-by: Matt Lord <mattalord@gmail.com> * Update vtadmin web protos Signed-off-by: Matt Lord <mattalord@gmail.com> * Get Reshard working as well Signed-off-by: Matt Lord <mattalord@gmail.com> * Make proto with go 1.18.7 Signed-off-by: Matt Lord <mattalord@gmail.com> * Elide bls.on_ddl in marshalled JSON output Signed-off-by: Matt Lord <mattalord@gmail.com> * Add TestVreplicationDDLHandling e2e test Signed-off-by: Matt Lord <mattalord@gmail.com> * Cleanup binlogdata[pb] import declarations Signed-off-by: Matt Lord <mattalord@gmail.com> * Fix argument handling for moveTables() Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor changes after self review Signed-off-by: Matt Lord <mattalord@gmail.com> * Address review comments Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor improvement to usage output Signed-off-by: Matt Lord <mattalord@gmail.com> * Test EXEC_IGNORE too Signed-off-by: Matt Lord <mattalord@gmail.com> * Update unit test Signed-off-by: Matt Lord <mattalord@gmail.com> * Address review comments Signed-off-by: Matt Lord <mattalord@gmail.com> Signed-off-by: Matt Lord <mattalord@gmail.com>

2022-10-21

opt in/out of query consolidation (#11080) * opt in/out of query consolidation Since Vitess query consolidation can break normal MySQL read committedguarantees outside of transactions, it can be difficult to strongly verify ifan application can handle this before switching query ignoredalready switch on query consolidation on the replica, but not the primary (via -enable-consolidator-replicas and -enable-consolidator=false). However, some applications need to send queries to the primary that can also benefit from query consolidation. This PR adds the ability to opt in or out of query consolidation with: mysql> SELECT /*vt+ CONSOLIDATOR=disabled */ 1 FROM customer LIMIT 1; mysql> SELECT /*vt+ CONSOLIDATOR=enabled */ 1 FROM customer LIMIT 1; mysql> SELECT /*vt+ CONSOLIDATOR=enabled_replicas */ 1 FROM customer LIMIT 1; When this directive is set on a query, it overrides the VTTablet-level setting. This allows disabling query consolidation on the primary by default, and opting into query consolidation for specific queries. Or, conversely, it allow enabling query consolidation by queries by default, and opting out for queries that are known to not behave well when consolidated. Signed-off-by: Max Englander <max@planetscale.com> * opt in/out of query consolidation: add test Signed-off-by: Max Englander <max@planetscale.com> * opt in/out of query consolidation: update vtadmin web proto Signed-off-by: Max Englander <max@planetscale.com> * address pr comments: comments test, always set directive Signed-off-by: Max Englander <max@planetscale.com> * address pr comments: e2e test Signed-off-by: Max Englander <max@planetscale.com> * update unit tests to account for vcursor.SetConsolidator(consolidator) on each query Signed-off-by: Max Englander <max@planetscale.com> * Revert "update unit tests to account for vcursor.SetConsolidator(consolidator) on each query" This reverts commit 0bee60cc656cbdb0e7b5d66a44e6c949e45db7f2. Signed-off-by: Max Englander <max@planetscale.com> * address pr comments: avoid creating empty session options Signed-off-by: Max Englander <max@planetscale.com> Signed-off-by: Max Englander <max@planetscale.com>

2022-10-17

[VTAdmin] Topology Browser (#11496) * Code freeze of release-15.0 (#11427) Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> * Add web side of topology components Signed-off-by: notfelineit <notfelineit@gmail.com> * Add GetTopologyPath Signed-off-by: notfelineit <notfelineit@gmail.com> * Add protos Signed-off-by: notfelineit <notfelineit@gmail.com> * Add comment explaining why name is not needed at toplevel topo path Signed-off-by: notfelineit <notfelineit@gmail.com> * Add GetTopologyPath to vtadmin Signed-off-by: notfelineit <notfelineit@gmail.com> * Modify node code for non recursive Signed-off-by: notfelineit <notfelineit@gmail.com> * Add expand button Signed-off-by: notfelineit <notfelineit@gmail.com> * Update logic for GetTopologyPath Signed-off-by: notfelineit <notfelineit@gmail.com> * Render correct depth for nodes Signed-off-by: notfelineit <notfelineit@gmail.com> * Add tests TestGetTopologyPath Signed-off-by: notfelineit <notfelineit@gmail.com> * Add GetTopologyPath command Signed-off-by: notfelineit <notfelineit@gmail.com> * Run lint prettier fix Signed-off-by: notfelineit <notfelineit@gmail.com> * Consolidate DecodeContent to vt/topo package - can't put in vt/topo/topoproto because of import cycle Signed-off-by: notfelineit <notfelineit@gmail.com> * Add rbac check for TopologyResource Signed-off-by: notfelineit <notfelineit@gmail.com> * Update web/vtadmin/src/api/http.ts Co-authored-by: David Graham <david.malcom.graham@gmail.com> Signed-off-by: Frances Thai <31225471+notfelineit@users.noreply.github.com> * Update DecodeContent of zkcmd.go Signed-off-by: notfelineit <notfelineit@gmail.com> * Update help text Signed-off-by: notfelineit <notfelineit@gmail.com> Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> Signed-off-by: notfelineit <notfelineit@gmail.com> Signed-off-by: Frances Thai <31225471+notfelineit@users.noreply.github.com> Co-authored-by: Rameez Sajwani <rameezwazirali@hotmail.com> Co-authored-by: David Graham <david.malcom.graham@gmail.com>

2022-10-06

[VTAdmin] `Validate`, `ValidateShard`, `ValidateVersionShard`, `GetFullStatus` (#11438) * Add validate Signed-off-by: notfelineit <notfelineit@gmail.com> * Add validate to cluster page Signed-off-by: notfelineit <notfelineit@gmail.com> * Run lint prettier fix Signed-off-by: notfelineit <notfelineit@gmail.com> * Add UI for validate shard Signed-off-by: notfelineit <notfelineit@gmail.com> * Add validateShard Signed-off-by: notfelineit <notfelineit@gmail.com> * Run make web proto Signed-off-by: notfelineit <notfelineit@gmail.com> * Refactor validation results and implement validateShard Signed-off-by: notfelineit <notfelineit@gmail.com> * add http shards Signed-off-by: notfelineit <notfelineit@gmail.com> * Add GetFullStatus Signed-off-by: notfelineit <notfelineit@gmail.com> * Add keys and comment Signed-off-by: notfelineit <notfelineit@gmail.com> * Run lint prettier fix Signed-off-by: notfelineit <notfelineit@gmail.com> * Add ValidateVersionShard to vtctldclient Signed-off-by: notfelineit <notfelineit@gmail.com> * Add ValidateVersionShard Signed-off-by: notfelineit <notfelineit@gmail.com> * Update test for vtctldclient Signed-off-by: notfelineit <notfelineit@gmail.com> * Add test for ValidateVersionShard Signed-off-by: notfelineit <notfelineit@gmail.com> * run lint prettier fix again Signed-off-by: notfelineit <notfelineit@gmail.com> * Modify during 1:1 with gomez Signed-off-by: notfelineit <notfelineit@gmail.com> * Run lint prettier fix Signed-off-by: notfelineit <notfelineit@gmail.com> * Address PR review comments Signed-off-by: notfelineit <notfelineit@gmail.com> * Update indentation Signed-off-by: notfelineit <notfelineit@gmail.com>

2022-10-04

feat: added query timeout to vtgate default and per session (#11429) * feat: added query timeout to vtgate default and per session Signed-off-by: Harshit Gangal <harshit@planetscale.com> * updated flag expectation Signed-off-by: Harshit Gangal <harshit@planetscale.com> * add in release notes Signed-off-by: Harshit Gangal <harshit@planetscale.com> Signed-off-by: Harshit Gangal <harshit@planetscale.com>

2022-09-30

VTAdmin: shard actions (#11328) * Setting up page for shard actions Signed-off-by: notfelineit <notfelineit@gmail.com> * Add delete shard to ui Signed-off-by: notfelineit <notfelineit@gmail.com> * Complete UI for delete shard Signed-off-by: notfelineit <notfelineit@gmail.com> * Make deleteShard redirect to shards list Signed-off-by: notfelineit <notfelineit@gmail.com> * Add externally reparent Signed-off-by: notfelineit <notfelineit@gmail.com> * Add TabletExternallyReparented Signed-off-by: notfelineit <notfelineit@gmail.com> * Disable externally reparent if no tablet selected Signed-off-by: notfelineit <notfelineit@gmail.com> * add all actions to rbac config when --no-rbac is configured' Signed-off-by: notfelineit <notfelineit@gmail.com> * Connect up PlannedReparent Signed-off-by: notfelineit <notfelineit@gmail.com> * Add tablet type to select box Signed-off-by: notfelineit <notfelineit@gmail.com> * Show resulting logs after planned reparent Signed-off-by: notfelineit <notfelineit@gmail.com> * Add emergency failover Signed-off-by: notfelineit <notfelineit@gmail.com> * add max height to line area Signed-off-by: notfelineit <notfelineit@gmail.com> * Run linter Signed-off-by: notfelineit <notfelineit@gmail.com> * Update go/vt/servenv/grpc_server.go Co-authored-by: Iheanyi Ekechukwu <iekechukwu@gmail.com> Signed-off-by: Frances Thai <31225471+notfelineit@users.noreply.github.com> * Address PR comments Signed-off-by: notfelineit <notfelineit@gmail.com> * Run gofmt again Signed-off-by: notfelineit <notfelineit@gmail.com> * Rename master --> primary Signed-off-by: notfelineit <notfelineit@gmail.com> * Run prettier Signed-off-by: notfelineit <notfelineit@gmail.com> * Update go/vt/servenv/grpc_auth.go Co-authored-by: Iheanyi Ekechukwu <iekechukwu@gmail.com> Signed-off-by: Frances Thai <31225471+notfelineit@users.noreply.github.com> * Update web/vtadmin/src/components/inputs/Select.tsx Co-authored-by: Iheanyi Ekechukwu <iekechukwu@gmail.com> Signed-off-by: Frances Thai <31225471+notfelineit@users.noreply.github.com> * Trigger GH actions Signed-off-by: notfelineit <notfelineit@gmail.com> Signed-off-by: notfelineit <notfelineit@gmail.com> Signed-off-by: Frances Thai <31225471+notfelineit@users.noreply.github.com> Co-authored-by: Iheanyi Ekechukwu <iekechukwu@gmail.com>

2022-09-29

Allow non-SSL callers of VTGate RPC APIs to specify group information for the CallerID (#11260) * Add Groups to vtrpc.CallerID Signed-off-by: phanatic <phanatic@github.com> * fix formatting Signed-off-by: phanatic <phanatic@github.com> * update vtadmin proto Signed-off-by: phanatic <phanatic@github.com> * Add end-to-end test Signed-off-by: phanatic <phanatic@github.com> * remove comment formatting Signed-off-by: phanatic <phanatic@github.com> * add release notes Signed-off-by: phanatic <phanatic@github.com> * fix code formatting Signed-off-by: phanatic <phanatic@github.com> * fix vtgate help text Signed-off-by: phanatic <phanatic@github.com> Signed-off-by: <phanatic@github.com> * refactor tests Signed-off-by: phanatic <phanatic@github.com> * run go mod tidy Signed-off-by: phanatic <phanatic@github.com> * bring back comment changes Signed-off-by: phanatic <phanatic@github.com> * rename flag to use dashes instead of underscore Signed-off-by: Phani Raj <phani@planetscale.com> * undo comment formatting by make proto Signed-off-by: Phani Raj <phani@planetscale.com> * fix formatting Signed-off-by: Phani Raj <phani@planetscale.com> * fix help text fixture Signed-off-by: Phani Raj <phani@planetscale.com> Signed-off-by: phanatic <phanatic@github.com> Signed-off-by: <phanatic@github.com> Signed-off-by: Phani Raj <phani@planetscale.com> Co-authored-by: phanatic <phanatic@github.com>

2022-09-23

[VTAdmin] RebuildKeyspaceGraph, RemoveKeyspaceCell, NewShard (#11249) * Add RebuildKeyspaceGraph Signed-off-by: notfelineit <notfelineit@gmail.com> * Run lint:prettier:fix Signed-off-by: notfelineit <notfelineit@gmail.com> * Connect to backend Signed-off-by: notfelineit <notfelineit@gmail.com> * Add skeleton for RemoveKeyspaceCell Signed-off-by: notfelineit <notfelineit@gmail.com> * Add javascript equivalent of removeKeyspaceCell Signed-off-by: notfelineit <notfelineit@gmail.com> * Connect remove keyspace cell in UI Signed-off-by: notfelineit <notfelineit@gmail.com> * Run prettier Signed-off-by: notfelineit <notfelineit@gmail.com> * Add skeleton of create shard Signed-off-by: notfelineit <notfelineit@gmail.com> * Connect CreateShard to UI Signed-off-by: notfelineit <notfelineit@gmail.com> * Use underscores and fix lint issue Signed-off-by: notfelineit <notfelineit@gmail.com> * Remove unused component Signed-off-by: notfelineit <notfelineit@gmail.com> * Run prettier Signed-off-by: notfelineit <notfelineit@gmail.com> * Restart the build Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> Signed-off-by: notfelineit <notfelineit@gmail.com> * Undo changes to unrelated protos Signed-off-by: notfelineit <notfelineit@gmail.com> Signed-off-by: notfelineit <notfelineit@gmail.com> Signed-off-by: Florent Poinsard <florent.poinsard@outlook.fr> Co-authored-by: Florent Poinsard <florent.poinsard@outlook.fr>

Partial Movetables: allow moving a table one shard at a time (#9987) * Initial Implementation of partial movetables. Implement cross-shard queries that obey shard routing rules. Refactor rss updation with routed shard and add to multi col resolution. Run vtgate and reservedconn tests for both regular and with a partially moved shard. Add ability to run tests with partial keyspaces by adding to a list in the CI generator. Implement that in test.go and cluster setup. Revert previous iteration of running partial keyspace tests in parallel. One test is failing: TestSetSystemVarAutocommitWithConnError. Add partial vtgate misc test Add partial keyspace tests to vtgate transaction Add vtgate queries partial keyspace tests Review and fix skipped tests where applicable. Refactor to allow conditional skipping of tests for partial keyspaces only so that other tests work exactly as earlier without specifying the DbName, for example Add move shards example Cache routing tables and minor refactor. Move move_shards example Signed-off-by: Rohit Nayak <rohit@planetscale.com> Add lookup index to move shards example. Comment error logs while trying to delete from missing _vt.vdiff table Signed-off-by: Rohit Nayak <rohit@planetscale.com> Remove move shards demo example Signed-off-by: Rohit Nayak <rohit@planetscale.com> Add vtgate flag to enable shard routing. Update tests to get vtparams based on partial keyspace setting, so that global routing continues to be tested as earlier without DbName being specified Signed-off-by: Rohit Nayak <rohit@planetscale.com> To create subsequent partial movetables workflows we should ignore previous partial movetables (now frozen) Signed-off-by: Rohit Nayak <rohit@planetscale.com> Remove vdiff2 deletes since they were failing unit tests etc Signed-off-by: Rohit Nayak <rohit@planetscale.com> Use vtgate flag to enable shard routing rules Signed-off-by: Rohit Nayak <rohit@planetscale.com> Fix incorrect test setup Signed-off-by: Rohit Nayak <rohit@planetscale.com> Expect modified frozen query Signed-off-by: Rohit Nayak <rohit@planetscale.com> set vtgate param --enable_shard_routing in partial movetables test Signed-off-by: Rohit Nayak <rohit@planetscale.com> Fix failing tests Signed-off-by: Rohit Nayak <rohit@planetscale.com> Fix flags test Signed-off-by: Rohit Nayak <rohit@planetscale.com> Use VTParams helper for partial keyspace tests Signed-off-by: Rohit Nayak <rohit@planetscale.com> Specify vtParams.DBName for partial keyspace queries Signed-off-by: Rohit Nayak <rohit@planetscale.com> Self-review. Remove bypass caching. Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Report partial SwitchTraffic results Signed-off-by: Matt Lord <mattalord@gmail.com> * Try to address most of the upstream PR review comments From: https://github.com/vitessio/vitess/pull/9987 Signed-off-by: Matt Lord <mattalord@gmail.com> * Enforce proper routing for partial migrations Signed-off-by: Matt Lord <mattalord@gmail.com> Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Minor changes after self review Signed-off-by: Matt Lord <mattalord@gmail.com> Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Account for local timezone in check for vdiff progress Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Update go version in workflows. Fix gofmt issue Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Fix failing CI tests Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Comment failing test for partial keyspaces Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Fix TestVtGateVtExplain by specifying dbname Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Fix leftover debug code Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Move partial keyspace tests to a single shard. Refactor test.go to use multiple config.json files Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Address review comments Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Address review comments Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Improve test skip comments Signed-off-by: Rohit Nayak <rohit@planetscale.com> * gofmt Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Fix failing tests Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Address review comments Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Try TestFunctionInDefault for partial keyspaces. Update skip comment. Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Delete temporary log lines Signed-off-by: Rohit Nayak <rohit@planetscale.com> Signed-off-by: Rohit Nayak <rohit@planetscale.com> Signed-off-by: Matt Lord <mattalord@gmail.com> Co-authored-by: Matt Lord <mattalord@gmail.com>

2022-08-22

Add tracking session state changes for transaction start (#11061) * Add tracking session state changes for transaction start This logic depends on the custom option that PlanetScale as in it's MySQL fork to get the current committed GTID at the start of a transaction snapshot. With this GTID, we can optimize vreplication to avoid having to lock tables around vreplication operations. See https://github.com/planetscale/mysql-server/commit/9fa67ec3ebdeda923a67963260a9e744aa635a3a for the change in MySQL that is published in our fork. The change here allows getting back this value through any query operation that might start a transaction. Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> * Use 'session_track_gtids = START_GTID' where possible, and avoid LOCK tables while still getting a transaction consistent GTID Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> Co-authored-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

2022-08-08

Simply Replication Status proto conversions (#10926) * feat: simplify replication status Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: reserve the two fields on proto Signed-off-by: Manan Gupta <manan@planetscale.com> * test: fix typing error to use correct variable Signed-off-by: Manan Gupta <manan@planetscale.com> * test: remove test which is no longer required Signed-off-by: Manan Gupta <manan@planetscale.com>

2022-08-01

feat: add getfullstatus rpc to vtctldserver Signed-off-by: Manan Gupta <manan@planetscale.com>

2022-07-26

VDiff2: Add Stop Action (#10830) * Minor refactoring and stop action handler scaffolding Signed-off-by: Matt Lord <mattalord@gmail.com> * Initial implementation of Stop action Signed-off-by: Matt Lord <mattalord@gmail.com> * Add e2e test for Stop action Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor changes after self review Signed-off-by: Matt Lord <mattalord@gmail.com> * Remove unnecessary conditional uuid.Parse() with an empty string returns an error. Signed-off-by: Matt Lord <mattalord@gmail.com> * Don't mark a completed VDiff as stopped Signed-off-by: Matt Lord <mattalord@gmail.com> * Need to use withDDL.ExecIgnore in the vdiff retry loop to avoid potentially doing DDL during tablet state transitions which can cause the transition to be slow or even hang. Signed-off-by: Matt Lord <mattalord@gmail.com>

2022-07-22

VDiff2: Auto retry to continue on error (#10639) * Don't allow resume if VDiff not completed. An uncompleted vdiff should only be retried, not resumed. Signed-off-by: Matt Lord <mattalord@gmail.com> * Record and display VDiff errors per shard Signed-off-by: Matt Lord <mattalord@gmail.com> * Better align errors in text based output Signed-off-by: Matt Lord <mattalord@gmail.com> * Work with underlying database errors Signed-off-by: Matt Lord <mattalord@gmail.com> * Retry when appropriate on vdiff engine startup Signed-off-by: Matt Lord <mattalord@gmail.com> * Address bugs in resume/retry logic after adding support for both Signed-off-by: Matt Lord <mattalord@gmail.com> * Make auto-retry the default Signed-off-by: Matt Lord <mattalord@gmail.com> * Ensure report is valid json before unmarshaling Signed-off-by: Matt Lord <mattalord@gmail.com> * Auto retry error'd VDiffs Signed-off-by: Matt Lord <mattalord@gmail.com> * Don't retry error'd VDiffs on engine start anymore We now have a goroutine that will do this periodically Signed-off-by: Matt Lord <mattalord@gmail.com> * Limit withDDL usage to entry points Signed-off-by: Matt Lord <mattalord@gmail.com> * Add e2e test and fix bugs it exposed Signed-off-by: Matt Lord <mattalord@gmail.com> * Ensure we always signal the retry goroutine to stop on engine.Close() Signed-off-by: Matt Lord <mattalord@gmail.com> * Use vdiff engine mutex during retries Signed-off-by: Matt Lord <mattalord@gmail.com> * Close retry goroutine on vde ctx cancel w/o done channel Signed-off-by: Matt Lord <mattalord@gmail.com> * Open & Close of VDiff engine controls retry goroutine Signed-off-by: Matt Lord <mattalord@gmail.com> * Rely on sync.Once to apply VDiff schema And avoid withDDL usage anywhere else. Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor change after self review Signed-off-by: Matt Lord <mattalord@gmail.com> * Tidy up vdiff retry goroutine mgmt Signed-off-by: Matt Lord <mattalord@gmail.com> * Improving gorouting mgmt -- trying to eliminate flakes Signed-off-by: Matt Lord <mattalord@gmail.com> * Moar safety Signed-off-by: Matt Lord <mattalord@gmail.com> * Making more tweaks to exec and term more quickly Signed-off-by: Matt Lord <mattalord@gmail.com> * Aye dios mio Signed-off-by: Matt Lord <mattalord@gmail.com> * Go back to 30s ticker Signed-off-by: Matt Lord <mattalord@gmail.com> * Make engine open more efficient to improve PRS times Otherwise PRS was timing out too often waiting for the replicas to catch up with the new promoted primary. Signed-off-by: Matt Lord <mattalord@gmail.com> * isOpen=true before initControllers to ensure proper Close Signed-off-by: Matt Lord <mattalord@gmail.com> * Do lazy init in 2 entry points when doing actual work. Check for cancelled context in the controller.run code path. Signed-off-by: Matt Lord <mattalord@gmail.com> * Bug fixes and improvements TODO: figure out how we should be tracking progress for merges where there's a single target shard -- and thus a single _vt.vdiff_table record -- but > 1 source shard. Signed-off-by: Matt Lord <mattalord@gmail.com> * Final (🤞) set of bug fixes Signed-off-by: Matt Lord <mattalord@gmail.com> * Add progress reporting Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor changes after self review Signed-off-by: Matt Lord <mattalord@gmail.com> * Improved template for error handling in text format Signed-off-by: Matt Lord <mattalord@gmail.com> * Bug fixes around progress calculation Signed-off-by: Matt Lord <mattalord@gmail.com> * Comment improvement Signed-off-by: Matt Lord <mattalord@gmail.com> * Ensure we get correct DB name from vreplication workflow. And quickly timeout the receive attempted on the controller's done channel when checking if it's active. Signed-off-by: Matt Lord <mattalord@gmail.com> * Bug fixes and improvements to progress reporting Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor changes after self review Signed-off-by: Matt Lord <mattalord@gmail.com> * Remove unnecessary case in select We'll never hit that as the first one will receive immediately as the channel was closed or we'll hit the default. Signed-off-by: Matt Lord <mattalord@gmail.com> * Add unit test for progress reporting Signed-off-by: Matt Lord <mattalord@gmail.com> * Be more realistic in half way unit test Signed-off-by: Matt Lord <mattalord@gmail.com> * Use more descriptive var names in unit test to make logic clear Signed-off-by: Matt Lord <mattalord@gmail.com> * Protect all access to vde.controllers Add comment about safe usage of vde.addController() Signed-off-by: Matt Lord <mattalord@gmail.com> * De-flake ers_prs tests using state waits with timeouts This is a test of the fix in: https://github.com/vitessio/vitess/pull/10720 Signed-off-by: Matt Lord <mattalord@gmail.com> * Minor improvement -- also to exec the test suite again Signed-off-by: Matt Lord <mattalord@gmail.com> * Use simpler defer stmt Signed-off-by: Matt Lord <mattalord@gmail.com> * Correct VDiff final state handling Signed-off-by: Matt Lord <mattalord@gmail.com> * Improve WorkflowDiffer logging & error handling Signed-off-by: Matt Lord <mattalord@gmail.com> * Use withDDL.ExecIgnore during engine startup The VReplication engine does the same. Signed-off-by: Matt Lord <mattalord@gmail.com>

2022-07-20

Parameterize BIT types and fixes in HEX types (#10689) * feat: add parsing for bitnums Signed-off-by: Manan Gupta <manan@planetscale.com> * test: add invalid cases to tests Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: normalize bitnums to bit vals too Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add normalization for bit literals Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: parameterize binary value to hex Signed-off-by: Harshit Gangal <harshit@planetscale.com> * test: added e2e test in vtgate and vttablet Signed-off-by: Harshit Gangal <harshit@planetscale.com> * fix: fix the type conversion from hexnum and hexval to binary Signed-off-by: Harshit Gangal <harshit@planetscale.com> * test: refactor test Signed-off-by: Harshit Gangal <harshit@planetscale.com> * fix: return varbinary and flaghex for hexval and hexnum in eval engine typeOf Signed-off-by: Harshit Gangal <harshit@planetscale.com> * test: separate expectation for mysql and vitess result Signed-off-by: Harshit Gangal <harshit@planetscale.com> * test: fix test expectation Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: added bitnum bind variable and test Signed-off-by: Harshit Gangal <harshit@planetscale.com> * proto: vtadmin side update Signed-off-by: Harshit Gangal <harshit@planetscale.com> * test: fixed test and added more bitnum bind var test Signed-off-by: Harshit Gangal <harshit@planetscale.com> * added license header Signed-off-by: Harshit Gangal <harshit@planetscale.com> Co-authored-by: Manan Gupta <manan@planetscale.com>

2022-07-07

Delete all legacy sharding related code (#10278) * Delete all legacy sharding related code Signed-off-by: Matt Lord <mattalord@gmail.com> * Move used until functions from initialsharding to cluster Signed-off-by: Matt Lord <mattalord@gmail.com> * Remove vtctl commands Signed-off-by: Matt Lord <mattalord@gmail.com> * Kill vtworker and SetKeyspaceServedFrom cmd Signed-off-by: Matt Lord <mattalord@gmail.com> * WaitForDrain related stragglers Signed-off-by: Matt Lord <mattalord@gmail.com> * Legacy local straggler workflow Signed-off-by: Matt Lord <mattalord@gmail.com> * Get rid of SetKeyspaceShardingInfo & wait for drain stragglers Signed-off-by: Matt Lord <mattalord@gmail.com> * Remove vtworker stragglers Signed-off-by: Matt Lord <mattalord@gmail.com> * Update throttlerservice protobuf Signed-off-by: Matt Lord <mattalord@gmail.com> * Rename test 24, add hashicorp vault test to it (now mysql_server_vault) Signed-off-by: Matt Lord <mattalord@gmail.com> * Remove last mentiones of legacy sharding in vtctl Signed-off-by: Matt Lord <mattalord@gmail.com> * remove binlog_use_v3_resharding_mode Signed-off-by: Matt Lord <mattalord@gmail.com> * Address review comments Signed-off-by: Matt Lord <mattalord@gmail.com> * Address review comments Signed-off-by: Matt Lord <mattalord@gmail.com> * Correct vtgate help output Signed-off-by: Matt Lord <mattalord@gmail.com>

2022-07-06

delete unused flags and handling of deprecated protobuf fields (#10612) * deprecation: delete unused flags and handling of deprecated protobuf fields Signed-off-by: deepthi <deepthi@planetscale.com> * test: fix TestHelpOutput assertion Signed-off-by: Manan Gupta <manan@planetscale.com> * endtoend: fix tests that were using deleted flag enable-autocommit Signed-off-by: deepthi <deepthi@planetscale.com> * deprecation: reserve multiple fields on same line, found and reserved more deprecated fields Signed-off-by: deepthi <deepthi@planetscale.com> * deprecation: delete handling of deprecated fields from ExecuteBatch and StreamExecute RPCs Signed-off-by: deepthi <deepthi@planetscale.com> * java: remove deleted asTransaction field from executeBatch Signed-off-by: deepthi <deepthi@planetscale.com> Co-authored-by: Manan Gupta <manan@planetscale.com>

2022-07-05

VReplication: throttling info for both source and target; Online DDL propagates said info (#10601) * vreplication throttling information Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * using RateLimiter Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * rowstreamer reports throttling status, captured by vcopier and logged to vreplication table Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * remove magic hint, add 'throttled' field Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * propagate throttling information into schema_migrations table Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * onlineddl/vrepl endtoend test: verify propagation of throttling info Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * test vstreamer throttling Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * simplify: reuse RateLimiter Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * vtadmin_web_proto_types Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * fix EOF return path Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * expose the two new column in 'vtctl Workflow ... Show' Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * adapt test queries to change in wrangler query Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * fix test Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * proto: add Heartbeat indicator in VStreamRowsResponse Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * fix unit tests Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * RateLimiter.Stop() Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * rowstreamer now sends heartbeats, received by vcopier, and updated in _vt.vreplication Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * whoops. Removed debug Sleep() Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * make vtadmin_web_proto_types Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * skip throttled events Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * f allowed to be nil Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * fix 'now' calculation Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * remove numeric precision Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * ix typo Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * indicate Unix timestamp in field name Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * grammar Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * refactor WaitForThrottledTimestamp Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * formalize ComponentName Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * throttled app is vstreamer Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

2022-06-22

Proto file lint fix and vtadmin generated file (#10563) * lint: add a new line to proto file and generate generated file Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: fix workflow to also setup go when there are proto changes Signed-off-by: Manan Gupta <manan@planetscale.com>

vtctl GetSchema --table_schema_only (#10552) * vtctl GetSchema --table_schema_only Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * release notes Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * no short form Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * no short form Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * updated flag comment Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * remove test; the test result is a fake, and therefore is meaningless Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * skip column introspection Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * skip column introspection Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com> * change to dashes Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

2022-06-20

Adds RPCs to vttablet that vtorc requires (#10464) * feat: add vttablet rpc to reset replication parameters Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: added end to end testing for the rpc and fixed bug Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: fix typing error Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add basic full status rpc functionality and add test for it Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add all the fields needed in full status Signed-off-by: Manan Gupta <manan@planetscale.com> * test: moved the test to reparent tests and improved it Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: bug fix for no replication status and no primary status Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add version to the full status output Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add binlog information to full status Signed-off-by: Manan Gupta <manan@planetscale.com> * docs: fix the comment explaining the binlog information Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add semi-sync statuses to full status Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: call the correct command Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add server uuid and id to full status Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: make server_id a uint32 to accept the correct range of values Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add few more fields to the full status like version comment, semi-sync settings, binlog_row_image Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: generate vtadmin proto files Signed-off-by: Manan Gupta <manan@planetscale.com> * test: add assertion to check binlog row format is read correctly Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: split GTID mode in its own function because mariadb doesn't support it Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: fix parsing of empty mariadb gtid set Signed-off-by: Manan Gupta <manan@planetscale.com> * docs: add doucmentation for existing fields in ReplicationStatus Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add relay log file position to the replication status output Signed-off-by: Manan Gupta <manan@planetscale.com> * test: augmented full status test to check all the different positions stored Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add additional fields to replication status and read source user Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: read sql delay from show replica status output Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: read ssl allowed from show replica status output Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: read has replication filters from show replica status output Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: read auto position and using gtid from show replica status output Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add replication lag unknown too to replication status Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: return nils from replication and primary postiion if it is not present Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: rename FileRelayLogPosition in replication status output to RelayLogSourceBinLogEquivalentPosition and augment test to make sure rpc changes are backward compatible Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: update vtadmin proto files Signed-off-by: Manan Gupta <manan@planetscale.com> * refactor: rename BinLog to binlog in renamed proto field Signed-off-by: Manan Gupta <manan@planetscale.com>

2022-06-19

Fix: handle all cases for consistent lookup unique on single transaction mode (#10493) * test: added failing e2e test Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: proto changes for vindex_only shard session Signed-off-by: Harshit Gangal <harshit@planetscale.com> * session to ignore multi-db transaction error if shard session is used only for vindex query Signed-off-by: Harshit Gangal <harshit@planetscale.com> * test: added additional e2e tests that fails Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: claim a vindex only session if other query uses it Signed-off-by: Harshit Gangal <harshit@planetscale.com> * refactor: addressed all review comments Signed-off-by: Harshit Gangal <harshit@planetscale.com>

2022-06-10

Remove v2 resharding fields (#10409) * cleanup: remove sharding_column_name and sharding_column_type Signed-off-by: Arvind Murty <arvind.murty@gmail.com> * cleanup: remove sharding_column_name and sharding_column_type Signed-off-by: Arvind Murty <arvind.murty@gmail.com> * cleanup: remove sharding_column_name and sharding_column_type Signed-off-by: Arvind Murty <arvind.murty@gmail.com> * cleanup: remove sharding_column_name and sharding_column_type Signed-off-by: Arvind Murty <arvind.murty@gmail.com> * generate vtadmin files Signed-off-by: Arvind Murty <arvind.murty@gmail.com> * cleanup: remove sharding_column_name and sharding_column_type from vtadmin Signed-off-by: Arvind Murty <arvind.murty@gmail.com> * Merge from main Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

2022-06-07

Vdiff2: initial release (#10382) * VDiff2: rearchitect VDiff to run on tablets Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Improve show summary output and add 'show last' Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Improve e2e test Signed-off-by: Rohit Nayak <rohit@planetscale.com> * go mod tidied Signed-off-by: Rohit Nayak <rohit@planetscale.com> * regenerate vtadmin protos Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Minor tweaks/improvements to the e2e test Signed-off-by: Matt Lord <mattalord@gmail.com> * Another round of minor tweaks Signed-off-by: Matt Lord <mattalord@gmail.com> * Use UUID types directly and remove string literals Signed-off-by: Matt Lord <mattalord@gmail.com> * Another round of minor tweaks Signed-off-by: Matt Lord <mattalord@gmail.com> * You guessed it!! More minor tweaks Signed-off-by: Matt Lord <mattalord@gmail.com> * Improve handling of last and all with no results Signed-off-by: Matt Lord <mattalord@gmail.com> * Remove double printing for show all output Signed-off-by: Matt Lord <mattalord@gmail.com> * Return empty json array for empty show all results Signed-off-by: Matt Lord <mattalord@gmail.com> * Add vdiff2 test to CI Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Trigger rebuild Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Another round of tweaks during review Signed-off-by: Matt Lord <mattalord@gmail.com> * Final round of code tweaks Signed-off-by: Matt Lord <mattalord@gmail.com> * Correct has_mismatch handling Signed-off-by: Matt Lord <mattalord@gmail.com> * Adjust reporting for extra/mismatched row samples Signed-off-by: Matt Lord <mattalord@gmail.com> * Use proto.Clone to copy VStreamRowsResponse And use small vstream_packet_size in e2e test. Signed-off-by: Matt Lord <mattalord@gmail.com> * Refactor tests. Call VDiff2 side by side with all VDiff1s. Currently failing m2m3 test workflow. Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Add doc.go to package and use a const for trunctated notation And rename readme.md to README.md so that it is listed first in lexicographical order. Signed-off-by: Matt Lord <mattalord@gmail.com> * Correct year in header for new doc.go file :facepalm: Signed-off-by: Matt Lord <mattalord@gmail.com> * Fix bug where we were not using in_keyrange() for filtering from source. Change logic to update table level mismatch flag to post-reconciliation of extra rows. Enhance vdiff2 test to include split/merge and full merge (working, but needs code refactoring). Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Use unbuffered channels in VDiff2 goroutine comms TL;DR — I think it was because we were using buffered channels in the table differ. So the order was not guaranteed and could be off by 1. I suspect the deep copy synchronized things as a side effect. I think it may also explain why even after the previous test correction to return in the wait after sending true to the results channel, vtctlclient would sometimes get another result after things were torn down. Because the channels were buffered, where the comment says that we block, we didn’t until there was 1+1 messages sent to the channel and the order was not enforced between the goroutines. Signed-off-by: Matt Lord <mattalord@gmail.com> * Increase vdiff timeout Signed-off-by: Matt Lord <mattalord@gmail.com> * VDiff2 does not support inter cluster diffs today Signed-off-by: Matt Lord <mattalord@gmail.com> * Go back to using buffered chans for the vstreams Added a comment explaining why we *want* to do the Clone here. If we need other synchronization points in the code paths then we can handle that independently. Signed-off-by: Matt Lord <mattalord@gmail.com> * Refactor vdiff2 tests Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Minor tweaks after final review Signed-off-by: Matt Lord <mattalord@gmail.com> * Correct test config after merging main branch Signed-off-by: Matt Lord <mattalord@gmail.com> * Add VDiff2 to the v14 release notes summary Signed-off-by: Matt Lord <mattalord@gmail.com> * Help output should not show --v2 as defaulting to true And add note about how to use VDiff2 with example usage to the release notes summary. Signed-off-by: Matt Lord <mattalord@gmail.com> Co-authored-by: Matt Lord <mattalord@gmail.com>

enhancement: handle advisory locks (#10186) * feat: parse advisory lock functions Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: added new tokens to keywords file Signed-off-by: Harshit Gangal <harshit@planetscale.com> * refactor: planner to use new locking func struct Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: add locking functions provided in the select expressions to the lock engine primitive Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: block mixing of locking function with other expression in dual query Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: proto changes to incluse advisory lock info Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: changed unsafe pooling logic based on new locking function parsing Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: advisory lock implementation to track and remove the lock and close lock session when no lock held Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: support only liternal string in lock name Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: allow lock functions to be executed outside of reserved connections except for get_lock Signed-off-by: Harshit Gangal <harshit@planetscale.com> * fix: use evalengine to evaluate the name from the ast expression Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: allow impossible query to execute out of reserved connection for pool unsafe queries Signed-off-by: Harshit Gangal <harshit@planetscale.com> * fix: getfields for lock function and name to be extracted as evalengine.Expr in plan time Signed-off-by: Harshit Gangal <harshit@planetscale.com> * test: added e2e test from #10130 Signed-off-by: Harshit Gangal <harshit@planetscale.com> * feat: change endAction to endLockAction Signed-off-by: Harshit Gangal <harshit@planetscale.com> * refactor: for backward compatibility Signed-off-by: Harshit Gangal <harshit@planetscale.com> * addressed review comments Signed-off-by: Harshit Gangal <harshit@planetscale.com>

2022-06-03

[vtadmin] Rename reparent-related RPCs, endpoints, methods, RBAC actions (#10404) * [vtadmin] Rename reparent-related RPCs, endpoints, methods, RBAC actions - `EmergencyReparentShard` => `EmergencyFailoverShard` - `PlannedReparentShard` => `PlannedFailoverShard` - `ReparentTablet` => `RefreshTabletReplicationSource` - `TabletExternallyReparented` => `TabletExternallyPromoted` Signed-off-by: Andrew Mason <andrew@planetscale.com> * Update vtadmin-web code that referenced old type Signed-off-by: Andrew Mason <andrew@planetscale.com> * fixup! Update vtadmin-web code that referenced old type Signed-off-by: Andrew Mason <andrew@planetscale.com> * Remove confusing comments Signed-off-by: Andrew Mason <andrew@planetscale.com> * figured out how to run the lint-fixer Signed-off-by: Andrew Mason <andrew@planetscale.com>

2022-05-27

VStream API: allow cells to be specified for picking source tablets to stream from (#10294) * Add parameter Cells in vstream api flags, for tablet picker to look for candidates in Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Add e2e test for the new Cells vstream flag Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Self-review Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Address review comments Signed-off-by: Rohit Nayak <rohit@planetscale.com>

2022-05-26

Adds DurabilityPolicy to the KeyspaceInfo in the topo and the associated RPCs (#10221) * feat: add durability_policy to the keyspace information in the topo server Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add durability_policy parameter to the CreateKeyspace command in vtctl and vtctld Signed-off-by: Manan Gupta <manan@planetscale.com> * test: add tests for durability_policy field in the keyspace information Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: add a new rpc to update the durability policy in the keyspace information Signed-off-by: Manan Gupta <manan@planetscale.com> * test: add testing for the newly added rpc SetKeyspaceDurabilityPolicy Signed-off-by: Manan Gupta <manan@planetscale.com> * test: changed the e2e test to also test the vtctldclient interface Signed-off-by: Manan Gupta <manan@planetscale.com> * docs: add a comment explaining the test objective Signed-off-by: Manan Gupta <manan@planetscale.com> * test: fix vtctld api test to expect durability_policy as a new returned value in the keyspace information Signed-off-by: Manan Gupta <manan@planetscale.com> * docs: fix the help messages for the new command Signed-off-by: Manan Gupta <manan@planetscale.com> * docs: fix casing and grammar in the flag description Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: change flag to be hyphenated instead of underscored Signed-off-by: Manan Gupta <manan@planetscale.com> * docs: fix the year of license of new file Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: remove the vtctl command Signed-off-by: Manan Gupta <manan@planetscale.com> * test: remove the test for the deleted command Signed-off-by: Manan Gupta <manan@planetscale.com> * refactor: fix the alphabetic order of functions and rpc registrations Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: fix ordering in vtctld file too Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: improve the long help message Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: fix typing error Signed-off-by: Manan Gupta <manan@planetscale.com> * docs: added documentation for keyspace type Signed-off-by: Manan Gupta <manan@planetscale.com>

2022-05-21

[vtadmin] reparenting actions (#10351) * Add protos for remaining reparent actions and run generators Signed-off-by: Andrew Mason <andrew@planetscale.com> * Initial reparent-actions implementations Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add pools and configs for reparent operations Signed-off-by: Andrew Mason <andrew@planetscale.com> * Update impls to use new pools Signed-off-by: Andrew Mason <andrew@planetscale.com> * jk, TER only updates the topo Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add http wrappers Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add unit tests Signed-off-by: Andrew Mason <andrew@planetscale.com> * Update local rbac to allow reparents Signed-off-by: Andrew Mason <andrew@planetscale.com>

2022-05-17

[vtadmin] tablet topopools + other cleanup (#10325)

[vtadmin] reload schema (#10300) * Start adding proto messages Signed-off-by: Andrew Mason <andrew@planetscale.com> * Fix ordering Signed-off-by: Andrew Mason <andrew@planetscale.com> * fixup! Start adding proto messages Signed-off-by: Andrew Mason <andrew@planetscale.com> * Implement cluster-level schema reloading Signed-off-by: Andrew Mason <andrew@planetscale.com> * Implement api method handler Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add http handler wrappers Signed-off-by: Andrew Mason <andrew@planetscale.com> * default to "all keyspaces" when nothing else set Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add tests for reloadKeyspaceSchemas Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add tests for reloadShardSchemas Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add tests for reloadSchemaTablets Signed-off-by: Andrew Mason <andrew@planetscale.com> * always forgetting about rbac Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add docs for proto sub-messages Signed-off-by: Andrew Mason <andrew@planetscale.com>

2022-05-16

[vtadmin] Add support for 'allow_primary' when deleting a tablet (#10304) * Add 'allow_primary' to vtadmin.proto's DeleteTabletRequest Signed-off-by: Sara Bee <855595+doeg@users.noreply.github.com> * Update vtadmin-api to pass through allow_primary Signed-off-by: Sara Bee <855595+doeg@users.noreply.github.com> * Update UI to set allow_primary=true Signed-off-by: Sara Bee <855595+doeg@users.noreply.github.com> * Update wording for notification on a successful DELETE request Signed-off-by: Sara Bee <855595+doeg@users.noreply.github.com> * Add Advanced.test.tsx + deletion tests Signed-off-by: Sara Bee <855595+doeg@users.noreply.github.com> * Update UI wording: allow_master -> allow_primary Signed-off-by: Sara Bee <855595+doeg@users.noreply.github.com> * Fix lint warnings Signed-off-by: Sara Bee <855595+doeg@users.noreply.github.com>

2022-05-13

[vtadmin] proper alias types for tablet RPCs (#10290) * Add Vars type Signed-off-by: Andrew Mason <andrew@planetscale.com> * [vtadmin] Tablet APIs string alias -> typed alias Signed-off-by: Andrew Mason <andrew@planetscale.com> * Fix assertion to handle proto messages Signed-off-by: Andrew Mason <andrew@planetscale.com> * Cleanup redundant filtering Signed-off-by: Andrew Mason <andrew@planetscale.com>

2022-05-09

[vtadmin] cell apis (#10227) * Add protos and generate Signed-off-by: Andrew Mason <andrew@planetscale.com> * Implement GetCellInfos Signed-off-by: Andrew Mason <andrew@planetscale.com> * Implement GetCellsAliases Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add http adapters Signed-off-by: Andrew Mason <andrew@planetscale.com> * simplify mutual-exclusive options check Signed-off-by: Andrew Mason <andrew@planetscale.com> * add mocking infra Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add unit tests Signed-off-by: Andrew Mason <andrew@planetscale.com> * Remove explicit dial calls after rebase Signed-off-by: Andrew Mason <andrew@planetscale.com>

2022-04-29

Backwards compatible replication status to state transition (#10167) * This effectively reverts 65226ad1d77629fe29e852393a19250a9a910024 It was NOT backwards compatible. Signed-off-by: Matt Lord <mattalord@gmail.com> * This adds the new state fields back so that we can switch to them in v15 Signed-off-by: Matt Lord <mattalord@gmail.com> * Allow for safe/smooth upgrades within 14.0-SNAPSHOT We are only appending the last io_thread_connecting field Signed-off-by: Matt Lord <mattalord@gmail.com> * Avoid intermediate io_thread_connecting protobuf field Signed-off-by: Matt Lord <mattalord@gmail.com> * Continue using replication states in v14 tests Signed-off-by: Matt Lord <mattalord@gmail.com> * Translate SQL running status to state as well Signed-off-by: Matt Lord <mattalord@gmail.com> * Use backward compat ReplicaWasRunning check Signed-off-by: Matt Lord <mattalord@gmail.com> * Add backward compat SQLThreadWasRunning function Signed-off-by: Matt Lord <mattalord@gmail.com> * Add comment about when backward compat can be removed Signed-off-by: Matt Lord <mattalord@gmail.com> * Support older clients with new tablets Signed-off-by: Matt Lord <mattalord@gmail.com> * feat: remove SQLThreadWasRunning unused function Signed-off-by: Manan Gupta <manan@planetscale.com> * test: add an upgrade test to verify the replicationstatus is backward compatible Signed-off-by: Manan Gupta <manan@planetscale.com> * feat: fix ReplicaWasRunning so that it doesn't have code dependent on the upgrade Signed-off-by: Manan Gupta <manan@planetscale.com> * refactor: rename function to reflect the output type Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: Manan Gupta <manan@planetscale.com>

2022-04-27

MoveTables: adjust datetimes when importing from non-UTC sources into UTC targets (#10102) * Add e2e test case for source and target in different time zones Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Added source_time_zone option to MoveTables v2, pass it through to table plan builder and convert datetime columns if option is provided. e2e test demonstrates this works. VDiff needs to be modified Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Add time zone support in VDiff. Improve tests. Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Add vdiff unit test Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Add SourceTimeZone to Workflow Show Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Update workflow test for added attribute SourceTimeZone Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Support SourceTimeZone in reverse workflows, update e2e test Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Update unit tests Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Refactor vdiff timezone-related changes Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Address review comment nits Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Fix: lower col name also for time zone cols Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Use field type for type comparison instead of the string representation Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Check if current primaries have time zone loaded Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Address review comments Signed-off-by: Rohit Nayak <rohit@planetscale.com> * Use same date for conversion and as date format Signed-off-by: Rohit Nayak <rohit@planetscale.com>

2022-04-26

refactor: removed last reserved field from vtgate session (#10140) Signed-off-by: Harshit Gangal <harshit@planetscale.com>

2022-04-25

fix: remove set_var session field Signed-off-by: Harshit Gangal <harshit@planetscale.com>

2022-04-20

Improve Tablet Refresh Behavior in VReplication Traffic Switch Handling (#10058) * Use lower tablet refresh timeout in vreplication traffic switching And refresh all tablets in dry runs to alert the user to potential issues. * Check primary tablet health in the traffic switcher pre-check * Add msgs about specific tablets that failed refresh to cmd output * Keep function semantics the same but provide details if wanted This keeps the previous behavior of ignoring partial refresh related errors, while providing the defails of WHY we had a partial refresh for anyone that's interested. * Add partial refresh details to response message This way the caller has that info and can decide what to do with it (if anything). * Update tests and vtadmin for updated refresh response message * Cancel switch writes on partial tablet refresh when safe Signed-off-by: Matt Lord <mattalord@gmail.com>

delete RPCs that were deprecated in 13.0 (#10099) * delete RPCs that were deprecated in 13.0 Signed-off-by: deepthi <deepthi@planetscale.com> * rename instead of deleting fuzzing tests Signed-off-by: deepthi <deepthi@planetscale.com> * rename tests instead of deleting Signed-off-by: deepthi <deepthi@planetscale.com> * fix imports Signed-off-by: deepthi <deepthi@planetscale.com>

2022-04-07

[vtctldserver] Migrate GetPermissions PR#9903 (#10013) * [vtctldserver] Migrate GetPermissions PR#9903 This PR migrates GetPermissions rpc to VtctldServer Run it locally: vtctldclient --server ":15999" GetPermissions zone1-100 unit test: go test vitess.io/vitess/go/vt/vtctl/grpcvtctldserver -run TestGetPermissions Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * [vtctldserver] Migrate GetPermissions PR#9903 This PR migrates GetPermissions rpc to VtctldServer Run it locally: vtctldclient --server ":15999" GetPermissions zone1-100 unit test: go test vitess.io/vitess/go/vt/vtctl/grpcvtctldserver -run TestGetPermissions Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * [vtctldserver] Migrate GetPermissions PR#9903 This PR migrates GetPermissions rpc to VtctldServer Run it locally: vtctldclient --server ":15999" GetPermissions zone1-100 unit test: go test vitess.io/vitess/go/vt/vtctl/grpcvtctldserver -run TestGetPermissions Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com> * [vtctldserver] Migrate GetPermissions PR#9903 This PR migrates GetPermissions rpc to VtctldServer Run it locally: vtctldclient --server ":15999" GetPermissions zone1-100 unit test: go test vitess.io/vitess/go/vt/vtctl/grpcvtctldserver -run TestGetPermissiions Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

2022-03-23

add VReplicationWorkflowType Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

2022-03-17

[vtctldserver] Migrate `ShardReplication{Add,Remove}` RPCs (#9899) * Add ShardReplication{Add,Remove} proto definitions and generate Signed-off-by: Andrew Mason <andrew@planetscale.com> * Port implementations to VtctldServer Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add unit tests Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add CLI entrypoints Signed-off-by: Andrew Mason <andrew@planetscale.com>

2022-03-16

[vtctldserver] Migrate `ShardReplicationFix` RPC (#9878) * Add ShardReplicationFix proto definitions and run generators Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add `ShardReplicationFix` implementation No unit tests, because `topo.FixShardReplication` is tested in isolation and we're just a passthrough Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add CLI entrypoint Signed-off-by: Andrew Mason <andrew@planetscale.com>

[vtctldserver] ExecuteFetchAs{App,DBA} (#9890)

2022-03-14

[topo] ShardReplicationFix typed responses (#9876) * Add ShardReplicationProblem protobuf message Signed-off-by: Andrew Mason <andrew@planetscale.com> * Update `FixShardReplication` to return problem info, update callers Signed-off-by: Andrew Mason <andrew@planetscale.com> * Tidy up imports Signed-off-by: Andrew Mason <andrew@planetscale.com> * Rename ShardReplicationProblem=>ShardReplicationError Signed-off-by: Andrew Mason <andrew@planetscale.com>

[vtctldserver] Migrate `SourceShard{Add,Delete}` RPCs (#9886) * Add SourceShard{Add,Delete} rpcs and generate Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add SourceShard{Add,Delete} implementations Signed-off-by: Andrew Mason <andrew@planetscale.com> * Port implementation back to wrangler, preserving old error behavior Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add unit tests Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add CLI entrypoints Signed-off-by: Andrew Mason <andrew@planetscale.com>

2022-03-12

[vtctldserver] Migrate `RestoreFromBackup` RPC (#9873) * Add RestoreFromBackup proto defs and regenerate Signed-off-by: Andrew Mason <andrew@planetscale.com> * Port implementation from legacy vtctl command Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add CLI entrypoint Signed-off-by: Andrew Mason <andrew@planetscale.com> * Add test cases Signed-off-by: Andrew Mason <andrew@planetscale.com>