These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)
| Commit: | 738546a | |
|---|---|---|
| Author: | Matthias Crauwels | |
| Committer: | Matthias Crauwels | |
vdiff update: make tablet_selection_preference optional vtctld could not tell whether a VDiffUpdateRequest set tablet_selection_preference, since ANY is the proto3 zero value, so a preference sent without tablet types was silently ignored for callers that use the RPC directly instead of vtctldclient. Mark the field optional, like tablet_selection_preference, on_ddl and state on UpdateVReplicationWorkflowRequest, and reject a preference without tablet types with INVALID_ARGUMENT. When the field is unset, ANY is used, as before. vtctldclient now sends the preference only together with --tablet-types. The repeated fields can't be optional in proto3. The cell lists keep using clear_source_cells and clear_target_cells to ask for "all cells", and an empty tablet_types list still means "leave unchanged". Signed-off-by: Matthias Crauwels <matthias.crauwels@planetscale.com>
| Commit: | 2884dc5 | |
|---|---|---|
| Author: | Matthias Crauwels | |
| Committer: | Matthias Crauwels | |
vdiff update: say tablet picker instead of picker "Picker" on its own is not clear to users. What the update command changes are the tablet picker options, since TabletPicker is both a type and a concept in Vitess, so say that everywhere this PR adds text: the changelog entry, the update command's Short and Long help, the "no tablet picker options specified to update" errors on the client, vtctld and tablet, the "Updated tablet picker options" _vt.vdiff_log message, the proto and Go doc comments, and the test names. The local helpers this PR added are renamed to match (tabletPicker, tabletPickerHasUpdates, mergeTabletPickerOptions, protoCloneTabletPickerOptions). The existing VDiffPickerOptions type, the PickerOptions field and the picker_options JSON key stored in _vt.vdiff.options are unchanged. Also pair the --target-cells help with --source-cells: "compare from" and "compare to". Signed-off-by: Matthias Crauwels <matthias.crauwels@planetscale.com>
| Commit: | 7817edf | |
|---|---|---|
| Author: | Arthur Schreiber | |
| Committer: | Arthur Schreiber | |
resultcharset: convert the text of a result between character sets This is the groundwork for reading from MySQL with character_set_results = NULL and converting results in Vitess. It changes no behavior. The new package go/mysql/collations/resultcharset converts the text columns of a query result the way MySQL does for character_set_results: - ToConnection takes a result that MySQL sent with character_set_results = NULL, with every text column in the character set of the column, and converts it to a connection collation. A latin1 VARCHAR(10) with 'Müller' (4D FC 6C 6C 65 72) becomes utf8mb4 with the length 40, which is what MySQL sends for character_set_results = utf8mb4. - ToClient takes a result with all text in the connection character set and converts it to the character_set_results of a client, with the names of the fields. MySQL converts JSON as well and leaves numbers and dates alone, also for ucs2, and so does the conversion. A character that the target character set does not have becomes "?", as in MySQL. The expected values in the tests were measured on MySQL 8.4.13 and 8.0.46. Two things are added to the protocol: - The requests that return rows have raw_result_charset, with which vtgate tells a tablet that it converts the text of the result itself. It is a field of the request and not of ExecuteOptions, for the reason that reserved_conn_keep_alive is: options travel through client sessions. It reaches the tablet through the context (queryservice.ContextWithRawResultCharset), and nothing sets it yet. - The vtgate session has character_set_results and character_set_results_null. Nothing reads them yet. Signed-off-by: Arthur Schreiber <arthur@planetscale.com>
| Commit: | 9d76a47 | |
|---|---|---|
| Author: | Matthias Crauwels | |
vtctldclient: describe the --clear-source-cells default for external clusters The help, changelog and proto comments said that --clear-source-cells and --clear-target-cells reset cells to the same default as VDiff create. For Mount/Migrate workflows, update resets source cells to the external cluster's cells, which create does not do. Describe what update does: source cells reset to all cells in the source cluster (the external cluster for Mount/Migrate), and target cells to all cells in the local cluster. No behavior change. Signed-off-by: Matthias Crauwels <matthias.crauwels@planetscale.com>
| Commit: | d29cd3b | |
|---|---|---|
| Author: | Arthur Schreiber | |
resultcharset: convert the text of a result between character sets This is the groundwork for reading from MySQL with character_set_results = NULL and converting results in Vitess. It changes no behavior. The new package go/mysql/collations/resultcharset converts the text columns of a query result the way MySQL does for character_set_results: - ToConnection takes a result that MySQL sent with character_set_results = NULL, with every text column in the character set of the column, and converts it to a connection collation. A latin1 VARCHAR(10) with 'Müller' (4D FC 6C 6C 65 72) becomes utf8mb4 with the length 40, which is what MySQL sends for character_set_results = utf8mb4. - ToClient takes a result with all text in the connection character set and converts it to the character_set_results of a client, with the names of the fields. MySQL converts JSON as well and leaves numbers and dates alone, also for ucs2, and so does the conversion. A character that the target character set does not have becomes "?", as in MySQL. The expected values in the tests were measured on MySQL 8.4.13 and 8.0.46. Two things are added to the protocol: - The requests that return rows have raw_result_charset, with which vtgate tells a tablet that it converts the text of the result itself. It is a field of the request and not of ExecuteOptions, for the reason that reserved_conn_keep_alive is: options travel through client sessions. It reaches the tablet through the context (queryservice.ContextWithRawResultCharset), and nothing sets it yet. - The vtgate session has character_set_results and character_set_results_null. Nothing reads them yet. Signed-off-by: Arthur Schreiber <arthur@planetscale.com>
| Commit: | cd85ec9 | |
|---|---|---|
| Author: | Matthias Crauwels | |
Merge remote-tracking branch 'upstream/main' into vdiff-update-tablet-types Signed-off-by: Matthias Crauwels <matthias.crauwels@planetscale.com> # Conflicts: # go/vt/proto/vtctldata/vtctldata.pb.go # go/vt/proto/vtctlservice/vtctlservice.pb.go
| Commit: | 4669c28 | |
|---|---|---|
| Author: | Matthias Crauwels | |
vtctldclient: validate tablet types and cells in VDiff update Reject tablet types other than PRIMARY, REPLICA, and RDONLY, and cells that are neither a cell nor a cell alias in the topology, before anything is stored. Otherwise a resumed VDiff could stop the workflow's target streams and then wait for a tablet that can never be picked. VDiff create is unchanged. Signed-off-by: Matthias Crauwels <matthias.crauwels@planetscale.com>
| Commit: | 8d3e417 | |
|---|---|---|
| Author: | Matthias Crauwels | |
vtctldclient: drop VDiffUpdateResponse.ShardResult.updated The field was always true: the tablet always reported 1 and any shard error fails the whole request. Remove it before it ships, along with the "unchanged" output, and document that rerunning update after stopping the remaining shards is safe when a shard rejects it. Signed-off-by: Matthias Crauwels <matthias.crauwels@planetscale.com>
| Commit: | a1ce42f | |
|---|---|---|
| Author: | Matthias Crauwels | |
vdiff: only allow update on stopped or completed VDiffs VDiff resume only accepts stopped or completed VDiffs, so an update of an errored VDiff that is not retried could never take effect. Reject update in the error state with an error that says so, and update the help text, proto comments, and release notes. Signed-off-by: Matthias Crauwels <matthias.crauwels@planetscale.com>
| Commit: | 804c514 | |
|---|---|---|
| Author: | Claude | |
Name only the registered group replication policy in MigrateReplicationMode's docs The MigrateReplicationModeRequest comment and a vtctldclient example still named group_replication, a policy that was removed before any release; a request with it fails as an unknown durability policy. They name group_replication_cross_cell now. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | c9b9214 | |
|---|---|---|
| Author: | Claude | |
Merge main into group-replication-prototype Resolve the conflicts with main's semi-sync durability changes (#21409), the backup restore changes (#21848) and the move to constant log messages: - changeTypeLocked threads main's topo context and DRAINED handling through the group replication serving decision. - setReplicationSourceLocked checks group primaryship before main's new demotion of a primary that is repointed. - The primary's semi-sync after an offline backup follows the shard's durability policy, like the replica's restore. - The PR's log calls flagged by logcheck use constant messages with slog attributes; main's new mysqlctl tests read the default channel's replica status. - The protos and the vtadmin bindings are regenerated. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 4b5bf94 | |
|---|---|---|
| Author: | siddharth16396 | |
| Committer: | GitHub | |
[QueryThrottler]Implement Tablet Throttler Strategy (#19919) Signed-off-by: siddharth16396 <siddharth16396@gmail.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The documentation is generated from this commit.
| Commit: | 3ad0821 | |
|---|---|---|
| Author: | Matthias Crauwels | |
vtctldclient: add VDiff update to change tablet types on an existing VDiff Signed-off-by: Matthias Crauwels <matthias.crauwels@planetscale.com>
| Commit: | a3e0207 | |
|---|---|---|
| Author: | Claude | |
Merge main into group-replication-prototype Resolves two conflicts: - go/test/endtoend/cluster/vttablet_process.go: main fixed the same exit-channel race of the process wrapper; its version is kept. - go/vt/proto/vtctldata/vtctldata.pb.go: regenerated from the merged protos. The vtadmin bindings regenerate identically. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 7331e02 | |
|---|---|---|
| Author: | Juozas Vainauskas | |
| Committer: | GitHub | |
`BackupShard`: choose the tablet type to back up from (#21302) Signed-off-by: JuozasVainauskas <vainauskasjuozas@gmail.com>
| Commit: | 8df9c24 | |
|---|---|---|
| Author: | Claude | |
Merge main into group-replication-prototype Resolves the conflict in changelog/25.0/25.0.0/summary.md by keeping both new sections, Group Replication and the binlog backups. The Go and vtadmin proto bindings, which merged textually, regenerate identically. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 7988b03 | |
|---|---|---|
| Author: | Matt Lord | |
| Committer: | GitHub | |
backup: optionally include binary logs in full builtin backups (#21616) Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | 954855b | |
|---|---|---|
| Author: | vitess-bot[bot] | |
| Committer: | GitHub | |
[release-23.0] Refer to flags by their dashed names in help and error text (#21339) (#21355) Signed-off-by: Arthur Schreiber <arthur@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Arthur Schreiber <arthur@planetscale.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
| Commit: | 3cdb1be | |
|---|---|---|
| Author: | vitess-bot[bot] | |
| Committer: | GitHub | |
[release-24.0] Refer to flags by their dashed names in help and error text (#21339) (#21356) Signed-off-by: Arthur Schreiber <arthur@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Arthur Schreiber <arthur@planetscale.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
| Commit: | 73715c0 | |
|---|---|---|
| Author: | Claude | |
Add the Group Replication fields and RPCs to the protos - topodata.Shard: the group's voters, its incarnation, a bootstrap intent, the shard's own durability policy (set while a keyspace is migrated shard by shard) and the voters' identities (address and server_uuid). All fields are additive; older components preserve them when they rewrite the record. - replicationdata: the Group Replication status of a member, reported in FullStatus, with whether the tablet runs Group Replication and which durability policy it applies. - tabletmanager: StartGroupReplication and StopGroupReplication. - vtctld: MigrateReplicationMode, and the forced new group option of EmergencyReparentShard. The vtadmin proto bindings are regenerated. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 88d9346 | |
|---|---|---|
| Author: | Matt Lord | |
vtctldclient: add --include-binlogs to Backup and BackupShard Add include_binlogs to tabletmanagerdata.BackupRequest (8) and to the vtctldata BackupRequest and BackupShardRequest (9). vtctldclient exposes it as --include-binlogs, vtctld passes it through to the tablet, and the tablet maps it to mysqlctl.BackupParams.IncludeBinlogs. The tablet rejects non-builtin and incremental backups, so vtctld does no check of its own. The fake TabletManagerClient records the Backup requests it receives so the vtctld tests can assert the field arrives. Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | 10bc300 | |
|---|---|---|
| Author: | Matthias Crauwels | |
| Committer: | GitHub | |
VStream: add projected after_data_columns bitmap and map partial images onto target expressions (#21157) Signed-off-by: Matthias Crauwels <matthias.crauwels@planetscale.com>
| Commit: | 853fea9 | |
|---|---|---|
| Author: | Claude | |
Persist the identity of group replication voters in the shard record VTOrc kept what it knew about a voter whose tablet record was deleted (its address, server_uuid and last-seen time) only in its backend, which is rebuilt at startup: a VTOrc that restarted after the deletion never removed the voter, and the shard failed closed until an operator acted. Each voter now publishes its identity in the shard record (new field Shard.group_replication_voter_identities): its address and its MySQL's server_uuid. Its vttablet's sync loop checks it every 10s and writes it, without the shard lock, when it is missing or outdated, and drops the entries of tablets no longer listed. After a restart, VTOrc restores a listed voter whose record is gone from its published tablet record, probes it there, identifies it by its published server_uuid until it reaches its MySQL, and counts it as down once its probes failed for the grace period. Unit tests for the tablet's write, VTOrc's restore, analysis and recovery; an end-to-end test that restarts VTOrc after the deletion. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | e103c4a | |
|---|---|---|
| Author: | Claude | |
Drop the demoted primary from the voters only while its demotion holds The model found the swap of a demoted primary whose demotion did not take: it had crashed and restarted as a REPLICA before PRS demoted it, so DemotePrimary did not set groupReplicationDemoted, and its sync loop made it PRIMARY and serving again before the swap dropped it from the voters. FullStatus now reports group_replication_demoted (field 30), and PRS drops the demoted primary only while it is set and MySQL is super_read_only; otherwise it fails without changing the voters, and undoes the demotion. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | ccb868e | |
|---|---|---|
| Author: | Claude | |
Add --group-replication-force-new-group to EmergencyReparentShard Once a group replication shard's group has lost its majority, its members leave it within group_replication_unreachable_majority_timeout, and VTOrc bootstraps it again only when every voter answers, or once the operator deleted the tablet records of the voters that are gone for good. group_replication_force_members does not apply: the members have left the group before an operator can act. EmergencyReparentShard --group-replication-force-new-group starts a new group from the voters that answer, under the shard lock and on one read of the shard record and of every tablet's status. It refuses, before anything changes, while a member has quorum, a tablet is an active member or runs a START GROUP_REPLICATION, a bootstrap intent is live, no voter or every voter answers, a voter is ignored, or no surviving voter holds every transaction the others executed or received (and --required-position). It then writes the surviving voters (a compare-and-swap on the voters and the incarnation), bootstraps the new group on the chosen voter with VTOrc's bootstrap mechanism (intent, required GTID set, token, expected incarnation, then the incarnation's compare-and-swap, adoption, or the intent's withdrawal), makes the other survivors join, promotes the primary and repoints the other tablets. What only the dropped voters held is lost, and they must be down: a dropped voter that still runs, cut off from the vtctld and the topology server, keeps serving. VTOrc never forces; an older vtctld ignores the field, and its ERS fails as before while the group has no quorum. GroupMemberGTIDSets moves from VTOrc to reparentutil, so that both share it. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | fc06bc8 | |
|---|---|---|
| Author: | Claude | |
Document the mixed-version guard of Group Replication The design document describes the keyspace record of a migration (step 0 names the target policy and keeps the migration source, step 8 removes it), the invariant that the keyspace's policy names a Group Replication policy whenever a shard may run a group, what an older VTOrc, vtctld and vttablet do under it, the capability checks before a group is initialized or its primary promoted, where a policy can be set and the guard of SetKeyspaceDurabilityPolicy, and the operator rules that no component can check. The comment of FullStatus field 29 now says that the tablet also resolves the keyspace's migration source. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 53b4299 | |
|---|---|---|
| Author: | Claude | |
Resolve a shard's policy through the keyspace's migration source Adds Keyspace.migration_source_durability_policy (field 13). While it is set, it is the policy of every shard of the keyspace that has no policy of its own, and the keyspace's durability_policy can already name the target of a migration to Group Replication: a component that does not know the field then reads a group replication policy for every shard of a keyspace in which a shard may run a group, and fails safe if it does not know that policy either. topo.ShardDurabilityPolicy takes the keyspace record and resolves the shard's own policy, else the migration source, else the keyspace's policy. VTOrc stores the migration source with its copy of the keyspace record and resolves it in the detection analysis and its recoveries. The vttablet resolves it through topo.GetShardInfoDurability. The field is additive: an older reader skips it, and a record without it encodes as before. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 914b2f4 | |
|---|---|---|
| Author: | Claude | |
Merge origin/main into the Group Replication prototype Conflicts: the help text of VTOrc's flags (both sides added flags), and the generated vtctldata vtproto code, which was regenerated with the Makefile's protoc command. The regeneration also brings the vtproto code of replicationdata, tabletmanagerdata and topodata back to the form that the Makefile generates. The required position of VTOrc's ERS reads the durability policy of the shard rather than the keyspace's, since a shard can have its own policy during a migration. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | b8c4855 | |
|---|---|---|
| Author: | Claude | |
| Committer: | Claude | |
Withdraw the bootstrap intent of a definitively refused bootstrap After a loss of majority, VTOrc records a bootstrap intent for the voter it chose, and the intent fences a bootstrap on any other voter for two minutes, so that the group of a bootstrap whose reply was lost can be adopted. If the chosen voter's mysqld restarted after VTOrc read it, its relay log is gone, and its tablet correctly refuses the bootstrap for the transactions it lost (47d5e41); but the intent still named it, and the voter that holds those transactions could not be bootstrapped until the intent expired. TestGroupReplicationWithdrawsRefusedBootstrapIntent reproduces it on MySQL 8.4: the primary was bootstrapped 99.8s after the refusal, 121.2s after VTOrc's intent. The tablet now reports a refusal as definitive when it proves that the request did not, and never will, start MySQL's bootstrap: - only the refusal for a transaction of required_gtid_set that MySQL lacks, also after the RPC stopped a START that was in progress; - only if a fresh read of MySQL's status under the action lock finds it in no group, with no START GROUP_REPLICATION in performance_schema.processlist: a START that an earlier RPC issued before its client gave up can still form a group of one. Every START of the tablet runs under the action lock, and MySQL starts none on its own, so none can start afterwards. A refusal of a superseded intent, of an active member, the UNAVAILABLE of a START in progress, and every other failure stay plain errors. The signal is structured: a new optional request field, report_definitive_refusal, asks the tablet to report such a refusal in a new response field, definitive_refusal, rather than as an error (gRPC carries one or the other). The client returns it as tmclient.GroupBootstrapRefusedError, with code FAILED_PRECONDITION. A tablet that does not know the request field refuses with an error, as before, and a client that does not set it gets one. The proto change is generated with protoc for tabletmanagerdata.proto only, additively. On a definitive refusal, VTOrc withdraws its intent under the shard lock, which it re-checks first (reparentutil.WithdrawGroupReplicationBootstrapIntent): a compare-and-swap that removes the intent only while the shard record holds its token, for the incarnation it was recorded for, so that a newer intent of another VTOrc, or an incarnation recorded since, stays. VTOrc sends one RPC per intent, and an older RPC that still waits for the tablet's lock is refused by the token check. Its next pass chooses again: the e2e test bootstraps the primary 1.7s after the refusal, 24.2s after the intent (the test holds the candidate's lock for 25s). A timeout, a transport error, a canceled context or a plain refusal keep the intent, for adoption or expiry, as before. Tests, each failing without its change: the tablet marks only the qualifying refusals as definitive, and none while a START runs; VTOrc withdraws on a definitive refusal and bootstraps another voter on its next pass, and keeps the intent on a timeout, a transport error, a canceled context and a plain refusal; the compare-and-swap leaves a newer intent, an incarnation recorded since, and needs the shard lock; the tmrpctest round trip carries the refusal through gRPC. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 8a5fdb3 | |
|---|---|---|
| Author: | Claude | |
| Committer: | Claude | |
Refuse a bootstrap whose intent was superseded The TLA+ model's integrated simulation found two groups bootstrapped from the same recorded incarnation, with every fix on. VTOrcs o1 and o2 both chose voter s1, o1's shard lock having expired, and o2's intent replaced o1's. o2's RPC bootstrapped s1 and o2 recorded the group. o1's RPC still waited for s1's action lock; s1's MySQL then left its group of one, and o1's RPC bootstrapped s1 again, while o2, finding every voter out of a group, bootstrapped s2. The compare-and-swap refuses to record s1's group, which stays read-only and unserved, but s1 is out of the shard's group until it leaves it, up to the one-minute grace. VTOrc now sends the intent's token, and the incarnation the intent was recorded for, with the bootstrap RPC (StartGroupReplicationRequest fields bootstrap_intent_token and expected_incarnation, optional and additive; empty means no check). Under the action lock the tablet reads the shard record and refuses with FAILED_PRECONDITION if it no longer holds that intent, or lists another incarnation: - when the RPC gets the lock, before it changes anything: a stale request no longer stops a PRIMARY tablet from serving, nor lifts the explicit stop of its rejoins; - before each STOP of a START GROUP_REPLICATION in progress: that START may be the bootstrap of the newer intent, whose group the STOP would make MySQL leave before VTOrc adopts it; - right before MySQL's START. Each read waits at most groupReplicationTopoReadTimeout (1s), like the bootstrap's other topology reads, so the lock is never held across an unbounded read. If the topology does not answer, the tablet bootstraps without the check, as before: the check protects the availability of the shard's group, which a bootstrap that cannot run would cost. With the token check, the model found a second path to the same state: a late bootstrap reply recorded an incarnation that another VTOrc had recorded already, and that write cleared the intent the other VTOrc had recorded since, for the group's next bootstrap, while that bootstrap still ran. Nothing then fenced a bootstrap on another voter, and the running bootstrap's group could not be adopted. Recording an incarnation that is recorded already now clears only an intent for an earlier incarnation. The proto change is generated with protoc for tabletmanagerdata.proto only, additively. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 47d5e41 | |
|---|---|---|
| Author: | Claude | |
| Committer: | Claude | |
Bootstrap a group only from a binlog that holds every voter's transactions The TLA+ model in doc/design-docs/group_replication_tla found that an acknowledged write could be lost through VTOrc's choice of the voter to bootstrap after a loss of majority. VTOrc ranked the voters by their executed and received GTID sets and broke ties by the lowest alias, so a voter that held an acknowledged transaction only in the relay log of its group_replication_applier channel could win over the old primary, which had it in its binlog. A restart of that voter's mysqld before the bootstrap's START discards the relay log (relay_log_recovery=ON), and the new group lacks the write; the old primary can then never join it. Lab, raw MySQL 8.4.11 (doc/failover-audit/GroupReplication.md, "Bootstrap candidate"): a member whose relay log holds transactions that the group it joins lacks applies them when its START begins, and MySQL then refuses the join for good; a member that bootstraps applies its relay log first; a restart discards it; and START REPLICA SQL_THREAD FOR CHANNEL 'group_replication_applier' applies it, into the binlog, while Group Replication is stopped. The rule: - VTOrc still requires a candidate whose executed and received set contains every voter's: a voter left with transactions that the new group lacks could never join it. A voter that executed them all can be missing: a transaction that a group decided while its primary crashed before committing it may sit in a relay log only, never acknowledged, and requiring it in a binlog would block the bootstrap for good. - Among the candidates, VTOrc prefers the target of a live intent, then a voter that executed every transaction (the old primary), then as before. - The bootstrap RPC carries the union of every voter's executed and received sets (StartGroupReplicationRequest.required_gtid_set, a new optional field). Under the action lock, right before START, the tablet applies its relay log if MySQL has not executed all of it yet, and refuses with FAILED_PRECONDITION unless the executed set then contains it; VTOrc re-reads every voter and chooses again on its next pass. Every acknowledged transaction is in the executed set of the primary that acknowledged it, so the check is on the binlog, which no restart can lose. An empty field means no check, as for the migration's and InitPrimary's bootstraps. The tmclient and tablet manager StartGroupReplication methods take the request instead of the bootstrap flag. The proto change is generated with protoc for tabletmanagerdata.proto only; both generated files change additively. TestBootstrapGroupReplicationPrefersTransactionsInTheBinlog reproduces the model's interleaving on VTOrc and now passes. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 160509b | |
|---|---|---|
| Author: | Claude | |
| Committer: | Claude | |
Disable mysql_disable_super_read_only_if_primary: only Vitess makes MySQL writable Group Replication's member action mysql_disable_super_read_only_if_primary clears super_read_only on the member it elects primary, after every election: a bootstrap, a failover, group_replication_set_as_primary, and the election of a member alone in a group of its own. A stray group that a join formed was therefore writable until the fence check fenced it, and a bootstrapped primary before its incarnation was recorded. With the action disabled, every election leaves the new primary super_read_only (sro-eval case B, MySQL 8.4.11), and the tablet makes MySQL writable only on a decision that it may serve, under the action lock, with the serving invariant holding. - go/mysql: the commands that disable and enable the action, a read of the action and the configuration version (performance_schema replication_group_member_actions and replication_group_configuration_ version), and GroupReplicationStatus.primary_election_in_progress (field 14, THD_primary_election_primary_process in performance_schema.threads). - ConfigureGroupReplication, which runs before every START, disables the action in the member's own configuration if it is enabled there, lifting super_read_only for the change like INSTALL PLUGIN: a group that a member forms on its own uses that configuration. - The serving, writable primary of a group disables it in the group's configuration if a group bootstrapped before has it enabled, once per incarnation and every minute, under the action lock. - Group Replication sets super_read_only again when an election ends, undoing an earlier clear. PromoteReplica and InitPrimary wait until MySQL is the group primary and its election ended; every decision to serve waits for the end of the election first (changeTypeLocked, UndoDemotePrimary, serveAgain, the end of a planned pause, the end of a join or bootstrap of a PRIMARY tablet), and a primary whose election still runs does not serve. Making MySQL writable during an election is refused (UNAVAILABLE). - liftGroupReplicationFenceLocked makes MySQL writable whenever it is read-only and the decision allows, not only when the fence check fenced it, and redoes the prepared transactions of a promotion that did not serve. A promotion refused by the serving invariant leaves MySQL read-only. - A serving PRIMARY whose MySQL is read-only is made writable again by the sync loop, woken by the fence check; a primary that DemotePrimary demoted is left to its caller. - SetReadWrite is refused unless the serving invariant holds. - VTOrc's PrimaryIsReadOnly and PrimaryCurrentTypeMismatch, whose recovery is UndoDemotePrimary, apply under a group replication policy only to the legitimate group's primary. - The fence check stays for the shrink case; its stray-group branch is defense in depth now. The migration back does not reset the action: the asynchronous paths never consult it. The replicationdata.pb.go change is generated with protoc for replicationdata.proto only; the vtprotobuf change is a purely additive hand patch, identical to protoc-gen-go-vtproto's output, and a round-trip test checks it. TestGroupReplicationLifecycle checks that every member has the action disabled after the migration and after a member rejoined. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 51e90e9 | |
|---|---|---|
| Author: | Claude | |
| Committer: | Claude | |
Track a START GROUP_REPLICATION in progress, and never join a PRIMARY tablet A START GROUP_REPLICATION that finds no group reports the member OFFLINE for about a minute, and can still end with the member alone in a group of its own. MySQL refuses a STOP, and a second START, meanwhile with errno 3663 ("Another instance of START/STOP GROUP_REPLICATION command is executing"), and accepts group_replication_bootstrap_group=ON (sro-eval case A, MySQL 8.4.11). - GroupReplicationStatus gains start_in_progress (field 13), read from performance_schema.processlist, which lists the START for as long as it runs, also after its client was killed. FullStatus ORs it with the tablet's own joins and bootstraps in progress. - VTOrc stores it (gr_start_in_progress). Such a voter counts as active for GroupNotBootstrapped, is not joined again by GroupMemberNotOnline, and BootstrapGroupReplication refuses to bootstrap while any tablet reports one. ClusterHasNoPrimary and PrimaryTabletDeleted keep ignoring a shard without an active member. A bootstrap may wait up to about a minute for a stuck START; MySQL refuses to stop it before it ends anyway. - The tablet recognizes 3663 with that message as a command in progress. stopOngoingGroupStartLocked tries the STOP again every second, for at most 10s or the caller's context, and then fails with UNAVAILABLE. - mysqlctl refuses (UNAVAILABLE) to set group_replication_bootstrap_group while a START runs, which MySQL could complete as a bootstrap. - The sync loop does not start a join while MySQL runs a START. - GroupMemberNotOnline never matches a PRIMARY tablet, in the topology or in its own state: the tablet demotes itself first. As defense in depth, StartGroupReplication(bootstrap=false) on a PRIMARY tablet under a group replication policy with voters, whose MySQL is not an active member, fails with FAILED_PRECONDITION. The migration only joins replicas. - The comment on readGroupReplicationFenceStatusFormat no longer claims that member_stats blocks for the whole START: it answers at once during a blocked START, and blocks about a second only at admission or group formation. The replicationdata.pb.go change is generated with protoc for replicationdata.proto only; the vtprotobuf change is a purely additive hand patch, identical to protoc-gen-go-vtproto's output, and a round-trip test checks it. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 8b512b7 | |
|---|---|---|
| Author: | Claude | |
| Committer: | Claude | |
Resolve the durability policy per shard during a replication mode migration MigrateReplicationMode converts a keyspace one shard at a time and switches the keyspace durability policy only after the last shard. Until then, a converted shard ran its group while every component managed it by the keyspace's semi-sync policy: ERS took the asynchronous path on a group, PRS promoted tablets without the voter checks, VTOrc ran none of the Group Replication analyses and recoveries and undid the conversion (StaleTopoPrimary configured asynchronous replication on an old primary that is a voter, ReplicaSemiSyncMustBeSet turned semi-sync back on), and the tablets applied neither the serving invariant nor rejoins, and trusted a stale PRIMARY record. The migration back had the mirror problem: it switched the keyspace policy first, so every shard not converted back yet lost the Group Replication rules while it still ran its group. A shard can now have a durability policy of its own (new optional field Shard.durability_policy). A single resolver, topo.ShardDurabilityPolicy ("the shard's own policy if set, else the keyspace's"), is used by every decision that depends on the policy: PRS, ERS, InitShardPrimary and the replica semi-sync of vtctld and the wrangler; VTOrc's detection analysis (vitess_shard.durability_policy) and every recovery; and the tablet's sync loop, startup, RPCs and member weight. vtctld resolves it from the shard record it read under the shard lock, VTOrc after refreshing the records under the shard lock, and the tablet reads the shard and keyspace records together; the sync loop drops its 10s policy cache as soon as a shard record it reads sets another own policy. MigrateReplicationMode changes a shard's policy only under its shard lock, at one step of its conversion: to Group Replication as the last step, once the group runs with every voter, its incarnation recorded and semi-sync superseded; back to semi-sync as the first step, before the group shrinks. The keyspace policy is switched in both directions only once every shard is converted to the target policy, so the switch changes no shard's policy, and the shards' own policies are then removed. Re-runs continue where a run stopped, and a shard that runs its group without a policy of its own is not counted as converted. Components that do not know the field use the keyspace policy, which is wrong for a converted shard of a half-migrated keyspace: vtctld, VTOrc and every vttablet must be upgraded before a migration starts. The tablets report that they know it (new optional field FullStatus.shard_durability_policy_supported), and the migration refuses voters and members that do not. The proto changes are generated with protoc for topodata.proto and replicationdata.proto only; the vtprotobuf output is purely additive and a round-trip test checks it. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 09586e6 | |
|---|---|---|
| Author: | Claude | |
| Committer: | Claude | |
Adopt a Group Replication group whose bootstrap reply was lost If VTOrc's BootstrapGroupReplication RPC succeeded but its reply was lost (timeout, partition, VTOrc crash), the new incarnation was never recorded. The tablet trusted the group it bootstrapped for a minute, the other voters did not join an unrecorded incarnation, and nothing else could bootstrap the group meanwhile (S7d run r3 in doc/failover-audit/GroupReplication.md). Before it bootstraps, VTOrc now records a bootstrap intent in the shard record, under the shard lock: the target tablet, the time, the incarnation the shard record listed, and a token (new optional field Shard.group_replication_bootstrap_intent; older readers ignore it, and an intent that a component unaware of it superseded no longer applies). If the RPC fails, or on a later pass (new analysis GroupBootstrapNotRecorded), VTOrc adopts the target's group and records its incarnation when the group's primary is the intent's target, with quorum, its incarnation differs from the intent's previous one, and it was created after the intent. A stray group of another voter, or the old incarnation, is never adopted. The intent also fences a second bootstrap: while an intent for another tablet is younger than two minutes, no other bootstrap starts, since the first one may still be running with its member not active yet. Incarnation writes are now a compare-and-swap against the incarnation the caller read before it changed the group, and clear the intent. The Shard change is generated with protoc for topodata.proto only; the vtprotobuf output is purely additive and a round-trip test checks it. Signed-off-by: Claude <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | c88aff6 | |
|---|---|---|
| Author: | vitess-bot[bot] | |
| Committer: | vitess-bot[bot] | |
Cherry-pick a240ed6c4bf6aae24b6f4a0b4c14bf2f5dd85aa5 with conflicts
| Commit: | a38864d | |
|---|---|---|
| Author: | vitess-bot[bot] | |
| Committer: | vitess-bot[bot] | |
Cherry-pick a240ed6c4bf6aae24b6f4a0b4c14bf2f5dd85aa5 with conflicts
| Commit: | a240ed6 | |
|---|---|---|
| Author: | Arthur Schreiber | |
| Committer: | GitHub | |
Refer to flags by their dashed names in help and error text (#21339) Signed-off-by: Arthur Schreiber <arthur@planetscale.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
| Commit: | 4689ec4 | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | GitHub | |
Add `--required-position` to `EmergencyReparentShard` (#21143) Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | 2b00fc5 | |
|---|---|---|
| Author: | Claude | |
Run Group Replication on the MySQL communication stack Groups now always use group_replication_communication_stack=MYSQL. Members connect to each other through the MySQL port of their tablet records, authenticated as the replication user, instead of XCom's separate port, allowlist and TLS settings. MySQL deprecates XCom in 9.7.2 and makes MYSQL the default in 26.7. - --enable-group-replication replaces --group-replication-port; the tablet record no longer carries a "gr" port. FullStatus reports the flag (group_replication_enabled), which the migration preflight checks. - The local address and seeds are the tablets' MySQL host:port. - The tablet stores the replication user's credentials on the group_replication_recovery channel, which the MySQL stack uses for the group's connections; START GROUP_REPLICATION no longer passes them. - The replication user needs GROUP_REPLICATION_STREAM and CONNECTION_ADMIN. init_db.sql grants them, and the tablet checks mysql.global_grants before a join and refuses with FAILED_PRECONDITION naming the GRANT to run. The GR e2e tests and the chaos scenarios S1, S2, S3, S9i and G3E pass on the MySQL stack with failover times matching the XCom runs and no acked writes lost. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89 Signed-off-by: Claude <noreply@anthropic.com>
| Commit: | a0acd14 | |
|---|---|---|
| Author: | Claude | |
Stop reading the group communication information, and bound status reads After SIGSTOP/SIGCONT, the expelled old primary wedged in ERROR in 3 of 3 S2 runs (NEW-5 in doc/failover-audit/GroupReplication.md): the status query on performance_schema.replication_group_communication_information, issued only to fill paxos_single_leader, waited forever in the group communication engine while holding the lock that the member's own rejoin needed. The query ran without a context, so a forced DemotePrimary from VTOrc hung inside it holding the tablet's action lock. - FullStatus.GroupReplicationStatus.paxos_single_leader now comes from @@global.group_replication_paxos_single_leader in the existing variables query. Vitess sets it on every member before it starts Group Replication, and MySQL refuses a joiner whose setting differs. - mysqlctl runs the status queries with the caller's context (the connection is killed and closed when it expires), in GroupReplicationStatus, ConfigureGroupReplication and FullStatus. - The tablet bounds every status read by 10s, whatever its caller's context, so the sync loop and DemotePrimary cannot hold the action lock on it indefinitely. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | d410495 | |
|---|---|---|
| Author: | Claude | |
Follow only the shard's legitimate replication group A member that left its group and was made to join again while the other members were leaving has been seen to end up alone in a new group incarnation, ONLINE and PRIMARY with quorum in its view of one (S7d in doc/failover-audit/GroupReplication.md, NEW-1). The tablet promoted itself and the transactions acknowledged by the other members were lost. Record the incarnation of the shard's group (the view id prefix) in the new Shard.group_replication_incarnation when Vitess bootstraps a group: MigrateReplicationMode, PlannedReparentShard's initial promotion, InitShardPrimary and VTOrc's BootstrapGroupReplication. The reverse migration clears it. policy.LegitimateGroup defines the primary Vitess may follow: the ONLINE primary of a group with quorum, in the recorded incarnation, whose view holds a majority of the listed voters (matched by server_uuid or MySQL address). The tablet sync loop, VTOrc (GroupPrimaryNotInTopo, PromoteGroupPrimary, voter selection, GroupMemberNotOnline), ERS and the PRS preflight use it. A member active in another incarnation makes its tablet leave that group and suspend its own rejoins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | eaa70b0 | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
vtctldclient: say that any candidate may hold the required position Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | cd70190 | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
proto: match the required position comment to the option doc Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | 4d63ad3 | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
vtctldclient: say that a wrong shard type fails after the primary is demoted Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | 43902d5 | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
vtctldclient: drop the MySQL56 prefix from the required position help The prefix is an internal Vitess detail, and MySQL prints a GTID set without it. The flag help and the proto comment now describe the value as the minimum MySQL GTID set. Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | e812828 | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
Add `--required-position` to `EmergencyReparentShard` `EmergencyReparentOptions.RequiredPosition` exists in reparentutil but nothing outside the process can set it. This adds `required_position` to `EmergencyReparentShardRequest` and a `--required-position` flag on the vtctldclient command. vtctld decodes the string before it takes the shard lock and returns `INVALID_ARGUMENT` on bad input. A bare GTID set without the `MySQL56/` prefix is accepted, since that is how MySQL prints it and the check supports MySQL56 only. Part of #21109. Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | b1cb5a4 | |
|---|---|---|
| Author: | Claude | |
Record Group Replication voters in the shard record and select them per cell Add Shard.group_replication_voters, the tablets that are the voting members of a shard's group, and SelectVoters, which picks them for a durability policy while changing the current voters as little as possible. The cross-cell policy allows one voter per cell. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 7413de0 | |
|---|---|---|
| Author: | Claude | |
Always apply the expel timeout and paxos single leader settings Vitess's Group Replication design depends on both: an expel timeout of 0 is what makes failovers take about 7 seconds instead of 22, and the paxos single leader setting must be identical on every member, since MySQL refuses a joiner whose setting differs from its group's. Neither is useful as a per-tablet flag, so apply them unconditionally and drop the flags and the logic that made joiners adopt their group's setting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | f264bf7 | |
|---|---|---|
| Author: | Claude | |
Default to the Group Replication settings of the consensus RFC Add --group-replication-member-expel-timeout (default 0) and --group-replication-paxos-single-leader (default ON), and lower the default --group-replication-unreachable-majority-timeout to 1s, matching the settings proposed in #18648. On MySQL 8.4.11 any expel timeout from 1 to 10 seconds delays the expulsion of a failed member by 16 seconds, so the group replaced a failed primary after about 22 seconds. With 0 the end-to-end test sees the new primary in the topology 6.6 seconds after the old primary's mysqld is killed. MySQL refuses a joining member whose paxos single leader setting differs from its group's, so the flag only applies when a tablet bootstraps a group. A joining tablet adopts the setting of the group, which FullStatus now reports. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 464f89a | |
|---|---|---|
| Author: | Claude | |
| Committer: | Claude | |
vtgate: name result columns in the session's charsets, as MySQL does MySQL reads select expressions in character_set_client and text literals in character_set_connection when it names result columns, and converts the names to character_set_results when it sends them. vtgate ignored these charsets, so clients that do not use utf8mb4 got different names, and it rejected SET character_set_results = binary as a syntax error. The session now keeps the client, connection and results charsets. The handshake, SET NAMES, SET CHARACTER SET, and SET of character_set_client, character_set_connection, character_set_results and collation_connection set them; they are still not set on the tablets. vtgate names result columns with the client and connection charsets, which are part of the plan cache key, since plans carry the names, and the MySQL protocol server converts result metadata to the results charset, copying fields only when a name changes. The parser accepts binary as the value of a system variable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Qgj8xZGox7XHv97eFvEJcP
| Commit: | b8cdf20 | |
|---|---|---|
| Author: | Claude | |
Add MigrateReplicationMode to convert shards to and from group replication A new vtctld RPC and vtctldclient command converts the shards of a keyspace between asynchronous (semi-sync) replication and MySQL Group Replication, online, with the logic in reparentutil.ReplicationModeMigrator so that VTOrc can reuse it. To group replication, each shard is converted under its shard lock: a preflight (reachability, GTID mode, ROW binlogs, MySQL 8.0.27+, 3-9 voting members with a gr port, cross-cell placement, primary keys and InnoDB), bootstrap on the primary, joins one replica at a time with cross-cell replicas first, a wait for the primary to disable semi-sync before its last semi-sync acker joins, and async non-voting replicas. The keyspace policy switches once every shard runs group replication. Back to asynchronous replication, the keyspace policy switches first, the secondaries leave the group and replicate from the primary, the migrator waits for the primary to re-enable semi-sync, and the primary leaves last. Every step inspects the current state first, so a failed migration can be run again; --dry-run returns the plan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | a04f45c | |
|---|---|---|
| Author: | Claude | |
Group Replication foundation: replication mode, MySQL primitives, FullStatus and RPC surface Add the building blocks for MySQL Group Replication as a first-class replication mode: - group_replication and group_replication_cross_cell durability policies, with an optional ReplicationModer interface so out-of-tree Durablers keep compiling. - GroupReplicationStatus in FullStatus, and MySQL/mysqlctl primitives to configure, start, stop and switch the primary of a group. - StartGroupReplication/StopGroupReplication tablet manager RPCs (tablet side stubbed). - Read only the default replication channel in the MySQL flavors, so the channels Group Replication creates do not break status parsing. - Design document. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HSaG275VLNRg3UfsJ3bY89
| Commit: | 1c31be4 | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
proto: match the required position comment to the option doc Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | 2bd899b | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
vtctldclient: drop the MySQL56 prefix from the required position help The prefix is an internal Vitess detail, and MySQL prints a GTID set without it. The flag help and the proto comment now describe the value as the minimum MySQL GTID set. Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | 7f035bf | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
Add `--required-position` to `EmergencyReparentShard` `EmergencyReparentOptions.RequiredPosition` exists in reparentutil but nothing outside the process can set it. This adds `required_position` to `EmergencyReparentShardRequest` and a `--required-position` flag on the vtctldclient command. vtctld decodes the string before it takes the shard lock and returns `INVALID_ARGUMENT` on bad input. A bare GTID set without the `MySQL56/` prefix is accepted, since that is how MySQL prints it and the check supports MySQL56 only. Part of #21109. Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | 15ec82a | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
proto: match the required position comment to the option doc Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | a9b5b9e | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
vtctldclient: drop the MySQL56 prefix from the required position help The prefix is an internal Vitess detail, and MySQL prints a GTID set without it. The flag help and the proto comment now describe the value as the minimum MySQL GTID set. Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | 15d5354 | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
Add `--required-position` to `EmergencyReparentShard` `EmergencyReparentOptions.RequiredPosition` exists in reparentutil but nothing outside the process can set it. This adds `required_position` to `EmergencyReparentShardRequest` and a `--required-position` flag on the vtctldclient command. vtctld decodes the string before it takes the shard lock and returns `INVALID_ARGUMENT` on bad input. A bare GTID set without the `MySQL56/` prefix is accepted, since that is how MySQL prints it and the check supports MySQL56 only. Part of #21109. Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | 269758e | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
Add `--required-position` to `EmergencyReparentShard` `EmergencyReparentOptions.RequiredPosition` exists in reparentutil but nothing outside the process can set it. This adds `required_position` to `EmergencyReparentShardRequest` and a `--required-position` flag on the vtctldclient command. vtctld decodes the string before it takes the shard lock and returns `INVALID_ARGUMENT` on bad input. A bare GTID set without the `MySQL56/` prefix is accepted, since that is how MySQL prints it and the check supports MySQL56 only. Part of #21109. Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | ed5c7f1 | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
Add `--required-position` to `EmergencyReparentShard` `EmergencyReparentOptions.RequiredPosition` exists in reparentutil but nothing outside the process can set it. This adds `required_position` to `EmergencyReparentShardRequest` and a `--required-position` flag on the vtctldclient command. vtctld decodes the string before it takes the shard lock and returns `INVALID_ARGUMENT` on bad input. A bare GTID set without the `MySQL56/` prefix is accepted, since that is how MySQL prints it and the check supports MySQL56 only. Part of #21109. Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | 0103300 | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
Add `--required-position` to `EmergencyReparentShard` `EmergencyReparentOptions.RequiredPosition` exists in reparentutil but nothing outside the process can set it. This adds `required_position` to `EmergencyReparentShardRequest` and a `--required-position` flag on the vtctldclient command. vtctld decodes the string with `replication.DecodePosition` before it takes the shard lock and returns `INVALID_ARGUMENT` on bad input. An e2e test covers the wiring from the CLI to ERS. Part of #21109. Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | 4c66b8c | |
|---|---|---|
| Author: | Mohamed Hamza | |
| Committer: | Mohamed Hamza | |
Add `--required-position` to `EmergencyReparentShard` `EmergencyReparentOptions.RequiredPosition` exists in reparentutil but nothing outside the process can set it. This adds `required_position` to `EmergencyReparentShardRequest` and a `--required-position` flag on the vtctldclient command. vtctld decodes the string with `replication.DecodePosition` before it takes the shard lock and returns `INVALID_ARGUMENT` on bad input. An e2e test covers the wiring from the CLI to ERS. Part of #21109. Signed-off-by: Mohamed Hamza <mhamza@fastmail.com>
| Commit: | e7ea479 | |
|---|---|---|
| Author: | Matthias Crauwels | |
| Committer: | GitHub | |
VStream: add before_data_columns bitmap for partial before images (#21067) Signed-off-by: Matthias Crauwels <matthias.crauwels@planetscale.com>
| Commit: | 2265cac | |
|---|---|---|
| Author: | Matt Lord | |
| Committer: | GitHub | |
VTGate: Keep idle connections holding temp tables alive with a heartbeat (#20429) Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | 01a25a7 | |
|---|---|---|
| Author: | Pedro Albuquerque | |
| Committer: | GitHub | |
VDiff: add --no-samples option to vdiff show (#20870) Signed-off-by: Pedro Albuquerque <pedro.albuquerque@slack-corp.com>
| Commit: | f625058 | |
|---|---|---|
| Author: | Matt Lord | |
Merge remote-tracking branch 'oss/main' into trx_temp_tables
| Commit: | 25b3457 | |
|---|---|---|
| Author: | Eduardo J. Ortega U. | |
| Committer: | GitHub | |
Make PRS & ERS new primary selection MySQL version aware (#20211) Signed-off-by: Eduardo Ortega <5791035+ejortegau@users.noreply.github.com> Signed-off-by: Eduardo J. Ortega U. <5791035+ejortegau@users.noreply.github.com> Co-authored-by: Claude <svc-devxp-claude@slack-corp.com>
| Commit: | 400013a | |
|---|---|---|
| Author: | Matt Lord | |
Merge remote-tracking branch 'oss/main' into trx_temp_tables
| Commit: | 18dc5a3 | |
|---|---|---|
| Author: | Tim Vaillancourt | |
| Committer: | GitHub | |
`EmergencyReparentShard`: add explicit split-brain recovery (#20780) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
| Commit: | 4425351 | |
|---|---|---|
| Author: | Max Englander | |
| Committer: | GitHub | |
ApplySchema: add session variable options (#20654) Signed-off-by: Max Englander <max@planetscale.com> Signed-off-by: Max Englander <max.englander@gmail.com> Co-authored-by: GPT-5.6 Sol <noreply@openai.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| Commit: | e5901c7 | |
|---|---|---|
| Author: | Matt Lord | |
Address more of Graham's comments Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | fbd0287 | |
|---|---|---|
| Author: | Matt Lord | |
Address Graham's comments Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | 4dcb41d | |
|---|---|---|
| Author: | Matt Lord | |
| Committer: | Matt Lord | |
More review comment fixes Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | 05eae69 | |
|---|---|---|
| Author: | Matt Lord | |
Address more review comments Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | 659ae9f | |
|---|---|---|
| Author: | Matt Lord | |
Don't reset mysqld's wait_timeout on keep-alives Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | 43fb69f | |
|---|---|---|
| Author: | Matt Lord | |
| Committer: | GitHub | |
Use quorum assessment of primary down and ERS (#20253) Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | 00fb91f | |
|---|---|---|
| Author: | Matt Lord | |
| Committer: | Matt Lord | |
Address review comment Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | c70e990 | |
|---|---|---|
| Author: | Matt Lord | |
Address fable review comments Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | 461b17c | |
|---|---|---|
| Author: | Matt Lord | |
Use quorum assessment of primary down and ERS Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | 6784781 | |
|---|---|---|
| Author: | Arthur Schreiber | |
Honor FetchLastInsertId on the raw streaming path MySQL hardcodes last_insert_id=0 in a streamed SELECT terminator, so the value a LAST_INSERT_ID(x) call set never rides the raw wire bytes. Mirror the non-raw path: reset the connection's last insert id before the query and refetch it with "select last_insert_id()" afterwards, returning it out-of-band so vtgate can surface it as an extra Result. Adds insert_id / insert_id_changed to the 4 *Raw response messages and a StreamExecuteRawState carrying the value out of StreamExecuteRaw; folds InsertID/InsertIDChanged into the Transaction/Reserved state structs. The explicit changed bit distinguishes a genuine LAST_INSERT_ID(0) from unset. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Arthur Schreiber <arthur@planetscale.com>
| Commit: | 2fd8492 | |
|---|---|---|
| Author: | Arthur Schreiber | |
StreamExecuteRaw: bidi *Raw streaming RPCs (no stream pool) Add four bidirectional streaming RPCs — StreamExecuteRaw, BeginStreamExecuteRaw, ReserveStreamExecuteRaw and ReserveBeginStreamExecuteRaw — that forward raw MySQL wire-protocol bytes from vttablet to vtgate, where a RawResultParser decodes them back into sqltypes.Result. This avoids the parse->serialize->deserialize cycle on the vttablet side. The gRPC client opens a fresh bidi stream per query: it sends the request, closes its send side, drains responses until the terminal done=true message, then closes. Stream pooling is intentionally left out of this change and will be added in a follow-up; doing so requires no wire-protocol or server changes. vtgate routing through the raw path is gated behind a new --experimental-raw-streaming flag, which defaults to on for now so the path gets exercised. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Arthur Schreiber <arthur@planetscale.com>
| Commit: | 56d5b99 | |
|---|---|---|
| Author: | Arthur Schreiber | |
Merge remote-tracking branch 'upstream/main' into arthur/stream-execute-raw Signed-off-by: Arthur Schreiber <arthur@planetscale.com> # Conflicts: # go/mysql/conn.go # go/vt/proto/queryservice/queryservice.pb.go # go/vt/proto/queryservice/queryservice_grpc.pb.go # go/vt/vttablet/tabletserver/tabletserver.go # proto/queryservice.proto
| Commit: | e2e08e8 | |
|---|---|---|
| Author: | Tim Vaillancourt | |
| Committer: | GitHub | |
`vtgate`: add controls for cross-keyspace reads (#19668) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| Commit: | be89d2c | |
|---|---|---|
| Author: | Thomas Thornton | |
| Committer: | GitHub | |
Add VStream MaxStreamAge (#19800) (#850) Cherry-picked from commit 3da1db263a6124344f3448500c58ae9337dc7178 This adds a new max_stream_age_seconds field to VStreamFlags that allows configuring a maximum duration for VStreams. When set, the server will terminate the stream with UNAVAILABLE after the configured duration (plus a random jitter of +/-10% to spread out reconnections). The implementation is best-effort: if a send to the client is in-flight when the age is reached, the server waits for it to complete before returning. Adapted for slack-22.0 by: - Using slack-22.0's wg.Add(1) + go func() pattern instead of wg.Go() - Proto files properly regenerated with make proto after fixing npm environment Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com>
| Commit: | d17e10b | |
|---|---|---|
| Author: | Matt Lord | |
Merge remote-tracking branch 'origin/main' into vtorc_ers_primary_tablet_down
| Commit: | 638053b | |
|---|---|---|
| Author: | Matt Lord | |
| Committer: | GitHub | |
VReplication: Add default data protection for cancel/complete on _reverse workflows (#19906) Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | c08a661 | |
|---|---|---|
| Author: | Matt Lord | |
| Committer: | Matt Lord | |
Changes from self review Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | bbc0060 | |
|---|---|---|
| Author: | Matt Lord | |
| Committer: | Matt Lord | |
Address Codex review feedback Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | 2a3c965 | |
|---|---|---|
| Author: | vitess-bot[bot] | |
| Committer: | GitHub | |
[release-23.0] proto: fix incorrect flag bits on `RAW` and `ROW_TUPLE` types (#19920) (#19949) Signed-off-by: Arthur Schreiber <arthur@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| Commit: | 39af295 | |
|---|---|---|
| Author: | vitess-bot[bot] | |
| Committer: | GitHub | |
[release-24.0] proto: fix incorrect flag bits on `RAW` and `ROW_TUPLE` types (#19920) (#19950) Signed-off-by: Arthur Schreiber <arthur@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| Commit: | 0c4cf11 | |
|---|---|---|
| Author: | Arthur Schreiber | |
| Committer: | GitHub | |
proto: fix incorrect flag bits on `RAW` and `ROW_TUPLE` types (#19920) Signed-off-by: Arthur Schreiber <arthur@planetscale.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| Commit: | 3da1db2 | |
|---|---|---|
| Author: | Thomas Thornton | |
| Committer: | GitHub | |
Add VStream MaxStreamAge (#19800) Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com>
| Commit: | c53e7dd | |
|---|---|---|
| Author: | Matt Lord | |
Merge remote-tracking branch 'origin/main' into vtorc_ers_primary_tablet_down
| Commit: | b7f8862 | |
|---|---|---|
| Author: | Matt Lord | |
Address zero value update issue for phi threshold Signed-off-by: Matt Lord <mattalord@gmail.com>
| Commit: | bc7bb3c | |
|---|---|---|
| Author: | Arthur Schreiber | |
| Committer: | GitHub | |
vtgate: expose `BinlogDumpGTID` via gRPC (#19689) Signed-off-by: Arthur Schreiber <arthur@planetscale.com> Signed-off-by: Arthur Schreiber <schreiber.arthur@googlemail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
| Commit: | 53eb081 | |
|---|---|---|
| Author: | Matt Lord | |
Merge remote-tracking branch 'origin/main' into vtorc_ers_primary_tablet_down
| Commit: | 9a3646f | |
|---|---|---|
| Author: | Max Englander | |
| Committer: | GitHub | |
movetables mirrortraffic: let read-only queries bypass denied tables (#18775) Signed-off-by: Max Englander <max@planetscale.com> Signed-off-by: Max Englander <max.englander@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
| Commit: | 7b2ad0d | |
|---|---|---|
| Author: | Arthur Schreiber | |
| Committer: | GitHub | |
Update proto/vtgate.proto Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Arthur Schreiber <schreiber.arthur@googlemail.com>
| Commit: | 9ef2f86 | |
|---|---|---|
| Author: | Arthur Schreiber | |
vtgate: fix BinlogDumpGTID validation for filename, position, and zero-value alias - Only require tablet_alias when binlog_filename is set (not position alone) - Treat zero-value TabletAlias as unset using TabletAliasIsZero - Update proto comments to document position >= 4 requirement - Add unit tests for the gRPC path Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Arthur Schreiber <arthur@planetscale.com>