package binlogdata

Get desktop application:
View/edit binary Protocol Buffers messages

BinlogSource specifies the source and filter parameters for Filtered Replication. KeyRange and Tables are legacy. Filter is the new way to specify the filtering rules.

• string keyspace = 1

  the source keyspace

• string shard = 2

  the source shard

• topodata.TabletType tablet_type = 3

  the source tablet type

• optional topodata.KeyRange key_range = 4

  KeyRange is set if the request is for a keyrange

• repeated string tables = 5

  Tables is set if the request is for a list of tables

• optional Filter filter = 6

  Filter is set if we're using the generalized representation for the filter.

• OnDDLAction on_ddl = 7

  OnDdl specifies the action to be taken when a DDL is encountered.

• string external_mysql = 8

  Source is an external mysql. This attribute should be set to the username to use in the connection

• bool stop_after_copy = 9

  StopAfterCopy specifies if vreplication should be stopped after copying is done.

BinlogTransaction describes a transaction inside the binlogs. It is streamed by vttablet for filtered replication, used during resharding.

Used in: StreamKeyRangeResponse, StreamTablesResponse

• repeated BinlogTransaction.Statement statements = 1

  the statements in this transaction

• optional query.EventToken event_token = 4

  The Event Token for this event.

Used in: BinlogTransaction

• Statement.Category category = 1

  what type of statement is this?

• optional Charset charset = 2

  charset of this statement, if different from pre-negotiated default.

• bytes sql = 3

  the sql

Used in: Statement

• BL_UNRECOGNIZED = 0

• BL_BEGIN = 1

• BL_COMMIT = 2

• BL_ROLLBACK = 3

• BL_DML_DEPRECATED = 4

  BL_DML is deprecated.

• BL_DDL = 5

• BL_SET = 6

• BL_INSERT = 7

• BL_UPDATE = 8

• BL_DELETE = 9

Charset is the per-statement charset info from a QUERY_EVENT binlog entry.

Used in: BinlogTransaction.Statement, StreamKeyRangeRequest, StreamTablesRequest

• int32 client = 1

  @@session.character_set_client

• int32 conn = 2

  @@session.collation_connection

• int32 server = 3

  @@session.collation_server

FieldEvent represents the field info for a table.

Used in: VEvent

• string table_name = 1

• repeated query.Field fields = 2

Filter represents a list of ordered rules. The first match wins.

Used in: BinlogSource, VStreamRequest, vtgate.VStreamRequest

• repeated Rule rules = 1

• Filter.FieldEventMode fieldEventMode = 2

  FieldEventMode specifies the behavior if there is a mismatch between the current schema and the fields in the binlog. This can happen if the binlog position is before a DDL that would cause the fields to change. If vstreamer detects such an inconsistency, the behavior depends on the FieldEventMode. If the value is ERR_ON_MISMATCH (default), then it errors out. If it's BEST_EFFORT, it sends a field event with fake column names as "@1", "@2", etc.

Used in: Filter

• ERR_ON_MISMATCH = 0

• BEST_EFFORT = 1

Journal contains the metadata for a journal event. The commit of a journal event indicates the point of no return for a migration.

Used in: VEvent

• int64 id = 1

  Id represents a unique journal id.

• MigrationType migration_type = 2

• repeated string tables = 3

  Tables is set if the journal represents a TABLES migration.

• string local_position = 4

  LocalPosition is the source position at which the migration happened.

• repeated ShardGtid shard_gtids = 5

  ShardGtids is the list of targets to which the migration took place.

• repeated KeyspaceShard participants = 6

  Participants is the list of source participants for a migration. Every participant is expected to have an identical journal entry. While streaming, the client must wait for the journal entry to be received from all pariticipants, and then replace them with new streams specified by ShardGtid. If a stream does not have all participants, a consistent migration is not possible.

• repeated string source_workflows = 7

  SourceWorkflows is the list of workflows in the source shard that were migrated to the target. If a migration fails after a Journal is committed, this information is used to start the target streams that were created prior to the creation of the journal.

KeyspaceShard represents a keyspace and shard.

Used in: Journal

• string keyspace = 1

• string shard = 2

Used in: VEvent

• optional TableLastPK table_last_p_k = 1

• bool completed = 2

MigrationType specifies the type of migration for the Journal.

Used in: Journal

• TABLES = 0

• SHARDS = 1

• repeated MinimalTable tables = 1

Used in: MinimalSchema

• string name = 1

• repeated query.Field fields = 2

• repeated int64 p_k_columns = 3

OnDDLAction lists the possible actions for DDLs.

Used in: BinlogSource

• IGNORE = 0

• STOP = 1

• EXEC = 2

• EXEC_IGNORE = 3

RowChange represents one row change. If Before is set and not After, it's a delete. If After is set and not Before, it's an insert. If both are set, it's an update.

Used in: RowEvent

• optional query.Row before = 1

• optional query.Row after = 2

RowEvent represent row events for one table.

Used in: VEvent

• string table_name = 1

• repeated RowChange row_changes = 2

Rule represents one rule in a Filter.

Used in: Filter

• string match = 1

  Match can be a table name or a regular expression. If it starts with a '/', it's a regular expression. For example, "t" matches a table named "t", whereas "/t.*" matches all tables that begin with 't'.

• string filter = 2

  Filter: If empty, all columns and rows of the matching tables are sent. If it's a keyrange like "-80", only rows that match the keyrange are sent. If Match is a table name instead of a regular expression, the Filter can also be a select expression like this: "select * from t", same as an empty Filter, or "select * from t where in_keyrange('-80')", same as "-80", or "select col1, col2 from t where in_keyrange(col1, 'hash', '-80'), or What is allowed in a select expression depends on whether it's a vstreamer or vreplication request. For more details, please refer to the specific package documentation. On the vreplication side, Filter can also accept a special "exclude" value, which will cause the matched tables to be excluded. TODO(sougou): support this on vstreamer side also.

ShardGtid contains the GTID position for one shard. It's used in a request for requesting a starting position. It's used in a response to transmit the current position of a shard. It's also used in a Journal to indicate the list of targets and shard positions to migrate to.

Used in: Journal, VGtid

• string keyspace = 1

• string shard = 2

• string gtid = 3

• repeated TableLastPK table_p_ks = 4

StreamKeyRangeRequest is the payload to StreamKeyRange

Used as request type in: binlogservice.UpdateStream.StreamKeyRange

• string position = 1

  where to start

• optional topodata.KeyRange key_range = 2

  what to get

• optional Charset charset = 3

  default charset on the player side

StreamKeyRangeResponse is the response from StreamKeyRange

Used as response type in: binlogservice.UpdateStream.StreamKeyRange

• optional BinlogTransaction binlog_transaction = 1

StreamTablesRequest is the payload to StreamTables

Used as request type in: binlogservice.UpdateStream.StreamTables

• string position = 1

  where to start

• repeated string tables = 2

  what to get

• optional Charset charset = 3

  default charset on the player side

StreamTablesResponse is the response from StreamTables

Used as response type in: binlogservice.UpdateStream.StreamTables

• optional BinlogTransaction binlog_transaction = 1

Used in: LastPKEvent, ShardGtid, VStreamRequest

• string table_name = 1

• optional query.QueryResult lastpk = 3

VEvent represents a vstream event. A FieldEvent is sent once for every table, just before the first event for that table. The client is expected to cache this information and match it against the RowEvent which contains the table name. A GTID event always precedes a commitable event, which can be COMMIT, DDL or OTHER. OTHER events are non-material events that have no additional metadata.

Used in: VStreamResponse, vtgate.VStreamResponse

• VEventType type = 1

• int64 timestamp = 2

  Timestamp is the binlog timestamp in seconds. The value should be ignored if 0.

• string gtid = 3

  Gtid is set if the event type is GTID.

• string statement = 4

  Statement is set if the event type is DDL, DML or SAVEPOINT.

• optional RowEvent row_event = 5

  RowEvent is set if the event type is ROW.

• optional FieldEvent field_event = 6

  FieldEvent is set if the event type is FIELD.

• optional VGtid vgtid = 7

  Vgtid is set if the event type is VGTID. This event is only generated by VTGate's VStream function.

• optional Journal journal = 8

  Journal is set if the event type is JOURNAL.

• string dml = 9

  Dml is set if the event type is INSERT, REPLACE, UPDATE or DELETE.

• int64 current_time = 20

  CurrentTime specifies the current time when the message was sent. This can be used to compenssate for clock skew.

• optional LastPKEvent last_p_k_event = 21

  LastPK is the last PK for a table

VEventType enumerates the event types. Many of these types will not be encountered in RBR mode.

Used in: VEvent

• UNKNOWN = 0

• GTID = 1

• BEGIN = 2

• COMMIT = 3

• ROLLBACK = 4

• DDL = 5

• INSERT = 6

  INSERT, REPLACE, UPDATE, DELETE and SET will not be seen in RBR mode.

• REPLACE = 7

• UPDATE = 8

• DELETE = 9

• SET = 10

• OTHER = 11

  OTHER is a dummy event. If encountered, the current GTID must be recorded by the client to be able to resume.

• ROW = 12

• FIELD = 13

• HEARTBEAT = 14

  HEARTBEAT is sent if there is inactivity. If a client does not receive events beyond the hearbeat interval, it can assume that it's lost connection to the vstreamer.

• VGTID = 15

  VGTID is generated by VTGate's VStream that combines multiple GTIDs.

• JOURNAL = 16

• VERSION = 17

• LASTPK = 18

• SAVEPOINT = 19

A VGtid is a list of ShardGtids.

Used in: VEvent, vtgate.VStreamRequest

• repeated ShardGtid shard_gtids = 1

VStreamRequest is the payload for VStreamer

Used as request type in: queryservice.Query.VStream

• optional vtrpc.CallerID effective_caller_id = 1

• optional query.VTGateCallerID immediate_caller_id = 2

• optional query.Target target = 3

• string position = 4

• optional Filter filter = 5

• repeated TableLastPK table_last_p_ks = 6

VStreamResponse is the response from VStreamer

Used as response type in: queryservice.Query.VStream

• repeated VEvent events = 1

VStreamResultsRequest is the payload for VStreamResults The ids match VStreamRows, in case we decide to merge the two. The ids match VStreamRows, in case we decide to merge the two.

Used as request type in: queryservice.Query.VStreamResults

• optional vtrpc.CallerID effective_caller_id = 1

• optional query.VTGateCallerID immediate_caller_id = 2

• optional query.Target target = 3

• string query = 4

VStreamResultsResponse is the response from VStreamResults The ids match VStreamRows, in case we decide to merge the two.

Used as response type in: queryservice.Query.VStreamResults

• repeated query.Field fields = 1

• string gtid = 3

• repeated query.Row rows = 4

VStreamRowsRequest is the payload for VStreamRows

Used as request type in: queryservice.Query.VStreamRows

• optional vtrpc.CallerID effective_caller_id = 1

• optional query.VTGateCallerID immediate_caller_id = 2

• optional query.Target target = 3

• string query = 4

• optional query.QueryResult lastpk = 5

VStreamRowsResponse is the response from VStreamRows

Used as response type in: queryservice.Query.VStreamRows

• repeated query.Field fields = 1

• repeated query.Field pkfields = 2

• string gtid = 3

• repeated query.Row rows = 4

• optional query.Row lastpk = 5