package proto.entity

Get desktop application:
View/edit binary Protocol Buffers messages

Set of {AppendCommit, CompactionCommit, UpdateCommit, MergeCommit}, which define the specific operation of this DataCommit

Used in: DataCommitInfo, PartitionInfo

• CompactionCommit = 0

  CompactionCommit: A commit type indicates that this DataCommit is to compact files in a specific table range partition

• AppendCommit = 1

  AppendCommit: A commit type indicates that this DataCommit is to append files to a specific table range partition without hash partition

• MergeCommit = 2

  MergeCommit: A commit type indicates that this DataCommit is to append files to a specific table range partition with hash partition

• UpdateCommit = 3

  UpdateCommit: A commit type indicates that this DataCommit is to update data(add new files and delete old invalid files) in a specific table range partition

• DeleteCommit = 4

  DeleteCommit: A commit type indicates that this DataCommit is to delete files in a specific table range partition

Data Files Commit information for specific table range partitions

Used in: JniWrapper

• string table_id = 1

  TableId of DataCommit

• string partition_desc = 2

  Range partition description, which defines a specific range partition of the table, in the formatted of comma-separated range_colum=range_value Especially, a table without range partitions use LAKESOUL_NON_PARTITION_TABLE_PART_DESC as partitionDesc

• optional Uuid commit_id = 3

  Global unique identifier of DataCommit

• repeated DataFileOp file_ops = 4

  Collection of DataFileOps included with DataCommit

• CommitOp commit_op = 5

  Specific operation of this DataCommit

• int64 timestamp = 6

  Creation timestamp of the files of the DataCommit

• bool committed = 7

  mark define if this DataCommit has already committed as PartitionInfo of table

• string domain = 8

Singleton Data File information

Used in: DataCommitInfo

• string path = 1

  Physical qualified non-percent-encoded path of a parquet file,

• FileOp file_op = 2

  Specific operation of this file

• int64 size = 3

  File size of byte-unit

• string file_exist_cols = 4

  Columns included with this parquet file, which should be equivalent of the meta of parquet file

Meta Information for LakeSoul DiscardCompressedFile

Used in: JniWrapper

• string file_path = 1

  File path

• string table_path = 2

  Table path of this file belong to

• string partition_desc = 3

  Partition info

• int64 timestamp = 4

  Creation timestamp when the date created

• string t_date = 5

  Date

Set of {add, delete}, which define the specific operation of this file

Used in: DataFileOp

• add = 0

  add: indicates that the parquet file is newly added

• del = 1

  del: indicates that the parquet file has been deleted

• repeated Namespace namespace = 1

• repeated TableInfo table_info = 2

• repeated TablePathId table_path_id = 3

• repeated TableNameId table_name_id = 4

• repeated PartitionInfo partition_info = 5

• repeated DataCommitInfo data_commit_info = 6

• repeated DiscardCompressedFileInfo discard_compressed_file_info = 7

A collection of PartitionInfo of one TableInfo

• repeated PartitionInfo list_partition = 1

• optional TableInfo table_info = 2

• repeated PartitionInfo read_partition_info = 3

Namespace of tables

Used in: JniWrapper

• string namespace = 1

  Dot-separated-formatted namespace

• string properties = 2

• string comment = 3

• string domain = 4

  Domain this entry belongs to. Only when rbac feature enabled will have contents different to 'public'

Version information for specific table range partitions

Used in: JniWrapper, MetaInfo

• string table_id = 1

  TableId of PartitionInfo

• string partition_desc = 2

  Range partition description, which defines a specific range partition of the table, in the formatted of comma-separated range_colum=range_value Especially, a table without range partitions use LAKESOUL_NON_PARTITION_TABLE_PART_DESC as partitionDesc

• int32 version = 3

  A version is defined as a number monotonically increasing by 1

• CommitOp commit_op = 4

  Specific operation of the version information

• int64 timestamp = 5

  Timestamp of the PartitionInfo successfully committed

• repeated Uuid snapshot = 7

  Collection of commitId of DataCommitInfo included with PartitionInfo

• string expression = 8

  Expression used to calculate or filter data, will be launched in the future. Now it's just a meaningless empty string.

• string domain = 9

  Domain this entry belongs to. Only when rbac feature enabled will have contents different to 'public'

Meta Information for LakeSoul Table

Used in: JniWrapper, MetaInfo

• string table_id = 1

  Global unique identifier of table

• string table_namespace = 2

  Namespace of table. A string of 'tableNamespace.tablePath' or 'tableNamespace.tableName' maps one unique table globally

• string table_name = 3

  Name of table, optional

• string table_path = 4

  Physical qualified path of table

• string table_schema = 5

  Spark-formatted schema of table

• string properties = 7

  Properties of table, used to tag table with information not tracked by SQL

• string partitions = 8

  Partition columns of table. Format of partitions is 'comma_separated_range_column;hash_column'

• string domain = 9

  Domain this entry belongs to. Only when rbac feature enabled will have contents different to 'public'

Relationship between 'TableNamespace.TableName' and TableId

Used in: JniWrapper

• string table_name = 1

  Name of table

• string tableId = 2

  Global unique identifier of table

• string table_namespace = 3

  Namespace of table

• string domain = 4

  Domain this entry belongs to. Only when rbac feature enabled will have contents different to 'public'

Relationship between 'TableNamespace.TablePath' and TableId

Used in: JniWrapper

• string table_path = 1

  Physical qualified path of table

• string table_id = 2

  Global unique identifier of table

• string table_namespace = 3

  Namespace of table

• string domain = 4

  Domain this entry belongs to. Only when rbac feature enabled will have contents different to 'public'

Used in: DataCommitInfo, PartitionInfo

• uint64 high = 1

• uint64 low = 2