package delta.connect

Get desktop application:
View/edit binary Protocol Buffers messages

Represents an assignment of a value to a field.

Used in: UpdateTable

• optional spark.connect.Expression field = 1

  (Required) Expression identifying the (struct) field that is assigned a new value.

• optional spark.connect.Expression value = 2

  (Required) Expression that produces the value to assign to the field.

Command that creates a copy of a DeltaTable in the specified target location.

Used in: DeltaCommand

• optional DeltaTable table = 1

  (Required) The source Delta table to clone.

• string target = 2

  (Required) Path to the location where the cloned table should be stored.

• oneof version_or_timestamp

  (Optional) Optional parameter to clone a previous state of the source table. The current state of the table is cloned when it is not specified.

  • int32 version = 3

    Clones the source table as of the provided version.

  • string timestamp = 4

    Clones the source table as of the provided timestamp.

• bool is_shallow = 5

  (Required) Performs a clone when true, this field should always be set to true.

• bool replace = 6

  (Required) Overwrites the target location when true.

• map<string, string> properties = 7

  (Required) User-defined table properties that override properties with the same key in the source table.

Command that turns a Parquet table into a Delta table. This needs to be a Relation as it returns the identifier of the resulting table. We cannot simply reuse the input identifier, as it could be a path-based identifier, and in that case we need to replace "parquet.`...`" with "delta.`...`".

Used in: DeltaRelation

• string identifier = 1

  (Required) Parquet table identifier formatted as "parquet.`path`"

• oneof partition_schema

  (Optional) Partitioning schema of the input table

  • string partition_schema_string = 2

    Hive DDL formatted string

  • spark.connect.DataType partition_schema_struct = 3

    Struct with names and types of partitioning columns

Command that deletes data from the target table that matches the given condition. Needs to be a Relation, as it returns a row containing the execution metrics.

Used in: DeltaRelation

• optional spark.connect.Relation target = 1

  (Required) Target table to delete data from. Must either be a DeltaRelation containing a Scan or a SubqueryAlias with a DeltaRelation containing a Scan as its input.

• optional spark.connect.Expression condition = 2

  (Optional) Expression returning a boolean.

Message to hold all command extensions in Delta Connect.

• oneof command_type

  • CloneTable clone_table = 1

  • VacuumTable vacuum_table = 2

  • UpgradeTableProtocol upgrade_table_protocol = 3

  • Generate generate = 4

Message to hold all relation extensions in Delta Connect.

• oneof relation_type

  • Scan scan = 1

  • DescribeHistory describe_history = 2

  • DescribeDetail describe_detail = 3

  • ConvertToDelta convert_to_delta = 4

  • RestoreTable restore_table = 5

  • IsDeltaTable is_delta_table = 6

  • DeleteFromTable delete_from_table = 7

  • UpdateTable update_table = 8

Information required to access a Delta table either by name or by path.

Used in: CloneTable, DescribeDetail, DescribeHistory, Generate, RestoreTable, Scan, UpgradeTableProtocol, VacuumTable

• oneof access_type

  (Required)

  • DeltaTable.Path path = 1

  • string table_or_view_name = 2

Used in: DeltaTable

• string path = 1

  (Required) Path to the Delta table.

• map<string, string> hadoop_conf = 2

  (Optional) Hadoop configuration used to access the file system.

Relation containing the details of a Delta table such as the format, name, and size.

Used in: DeltaRelation

• optional DeltaTable table = 1

  (Required) The Delta table to describe the details of.

Relation containing information of the latest commits on a Delta table. The information is in reverse chronological order.

Used in: DeltaRelation

• optional DeltaTable table = 1

  (Required) The Delta table to read the history of.

Command that generates manifest files for a given Delta table.

Used in: DeltaCommand

• optional DeltaTable table = 1

  (Required) The Delta table to generate the manifest files for.

• string mode = 2

  (Required) The type of manifest file to be generated.

Relation containing a single row containing a single boolean that indicates whether the provided path contains a Delta table.

Used in: DeltaRelation

• string path = 1

  (Required) The path to check.

Command that restores the DeltaTable to an older version of the table specified by either a version number or a timestamp. Needs to be a Relation, as it returns a row containing the execution metrics.

Used in: DeltaRelation

• optional DeltaTable table = 1

  (Required) The Delta table to restore to an earlier version.

• oneof version_or_timestamp

  (Required) Version to restore to.

  • int64 version = 2

    The version number to restore to.

  • string timestamp = 3

    The timestamp to restore to.

Relation that reads from a Delta table.

Used in: DeltaRelation

• optional DeltaTable table = 1

  (Required) The Delta table to scan.

Command that updates data in the target table using the given assignments for rows that matches the given condition. Needs to be a Relation, as it returns a row containing the execution metrics.

Used in: DeltaRelation

• optional spark.connect.Relation target = 1

  (Required) Target table to delete data from. Must either be a DeltaRelation containing a Scan or a SubqueryAlias with a DeltaRelation containing a Scan as its input.

• optional spark.connect.Expression condition = 2

  (Optional) Condition that determines which rows must be updated. Must be an expression returning a boolean.

• repeated Assignment assignments = 3

  (Optional) Set of assignments to apply to the rows matching the condition.

Command to updates the protocol version of the table so that new features can be used.

Used in: DeltaCommand

• optional DeltaTable table = 1

  (Required) The Delta table to upgrade the protocol of.

• int32 reader_version = 2

  (Required) The minimum required reader protocol version.

• int32 writer_version = 3

  (Required) The minimum required writer protocol version.

Command that deletes files and directories in the table that are not needed by the table for maintaining older versions up to the given retention threshold.

Used in: DeltaCommand

• optional DeltaTable table = 1

  (Required) The Delta table to vacuum.

• optional double retention_hours = 2

  (Optional) Number of hours retain history for. If not specified, then the default retention period will be used.