package org.apache.flink.fn_execution.v1

Get desktop application:
View/edit binary Protocol Buffers messages

A representation of the coder

• oneof data_type

  • CoderInfoDescriptor.FlattenRowType flatten_row_type = 1

    for Table & SQL

  • CoderInfoDescriptor.RowType row_type = 2

  • CoderInfoDescriptor.ArrowType arrow_type = 3

  • CoderInfoDescriptor.OverWindowArrowType over_window_arrow_type = 4

  • CoderInfoDescriptor.RawType raw_type = 5

    for DataStream

• CoderInfoDescriptor.Mode mode = 6

• bool separated_with_end_message = 7

  append an end message (0x00) after serializing the records belonging to the same batch.

Used in: CoderInfoDescriptor

• optional Schema schema = 1

for Table & SQL

Used in: CoderInfoDescriptor

• optional Schema schema = 1

Used in: CoderInfoDescriptor

• SINGLE = 0

  process one record at a time

• MULTIPLE = 1

  process multiple records at a time

only used in batch over window the data consists of [window data][arrow data]

Used in: CoderInfoDescriptor

• optional Schema schema = 1

for DataStream

Used in: CoderInfoDescriptor

• optional TypeInfo type_info = 1

Used in: CoderInfoDescriptor

• optional Schema schema = 1

Used in: UserDefinedAggregateFunctions

• GroupWindow.WindowType window_type = 1

• bool is_time_window = 2

• int64 window_slide = 3

• int64 window_size = 4

• int64 window_gap = 5

• bool is_row_time = 6

• int32 time_field_index = 7

• int64 allowedLateness = 8

• repeated GroupWindow.WindowProperty namedProperties = 9

• string shift_timezone = 10

Used in: GroupWindow

• WINDOW_START = 0

• WINDOW_END = 1

• ROW_TIME_ATTRIBUTE = 2

• PROC_TIME_ATTRIBUTE = 3

Used in: GroupWindow

• TUMBLING_GROUP_WINDOW = 0

• SLIDING_GROUP_WINDOW = 1

• SESSION_GROUP_WINDOW = 2

Used in: UserDefinedAggregateFunction, UserDefinedFunction

• oneof input

  • UserDefinedFunction udf = 1

  • int32 inputOffset = 2

  • bytes inputConstant = 3

Used in: UserDefinedAggregateFunctions, UserDefinedDataStreamFunction.RuntimeContext, UserDefinedFunctions

• string key = 1

• string value = 2

Used to describe the info of over window in pandas batch over window aggregation

Used in: UserDefinedFunctions

• OverWindow.WindowType window_type = 1

• int64 lower_boundary = 2

• int64 upper_boundary = 3

Used in: OverWindow

• RANGE_UNBOUNDED = 0

• RANGE_UNBOUNDED_PRECEDING = 1

• RANGE_UNBOUNDED_FOLLOWING = 2

• RANGE_SLIDING = 3

• ROW_UNBOUNDED = 4

• ROW_UNBOUNDED_PRECEDING = 5

• ROW_UNBOUNDED_FOLLOWING = 6

• ROW_SLIDING = 7

A representation of the data schema.

Used in: CoderInfoDescriptor.ArrowType, CoderInfoDescriptor.FlattenRowType, CoderInfoDescriptor.OverWindowArrowType, CoderInfoDescriptor.RowType, Schema.FieldType

• repeated Schema.Field fields = 1

Used in: FieldType

• int32 length = 1

Used in: FieldType

• int32 length = 1

Used in: FieldType

• int32 precision = 1

• int32 scale = 2

Used in: Schema

• string name = 1

• string description = 2

• optional FieldType type = 3

Used in: Field, MapInfo, UserDefinedAggregateFunction.DataViewSpec.ListView, UserDefinedAggregateFunction.DataViewSpec.MapView, UserDefinedAggregateFunctions

• TypeName type_name = 1

• bool nullable = 2

• oneof type_info

  • FieldType collection_element_type = 3

  • MapInfo map_info = 4

  • Schema row_schema = 5

  • DecimalInfo decimal_info = 6

  • TimeInfo time_info = 7

  • TimestampInfo timestamp_info = 8

  • LocalZonedTimestampInfo local_zoned_timestamp_info = 9

  • ZonedTimestampInfo zoned_timestamp_info = 10

  • BinaryInfo binary_info = 11

  • VarBinaryInfo var_binary_info = 12

  • CharInfo char_info = 13

  • VarCharInfo var_char_info = 14

Used in: FieldType

• int32 precision = 1

Used in: FieldType

• optional FieldType key_type = 1

• optional FieldType value_type = 2

Used in: FieldType

• int32 precision = 1

Used in: FieldType

• int32 precision = 1

Used in: FieldType

• ROW = 0

• TINYINT = 1

• SMALLINT = 2

• INT = 3

• BIGINT = 4

• DECIMAL = 5

• FLOAT = 6

• DOUBLE = 7

• DATE = 8

• TIME = 9

• TIMESTAMP = 10

• BOOLEAN = 11

• BINARY = 12

• VARBINARY = 13

• CHAR = 14

• VARCHAR = 15

• BASIC_ARRAY = 16

• MAP = 17

• MULTISET = 18

• LOCAL_ZONED_TIMESTAMP = 19

• ZONED_TIMESTAMP = 20

• NULL = 21

Used in: FieldType

• int32 length = 1

Used in: FieldType

• int32 length = 1

Used in: FieldType

• int32 precision = 1

A representation of State

• string state_name = 1

• optional StateDescriptor.StateTTLConfig state_ttl_config = 2

Used in: StateDescriptor

• StateTTLConfig.UpdateType update_type = 1

• StateTTLConfig.StateVisibility state_visibility = 2

• StateTTLConfig.TtlTimeCharacteristic ttl_time_characteristic = 3

• int64 ttl = 4

• optional StateTTLConfig.CleanupStrategies cleanup_strategies = 5

TTL cleanup strategies.

Used in: StateTTLConfig

• bool is_cleanup_in_background = 1

• repeated CleanupStrategies.MapStrategiesEntry strategies = 2

Used in: MapStrategiesEntry

• EMPTY_STRATEGY = 0

Configuration of cleanup strategy while taking the full snapshot.

Used in: MapStrategiesEntry

• int32 cleanup_size = 1

  Max number of keys pulled from queue for clean up upon state touch for any key.

• bool run_cleanup_for_every_record = 2

  Whether to run incremental cleanup per each processed record.

Used in: CleanupStrategies

• Strategies strategy = 1

• oneof CleanupStrategy

  • EmptyCleanupStrategy empty_strategy = 2

  • IncrementalCleanupStrategy incremental_cleanup_strategy = 3

  • RocksdbCompactFilterCleanupStrategy rocksdb_compact_filter_cleanup_strategy = 4

Configuration of cleanup strategy using custom compaction filter in RocksDB.

Used in: MapStrategiesEntry

• int64 query_time_after_num_entries = 1

  Number of state entries to process by compaction filter before updating current timestamp.

Fixed strategies ordinals in strategies config field.

Used in: MapStrategiesEntry

• FULL_STATE_SCAN_SNAPSHOT = 0

• INCREMENTAL_CLEANUP = 1

• ROCKSDB_COMPACTION_FILTER = 2

This option configures whether expired user value can be returned or not.

Used in: StateTTLConfig

• ReturnExpiredIfNotCleanedUp = 0

  Return expired user value if it is not cleaned up yet.

• NeverReturnExpired = 1

  Never return expired user value.

This option configures time scale to use for ttl.

Used in: StateTTLConfig

• ProcessingTime = 0

  Processing time

This option value configures when to update last access timestamp which prolongs state TTL.

Used in: StateTTLConfig

• Disabled = 0

  TTL is disabled. State does not expire.

• OnCreateAndWrite = 1

  Last access timestamp is initialised when state is created and updated on every write operation.

• OnReadAndWrite = 2

  The same as OnCreateAndWrite but also updated on read.

A representation of the data type information in DataStream.

Used in: CoderInfoDescriptor.RawType, TypeInfo.MapTypeInfo, TypeInfo.RowTypeInfo.Field, TypeInfo.TupleTypeInfo, UserDefinedDataStreamFunction

• TypeInfo.TypeName type_name = 1

• oneof type_info

  • TypeInfo collection_element_type = 2

  • TypeInfo.RowTypeInfo row_type_info = 3

  • TypeInfo.TupleTypeInfo tuple_type_info = 4

  • TypeInfo.MapTypeInfo map_type_info = 5

  • TypeInfo.AvroTypeInfo avro_type_info = 6

Used in: TypeInfo

• string schema = 1

Used in: TypeInfo

• optional TypeInfo key_type = 1

• optional TypeInfo value_type = 2

Used in: TypeInfo

• repeated RowTypeInfo.Field fields = 1

Used in: RowTypeInfo

• string field_name = 1

• optional TypeInfo field_type = 2

Used in: TypeInfo

• repeated TypeInfo field_types = 1

Used in: TypeInfo

• ROW = 0

• STRING = 1

• BYTE = 2

• BOOLEAN = 3

• SHORT = 4

• INT = 5

• LONG = 6

• FLOAT = 7

• DOUBLE = 8

• CHAR = 9

• BIG_INT = 10

• BIG_DEC = 11

• SQL_DATE = 12

• SQL_TIME = 13

• SQL_TIMESTAMP = 14

• BASIC_ARRAY = 15

• PRIMITIVE_ARRAY = 16

• TUPLE = 17

• LIST = 18

• MAP = 19

• PICKLED_BYTES = 20

• OBJECT_ARRAY = 21

• INSTANT = 22

• AVRO = 23

• LOCAL_DATE = 24

• LOCAL_TIME = 25

• LOCAL_DATETIME = 26

• LOCAL_ZONED_TIMESTAMP = 27

Used in: UserDefinedAggregateFunctions

• bytes payload = 1

  The serialized representation of the user-defined function

• repeated Input inputs = 2

  The input arguments of the user-defined function, it could be one of the following: 1. A column from the input row 2. The result of another user-defined function 3. The constant value of the column

• repeated UserDefinedAggregateFunction.DataViewSpec specs = 3

• int32 filter_arg = 4

• bool distinct = 5

• bool takes_row_as_input = 6

  Whether the UDF takes row as input instead of each columns of a row

Used in: UserDefinedAggregateFunction

• string name = 1

• int32 field_index = 2

• oneof data_view

  • DataViewSpec.ListView list_view = 3

  • DataViewSpec.MapView map_view = 4

Used in: DataViewSpec

• optional Schema.FieldType element_type = 1

Used in: DataViewSpec

• optional Schema.FieldType key_type = 1

• optional Schema.FieldType value_type = 2

A list of the user-defined aggregate functions to be executed in a group aggregate operation.

• repeated UserDefinedAggregateFunction udfs = 1

• bool metric_enabled = 2

• repeated int32 grouping = 3

  The array of the key indexes.

• bool generate_update_before = 4

  Generate retract messages if true.

• optional Schema.FieldType key_type = 5

  The schema of the grouping keys.

• int32 index_of_count_star = 6

  The index of a count aggregate used to calculate the number of accumulated rows.

• bool state_cleaning_enabled = 7

  Cleanup the expired state if true.

• int32 state_cache_size = 8

  The state cache size.

• int32 map_state_read_cache_size = 9

  The map state read cache size.

• int32 map_state_write_cache_size = 10

  The map_state_write_cache_size.

• bool count_star_inserted = 11

  True if the count(*) agg is inserted by the planner.

• optional GroupWindow group_window = 12

  Group Window.

• bool profile_enabled = 13

• repeated JobParameter job_parameters = 14

User defined DataStream function definition.

• UserDefinedDataStreamFunction.FunctionType function_type = 1

• optional UserDefinedDataStreamFunction.RuntimeContext runtime_context = 2

• bytes payload = 3

• bool metric_enabled = 4

• optional TypeInfo key_type_info = 5

• bool profile_enabled = 6

• bool has_side_output = 7

• int32 state_cache_size = 8

  The state cache size.

• int32 map_state_read_cache_size = 9

  The map state read cache size.

• int32 map_state_write_cache_size = 10

  The map_state_write_cache_size.

Used in: UserDefinedDataStreamFunction

• PROCESS = 0

• CO_PROCESS = 1

• KEYED_PROCESS = 2

• KEYED_CO_PROCESS = 3

• WINDOW = 4

• CO_BROADCAST_PROCESS = 5

• KEYED_CO_BROADCAST_PROCESS = 6

• REVISE_OUTPUT = 100

Used in: UserDefinedDataStreamFunction

• string task_name = 1

• string task_name_with_subtasks = 2

• int32 number_of_parallel_subtasks = 3

• int32 max_number_of_parallel_subtasks = 4

• int32 index_of_this_subtask = 5

• int32 attempt_number = 6

• repeated JobParameter job_parameters = 7

• bool in_batch_execution_mode = 8

User-defined function definition. It supports chaining functions, that's, the execution result of one user-defined function as the input of another user-defined function.

Used in: Input, UserDefinedFunctions

• bytes payload = 1

  The serialized representation of the user-defined function

• repeated Input inputs = 2

  The input arguments of the user-defined function, it could be one of the following: 1. A column from the input row 2. The result of another user-defined function 3. The constant value of the column

• int32 window_index = 3

  The index of the over window used in pandas batch over window aggregation

• bool takes_row_as_input = 4

  Whether the UDF takes row as input instead of each columns of a row

• bool is_pandas_udf = 5

  Whether it's pandas UDF

A list of user-defined functions to be executed in a batch.

• repeated UserDefinedFunction udfs = 1

• bool metric_enabled = 2

• repeated OverWindow windows = 3

• bool profile_enabled = 4

• repeated JobParameter job_parameters = 5