package apsara.odps.lot

Get desktop application:
View/edit binary Protocol Buffers messages

for adhoc query. e.g.: select xx from table1;

Used in: DataSink

(message has no fields)

Group By会输出所有的group by columns + 所有的Function的alias。有重复报错

Used in: LogicalOperator

• required string Id = 1

• required string ParentId = 2

• repeated Reference GroupByColumns = 3

• repeated group apsara.odps.lot.Aggregate.Functions = 4

Used in: Aggregate

• required AggregationFunction Function = 1

• required string Alias = 2

Used in: Aggregate.Functions

• required string Project = 1

• required string Name = 2

• required bool IsDistinct = 3

• repeated ScalarExpression Parameters = 4

• optional bool requireBuiltin = 5

用来实现UDTF，SQLServer术语 它的输出时所有TVF的输出列 + 指定的输入列 select udtf(a, b) as (a1, b1), udtf(c, d) as (c1, d1), e, f from table

Used in: LogicalOperator

• required string Id = 1

• required string ParentId = 2

• repeated TableValuedFunction Udtfs = 3

Used in: DDL

• required LogicalOperatorTree Lot = 1

• repeated group apsara.odps.lot.CreateTableAs.Properties = 2

• optional bool IfNotExists = 3

• optional int32 LifeCycle = 4

Used in: CreateTableAs

• required string Key = 1

• required string Value = 2

Used in: DDL

• required LogicalOperatorTree Lot = 1

• required string Project = 2

• required string Name = 3

Used in: LotTask

• optional CreateTable CreateTable = 1

• optional CreateTableAs CreateTableAs = 2

• optional CreateTableLike CreateTableLike = 3

• optional DropTable DropTable = 4

• optional CreateViewAs CreateViewAs = 5

• optional CreateStreamTable CreateStreamTable = 6

• optional CreateDimensionTable CreateDimensionTable = 7

• optional CreateResultTable CreateResultTable = 8

• optional CreateTempTable CreateTempTable = 9

Used in: LogicalOperator

• required string Id = 1

• required string ParentId = 2

• optional TableSink TableSink = 3

• optional AdhocSink AdhocSink = 4

• optional LanguageSink LanguageSink = 5

• optional FakeSink FakeSink = 6

Used in: LogicalOperator

• required string Id = 1

• optional TableScan TableScan = 2

• optional LanguageSource LanguageSource = 3

• optional ViewSource ViewSource = 4

Used in: LogicalOperator

• required string Id = 1

• required string ParentId = 2

• repeated Reference Columns = 3

什么都不干的sink，包括屏显也不干

Used in: DataSink

(message has no fields)

Used in: LogicalOperator

• required string Id = 1

• required string ParentId = 2

• required ScalarExpression Condition = 3

Used in: LogicalOperator

• required string Id = 1

• required string LeftParentId = 2

• required string RightParentId = 3

• required JoinType JoinType = 4

• optional ScalarExpression OnCondition = 5

  when CartesianProduct, on condition'll be ignored

• optional Join.ExecutionType ExeType = 6

• repeated string SmallParents = 7

  just used for hash join

Used in: Join

• Auto = 0

• MergeJoin = 1

• FullHashJoin = 2

  现在odps的map join

• ShuffledHashJoin = 3

  两边输入做shuffle，然后做map join

Used in: DataSink

• required Language Language = 1

• repeated group apsara.odps.lot.LanguageSink.Resources = 2

• required string ClassName = 3

• repeated ReferencedURI URIs = 4

• repeated group apsara.odps.lot.LanguageSink.Output = 5

• required uint32 InstanceCount = 6

• repeated group apsara.odps.lot.LanguageSink.Properties = 7

Used in: LanguageSink

• required Reference Column = 1

• required Type Type = 2

Used in: LanguageSink

• required string Key = 1

• required string Value = 2

Used in: LanguageSink

• required string Project = 1

• required string ResourceName = 2

Used in: DataSource

• required Language Language = 1

• repeated group apsara.odps.lot.LanguageSource.Resources = 2

• required string ClassName = 3

• repeated ReferencedURI URIs = 4

• required Schema Schema = 5

• required uint32 InstanceCount = 6

• repeated group apsara.odps.lot.LanguageSource.Properties = 7

• repeated Volume Volumes = 8

Used in: LanguageSource

• required string Key = 1

• required string Value = 2

Used in: LanguageSource

• required string Project = 1

• required string ResourceName = 2

Used in: Transform

• required string ClassName = 1

• repeated ReferencedURI URIs = 2

• required Language Language = 3

Used in: LogicalOperator

• required string Id = 1

• required string ParentId = 2

• required uint64 Count = 3

Used in: LogicalOperatorTree

• optional Apply Apply = 1

• optional DistributeBy DistributeBy = 2

• optional Filter Filter = 3

• optional Aggregate Aggregate = 4

• optional Join Join = 5

• optional Select Select = 6

• optional SortBy SortBy = 7

• optional DataSource DataSource = 8

• optional DataSink DataSink = 9

• optional Limit Limit = 10

• optional UnionAll UnionAll = 11

• optional Window Window = 12

• optional Transform Transform = 13

Used in: CreateTableAs, CreateViewAs, LotTask

• repeated LogicalOperator Operators = 1

• optional int64 Version = 1

• repeated group apsara.odps.lot.LotTask.Environment = 2

• optional DDL DDL = 3

• optional LogicalOperatorTree Lot = 4

• repeated TemporaryFunction TempFunctions = 5

Used in: LotTask

• required string Key = 1

• required string Value = 2

• repeated group apsara.odps.lot.PythonTransform.Resources = 1

• required string ClassName = 2

  it's the class name in "create function xx as '<class name>'..."

• repeated ReferencedURI URIs = 3

Used in: PythonTransform

• required string Project = 1

• required string ResourceName = 2

Used in: AggregationFunction, Filter, Join, ScalarFunction, Select.Expressions, TableValuedFunction, Transform, WindowFunction, WindowFunction.OrderBy

• optional Reference Reference = 1

• optional Constant Constant = 2

• optional Null Null = 3

• optional ScalarFunction Expression = 4

Used in: ScalarExpression

• required string Project = 1

• required string Name = 2

• repeated ScalarExpression Parameters = 3

• optional bool requireBuiltin = 4

Used in: LanguageSource, Transform

• repeated group apsara.odps.lot.Schema.Columns = 1

Used in: Schema

• required string Name = 1

• required Type Type = 2

Used in: LogicalOperator

• required string Id = 1

• required string ParentId = 2

• repeated group apsara.odps.lot.Select.Expressions = 3

• optional string TempTableName = 4

  empty string will be treated as not set.

Used in: Select

• required ScalarExpression Expression = 1

• required string Alias = 2

Used in: LogicalOperator

• required string Id = 1

• required string ParentId = 2

• repeated Order Orders = 3

• required bool IsPartial = 4

  true: sort rows in each instance, false: sort all rows

Used in: Transform

• required string Cmd = 1

• repeated group apsara.odps.lot.StreamingTransform.Properties = 2

Used in: StreamingTransform

• required string Key = 1

• required string Value = 2

Used in: DataSource

• required string Project = 1

• required string Table = 2

当一个LOT中有多个对同一张表的TableSink时的检查

Used in: DataSink

• required string Project = 1

• required string Table = 2

• required bool IsOverwrite = 3

• optional PartitionSpec Partition = 4

• optional DynamicPartitionSpec DynamicPartition = 5

UDTF,实际上聚合函数、WindowFunction都是它的特例

Used in: Apply

• required string Project = 1

• required string Name = 2

• repeated ScalarExpression Parameters = 3

• repeated string OutputColumns = 4

• repeated group apsara.odps.lot.TableValuedFunction.Properties = 5

• optional bool requireBuiltin = 6

Used in: TableValuedFunction

• required string Key = 1

• required string Value = 2

Used in: LotTask

• required string Name = 1

• repeated group apsara.odps.lot.TemporaryFunction.Resources = 2

• required string ClassName = 3

• required Language Language = 4

Used in: TemporaryFunction

• required string Project = 1

• required string ResourceName = 2

Used in: LogicalOperator

• required string Id = 1

• required string ParentId = 2

• repeated ScalarExpression Parameters = 3

• repeated group apsara.odps.lot.Transform.Resources = 4

• optional LanguageTransform LanguageTransform = 5

• optional StreamingTransform StreamingTransform = 6

• required Schema Schema = 7

• repeated Volume Volumes = 8

Used in: Transform

• required string Project = 1

• required string ResourceName = 2

Used in: LogicalOperator

• required string Id = 1

• repeated string Parents = 2

Used in: DataSource

• required string Project = 1

• required string View = 2

Used in: LanguageSource, Transform

• required string Project = 1

• required string VolumeName = 2

• required string Partition = 3

• required string Label = 4

• required bool IsInput = 5

  true for input, false for output

Used in: LogicalOperator

• required string Id = 1

• required string ParentId = 2

• repeated group apsara.odps.lot.Window.Functions = 3

Used in: Window

• required WindowFunction Function = 1

• required string Alias = 2

Used in: Window.Functions

• required string Project = 1

• required string Name = 2

• optional bool IsDistinct = 3

• repeated ScalarExpression Parameters = 4

• repeated ScalarExpression PartitionBy = 5

• repeated group apsara.odps.lot.WindowFunction.OrderBy = 6

• optional group apsara.odps.lot.WindowFunction.WindowingClause = 7

• optional bool requireBuiltin = 8

  TODO: add other properties.

Used in: WindowFunction

• required ScalarExpression Expression = 1

• optional bool IsAsc = 2

Used in: WindowFunction

• required int32 Begin = 1

• required int32 End = 2