package autodist.proto

Get desktop application:
View/edit binary Protocol Buffers messages

* Synchronization using AllReduce.

Used in: Strategy.Node

• AllReduceSynchronizer.Spec spec = 1

  Specification for collective communication

• AllReduceSynchronizer.Compressor compressor = 2

  One of the compressors to choose

• int32 group = 3

  * The allreduce group to merge with. The group index should be less than the number of variables

* Which gradient compression method to use

Used in: AllReduceSynchronizer

• NoneCompressor = 0

  No compression

• HorovodCompressor = 1

  Horovod's Compression

• HorovodCompressorEF = 2

  Horovod's Compression but with Error Feedback.

* Which communication method to use

Used in: AllReduceSynchronizer

• AUTO = 0

  Runtime's automatic choices

• NCCL = 1

  Use ncclAllReduce for all-reduce, and ring algorithms for all-gather

• RING = 2

  TensorFlow's ring algorithms for all-reduce and all-gather

* Represents the strategy the AutoDist backend will implement.

• optional google.protobuf.Any graph_def = 1

  / TensorFlow graph_def

• map<string, string> grad_target_pairs = 2

  / Mapping from grad tensor name to variable name

• optional GraphItem.Info info = 3

* Represents the essential transformed subset of TensorFlow MetaGraph Right now, it represents a essential AutoDist subset of collections of MetaGraph. In the future, it will generalize to captures.

Used in: GraphItem

• repeated google.protobuf.Any variables = 1

• repeated string table_initializers = 2

• repeated google.protobuf.Any savers = 3

* Synchronization using a Parameter Server.

Used in: Strategy.Node

• string reduction_destination = 1

  Parameter Server to use

• bool local_replication = 2

  Whether to create local proxies of each PS variable

• bool sync = 3

  Whether to sync gradients across between-graph replications

• int32 staleness = 4

  Staleness

* Represents the strategy the AutoDist backend will implement.

• string id = 1

  / unique strategy identifier

• string path = 2

  / optional serialized strategy message temp path

• repeated Strategy.Node node_config = 3

  * configuration of some individual nodes of the computational graph

• optional Strategy.GraphConfig graph_config = 4

  * configuration of the computational graph as a whole

* Represents the configuration of the graph as a whole. Based on the list of replicas, the AutoDist backend does a combination of in-graph and between-graph distribution.

Used in: Strategy

• repeated string replicas = 1

  * the number of batch-splitting/data-parallel replicas

* Represents the configuration of an individual node in the graph. Right now, these nodes are just variables in the graph, so the only information they contain is how to synchronize the variable's gradients. In the future, for node partitioning, these could be any node in the graph. In that case, they would also have more logic for partitioning the op.

Used in: Strategy

• string var_name = 1

  / variable name

• oneof synchronizer

  • PSSynchronizer PSSynchronizer = 2

    One of a synchronizer to choose

  • AllReduceSynchronizer AllReduceSynchronizer = 3

    One of a synchronizer to choose

• string partitioner = 4

  Optional partitioner configuration, e.g. `1, 2, 1`

• repeated Node part_config = 5

  Optional node configs for each node partition (if partitioned)