package tensorflow.eager

Get desktop application:
View/edit binary Protocol Buffers messages

////////////////////////////////////////////////////////////////////////////// Eager Service defines a TensorFlow service that executes operations eagerly on a set of local devices, on behalf of a remote Eager executor. The service impl will keep track of the various clients and devices it has access to and allows the client to enqueue ops on any devices that it is able to access and schedule data transfers from/to any of the peers. A client can generate multiple contexts to be able to independently execute operations, but cannot share data between the two contexts. NOTE: Even though contexts generated by clients should be independent, the lower level tensorflow execution engine is not, so they might share some data (e.g. a Device's ResourceMgr). //////////////////////////////////////////////////////////////////////////////

• rpc CloseContext (CloseContextRequest, CloseContextResponse)

  eager_service.proto:343

  Closes the context. No calls to other methods using the existing context ID are valid after this.

  message CloseContextRequest

  eager_service.proto:203

  • fixed64 context_id = 1

  • fixed64 context_view_id = 2

  message CloseContextResponse

  eager_service.proto:208

  (message has no fields)

• rpc CreateContext (CreateContextRequest, CreateContextResponse)

  eager_service.proto:291

  This initializes the worker, informing it about the other workers in the cluster and exchanging authentication tokens which will be used in all other RPCs to detect whether the worker has restarted.

  message CreateContextRequest

  eager_service.proto:87

  • optional ServerDef server_def = 1

    Identifies the full cluster, and this particular worker's position within.

  • bool async = 2

    Whether the ops on the worker should be executed synchronously or asynchronously. By default, ops are executed synchronously.

  • int64 keep_alive_secs = 3

    Number of seconds to keep the context alive. If more than keep_alive_secs has passed since a particular context has been communicated with, it will be garbage collected.

  • optional VersionDef version_def = 4

    This is the version for all the ops that will be enqueued by the client.

  • repeated DeviceAttributes cluster_device_attributes = 6

    Device attributes in the cluster

  • fixed64 context_id = 7

    The ID of the created context. This is usually a randomly generated number, that will be used to identify the context in future requests to the service. Contexts are not persisted through server restarts. This ID will be used for all future communications as well. It is essential that both ends use this ID for selecting a rendezvous to get everything to match.

  • fixed64 context_view_id = 8

    The view ID of the context.

  • bool lazy_copy_remote_function_inputs = 9

    For a multi device function, if false, eagerly copy all remote inputs to the default function device; if true, lazily copy remote inputs to their target devices after function instantiation to avoid redundant copies.

  message CreateContextResponse

  eager_service.proto:125

  • repeated DeviceAttributes device_attributes = 2

    List of devices that are locally accessible to the worker.

• rpc Enqueue (EnqueueRequest, EnqueueResponse)

  eager_service.proto:302

  This takes a list of Execute and DeleteTensorHandle operations and enqueues (in async mode) or executes (in sync mode) them on the remote server. All outputs of ops which were not explicitly deleted with DeleteTensorHandle entries will be assumed to be alive and are usable by future calls to Enqueue.

• rpc KeepAlive (KeepAliveRequest, KeepAliveResponse)

  eager_service.proto:339

  Contexts are always created with a deadline and no RPCs within a deadline will trigger a context garbage collection. KeepAlive calls can be used to delay this. It can also be used to validate the existence of a context ID on remote eager worker. If the context is on remote worker, return the same ID and the current context view ID. This is useful for checking if the remote worker (potentially with the same task name and hostname / port) is replaced with a new process.

  message KeepAliveRequest

  eager_service.proto:194

  • fixed64 context_id = 1

  message KeepAliveResponse

  eager_service.proto:198

  • fixed64 context_view_id = 1

    If the requested context_id is on the remote host, set the context view ID.

• rpc RunComponentFunction (RunComponentFunctionRequest, RunComponentFunctionResponse)

  eager_service.proto:329

  This takes an Eager operation and executes it in async mode on the remote server. Different from EnqueueRequest, ops/functions sent through this type of requests are allowed to execute in parallel and no ordering is preserved by RPC stream or executor. This request type should only be used for executing component functions. Ordering of component functions should be enforced by their corresponding main functions. The runtime ensures the following invarients for component functions (CFs) and their main functions (MFs): (1) MF1 -> MF2 ==> CF1 -> CF2 ("->" indicates order of execution); (2) MF1 || MF2 ==> CF1 || CF2 ("||" indicates possible parallel execution); (3) For CF1 and CF2 that come from the same MF, CF1 || CF2 For executing ops/main functions, use Enqueue or StreamingEnqueue instead for correct ordering.

  message RunComponentFunctionRequest

  eager_service.proto:179

  • fixed64 context_id = 1

  • optional Operation operation = 2

  • repeated int32 output_num = 3

    The output indices of its parent function.

  message RunComponentFunctionResponse

  eager_service.proto:188

  • repeated TensorShapeProto shape = 1

  • repeated TensorProto tensor = 2

• rpc StreamingEnqueue (stream EnqueueRequest, stream EnqueueResponse)

  eager_service.proto:310

  A streaming version of Enqueue. Current server implementation sends one response per received request. The benefit for using a streaming version is that subsequent requests can be sent without waiting for a response to the previous request. This synchronization is required in the regular Enqueue call because gRPC does not guarantee to preserve request order.

• rpc UpdateContext (UpdateContextRequest, UpdateContextResponse)

  eager_service.proto:295

  This updates the eager context on an existing worker when updating the set of servers in a distributed eager cluster.

  message UpdateContextRequest

  eager_service.proto:132

  • optional ServerDef server_def = 1

    Identifies the full cluster, and this particular worker's position within.

  • repeated DeviceAttributes cluster_device_attributes = 2

    Device attributes in the cluster. If this field is empty, it indicates that this is a simple update request that only increments the cluster view ID and does not require changes to the workers it connects to.

  • fixed64 context_id = 3

    The ID of the context to be updated. A context with the specified ID must already exist on the recepient server of this request.

  • fixed64 context_view_id = 4

    The view ID of the context, which should be contiguously incremented when updating the same context.

  message UpdateContextResponse

  eager_service.proto:151

  • repeated DeviceAttributes device_attributes = 1

    List of devices that are locally accessible to the worker.

• rpc WaitQueueDone (WaitQueueDoneRequest, WaitQueueDoneResponse)

  eager_service.proto:314

  Takes a set of op IDs and waits until those ops are done. Returns any error in the stream so far.

  message WaitQueueDoneRequest

  eager_service.proto:167

  • fixed64 context_id = 1

  • repeated int64 op_id = 2

    Ids to wait on. If empty, wait on everything currently pending.

  message WaitQueueDoneResponse

  eager_service.proto:174

  TODO(nareshmodi): Consider adding NodeExecStats here to be able to propagate some stats.

  (message has no fields)

Cleanup the step state of a multi-device function (e.g. tensors buffered by a `Send` op but not picked up by its corresponding `Recv` op).

Used in: QueueItem

• int64 step_id = 1

Used as request type in: EagerService.Enqueue, EagerService.StreamingEnqueue

• fixed64 context_id = 1

• repeated QueueItem queue = 3

Used as response type in: EagerService.Enqueue, EagerService.StreamingEnqueue

• repeated QueueResponse queue_response = 1

  A single operation response for every item in the request.

A proto representation of an eager operation.

Used in: QueueItem, RunComponentFunctionRequest

• int64 id = 1

  A unique identifier for the operation. Set by the client so that the client can uniquely identify the outputs of the scheduled operation. In the initial implementation, sending duplicate IDs has undefined behaviour, but additional constraints may be placed upon this in the future.

• string name = 2

• repeated Operation.Input op_inputs = 10

• repeated int64 control_op_ids = 4

  Control Operation IDs that will be respected when ops are re-ordered by async execution. If async execution (+ op re-ordering) is not enabled, this should have no effect.

• map<string, AttrValue> attrs = 5

• string device = 6

• bool is_component_function = 7

  Indicates whether the op is a component of a multi-device function.

• int64 func_step_id = 8

  Set when is_component_function is true. It's initially generated when we create an FunctionLibraryRuntime::Options (negative value) and used to create Rendezvous for function execution. All components of a multi-device function should use the same step id to make sure that they can communicate through Send/Recv ops.

• bool is_function = 9

  Indicates whether the op is a function.

Used in: Operation

• oneof item

  • RemoteTensorHandle remote_handle = 1

  • TensorProto tensor = 2

Used in: EnqueueRequest

• oneof item

  The remote executor should be able to handle either executing ops directly, or releasing any unused tensor handles, since the tensor lifetime is maintained by the client.

  • RemoteTensorHandle handle_to_decref = 1

  • Operation operation = 2

  • SendTensorOp send_tensor = 3

  • RegisterFunctionOp register_function = 4

    Takes a FunctionDef and makes it enqueable on the remote worker.

  • CleanupFunctionOp cleanup_function = 5

  • SyncRemoteExecutorForStream sync_remote_executor_for_stream = 6

    A remote executor is created to execute ops/functions asynchronously enqueued in streaming call. Request with this item type waits for pending nodes to finish on the remote executor and report status.

  • SendPackedHandleOp send_packed_handle = 7

Used in: EnqueueResponse

• repeated TensorShapeProto shape = 1

  `shape` and `tensor` cannot be set in the same response. Shapes of output tensors for creating remote TensorHandles.

• repeated string device = 3

  Optional. If set, represents the output devices of a function.

• repeated TensorProto tensor = 2

  Output tensors of a remote function. Set when Operation.id is invalid.

Used in: QueueItem

• optional FunctionDef function_def = 1

• bool is_component_function = 2

  If true, it means that function_def is produced by graph partition during multi-device function instantiation.

• optional FunctionDefLibrary library = 3

  All necessary FunctionDefs and GradientDefs to expand `function_def`. When is_component_function is true, `function_def` could be a nested function, since some nodes in its parent's function body could be replaced with a new function by the graph optimization passes. No need to add FunctionDefs here to the function cache in EagerContext since they won't be executed as KernelAndDevices.

Used in: Operation.Input, QueueItem, SendPackedHandleOp.Handle

• int64 op_id = 1

  The ID of the operation that produced this tensor.

• int32 output_num = 2

  The index into the outputs of the operation that produced this tensor.

• string device = 3

  Device where the tensor is located. Cannot be empty. For multi-device functions, it's the default device passed to placer.

• string op_device = 4

  Device of the operation producing this tensor. Can be empty if the operation producing this tensor is a multi-device function.

• DataType dtype = 5

  Tensor type.

• repeated ResourceDtypeAndShape resource_dtypes_and_shapes = 6

  Optional data types and shapes of a remote resource variable.

Used in: RemoteTensorHandle

• DataType dtype = 1

• optional TensorShapeProto shape = 2

Send a packed TensorHandle to a remote worker.

Used in: QueueItem

• int64 op_id = 1

  Op id of the remote packed TensorHandle.

• repeated SendPackedHandleOp.Handle handles = 2

• string device_name = 3

Used in: SendPackedHandleOp

• oneof item

  • LocalTensorHandle local_handle = 1

  • RemoteTensorHandle remote_handle = 2

Used in: Handle

• optional TensorProto tensor = 1

• string device = 2

  Device where the tensor is produced.

Used in: QueueItem

• int64 op_id = 1

  All remote tensors are identified by <Op ID, Output num>. To mimic this situation when directly sending tensors, we include an "artificial" op ID (which would have corresponded to the _Recv op when not using SendTensor).

• repeated TensorProto tensors = 2

  The index within the repeated field is the output number that will help uniquely identify (along with the above op_id) the particular tensor.

• string device_name = 3

  The device on which the tensors should be resident.

Used in: QueueItem

(message has no fields)