package google.devtools.remoteworkers.v1test2
Get desktop application:
View/edit binary Protocol Buffers messages
service Bots
bots.proto:64Design doc: https://goo.gl/oojM5H Loosely speaking, the Bots interface monitors a collection of workers (think of them as "computers" for a moment). This collection is known as a "farm," and its purpose is to perform work on behalf of a client. Each worker runs a small program known as a "bot" that allows it to be controlled by the server. This interface contains only methods that are called by the bots themselves; admin functionality is out of scope for this interface. More precisely, we use the term "worker" to refer to the physical "thing" running the bot. We use the term "worker," and not "machine" or "computer," since a worker may consist of more than one machine - e.g., a computer with multiple attached devices, or even a cluster of computers, with only one of them running the bot. Conversely, a single machine may host several bots, in which case each bot has a "worker" corresponding to the slice of the machine being managed by that bot. The main resource in the Bots interface is not, surprisingly, a Bot - it is a BotSession, which represents a period of time in which a bot is in continuous contact with the server (see the BotSession message for more information). The parent of a bot session can be thought of as an instance of a farm. That is, one endpoint may be able to manage many farms for many users. For example, for a farm managed through GCP, the parent resource will typically take the form "projects/{project_id}". This is referred to below as "the farm resource."
rpc CreateBotSession (, )
bots.proto:70CreateBotSession is called when the bot first joins the farm, and establishes a session ID to ensure that multiple machines do not register using the same name accidentally.
message
bots.proto:314Request message for CreateBotSession.
string parent = 1
Required. The farm resource.
optional bot_session = 2
Required. The bot session to create. Server-assigned fields like name must be unset.
rpc UpdateBotSession (, )
bots.proto:81UpdateBotSession must be called periodically by the bot (on a schedule determined by the server) to let the server know about its status, and to pick up new lease requests from the server.
message
bots.proto:323Request message for UpdateBotSession.
string name = 1
Required. The bot session name. Must match bot_session.name.
optional bot_session = 2
Required. The bot session resource to update.
optional update_mask = 3
Required. The fields on the bot that should be updated. See the BotSession resource for which fields are updatable by which caller.
service Tasks
tasks.proto:38DEPRECATED. GetTask should be replaced by Lease.payload, UpdateTaskResult by Lease.result and logs should be precreated prior to sending to the bot (eg, via CommandTask.expected_outputs.stdout_destination).
rpc GetTask (, )
tasks.proto:45DEPRECATED - use Lease.payload instead. GetTask reads the current state of the task. Tasks must be created through some other interface, and should be immutable once created and exposed to the bots.
message
tasks.proto:142Request message for `GetTask`.
string name = 1
Required. The task name.
message
tasks.proto:86DEPRECATED - use Lease.payload instead. A Task represents a unit of work. Its result and logs are defined as subresources. If all the `Any` fields are populated, this can be a very large message, and clients may not want the entire message returned on every call to every method. Such clients should request partial responses (https://cloud.google.com/apis/design/design_patterns#partial_response) and servers should implement partial responses in order to reduce unnecessry overhead.
string name = 1
The name of this task. Output only.
optional description = 2
The actual task to perform. For example, this could be CommandTask to run a command line.
map<string, string> logs = 3
Handles to logs. The key is a human-readable name like `stdout`, and the handle is a resource name that can be passed to ByteStream or other accessors. An implementation may define some logs by default (like `stdout`), and may allow clients to add new logs via AddTaskLog.
rpc UpdateTaskResult (, )
tasks.proto:54DEPRECATED - use Lease.result instead. UpdateTaskResult updates the result.
message
tasks.proto:153Request message for `UpdateTaskResult`.
string name = 1
Required. The task result name; must match `result.name`.
optional result = 2
Required. The result being updated.
optional update_mask = 3
Required. The fields within `result` that are specified.
string source = 4
Required. If this is being updated by a bot from BotManager, the source should be bot.session_id. That way, if two bots accidentally get the same name, we'll know to reject updates from the older one.
rpc AddTaskLog (, )
tasks.proto:67DEPRECATED - precreate logs prior to sending to bot. AddTaskLog creates a new streaming log. The log is streamed and marked as completed through other interfaces (i.e., ByteStream). This can be called by the bot if it wants to create a new log; the server can also predefine logs that do not need to be created (e.g. `stdout`).
message
tasks.proto:175Request message for `AddTaskLog`.
string name = 1
Required. The name of the task that will own the new log.
string log_id = 2
Required. The human-readable name of the log, like `stdout` or a relative file path.
message
tasks.proto:189Response message for `AddTaskLog`.
string handle = 1
The handle for the new log, as would be returned in Task.logs.
message
bots.proto:226AdminTemp is a prelimiary set of administration tasks. It's called "Temp" because we do not yet know the best way to represent admin tasks; it's possible that this will be entirely replaced in later versions of this API. If this message proves to be sufficient, it will be renamed in the alpha or beta release of this API. This message (suitably marshalled into a protobuf.Any) can be used as the inline_assignment field in a lease; the lease assignment field should simply be `"admin"` in these cases. This message is heavily based on Swarming administration tasks from the LUCI project (http://github.com/luci/luci-py/appengine/swarming).
AdminTemp.Command command = 1
The admin action; see `Command` for legal values.
string arg = 2
The argument to the admin action; see `Command` for semantics.
Possible administration actions.
Used in:
UNSPECIFIED = 0
Illegal value.
BOT_UPDATE = 1
Download and run a new version of the bot. `arg` will be a resource accessible via `ByteStream.Read` to obtain the new bot code.
BOT_RESTART = 2
Restart the bot without downloading a new version. `arg` will be a message to log.
BOT_TERMINATE = 3
Shut down the bot. `arg` will be a task resource name (similar to those in tasks.proto) that the bot can use to tell the server that it is terminating.
HOST_RESTART = 4
Restart the host computer. `arg` will be a message to log.
message
command.proto:264Describes a blob of binary content with its digest.
Used in:
optional digest = 1
The digest of the blob. This should be verified by the receiver.
bytes contents = 2
The contents of the blob.
message
bots.proto:97A bot session represents the state of a bot while in continuous contact with the server for a period of time. The session includes information about the worker - that is, the *worker* (the physical or virtual hardware) is considered to be a property of the bot (the software agent running on that hardware), which is the reverse of real life, but more natural from the point of the view of this API, which communicates solely with the bot and not directly with the underlying worker.
Used as response type in: Bots.CreateBotSession, Bots.UpdateBotSession
Used as field type in: ,
string name = 1
The bot session name, as selected by the server. Output only during a call to CreateBotSession.
string bot_id = 2
A unique bot ID within the farm used to persistently identify this bot over time (i.e., over multiple sessions). This ID must be unique within a farm. Typically, the bot ID will be the same as the name of the primary device in the worker (e.g., what you'd get from typing `uname -n` on *nix), but this is not required since a single device may allow multiple bots to run on it, each with access to different resources. What is important is that this ID is meaningful to humans, who might need to hunt a physical machine down to fix it. When CreateBotSession is successfully called with a bot_id, all prior sessions with the same ID are invalidated. If a bot attempts to update an invalid session, the server must reject that request, and may also quarantine the other bot with the same bot IDs (ie, stop sending it new leases and alert an admin).
BotStatus status = 3
The status of the bot. This must be populated in every call to UpdateBotSession.
optional worker = 4
A description of the worker hosting this bot. The Worker message is used here in the Status context (see Worker for more information). If multiple bots are running on the worker, this field should only describe the resources accessible from this bot. During the call to CreateBotSession, the server may make arbitrary changes to the worker's `server_properties` field (see that field for more information). Otherwise, this field is input-only.
repeated leases = 5
A list of all leases that are a part of this session. See the Lease message for details.
optional expire_time = 6
The time at which this bot session will expire, unless the bot calls UpdateBotSession again. Output only.
string version = 7
The version of the bot code currently running. The server may use this information to issue an admin action to tell the bot to update itself.
A coarse description of the status of the bot that the server uses to determine whether to assign the bot new leases.
Used in:
BOT_STATUS_UNSPECIFIED = 0
Default value; do not use.
OK = 1
The bot is healthy, and will accept leases as normal.
UNHEALTHY = 2
The bot is unhealthy and will not accept new leases. For example, the bot may have detected that available disk space is too low. This situation may resolve itself, but will typically require human intervention.
HOST_REBOOTING = 3
The bot has been asked to reboot the host. The bot will not accept new leases; once all leases are complete, this session will no longer be updated but the bot will be expected to establish a new session after the reboot completes.
BOT_TERMINATING = 4
The bot has been asked to shut down. As with HOST_REBOOTING, once all leases are completed, the session will no longer be updated and the bot will not be expected to establish a new session. Bots are typically only asked to shut down if its host computer will be modified in some way, such as deleting a VM.
INITIALIZING = 5
The bot is initializing and is not ready to accept leases.
message
command.proto:145DEPRECATED - use CommandResult instead. Describes the actual outputs from the task.
int32 exit_code = 1
exit_code is only fully reliable if the status' code is OK. If the task exceeded its deadline or was cancelled, the process may still produce an exit code as it is cancelled, and this will be populated, but a successful (zero) is unlikely to be correct unless the status code is OK.
optional outputs = 2
The output files. The blob referenced by the digest should contain one of the following (implementation-dependent): * A marshalled DirectoryMetadata of the returned filesystem * A LUCI-style .isolated file
message
command.proto:162DEPRECATED - use CommandResult instead. Can be used as part of CompleteRequest.metadata, or are part of a more sophisticated message.
optional duration = 1
The elapsed time between calling Accept and Complete. The server will also have its own idea of what this should be, but this excludes the overhead of the RPCs and the bot response time.
optional overhead = 2
The amount of time *not* spent executing the command (ie uploading/downloading files).
message
command.proto:175All information about the execution of a command, suitable for providing as the Bots interface's `Lease.result` field.
optional status = 1
An overall status for the command. For example, if the command timed out, this might have a code of DEADLINE_EXCEEDED; if it was killed by the OS for memory exhaustion, it might have a code of RESOURCE_EXHAUSTED.
int32 exit_code = 2
The exit code of the process. An exit code of "0" should only be trusted if `status` has a code of OK (otherwise it may simply be unset).
optional outputs = 3
The output files. The blob referenced by the digest should contain one of the following (implementation-dependent): * A marshalled DirectoryMetadata of the returned filesystem * A LUCI-style .isolated file
optional duration = 4
The elapsed time between calling Accept and Complete. The server will also have its own idea of what this should be, but this excludes the overhead of the RPCs and the bot response time.
optional overhead = 5
The amount of time *not* spent executing the command (ie uploading/downloading files).
repeated metadata = 6
Implementation-dependent metadata about the task. Both servers and bots may define messages which can be encoded here; bots are free to provide metadata in multiple formats, and servers are free to choose one or more of the values to process and ignore others. In particular, it is *not* considered an error for the bot to provide the server with a field that it doesn't know about.
message
command.proto:33Describes a shell-style task to execute, suitable for providing as the Bots interface's `Lease.payload` field.
optional inputs = 1
The inputs to the task.
optional expected_outputs = 4
The expected outputs from the task.
optional timeouts = 5
The timeouts of this task.
message
command.proto:35Describes the inputs to a shell-style task.
Used in:
repeated string arguments = 1
The command itself to run (e.g., argv). This field should be passed directly to the underlying operating system, and so it must be sensible to that operating system. For example, on Windows, the first argument might be "C:\Windows\System32\ping.exe" - that is, using drive letters and backslashes. A command for a *nix system, on the other hand, would use forward slashes. All other fields in the RWAPI must consistently use forward slashes, since those fields may be interpretted by both the service and the bot.
repeated files = 2
The input filesystem to be set up prior to the task beginning. The contents should be a repeated set of FileMetadata messages though other formats are allowed if better for the implementation (eg, a LUCI-style .isolated file). This field is repeated since implementations might want to cache the metadata, in which case it may be useful to break up portions of the filesystem that change frequently (eg, specific input files) from those that don't (eg, standard header files).
repeated inline_blobs = 4
Inline contents for blobs expected to be needed by the bot to execute the task. For example, contents of entries in `files` or blobs that are indirectly referenced by an entry there. The bot should check against this list before downloading required task inputs to reduce the number of communications between itself and the remote CAS server.
repeated environment_variables = 3
All environment variables required by the task.
string working_directory = 5
Directory from which a command is executed. It is a relative directory with respect to the bot's working directory (i.e., "./"). If it is non-empty, then it must exist under "./". Otherwise, "./" will be used.
message
command.proto:37An environment variable required by this task.
Used in:
string name = 1
The envvar name.
string value = 2
The envvar value.
message
command.proto:87Describes the expected outputs of the command.
Used in:
repeated string files = 1
A list of expected files, relative to the execution root. All paths MUST be delimited by forward slashes.
repeated string directories = 2
A list of expected directories, relative to the execution root. All paths MUST be delimited by forward slashes.
string stdout_destination = 3
The destination to which any stdout should be sent. The method by which the bot should send the stream contents to that destination is not defined in this API. As examples, the destination could be a file referenced in the `files` field in this message, or it could be a URI that must be written via the ByteStream API.
string stderr_destination = 4
The destination to which any stderr should be sent. The method by which the bot should send the stream contents to that destination is not defined in this API. As examples, the destination could be a file referenced in the `files` field in this message, or it could be a URI that must be written via the ByteStream API.
message
command.proto:112Describes the timeouts associated with this task.
Used in:
optional execution = 1
This specifies the maximum time that the task can run, excluding the time required to download inputs or upload outputs. That is, the worker will terminate the task if it runs longer than this.
optional idle = 2
This specifies the maximum amount of time the task can be idle - that is, go without generating some output in either stdout or stderr. If the process is silent for more than the specified time, the worker will terminate the task.
optional shutdown = 3
If the execution or IO timeouts are exceeded, the worker will try to gracefully terminate the task and return any existing logs. However, tasks may be hard-frozen in which case this process will fail. This timeout specifies how long to wait for a terminated task to shut down gracefully (e.g. via SIGTERM) before we bring down the hammer (e.g. SIGKILL on *nix, CTRL_BREAK_EVENT on Windows).
message
worker.proto:133Any device, including computers, phones, accelerators (e.g. GPUs), etc. All names must be unique.
Used in:
string handle = 1
The handle can be thought of as the "name" of the device, and must be unique within a Worker. In the Status context, the handle should be some human-understandable name, perhaps corresponding to a label physically written on the device to make it easy to locate. In the Request context, the name should be the *logical* name expected by the task. The bot is responsible for mapping the logical name expected by the task to a machine-readable name that the task can actually use, such as a USB address. The method by which this mapping is communicated to the task is not covered in this API.
repeated properties = 2
Properties of this device that don't change based on the tasks that are running on it, e.g. OS, CPU architecture, etc. Keys may be repeated, and have the following interpretation: * Status context: the device can support *any* the listed values. For example, an "ISA" property might include "x86", "x86-64" and "sse4". * Request context: the device *must* support *all* of the listed values.
message
worker.proto:135A device property; see `properties` for more information.
Used in:
string key = 1
For general information on keys, see the documentation to `Worker`. The current set of standard keys are: * os: a human-readable description of the OS. Examples include `linux`, `ubuntu` and `ubuntu 14.04` (note that a bot may advertise itself as more than one). This will be replaced in the future by more well-structured keys and values to represent OS variants. * has-docker: "true" if the bot has Docker installed. This will be replaced in the future by a more structured message for Docker support.
string value = 2
The property's value.
message
command.proto:251The CommandTask and CommandResult messages assume the existence of a service that can serve blobs of content, identified by a hash and size known as a "digest." The method by which these blobs may be retrieved is not specified here, but a model implementation is in the Remote Execution API's "ContentAddressibleStorage" interface. In the context of the RWAPI, a Digest will virtually always refer to the contents of a file or a directory. The latter is represented by the byte-encoded Directory message.
Used in: , , , , ,
string hash = 1
A string-encoded hash (eg "1a2b3c", not the byte array [0x1a, 0x2b, 0x3c]) using an implementation-defined hash algorithm (eg SHA-256).
int64 size_bytes = 2
The size of the contents. While this is not strictly required as part of an identifier (after all, any given hash will have exactly one canonical size), it's useful in almost all cases when one might want to send or retrieve blobs of content and is included here for this reason.
message
command.proto:274The contents of a directory. Similar to the equivalent message in the Remote Execution API.
repeated files = 1
The files in this directory
repeated directories = 2
Any subdirectories
message
command.proto:233The metadata for a directory. Similar to the equivalent message in the Remote Execution API.
Used in:
string path = 1
The path of the directory, as in [FileMetadata.path][google.devtools.remoteworkers.v1test2.FileMetadata.path].
optional digest = 2
A pointer to the contents of the directory, in the form of a marshalled Directory message.
message
command.proto:211The metadata for a file. Similar to the equivalent message in the Remote Execution API.
Used in:
string path = 1
The path of this file. If this message is part of the CommandOutputs.outputs fields, the path is relative to the execution root and must correspond to an entry in CommandTask.outputs.files. If this message is part of a Directory message, then the path is relative to the root of that directory. All paths MUST be delimited by forward slashes.
optional digest = 2
A pointer to the contents of the file. The method by which a client retrieves the contents from a CAS system is not defined here.
bytes contents = 3
If the file is small enough, its contents may also or alternatively be listed here.
bool is_executable = 4
Properties of the file
message
bots.proto:168A Lease is a lease that the scheduler has assigned to this bot. If the bot notices (by UpdateBotSession) that it has any leases in the PENDING state, it should call UpdateBotSession to put the leases into the ACTIVE state and start executing their assignments. All fields in this message are output-only, *except* the `state` and `status` fields. Note that repeated fields can only be updated as a unit, so on every update the bot must provide an update for *all* the leases the server expects it to report on. The scheduler *should* ensure that all leases scheduled to a bot can actually be accepted, but race conditions may occur. In such cases, the bot should attempt to accept the leases in the order they are listed by the server, to allow the server to control priorities. The server will remove COMPLETED leases from time to time, after which the bot shouldn't report on them any more (the server will ignore superfluous COMPLETED records).
Used in:
string id = 7
A short string uniquely identifing the lease within this bot session.
optional payload = 8
The actual work to be performed, if any. May be omitted by the server if the lease is not in the `PENDING` state. The message must be meaningful to the bot. Output only (must only be set by the server).
optional result = 9
Any result the bot wishes to provide about the lease. Must not be changed after the first call with the lease in the `COMPLETED` or `CANCELLED` state. Input only (must only be set by the bot, will not be echoed by the server).
LeaseState state = 2
The state of the lease. See LeaseState for more information.
optional status = 3
The final status of the lease (should be populated by the bot if the state is completed). This is the status of the lease, not of any task represented by the lease. For example, if the bot could not accept the lease because it asked for some resource the bot didn't have, this status will be FAILED_PRECONDITION. But if the assignment in the lease didn't execute correctly, this field will be `OK` while the failure of the assignment must communicated via the `result` field.
optional requirements = 4
The requirements that are being claimed by this lease. This field may be omitted by the server if the lease is not pending.
optional expire_time = 5
The time at which this lease expires. The server *may* extend this over time, but due to race conditions, the bot is not *required* to respect any expiry date except the first one.
string assignment = 1
DEPRECATED. The assignment should be provided to the bot via the `payload` field. Clients that wish to use a simple name (such as a queue of work provided elsewhere) should define a custom message type and encode it into `payload`.
optional inline_assignment = 6
DEPRECATED. Use `payload` instead.
The state of the lease. All leases start in the PENDING state. A bot can change PENDING to ACTIVE or (in the case of an error) COMPLETED, or from ACTIVE to COMPLETED. The server can change PENDING or ACTIVE to CANCELLED if it wants the bot to release its resources - for example, if the bot needs to be quarantined (it's producing bad output) or a cell needs to be drained.
Used in:
LEASE_STATE_UNSPECIFIED = 0
Default value; do not use.
PENDING = 1
Pending: the server expects the bot to accept this lease. This may only be set by the server.
ACTIVE = 2
Active: the bot has accepted this lease. This may only be set by the bot.
COMPLETED = 4
Completed: the bot is no longer leased. This may only be set by the bot, and the status field must be populated iff the state is COMPLETED.
CANCELLED = 5
Cancelled: The bot should immediately release all resources associated with the lease. This may only be set by the server.
message
tasks.proto:110DEPRECATED - use Lease.assignment_result instead. The result and metadata of the task.
Used as response type in: Tasks.UpdateTaskResult
Used as field type in:
string name = 1
bool complete = 2
The result may be updated several times; the client must only set `complete` to true to indicate that no further updates are allowed. If this is not true, the `status` field must not be examined since its zero value is equivalent to `OK`. Once a task is completed, it must not be updated with further results, though the implementation may choose to continue to receive logs.
optional status = 3
The final status of the task itself. For example, if task.description included a timeout which was violated, status.code may be DEADLINE_EXCEEDED. This field can only be read if `complete` is true.
optional output = 4
Any non-log output, such as output files and exit codes. See CommandResult as an example.
optional meta = 5
Any information about how the command was executed, eg runtime. See CommandOverhead as an example.
message
worker.proto:62Describes a worker, which is a list of one or more devices and the connections between them. A device could be a computer, a phone, or even an accelerator like a GPU; it's up to the farm administrator to decide how to model their farm. For example, if a farm only has one type of GPU, the GPU could be modelled as a "has_gpu" property on its host computer; if it has many subproperties itself, it might be better to model it as a separate device. The first device in the worker is the "primary device" - that is, the device running a bot and which is responsible for actually executing commands. All other devices are considered to be attached devices, and must be controllable by the primary device. This message (and all its submessages) can be used in two contexts: * Status: sent by the bot to report the current capabilities of the device to allow reservation matching. * Request: sent by a client to request a device with certain capabilities in a reservation. Several of the fields in this message have different semantics depending on which of which of these contexts it is used. These semantics are described below. Several messages in Worker and its submessages have the concept of keys and values, such as `Worker.Property` and `Device.Property`. All keys are simple strings, but certain keys are "standard" keys and should be broadly supported across farms and implementations; these are listed below each relevant message. Bot implementations or farm admins may add *additional* keys, but these SHOULD all begin with an underscore so they do not conflict with standard keys that may be added in the future. Keys are not context sensitive. See http://goo.gl/NurY8g for more information on the Worker message.
Used in: ,
repeated devices = 1
A list of devices; the first device is the primary device. See the `Device` message for more information.
repeated properties = 2
A worker may contain "global" properties. For example, certain machines might be reserved for certain types of jobs, like short-running compilation versus long-running integration tests. This property is known as a "pool" and is not related to any one device within the worker; rather, it applies to the worker as a whole. The behaviour of repeated keys is identical to that of Device.Property.
repeated configs = 3
Bots can be configured in certain ways when accepting leases. For example, many leases are executed inside a Docker container. To support this, the bot needs to be able to report that it has Docker installed (and knows how to execute something inside a container), and the task submitter needs to specify which image should be used to start the container. Similarly, a lease may be able to run as one of several users on the worker; in such cases, the bot needs to report what users are available, and the submitter needs to choose one. Therefore, when this message is reported by the bot to the service, each key represents a *type* of configuration that the bot knows how to set, while each *value* represents a legal value for that configuration (the empty string is interpretted as a wildcard, such as for Docker images). When this message is sent by the server to the bot in the context of a lease, it represents a command to the bot to apply the setting. Keys may be repeated during reporting but not in a lease.
message
worker.proto:83A configuration request or report; see the `configs` field for more information.
Used in:
string key = 1
For general information on keys, see the documentation to `Worker`. The current set of standard keys are: * DockerImage: the image of the container. When being reported by the bot, the empty value should always be included if the bot is able to pull its own images; the bot may optionally *also* report images that are present in its cache. When being requested in a lease, the value is the URI of the image (eg `gcr.io/user/image@sha256:hash`).
string value = 2
The configuration's value.
message
worker.proto:64A global property; see the `properties` field for more information.
Used in:
string key = 1
For general information on keys, see the documentation to `Worker`. The current set of standard keys are: * pool: different workers can be reserved for different purposes. For example, an admin might want to segregate long-running integration tests from short-running unit tests, so unit tests will always get some throughput. To support this, the server can assign different values for `pool` (such as "itest" and "utest") to different workers, and then have jobs request workers from those pools.
string value = 2
The property's value.