package protobuf

Get desktop application:
View/edit binary Protocol Buffers messages

A grpc service for implementing org.apache.nemo.runtime.common.message components using grpc

• rpc send (Message, Void)

  GrcpMessageService.proto:32

  See org.apache.nemo.runtime.common.message.MessageSender#send for more details

  message Void

  GrcpMessageService.proto:39

  (message has no fields)

• rpc request (Message, Message)

  GrcpMessageService.proto:35

  See org.apache.nemo.runtime.common.message.MessageSender#request for more details

Used in: Message

• required int64 requestId = 1

  To find the matching request msg

• required string blockId = 2

• optional BlockStateFromExecutor state = 3

• optional string ownerExecutorId = 4

Used in: Message

• required string executorId = 1

• required string blockId = 2

• required BlockStateFromExecutor state = 3

• required string location = 4

  The location of the block (e.g., executor id or remote store).

Used in: BlockLocationInfoMsg, BlockStateChangedMsg

• NOT_AVAILABLE = 0

• IN_PROGRESS = 1

• AVAILABLE = 2

Used in: BlockTransferContextDescriptor

• MEMORY = 0

• SER_MEMORY = 1

• LOCAL_FILE = 2

• REMOTE_FILE = 3

• required string blockId = 1

• required BlockStore blockStore = 2

• required string runtimeEdgeId = 3

• optional bytes keyRange = 4

• required string initiatorExecutorId = 1

• required int32 transferIndex = 2

• required ByteTransferDataDirection dataDirection = 3

• required bytes contextDescriptor = 4

• required bool isPipe = 5

Messages between Executors

Used in: ByteTransferContextSetupMessage

• INITIATOR_SENDS_DATA = 0

• INITIATOR_RECEIVES_DATA = 1

• required ClientToDriverMessageType type = 1

• optional LaunchDAGMessage launchDAG = 2

• optional NotificationMessage message = 3

Messages from client to driver

Used in: ClientToDriverMessage

• LaunchDAG = 0

• DriverShutdown = 1

• Notification = 2

Used in: Message

• required string executorId = 1

Used in: DriverToClientMessage, Message

• required string data = 1

• required DriverToClientMessageType type = 1

• optional DataCollectMessage dataCollected = 2

• optional OptimizationType optimizationType = 3

Messages from driver to client

Used in: DriverToClientMessage

• DriverStarted = 0

• DriverReady = 1

• DataCollected = 2

• ExecutionDone = 3

• LaunchOptimization = 4

Used in: Message

• required string executorId = 1

• required bytes exception = 2

Used in: Message

• required int64 requestId = 1

  To find the matching request msg

• required bytes variable = 2

Used in: ClientToDriverMessage

• required string dag = 1

• optional bytes broadcastVars = 2

Used as request type in: MessageService.request, MessageService.send

Used as response type in: MessageService.request

• required MessageType type = 1

• required int64 id = 2

• required string listenerId = 3

  The id of the message listener (handler).

• optional TaskStateChangedMsg taskStateChangedMsg = 4

• optional ScheduleTaskMsg scheduleTaskMsg = 5

• optional BlockStateChangedMsg blockStateChangedMsg = 6

• optional RunTimePassMessageMsg runTimePassMessageMsg = 7

• optional RequestBlockLocationMsg requestBlockLocationMsg = 8

• optional BlockLocationInfoMsg blockLocationInfoMsg = 9

• optional ExecutorFailedMsg executorFailedMsg = 10

• optional ContainerFailedMsg containerFailedMsg = 11

• optional MetricMsg metricMsg = 12

• optional DataCollectMessage dataCollected = 13

• optional RequestBroadcastVariableMessage requestbroadcastVariableMsg = 14

• optional InMasterBroadcastVariableMessage broadcastVariableMsg = 15

• optional PipeInitMessage pipeInitMsg = 16

• optional RequestPipeLocationMessage requestPipeLocMsg = 17

• optional PipeLocationInfoMessage pipeLocInfoMsg = 18

Used in: Message

• TaskStateChanged = 0

• ScheduleTask = 1

• BlockStateChanged = 2

• RunTimePassMessage = 3

• RequestBlockLocation = 4

• BlockLocationInfo = 5

• ExecutorFailed = 6

• ExecutorDataCollected = 7

• MetricMessageReceived = 8

• RequestMetricFlush = 9

• MetricFlushed = 10

• RequestBroadcastVariable = 11

• InMasterBroadcastVariable = 12

• PipeInit = 13

• RequestPipeLoc = 14

• PipeLocInfo = 15

Common messages

Used in: MetricMsg

• required string metricType = 1

• required string metricId = 2

• required string metricField = 3

• required bytes metricValue = 4

Used in: Message

• repeated Metric metric = 1

Used in: ClientToDriverMessage

• required OptimizationType optimizationType = 1

• required string data = 2

Used in: DriverToClientMessage, NotificationMessage

• XGBoost = 0

Used in: Message

• required int64 srcTaskIndex = 1

• required string runtimeEdgeId = 2

• required string executorId = 3

Used in: Message

• required int64 requestId = 1

  To find the matching request msg

• required string executorId = 2

• required int64 srcTaskIndex = 1

• required string runtimeEdgeId = 2

• required int64 dstTaskIndex = 3

• required int64 numPipeToWait = 4

Used in: TaskStateChangedMsg

• InputReadFailure = 0

• OutputWriteFailure = 1

  There is a 3rd cause: container_failure, but this is omitted here as it is never propagated with a control msg.

Used in: Message

• required string executorId = 1

• required string blockIdWildcard = 2

Used in: Message

• required string executorId = 1

• required bytes broadcastId = 2

Used in: Message

• required string executorId = 1

• required int64 srcTaskIndex = 2

• required string runtimeEdgeId = 3

Used in: RunTimePassMessageMsg

• required string key = 1

• required int64 value = 2

Used in: Message

• required string taskId = 1

• repeated RunTimePassMessageEntry entry = 2

Messages from Master to Executors

Used in: Message

• required bytes task = 1

Messages from Executors to Master

Used in: Message

• required string executorId = 1

• required string taskId = 2

• required TaskStateFromExecutor state = 3

• optional string vertexPutOnHoldId = 4

• optional RecoverableFailureCause failureCause = 5

• required int32 attemptIdx = 6

Used in: TaskStateChangedMsg

• READY = 0

• EXECUTING = 1

• COMPLETE = 2

• FAILED_RECOVERABLE = 3

• FAILED_UNRECOVERABLE = 4

• ON_HOLD = 5