Default package

Get desktop application:
View/edit binary Protocol Buffers messages

Used in: KlioJobAuditLogItem, KlioMessage.Metadata, KlioMessage.Metadata.Recipients.Limited

• string job_name = 1

  required

• string gcp_project = 2

  required for Dataflow (job_name + gcp_project ensures unique names)

• repeated KlioJob.JobInput inputs = 3

  marked for deprecation for v2

marked for deprecation for v2

Used in: KlioJob

• string topic = 1

• string subscription = 2

• string data_location = 3

Used in: KlioMessage.Metadata

• optional google.protobuf.Timestamp timestamp = 1

  required - timestamp of when the log item was created

• optional KlioJob klio_job = 2

  required

• optional KlioMessage.Metadata metadata = 1

• optional KlioMessage.Data data = 2

• Version version = 3

  message version

Used in: KlioMessage

• string entity_id = 1

  required for v1 messages

• bytes payload = 2

  optional - (v1) used for non-klio messages, (v2) used for intra-pipeline state

• bytes element = 3

  required for v2 messages

Used in: KlioMessage

• repeated KlioJob downstream = 1

  marked for deprecation for v2

• repeated KlioJob visited = 2

  optional - unique jobs this message has visited

• repeated KlioJobAuditLogItem job_audit_log = 3

  optional - audit log for all jobs message has visited (complete)

• bool force = 4

  optional - force computation even if output exists; default = false

• bool ping = 5

  optional - if true, then no transformation work is done; default = false

• optional Metadata.Recipients intended_recipients = 6

  required for V2 messages

Used in: Metadata

• oneof recipients

  • Recipients.Anyone anyone = 1

  • Recipients.Limited limited = 2

FYI: since this is an empty field, intended_recipients.anyone can't simply be assigned. To set `anyone` as the intended_recipients in Python, use kmsg.metadata.intended_recipients.anyone.SetInParent()` https://stackoverflow.com/a/29651069

Used in: Recipients

(message has no fields)

Used in: Recipients

• repeated KlioJob recipients = 1

  "recipients" is effectively what "downstream" is in v1 but fixes some issues. If a user only wants to trigger a particular job w/o the child jobs, the publisher of the message will need to add the job's name into recipients (like how `klio message publish --bottom-up` does). In that case, if there is no input, then the job will add its parent job to this recipients list, then publish to parent. It would not set `trigger_children_of`. Otherwise, if the message first starts in top-down, and input is missing, the job will add itself + its parent to recipients **as well as** assign `trigger_children_of` to itself. That would signify to other child jobs of the parent job to not do any work and ignore/drop the message.

• optional KlioJob trigger_children_of = 2

  If this isn't set, then the whole graph execution mode is bottom-up. If set, then when the message gets to this job, it should update `metadata.recipients` to `anyone` (clearing out `limited`) so that when it publishes the output message, all children treat it as top-down

Version maps klio config version to message version when actually handling/parsing klio messages

Used in: KlioMessage

• UNKNOWN = 0

  default (per proto3 requirements)

• V1 = 1

  for entity_id+payload

• V2 = 2

  for element+payload