package redpanda.runtime.v1alpha1
Get desktop application:
View/edit binary Protocol Buffers messages
service AgentRuntime
agent.proto:56`AgentRuntime` is the service that provides the ability to invoke an agent.
rpc InvokeAgent (, )
agent.proto:57message
agent.proto:42InvokeAgentRequest is the request message for the `InvokeAgent` method.
optional message = 1
optional trace_context = 2
message
agent.proto:49InvokeAgentResponse is the response message for the `InvokeAgent` method.
optional message = 1
optional trace = 2
service BatchInputService
input.proto:29BatchInput is an interface implemented by Benthos inputs that produce messages in batches, where there is a desire to process and send the batch as a logical group rather than as individual messages. Calls to ReadBatch should block until either a message batch is ready to process, the connection is lost, or the RPC deadline is reached.
rpc Ack (, )
input.proto:67Acknowledge a message batch. This function ensures that the source of the message receives either an acknowledgement (error is missing) or an error that can either be propagated upstream as a nack, or trigger a reattempt at delivering the same message. If your input implementation doesn't have a specific mechanism for dealing with a nack then you can wrap your input implementation with AutoRetryNacks to get automatic retries, and noop this function.
message
input.proto:106uint64 batch_id = 1
The ID of the batch.
optional error = 2
If present, then this is a nack request. If auto_replay_nacks is enabled in the InitResponse, then this should never be present.
message
input.proto:114optional error = 2
If present, then this ack/nack request failed.
rpc Close (, )
input.proto:70Close the component, blocks until either the underlying resources are cleaned up or the RPC deadline is reached.
message
input.proto:119(message has no fields)
message
input.proto:120optional error = 1
If present, then the close attempt failed.
rpc Connect (, )
input.proto:42Establish a connection to the upstream service. Connect will always be called first when a reader is instantiated, and will be continuously called with back off until a nil error is returned. Once Connect returns a nil error the Read method will be called until either ErrNotConnected is returned, or the reader is closed.
message
input.proto:88(message has no fields)
message
input.proto:89optional error = 1
If present, then the connect attempt failed.
rpc Init (, )
input.proto:35Init is the first method called for a batch input and it passes the user's configuration to the input. The schema for the input configuration is specified in the `plugin.yaml` file provided to Redpanda Connect.
message
input.proto:73optional config = 1
The parsed configuration from the user based on the registered schema in `plugin.yaml`.
message
input.proto:78optional error = 1
If present, then the input configuration is invalid and an error should be surfaced at pipeline construction time.
bool auto_replay_nacks = 2
If true, then any nacks are automatically retried. This is useful for inputs that don't have a mechanism for dealing with nacks, and want to just automatically retry them until they succeed.
rpc ReadBatch (, )
input.proto:58Read a message batch from a source, along with a function to be called once the entire batch can be either acked (successfully sent or intentionally filtered) or nacked (failed to be processed or dispatched to the output). The Ack will be called for every message batch at least once, but there are no guarantees as to when this will occur. If your input implementation doesn't have a specific mechanism for dealing with a nack then you can instruct the Connect framework to auto_replay_nacks in the InitResponse to get automatic retries. If this method returns Error.NotConnected then ReadBatch will not be called again until Connect has returned a nil error. If Error.EndOfInput is returned then Read will no longer be called and the pipeline will gracefully terminate.
message
input.proto:94(message has no fields)
message
input.proto:95uint64 batch_id = 1
The ID of the batch, which is used in the ack request to identify the batch used. These IDs are opaque to the connect framework but IDs should be unique per process.
optional batch = 2
The batch of messages to be processed.
optional error = 3
If present, then there was an error reading messages.
BatchOutput is an interface implemented by Benthos outputs that require Benthos to batch messages before dispatch in order to improve throughput. Each call to WriteBatch should block until either all messages in the batch have been successfully or unsuccessfully sent, or the RPC deadline is reached. Multiple write calls can be performed in parallel, and the constructor of an output must provide a MaxInFlight parameter indicating the maximum number of parallel batched write calls the output supports.
rpc Close (, )
output.proto:53Close the component, blocks until either the underlying resources are cleaned up or the RPC deadline is reached.
message
output.proto:100(message has no fields)
message
output.proto:101optional error = 1
If present, then the close attempt failed.
rpc Connect (, )
output.proto:44Establish a connection to the downstream service. Connect will always be called first when a writer is instantiated, and will be continuously called with back off until a nil error is returned. Once Connect returns a nil error the write method will be called until either Error.NotConnected is returned, or the writer is closed.
message
output.proto:85(message has no fields)
message
output.proto:86optional error = 1
If present, then the connect attempt failed.
rpc Init (, )
output.proto:37Init is the first method called for a batch output and it passes the user's configuration to the output. The schema for the output configuration is specified in the `plugin.yaml` file provided to Redpanda Connect.
message
output.proto:67optional config = 1
The parsed configuration from the user based on the register schema in `plugin.yaml`.
message
output.proto:72optional error = 1
If present, then the input configuration is invalid and an error should be surfaced at pipeline construction time.
int32 max_in_flight = 2
The maximum number of write calls can be performed in parallel. Must be > 0.
optional batch_policy = 3
The batching policy for messages sent to this output. If omitted then no additional batching will be performed on top of the batches that already exist in the pipeline.
rpc Send (, )
output.proto:50Write a batch of messages to a sink, or return an error if delivery is not possible. If this method returns Error.NotConnected then write will not be called again until Connect has returned a nil error.
message
output.proto:91optional batch = 1
The batch of messages to send to the output
message
output.proto:95optional error = 1
If present, then the send attempt failed.
BatchProcessor is a Benthos processor implementation that works against batches of messages, which allows windowed processing. Message batches must be created by upstream components (inputs, buffers, etc) otherwise this processor will simply receive batches containing single messages.
rpc Close (, )
processor.proto:58Close the component, blocks until either the underlying resources are cleaned up or the RPC deadline is reached.
message
processor.proto:80(message has no fields)
message
processor.proto:81optional error = 1
If present, then the close attempt failed.
rpc Init (, )
processor.proto:35Init is the first method called for a batch processor and it passes the user's configuration to the input. The schema for the processor configuration is specified in the `plugin.yaml` file provided to Redpanda Connect.
message
processor.proto:61optional config = 1
message
processor.proto:62optional error = 1
If present, then the input configuration is invalid and an error should be surfaced at pipeline construction time.
rpc ProcessBatch (, )
processor.proto:54Process a batch of messages into one or more resulting batches, or return an error if the entire batch could not be processed. If zero messages are returned and the error is nil then all messages are filtered. The provided MessageBatch should NOT be modified, in order to return a mutated batch a copy of the slice should be created instead. When an error is returned all of the input messages will continue down the pipeline but will be marked with the error with *message.SetError, and metrics and logs will be emitted. In order to add errors to individual messages of the batch for downstream handling use message.SetError(err) and return it in the resulting batch with a nil error. The Message types returned MUST be derived from the provided messages, and CANNOT be custom instantiations of Message. In order to copy the provided messages use the Copy method.
message
processor.proto:68optional batch = 1
The input batch to the processor.
message
processor.proto:72repeated batches = 1
The resulting batch of messages. Returning multiple batches allows for splitting a single batch into multiple batches.
optional error = 2
If present, then the processing failed.
message
output.proto:60BatchPolicy describes the mechanisms by which batching should be performed of messages destined for a Batch output. This is returned by Init RPC of batch outputs.
Used in:
int64 byte_size = 1
int64 count = 2
string check = 3
string period = 4
message
message.proto:54An error in the context of a data pipeline.
Used in: , , , , , , , , , , , , ,
string message = 1
The error message. If non empty, then the error is valid and if empty the error is ignored as if a success (due to proto3 empty semantics).
oneof detail
Additional error details for specific Redpanda Connect behavior. If one of these fields is set, then message must be non-empty.
backoff = 2
BackOff is an error that plugins can optionally wrap another error with which instructs upstream components to wait for a specified period of time before retrying the errored call. Only supported by Connect methods in the Input and Output services.
not_connected = 3
end_of_input = 4
message
message.proto:68EndOfInput is returned by inputs that have exhausted their source of data to the point where subsequent Read calls will be ineffective. This error prompts the upstream component to gracefully terminate the pipeline.
Used in:
(message has no fields)
message
message.proto:63NotConnected is returned by inputs and outputs when their Read or Write methods are called and the connection that they maintain is lost. This error prompts the upstream component to call Connect until the connection is re-established.
Used in:
(message has no fields)
message
message.proto:35`ListValue` represents a list value which can be used to represent a list of values.
Used in:
repeated values = 1
message
message.proto:85Message represents a piece of data or an event that flows through the runtime.
Used in: , ,
oneof payload
bytes bytes = 1
structured = 2
optional metadata = 3
optional error = 4
message
message.proto:94Used in: , , ,
repeated messages = 1
`NullValue` is a representation of a null value.
Used in:
NULL_VALUE = 0
message
agent.proto:32Used in:
string span_id = 1
string name = 2
optional start_time = 3
optional end_time = 4
map<string, > attributes = 5
repeated child_spans = 6
message
message.proto:31`StructValue` represents a struct value which can be used to represent a structured data value.
Used in: ,
map<string, > fields = 1
message
agent.proto:30Used in:
repeated spans = 1
message
agent.proto:24Used in:
string trace_id = 1
string span_id = 2
string trace_flags = 4
message
message.proto:39`Value` represents a dynamically typed value which can be used to represent a value within a Redpanda Connect pipeline.
Used in: , , , , , ,
oneof kind
NullValue null_value = 1
string string_value = 2
int64 integer_value = 3
double double_value = 4
bool bool_value = 5
timestamp_value = 6
bytes bytes_value = 7
struct_value = 8
list_value = 9