package rerun.cloud.v1alpha1
Get desktop application:
View/edit binary Protocol Buffers messages
service RerunCloudService
cloud.proto:23The catalog server public API. ## Headers Most endpoints in the catalog server service require specific gRPC headers to be set. The so-called "standard dataset headers" correspond to at least one of the following headers: * x-rerun-entry-id: ID of the entry of interest, e.g. `1860390B087BC65F602d68eb646c385c`. * x-rerun-entry-name-bin: Name of the entry of interest, e.g. `droid:sample2k`. Headers with a -bin suffix must be base64-encoded (HTTP only supports ASCII values, UTF8 strings must binary encoded).
rpc CancelTasks (, )
cloud.proto:218Cancel existing tasks
message
cloud.proto:886`CancelTasksRequest` is the request message for cancelling a number of tasks
repeated ids = 1
Unique identifiers for the tasks
message
cloud.proto:892`CancelTasksResponse` is the response message for cancelling a number of tasks
(message has no fields)
rpc CreateDatasetEntry (, )
cloud.proto:53message
cloud.proto:940optional string name = 1
Name of the dataset entry to create. The name should be a short human-readable string. It must be unique within all entries in the catalog. If an entry with the same name already exists, the request will fail. Entry names ending with `__manifest` are reserved.
optional id = 2
If specified, create the entry using this specific ID. Use at your own risk.
message
cloud.proto:953optional dataset = 1
rpc CreateTableEntry (, )
cloud.proto:55message
cloud.proto:959string name = 1
Name of the dataset entry to create. The name should be a short human-readable string. It must be unique within all entries in the catalog. If an entry with the same name already exists, the request will fail. Entry names ending with `__manifest` are reserved.
optional provider_details = 2
Information about the table to register. If not provided, a Lance-backed table will be created using at an auto-generated URL based on the server's configuration. If provided, it must be an encoded message of the `LanceTable` type, which contains the intended storage URL. The caller is responsible for ensuring that the storage is writable and free to use.
optional schema = 3
Schema of the table to create
message
cloud.proto:979optional table = 1
rpc DeleteEntry (, )
cloud.proto:49message
cloud.proto:909DeleteDatasetEntry This endpoint requires the standard dataset headers.
optional id = 1
Deprecated: use x-rerun-entry-id header instead. This field should be dropped once all servers are updated to 0.14.x or later.
message
cloud.proto:916(message has no fields)
rpc DoBandwidthTest (, stream )
cloud.proto:41Returns a payload of pseudo-random (incompressible) bytes of the requested size. Intended for measuring round-trip time and bandwidth between client and server: * Send a few small requests (e.g. `num_bytes = 1`) to estimate RTT. * Send a larger request (e.g. `num_bytes = 1 MiB`), subtract the RTT, and divide `num_bytes` by the remaining time to estimate bandwidth. The response is streamed as one or more chunks; the total number of bytes across all chunks is exactly `num_bytes`.
message
cloud.proto:278uint64 num_bytes = 1
Total number of pseudo-random (incompressible) bytes the server should return, summed across all streamed response chunks.
message
cloud.proto:284bytes payload = 1
A chunk of pseudo-random (incompressible) bytes. The total length across all streamed chunks equals `DoBandwidthTestRequest.num_bytes`.
rpc DoGlobalMaintenance (, )
cloud.proto:229Run global maintenance operations on the platform: this includes optimization of all datasets, garbage collection of unused data, and can include more in the future.
message
cloud.proto:851Request all maintenance operations to run on all datasets
(message has no fields)
message
cloud.proto:853(message has no fields)
rpc DoMaintenance (, )
cloud.proto:225Rerun Manifests maintenance operations: scalar index creation, compaction, etc. This endpoint requires the standard dataset headers.
message
cloud.proto:800bool optimize_indexes = 2
Optimize all builtin indexes on this dataset. This merges all individual index deltas back in the main index, improving runtime performance of all indexes.
bool retrain_indexes = 6
Retrain all builtin indexes on this dataset from scratch. This retrains all builtin indexes from scratch for optimal runtime performance. This is faster than re-creating the indexes, and automatically keeps track of their configurations. This implies `optimize_indexes`.
bool compact_fragments = 3
Compact the underlying Lance fragments, for all Rerun Manifests. Hardcoded to the default (optimal) settings.
optional cleanup_before = 4
If set, all Lance fragments older than this date will be removed, for all Rerun Manifests. In case requested date is more recent than 1 hour, it will be ignored and 1 hour ago timestamp will be used. This is to prevent still used files (like recent transaction files) to be removed and cause Lance Dataset update issues. See https://docs.rs/lance/latest/lance/dataset/cleanup/index.html and https://docs.rs/lance/latest/lance/dataset/cleanup/fn.cleanup_old_versions.html
bool gc_object_store = 7
List the replicated dataset in its object store, such as S3, and remove objects that are no longer present in the local dataset. This performs a full remote reconciliation, unlike normal cleanup, which only removes replica objects corresponding to local files deleted during that cleanup.
bool unsafe_allow_recent_cleanup = 5
Override default platform behavior and allow cleanup of recent files. This will respect the value of `cleanup_before` timestamp even if it's more recent than 1 hour. ⚠️ Do not ever use this unless you know exactly what you're doing. Improper use will lead to data loss.
message
cloud.proto:846string report = 1
rpc FetchChunks (, stream )
cloud.proto:189Fetch specific chunks from the catalog server. In a 2-step query process, result of 1st phase, that is, the result of `QueryDataset` should include all the necessary information to send the actual chunk requests, which is the 2nd step of the query process. See `FetchChunksRequest` for details on the fields that describe each individual chunk.
message
cloud.proto:729repeated chunk_infos = 1
Information about the chunks to fetch. These dataframes have to include the following columns: * `chunk_id` - Chunk unique identifier * `segment_id` - segment this chunk belongs to. Currently needed as we pass this metadata back and forth * `segment_layer` - specific segment layer. Currently needed as we pass this metadata back and forth * `chunk_key` - chunk location details
message
cloud.proto:738repeated chunks = 1
Every gRPC response, even within the confines of a stream, involves HTTP2 overhead, which isn't cheap by any means, which is why we're returning a batch of `ArrowMsg` rather than a single one.
rpc FindEntries (, )
cloud.proto:47message
cloud.proto:898optional filter = 1
message
cloud.proto:902repeated entries = 1
rpc GetAssetsForSegment (, stream )
cloud.proto:150Get the assets that apply to this dataset. Returns the dataset's asset dataset and the asset segments within it. The asset segment ids may be spread over multiple responses, at the discretion of the server. This endpoint requires the standard dataset headers.
message
cloud.proto:537(message has no fields)
message
cloud.proto:539optional assets_entry = 1
The asset dataset. Set on every response in the stream.
repeated asset_segment_ids = 2
The segments of the asset dataset. Concatenate across all responses in the stream.
Returns the schema of the dataset manifest. To inspect the data of the dataset manifest, which is guaranteed to match the schema returned by this endpoint, check out `ScanDatasetManifest`. This endpoint requires the standard dataset headers.
message
cloud.proto:487(message has no fields)
message
cloud.proto:489optional schema = 1
rpc GetDatasetSchema (, )
cloud.proto:132Returns the schema of the dataset. This is the union of all the schemas from all the underlying segments. It will contain all the indexes, entities and components present in the dataset. This endpoint requires the standard dataset headers.
message
cloud.proto:520(message has no fields)
message
cloud.proto:525optional schema = 1
rpc GetRrdManifest (, stream )
cloud.proto:142Get the RRD Footer manifest. This includes details about what chunks there are, and what kind of data they contain. The manifest might be returned in multiple parts, at the discretion of the server. When that happens, it is guaranteed that all parts have the same exact Sorbet schemas (and therefore identical Sorbet schema hashes too). That means it is always semantically valid to concatenate the data from these RRD manifests.
message
cloud.proto:529optional segment_id = 1
message
cloud.proto:533optional rrd_manifest = 1
rpc GetSegmentTableSchema (, )
cloud.proto:102Returns the schema of the segment table. This is not to be confused with the schema of the dataset itself. For that, refer to `GetDatasetSchema`. To inspect the data of the segment table, which is guaranteed to match the schema returned by this endpoint, check out `ScanSegmentTable`. This endpoint requires the standard dataset headers.
message
cloud.proto:438(message has no fields)
message
cloud.proto:440optional schema = 1
rpc GetTableSchema (, )
cloud.proto:197message
cloud.proto:746optional table_id = 1
Deprecated: use x-rerun-entry-id header instead. This field should be dropped once all servers are updated to 0.14.x or later.
message
cloud.proto:753optional schema = 1
rpc QueryDataset (, stream )
cloud.proto:182Perform Rerun-native queries on a dataset, returning the matching chunk IDs, as well as information that can be sent back to the catalog server to fetch the actual chunks as part of `FetchChunks` request. In this 2-step query process, 1st step is getting information from the server about the chunks that contain relevant information. 2nd step is fetching those chunks (the actual data). These Rerun-native queries include: * Filtering by specific segment and chunk IDs. * Latest-at, range and dataframe queries. * Arbitrary Lance filters. To fetch the actual chunks themselves, see `FetchChunks`. Passing chunk IDs to this method effectively acts as a IF_EXIST filter. For latest-at queries, the returned chunks are a **correctness-preserving superset** of the minimum set needed to answer the query: the server may include additional candidate chunks (e.g., overlap siblings within the global max chunk length, or chunks that pass segment-level bounds without per-value narrowing). Clients are expected to run their own latest-at resolution on the returned chunks. This endpoint requires the standard dataset headers.
message
cloud.proto:558repeated segment_ids = 11
Client can specify what segments are queried. If left unspecified (empty list), all segments will be queried.
bool generate_direct_urls = 12
Will ask the server to generate direct URLs for the requested segments. It is not guaranteed that the server will return all (or any) of them.
repeated chunk_ids = 3
Client can specify specific chunk ids to include. If left unspecified (empty list), all chunks that match other query parameters will be included.
repeated entity_paths = 4
Which entity paths are we interested in? Leave empty, and set `select_all_entity_paths`, in order to query all of them.
bool select_all_entity_paths = 7
If set, the query will cover all existing entity paths. `entity_paths` must be empty, otherwise an error will be raised. Truth table: ```text select_all_entity_paths | entity_paths | result ------------------------+----------------+-------- false | [] | valid query, empty results (no entity paths selected) false | ['foo', 'bar'] | valid query, 'foo' & 'bar' selected true | [] | valid query, all entity paths selected true | ['foo', 'bar'] | invalid query, error ```
repeated string fuzzy_descriptors = 10
Which components are we interested in? If left unspecified, all existing components are considered of interest. This will perform a basic fuzzy match on the available columns' descriptors. The fuzzy logic is a simple case-sensitive `contains()` query. For example, given a `log_tick__SeriesLines:width` index, all of the following would match: `SeriesLines:width`, `Width`, `SeriesLines`, etc.
bool exclude_static_data = 8
If set, static data will be excluded from the results.
bool exclude_temporal_data = 9
If set, temporal data will be excluded from the results.
optional scan_parameters = 5
Generic parameters that will influence the behavior of the Lance scanner.
optional query = 6
message
cloud.proto:617optional data = 1
rpc QueryTasks (, )
cloud.proto:212Query the status of submitted tasks
message
cloud.proto:858`QueryTasksRequest` is the request message for querying tasks status
repeated ids = 1
Empty queries for all tasks if the server allows it.
message
cloud.proto:865`QueryTasksResponse` is the response message for querying tasks status encoded as a record batch
optional data = 1
rpc QueryTasksOnCompletion (, stream )
cloud.proto:215Query the status of submitted tasks as soon as they are no longer pending
message
cloud.proto:871`QueryTasksOnCompletionRequest` is the request message for querying tasks status. This is close-to-a-copy of `QueryTasksRequest`, with the addition of a timeout.
repeated ids = 1
Empty queries for all tasks if the server allows it.
optional timeout = 2
Time limit for the server to wait for task completion. The actual maximum time may be arbitrarily capped by the server.
message
cloud.proto:881`QueryTaskOnCompletionResponse` is the response message for querying tasks status encoded as a record batch. This is a copy of `QueryTasksResponse`.
optional data = 1
rpc ReadDatasetEntry (, )
cloud.proto:60Fetch metadata about a specific dataset. This endpoint requires the standard dataset headers.
message
cloud.proto:985(message has no fields)
message
cloud.proto:990optional dataset = 1
rpc ReadTableEntry (, )
cloud.proto:64message
cloud.proto:1040ReadTableEntry This endpoint requires the standard dataset headers.
optional id = 1
Deprecated: use x-rerun-entry-id header instead. This field should be dropped once all servers are updated to 0.14.x or later.
message
cloud.proto:1047optional table = 1
rpc RegisterTable (, )
cloud.proto:195Register a foreign table as a new table entry in the catalog.
message
cloud.proto:1016string name = 1
Name of the table entry to create. The name should be a short human-readable string. It must be unique within all entries in the catalog. If an entry with the same name already exists, the request will fail. Entry names ending with `__manifest` are reserved.
optional provider_details = 2
Information about the table to register. This must be encoded message of one of the following supported types: - LanceTable
message
cloud.proto:1032optional table_entry = 1
Details about the table that was created and registered.
rpc RegisterWithDataset (, )
cloud.proto:75Register new segments with the Dataset. This endpoint requires the standard dataset headers.
message
cloud.proto:385repeated data_sources = 2
common.v1alpha1.IfDuplicateBehavior on_duplicate = 3
message
cloud.proto:393optional data = 1
rpc ScanDatasetManifest (, stream )
cloud.proto:124Inspect the contents of the dataset manifest. The data will follow the schema returned by `GetDatasetManifestSchema`. This endpoint requires the standard dataset headers.
message
cloud.proto:493repeated string columns = 3
A list of column names to be projected server-side. If empty, all columns are returned. If not empty, the returned `RecordBatch` are guaranteed to only have the requested column, in the order they were requested. If a projected column does not exist, or is projected more than once, the `ScanDatasetManifest` call will fail with an `InvalidArgument` error.
optional segment_id_filter = 5
An optional best-effort hint for pruning segment IDs from the server-side scan. Servers may apply or ignore this filter. Clients must re-apply filters locally if correctness depends on them.
message
cloud.proto:515optional data = 1
The contents of the dataset manifest (i.e. information about layers) as Arrow RecordBatch.
rpc ScanSegmentTable (, stream )
cloud.proto:109Inspect the contents of the segment table. The data will follow the schema returned by `GetSegmentTableSchema`. This endpoint requires the standard dataset headers.
message
cloud.proto:460repeated string columns = 1
A list of column names to be projected server-side. If empty, all columns are returned. If not empty, the returned `RecordBatch` are guaranteed to only have the requested column, in the order they were requested. If a projected column does not exist, or is projected more than once, the `ScanSegmentTable` call will fail with an `InvalidArgument` error.
optional segment_id_filter = 3
An optional best-effort hint for pruning segment IDs from the server-side scan. Servers may apply or ignore this filter. Clients must re-apply filters locally if correctness depends on them.
message
cloud.proto:482optional data = 1
Segments metadata as Arrow RecordBatch.
rpc ScanTable (, stream )
cloud.proto:199message
cloud.proto:757optional table_id = 1
Deprecated: use x-rerun-entry-id header instead. This field should be dropped once all servers are updated to 0.14.x or later.
TODO(jleibs): support ScanParameters iff we can plumb them into Datafusion TableProvider Otherwise, just wait for Arrow Flight rerun.common.v1alpha1.ScanParameters scan_parameters = 2;
message
cloud.proto:767optional dataframe_part = 1
rpc UnregisterFromDataset (, stream )
cloud.proto:83Unregisters segments and layers from the Dataset. This is an asynchronous operation, and returns a list of task ids. The response is a stream only for historical reasons. This endpoint requires the standard dataset headers.
message
cloud.proto:402This request acts as a *product* filter: * empty `segments_to_drop` + empty `layers_to_drop`: invalid argument error * empty `segments_to_drop` + non-empty `layers_to_drop`: remove specified layers for *all* segments * non-empty `segments_to_drop` + empty `layers_to_drop`: remove *all* layers for specified segments * non-empty `segments_to_drop` + non-empty `layers_to_drop`: delete *all* specified layers for *all* specified segments
repeated segments_to_drop = 1
The segment IDs to drop. All of them if empty. The final filter will be the *outer product* of this and `layers_to_drop`.
repeated string layers_to_drop = 2
The layer names to drop. All of them if empty. The final filter will be the *outer product* of this and `segments_to_drop`.
bool force = 3
If true, deletion will go through regardless of the segments/layers' current statuses. This is only useful in the very specific, catatrophic scenario where the contents of the task queue were lost and some tasks are now stuck in `status=pending` forever. Do not use this unless you know exactly what you're doing.
message
cloud.proto:422optional data = 1
This dataframe is always empty. It remains here for format compatibility with older SDKs. TODO(ilya): remove this once the clients are all new enough
optional task_id = 2
Id of the task that performs the unregistration. The data is not deleted until the task completes.
rpc UpdateDatasetEntry (, )
cloud.proto:62message
cloud.proto:997UpdateDatasetEntry This endpoint requires the standard dataset headers.
optional id = 1
The dataset to modify. Deprecated: use x-rerun-entry-id header instead. This field should be dropped once all servers are updated to 0.14.x or later.
optional dataset_details = 2
The new values.
message
cloud.proto:1009optional dataset = 1
The updated dataset entry
rpc UpdateEntry (, )
cloud.proto:51message
cloud.proto:921UpdateEntry This endpoint requires the standard dataset headers.
optional id = 1
The entry to modify. Deprecated: use x-rerun-entry-id header instead. This field should be dropped once all servers are updated to 0.14.x or later.
optional entry_details_update = 2
The new values for updatable fields.
message
cloud.proto:933optional entry_details = 1
The updated entry details
rpc UpdateTableEntry (, )
cloud.proto:66message
cloud.proto:1053optional id = 1
The table to modify.
optional table_details = 2
The new values.
message
cloud.proto:1061optional table = 1
The updated table entry.
rpc Version (, )
cloud.proto:24message
cloud.proto:234(message has no fields)
message
cloud.proto:236optional build_info = 1
string version = 2
A single version string representing the version of the whole stack.
optional string cloud_provider = 3
Cloud provider hosting this instance (e.g. "aws", "azure"). Null if not deployed on cloud.
optional string cloud_region = 4
Cloud region where this instance is deployed (e.g. "us-west-2", "eastus"). Null if not deployed on cloud.
repeated string features = 5
Server-supported feature flags. Clients use this to gate optional behavior and stay safe against rolling deployments where the server may be older than the client. The format is a list of opaque feature strings. Known features: * `per_segment_index_values` — the server consumes `QueryLatestAt.per_segment_values` (RR-4355). Clients that omit this capability check and send `per_segment_values` to an old server will silently get static-only data. Unknown features must be ignored. Adding new features is additive and never breaks compatibility — old clients see an empty list and fall back to today's slow-but-correct path.
rpc WatchEvents (, stream )
cloud.proto:45message
cloud.proto:293repeated kinds = 1
Which kinds of events to subscribe to. If empty, the server subscribes to all event kinds.
message
cloud.proto:313oneof kind
entry_created = 1
entry_deleted = 2
rpc WhoAmI (, )
cloud.proto:30Returns information about the currently authenticated user. This is a lightweight endpoint that can be used to verify that authentication is successful and to retrieve the user's identity.
message
cloud.proto:265(message has no fields)
message
cloud.proto:267optional string user_id = 1
The user ID of the authenticated user, if any.
bool can_read = 2
Whether the user has read access.
bool can_write = 3
Whether the user has write access.
rpc WriteChunks (stream , )
cloud.proto:90Write chunks to one or more segments. The segment ID for each individual chunk is extracted from their metadata (`rerun:segment_id`). This endpoint requires the standard dataset headers.
message
cloud.proto:430optional chunk = 1
message
cloud.proto:434(message has no fields)
rpc WriteTable (stream , )
cloud.proto:207Write record batches to a table. This endpoint requires the standard dataset headers. TODO(#11645): endpoints with streaming input are not supported by `grpc-web`. A non-streaming shim will need to be added if/when the viewer uses this endpoint.
message
cloud.proto:791optional dataframe_part = 1
TableInsertMode insert_mode = 2
message
cloud.proto:796(message has no fields)
message
cloud.proto:1210`ChunkKey` provides chunk location details in the data store. Returned by `QueryDataset` (in the `chunk_key` Arrow column) and forwarded back to the server on `FetchChunks`. The `location` payload is opaque and interpreted per `data_source_kind` (e.g. `RrdChunkLocation` for RRD).
optional chunk_id = 1
Chunk unique identifier
DataSourceKind data_source_kind = 2
What type of partition is this, rrd, mcap, etc. This information determines how to interpret the `location` payload.
optional bytes location = 3
The location of the chunk in the data store. Opaque bytes; readers decode based on `data_source_kind`.
optional string etag = 4
ETag of the source object (segment) as observed at registration time. Optional: legacy registrations and stores that do not return an ETag leave this unset.
optional int64 registration_time_nanos = 5
Wall-clock registration time of the parent segment (nanoseconds since the Unix epoch), as recorded in the dataset manifest. Diagnostic only: not used as a precondition. Lets a client correlate a decode failure on a stale chunk handle with a subsequent re-registration event without consulting the dataset manifest. Optional: legacy registrations leave this unset.
message
cloud.proto:358Used in:
optional string storage_url = 1
Where is the data for this data source stored (e.g. s3://bucket/file or file:///path/to/file)?
optional string layer = 3
Which segment layer should this data source be registered to? / Defaults to `base` if unspecified.
bool prefix = 4
Is this a prefix URL (a directory)? If true, all files of `typ` under this prefix will be considered part of this data source.
DataSourceKind typ = 2
What kind of data is it (e.g. rrd, mcap, Lance, etc)?
Used in: ,
DATA_SOURCE_KIND_UNSPECIFIED = 0
DATA_SOURCE_KIND_RRD = 1
message
cloud.proto:1123Used in: ,
optional blueprint_dataset = 3
The blueprint dataset associated with this dataset (if any). This association is owned for lifecycle purposes: deleting this dataset also deletes the associated blueprint dataset.
optional default_blueprint_segment = 5
The segment of the blueprint dataset corresponding to the default blueprint (if any).
optional default_segment_table_blueprint_segment = 6
The segment of the blueprint dataset corresponding to the default blueprint for this dataset's segment table (if any).
optional asset_dataset = 7
The asset dataset associated with this dataset (if any). This association is owned for lifecycle purposes: deleting this dataset also deletes the associated asset dataset.
message
cloud.proto:1143Used in: , ,
optional details = 1
optional dataset_details = 4
Dataset-specific information, may be update with `UpdateDatasetEntry`
optional dataset_handle = 2
Read-only
message
cloud.proto:1200Optional debug info
optional uint64 memory_used = 1
The amount of memory used by the task or service call in bytes
message
cloud.proto:322Used in:
optional id = 1
message
cloud.proto:325Used in:
optional id = 1
message
cloud.proto:1103Minimal info about an Entry for high-level catalog summary
Used in: , , ,
optional id = 1
The EntryId is immutable
optional string name = 2
The name of this entry.
EntryKind entry_kind = 3
The kind of entry
optional created_at = 4
optional updated_at = 5
message
cloud.proto:1118Updatable fields of an Entry
Used in:
optional string name = 2
The name of this entry.
message
cloud.proto:311Subscribe to all catalog entry lifecycle events.
Used in:
(message has no fields)
message
cloud.proto:1066Used in:
optional id = 1
optional string name = 2
optional EntryKind entry_kind = 3
Deprecated: use `entry_kinds` instead. Ignored by servers when `entry_kinds` is non-empty.
repeated EntryKind entry_kinds = 4
The entry kinds to include. If non-empty, the server returns exactly entries of these kinds (ENTRY_KIND_UNSPECIFIED is rejected with an `invalid_argument` error). It is recommended to always explicitly ask for a set of entry kinds. When unset the server returns a default set of kinds to ensure backwards compatibility with old clients that did not set this. This behavior may be dropped in future releases. (condition: after rerun-sdk 0.34 is out of its support cycle)
What type of entry. This has strong implication on which APIs are available for this entry.
Used in: ,
ENTRY_KIND_UNSPECIFIED = 0
Always reserve unspecified as default value
ENTRY_KIND_DATASET = 1
Order as TYPE, TYPE_VIEW so things stay consistent as we introduce new types.
ENTRY_KIND_DATASET_VIEW = 2
ENTRY_KIND_TABLE = 3
ENTRY_KIND_TABLE_VIEW = 4
ENTRY_KIND_BLUEPRINT_DATASET = 5
ENTRY_KIND_ASSET_DATASET = 6
message
cloud.proto:330Application level error - used as `details` in the `google.rpc.Status` message
ErrorCode code = 1
error code
string id = 2
unique identifier associated with the request (e.g. recording id, recording storage url)
string message = 3
human readable details about the error
Error codes for application level errors
Used in:
ERROR_CODE_UNSPECIFIED = 0
unused
ERROR_CODE_OBJECT_STORE_ERROR = 1
object store access error
ERROR_CODE_METADATA_DB_ERROR = 2
metadata database access error
ERROR_CODE_CODEC_ERROR = 3
Encoding / decoding error
message
cloud.proto:301Selects a category of events to subscribe to in `WatchEvents`.
Used in:
oneof kind
entry = 1
Catalog entry lifecycle events (created, deleted).
message
cloud.proto:706A list of index values for a single segment. Used in `QueryLatestAt.per_segment_values`.
Used in:
repeated int64 values = 1
message
cloud.proto:1194A foreign table stored as a Lance table.
string table_url = 1
The URL of the Lance table.
message
cloud.proto:621Used in:
optional latest_at = 1
If specified, will perform a latest-at query with the given parameters. You can combine this with a `QueryRange` in order to gather all the relevant chunks for a full-fledged dataframe query (i.e. they get OR'd together).
optional range = 2
If specified, will perform a range query with the given parameters. You can combine this with a `QueryLatestAt` in order to gather all the relevant chunks for a full-fledged dataframe query (i.e. they get OR'd together).
bool columns_always_include_everything = 3
If true, `columns` will contain the entire schema.
bool columns_always_include_byte_offsets = 5
If true, `columns` always includes `byte_offset` and `byte_size`.
bool columns_always_include_entity_paths = 6
If true, `columns` always includes `entity_path`.
bool columns_always_include_static_indexes = 7
If true, `columns` always includes all static component-level indexes.
bool columns_always_include_global_indexes = 8
If true, `columns` always includes all temporal chunk-level indexes.
bool columns_always_include_component_indexes = 9
If true, `columns` always includes all component-level indexes.
message
cloud.proto:666A chunk-level latest-at query, aka `LatestAtRelevantChunks`. This has the exact same semantics as the query of the same name on our `ChunkStore`. Two modes of use (mutually exclusive): 1. Global: set `at`. Applied to all segments. 2. Per-segment: set `per_segment_values`. Each entry is positionally matched to `QueryDatasetRequest.segment_ids`. `at` and `Query.range` must be unset — the server reconstructs global bounds internally from the per-segment values.
Used in:
optional index = 1
Which index column should we perform the query on? E.g. `log_time`. Leave this empty to query for static data.
optional int64 at = 2
Global query value — applied to all segments. Omit this field to query for static data only. Mutually exclusive with `per_segment_values`.
repeated per_segment_values = 4
Per-segment index values, positionally matched to `QueryDatasetRequest.segment_ids`. When non-empty, `at` and `Query.range` must be unset — the server reconstructs global bounds from the values for Lance pre-filtering, then applies a per-segment post-filter for precision. Constraints (server returns `invalid_argument` on violation): - `segment_ids` must be non-empty. - `per_segment_values.len()` must equal `segment_ids.len()`. - `segment_ids` must contain no duplicates. Each entry's values list corresponds to the segment at the same position in `segment_ids`. An empty values list for a segment means no temporal chunks are returned for that segment (only static data).
message
cloud.proto:713A chunk-level range query, aka `RangeRelevantChunks`. This has the exact same semantics as the query of the same name on our `ChunkStore`.
Used in:
optional index = 1
Which index column should we perform the query on? E.g. `log_time`.
optional index_range = 2
What index range are we looking for?
message
cloud.proto:1239RRD-specific decoding of `ChunkKey.location`.
optional string url = 1
Where the chunk is stored (e.g. s3://bucket/file, file:///path/to/file).
optional uint64 offset = 2
Byte offset of the chunk within the data source.
optional uint64 length = 3
Chunk length in bytes.
message
cloud.proto:448Used in: ,
oneof strategy
scan_only = 1
Scan only manifest rows belonging to these segment IDs. An empty list scans no segment rows.
skip = 2
Scan all manifest rows except those belonging to these segment IDs. An empty list skips no segment rows.
message
cloud.proto:444Used in:
repeated string segment_ids = 1
message
cloud.proto:1189SystemTableKind kind = 1
Used in:
SYSTEM_TABLE_KIND_UNSPECIFIED = 0
Always reserve unspecified as default value
SYSTEM_TABLE_KIND_NAMESPACES = 1
Not used yet
SYSTEM_TABLE_KIND_ENTRIES = 2
All of the entries in the associated namespace
message
cloud.proto:1156Used in: ,
optional blueprint_dataset = 1
Blueprint dataset associated with this table. This association is owned for lifecycle purposes: deleting this table also deletes the associated blueprint dataset. Like recording datasets, tables get an associated blueprint dataset automatically when they are created.
optional default_blueprint_segment = 2
Segment of the blueprint dataset corresponding to the default table blueprint.
message
cloud.proto:1167Used in: , , ,
optional details = 1
optional provider_details = 3
Details specific to the table-provider
optional table_details = 4
Table-specific information, may be updated with `UpdateTableEntry`.
Used in:
TABLE_INSERT_MODE_UNSPECIFIED = 0
Always reserve unspecified as default value
TABLE_INSERT_MODE_APPEND = 1
Appends new rows to the existing table without modifying any existing rows.
TABLE_INSERT_MODE_OVERWRITE = 2
Overwrites all existing rows in the table with the new rows.
TABLE_INSERT_MODE_REPLACE = 3
Overwrite rows based on the rerun_table_index fields.
TABLE_INSERT_MODE_UPDATE = 4
Update existing rows matched by rerun_table_index fields. Source rows that do not match any existing row are dropped. The source schema may be a subset of the table schema (only the key columns plus the columns to update). Maps to SQL `UPDATE`.