package lance.index.pb

Get desktop application:
View/edit binary Protocol Buffers messages

DiskAnn Index

Used in: VectorIndexStage

• uint32 spec = 1

  Graph spec version

• string filename = 2

  Graph file

• uint32 r = 3

  r parameter

• float alpha = 4

  alpha parameter

• uint32 L = 5

  L parameter

• repeated uint64 entries = 6

  / Entry points to the graph

Flat Index

Used in: VectorIndexStage

(message has no fields)

Inverted Index File Metadata.

Used in: VectorIndexStage

• repeated float centroids = 1

  Centroids of partitions. `dimension * num_partitions` of float32s. Deprecated, use centroids_tensor instead.

  [deprecated = true];

• repeated uint64 offsets = 2

  File offset of each partition.

• repeated uint32 lengths = 3

  Number of records in the partition.

• optional Tensor centroids_tensor = 4

  Tensor of centroids. `num_partitions * dimension` of float32s.

• optional double loss = 5

  KMeans loss.

• string name = 1

  The unique index name in the dataset.

• repeated string columns = 2

  Columns to be used to build the index.

• uint64 dataset_version = 3

  The version of the dataset this index was built from.

• IndexType index_type = 4

  The [`IndexType`] of the index.

• oneof implementation

  / Index implementation details.

  • VectorIndex vector_index = 5

The type of an index.

Used in: Index

• VECTOR = 0

  Vector index

Product Quantization.

Used in: VectorIndexStage

• uint32 num_bits = 1

  The number of bits to present a centroid.

• uint32 num_sub_vectors = 2

  Number of sub vectors.

• uint32 dimension = 3

  Vector dimension

• repeated float codebook = 4

  Codebook. `dimension * 2 ^ num_bits` of float32s.

• optional Tensor codebook_tensor = 5

  Tensor of codebook. `2 ^ num_bits * dimension` of floats.

Used in: IVF, PQ

• Tensor.DataType data_type = 1

• repeated uint32 shape = 2

  Data shape, [dim1, dim2, ...]

• bytes data = 3

  Data buffer

Used in: Tensor

• BFLOAT16 = 0

• FLOAT16 = 1

• FLOAT32 = 2

• FLOAT64 = 3

• UINT8 = 4

• UINT16 = 5

• UINT32 = 6

• UINT64 = 7

A transform matrix to apply to a vector or vectors.

Used in: VectorIndexStage

• uint64 position = 1

  The file offset the matrix is stored

• repeated uint32 shape = 2

  Data shape of the matrix, [rows, cols].

• TransformType type = 3

  Transform type.

Transform type

Used in: Transform

• OPQ = 0

Vector Index Metadata

Used in: Index

• uint32 spec_version = 1

  Index specification version.

• uint32 dimension = 2

  Vector dimension;

• repeated VectorIndexStage stages = 3

  Composed vector index stages. For example, `IVF_PQ` index type can be expressed as: ```no_run,ignore let stages = vec![Ivf{}, PQ{num_bits: 8, num_sub_vectors: 16}] ```

• VectorMetricType metric_type = 4

  Vector distance metrics type

One stage in the vector index pipeline.

Used in: VectorIndex

• oneof stage

  • Flat flat = 1

    Flat index

  • IVF ivf = 2

    `IVF` - Inverted File

  • PQ pq = 3

    Product Quantization

  • Transform transform = 4

    Transformer

  • DiskAnn diskann = 5

    DiskANN

Metric Type for Vector Index

Used in: VectorIndex

• L2 = 0

  L2 (Euclidean) Distance

• Cosine = 1

  Cosine Distance

• Dot = 2

  Dot Product

• Hamming = 3

  Hamming Distance