Proto commits in apache/horaedb

These 30 commits are when the Protocol Buffers files have changed:

2024-12-18

feat: manifest support delete (#1610) ## Rationale When compact finished, we need to delete the old input sst and expired sst. ## Detailed Changes - The delta file use `ManifestUpdate` struct. - Refactor compact scheduler, to make it more modular. ## Test Plan CI

The documentation is generated from this commit.

2024-12-13

chore: move old engine into analytic-engine branch (#1607) ## Rationale To maintain system stability and code clarity, it would be advisable to relocate the legacy engine to a separate branch for dedicated maintenance, preventing potential complications from mixing old and new engine implementations. ## Detailed Changes ## Test Plan CI

2024-11-05

feat: init scan implementation for time merge storage (#1586) ## Rationale ## Detailed Changes ## Test Plan CI

2024-10-28

feat: impl write procedure scaffold (#1580) ## Rationale Implement write procedure for TimeMergeStorage ## Detailed Changes - Add basic write implementation. ## Test Plan CI

2023-02-14

chore: move `proto` to `ceresdbproto` repo (#644) * chore: use proto in ceresdbproto * cargo fmt * refactor code

This commit does not contain any .proto files.

2023-02-13

feat: replace bloom with xor8 filter (#631) * feat: add xor8 * fix test * rename bloom * remove unsed fns * rename files * rename sst filter to parquet filter

2023-02-09

feat: store manifest to oss (#607) * feat: store snapshot to object store * Store snapshot to oss * adapt to the new implementation * support convestion for snapshot and current snapshot * implement the SnapshotStore * update unit test * fix bug found by unit test * fix some comments * fix test_table_write_get_override_mem_wal caused by resetting seq of mem wal * add TODO for delete logs after snapshotting * refactor the manifest api: load_data * remove debug println * fix failure of ut * rename mod manifest to meta * address CR * avoid indirect current snapshot file * handle snapshot not exist

2023-02-07

feat: make bloom filter optional on column level (#615) * feat: make bloom filter optional in both row-group level and column level * make bloom-filter row_group level required * fix relative importing * add UT for conversion of BloomFilter * fix ut of conversion for bloom filter

2023-01-18

refactor: simpify sst meta data in manifest (#538) * Remove SstMetaData from FileMeta * support read sst metadata when prepare for compaction * make things work * remove util mod from sst mod

fix: remove size from sst metadata (#513) * remove size from sst metadata * pass file size from file handler * run CI for breaking-changes branch * fix CR

refactor: make storage format more elegant (#556) * refactor: make storage format more elegant * polish up some comments * separate SstMetaData from parquet sst

2023-01-10

feat: add timeout for query (#539) * add timeout when collect * add timeout control for memtable scan and sst scan * add timeout for user-facing service, like grpc, http * add timeout in top level execute * remove deadline from sst options * timeout http handle at top level * refactor timeout to Option * add check_deadline in InstantExt

feat: use arrow-ipc to communicate between remote server and client (#552) * feat: use arrow-ipc to communicate bewteen remote server and client * add benchmarks * add zstd for ipc * add RemoteEngineVersion * fix message

2023-01-05

chore: add version in PartitionInfo (#537) * chore: add version in PartitionInfo * refactor by CR

2022-12-29

feat: update ceresdb proto & adapter to partition info (#515) * update ceresdb proto & adapter to partition info * refactor by cr * fix CI * refactor by cr * add encoder test Co-authored-by: jiacai2050 <dev@liujiacai.net>

2022-12-27

feat: impl remote_engine grpc service (#508) * feat: impl remote_engine grpc service * chore: refactor code * chore: refactor code * refactor by CR * refactor by CR * make CI happy

2022-12-25

chore: define remote_engine grpc service (#505) Co-authored-by: kamille <34352236+Rachelint@users.noreply.github.com>

2022-12-23

feat: support parse key partition (#506) * add key partition parser * refactor the codes * add unit test * fix show create table for partition table * address CR issues * supplement the error msg when fail to parse partition key * fix unit tests Co-authored-by: CooooolFrog <zuliangwanghust@gmail.com>

2022-12-20

feat: support create partition table (#485) * feat: support creating partition table * feat: show create table display partition info * refactor code * make CI happy * refactor by CR * add test_render_partition_info * install protoc in CI Co-authored-by: kamille <34352236+Rachelint@users.noreply.github.com>

2022-12-07

refactor: remove `enable_tsid_primary_key` (#459) * refactor: remove useless enable_tsid_primary_key * chore: add new test cases for complex primary key * chore: disallow complex primary key with tsid

2022-12-02

feat: add disk based object store cache (#420) * feat: add disk based object store * add cache bytes proto * add get range test * remove drop * add manifest * impl recover cache * impl setup * fix tests * add more comments * fix lock * fix CR comments

2022-11-18

feat: correct column order (#340) * edit: column order use user define * edit: update commit * fix: some error * fix: key comparableInnternalKey encode * edit: select * fix: insert data compare use primary key * fix: unit test * edit: remote annotation, fix large enum variant * fix * fix: unit test * fix: unit test * fix: project_record_schema_with_key * fix: unit test * edit: more simple code * edit: modification of description

2022-11-08

feat: add bloom filter when write sst (#370) * add bloom filter when write sst * fix compile * remove unwrap, add more error types * fix test * make pending batch local var * Address comments * fix partition record batch * fix unittest * add break * add more test cases

2022-11-01

feat: encoding in wal's message queue impl. (#358) * meta encoding in wal's message queue impl. * add new common log key. * address CR.

2022-10-30

Refactor replace protobuf with prost (#352) * chore: fix error message for DataNotLegal * chore: fix lock checking * refactor: replace protobuf with prost * feat: impl MemBufMut/MemBuf for BufMut/Buf * chore: fix comments for MemBuf/MemBufMut module * chore: refactor MemBuf/MemBufMut * chore: format proto files * chore: fix default_value type is not optional yet * chore: fix generated directory not created * chore: ignore generated protobuf files * chore: fix ut failure

2022-09-23

feat: persist collapsible info in parquet meta_data (#261) * feat: persist collapsible info in parquet meta_data * refactor code * refactor by CR

2022-09-21

feat: base implementation of default value (#246) * feat: base implemment of default value * Add some unit tests * Make lint happy * replace CastExpr with TryCastExpr when filling default value * Update df_operator/src/visitor.rs Co-authored-by: Ruihang Xia <waynestxia@gmail.com> * Update interpreters/src/insert.rs Co-authored-by: Ruihang Xia <waynestxia@gmail.com> * Update sql/src/plan.rs Co-authored-by: Ruihang Xia <waynestxia@gmail.com> * Add integration tests * Fix minor comments * Fix integration test * Merge remote-tracking branch 'upstream/main' into simple-default-value * Fix minor comments * Improve conversion from Output to inner records Co-authored-by: Ruihang Xia <waynestxia@gmail.com>

2022-08-26

feat: add new table_option storage format (#218) * add new table_option: storage format * fix clippy * add integration tests * add storage format in table schema proto * move storage format to table options module * fix clippy * fix CR comments

2022-05-27

feat: cherry-pick changeset about scalability

2022-05-26

initial commit