Proto commits in JasonThon/lightflus

These 29 commits are when the Protocol Buffers files have changed:

2023-04-01

LIG-61 1. make redis set, get (key value) type more common; 2. optimize proto api: rename message and remove unused methods; 3. implement get_dataflow for scheduler; 4. implement get_states for coordinator; 3. should insert local operator edge builder while iterating; 4. add e2e test; 5. optimize taskmanager integration test; 6. implement Display for BatchSinkException; 7. impl From<RedisException> for BatchSinkException 8. impl From<&mut RedisException> for BatchSinkException 9. share Executor states in multi-threads; 10. rename poll_recv_data_stream fn into poll_next 11. modify workflow.yml

The documentation is generated from this commit.

2023-03-28

LIG-61 1. merge ApiServer into Coordinator; 2. refactor Error into the standard form; 3. optimize dependencies import 4. modify workflow; 5. remove window module;

2023-03-26

1. remove unused error; 2. use RpcError to unify all Rpc api returned error; 3. remove in_edge map, wrap in_edge in Task; 4. remove unused functions;

2023-03-23

LIG-61 1. move coordinator, taskmanager -> ligflus-core 2. fix StreamExecutor Poll data stream twice will be locked 3. remove window API

2023-03-10

LIG-61 1. add unittests 2. merge some impl of common proto into macro 3. add batch sink api for Sink trait 4. implement OutEdge write() method; 5. TaskWorker does not need to terminate because: * drop() will do this when remove from SkipMap * TaskWorker's props are all droppable

2023-02-19

[LIG-51] task refactor done

2023-02-06

[feat/LIG-61] complete part of refactor works

2023-01-31

[feat/LIG-61] rename all TaskWorker -> TaskManager

2023-01-29

[feat/LIG-58] 1. refactor configuration into builder mode 2. modify Dataflow proto structure -> add executionId 3. Delete `DataflowStorageImpl`, replace by `Box<dyn DataflowStorage>` 4. add some rustdoc comments 5. modify coordinator.proto and worker.proto go_grpc-out package path

2023-01-20

[feat/LIG-58] 1. refactor coordinator rpc api 2. remove healthcheck in docker-compose

2023-01-19

[feat/LIG-58] 1. complete heartbeat & ack architecture; 2. refactor coordinator & worker heartbeat api; 3. add ack api in coordinator & worker

2023-01-07

[feat/LIG-58] add Dispatcher and JobManager

2023-01-03

[feat/LIG-57] done

2022-12-16

Revert "Revert "Merge branch 'feat/LIG-10'"" This reverts commit 95155ffa09dd01729342c5eaee5bdf4e3bb75c89.

Revert "Merge branch 'feat/LIG-10'" This reverts commit f59d8b8ebf5f910040cb4d731875e7902b14e801, reversing changes made to b0263f0176e284d8bd80a49c1f7211d20a3b657c.

2022-12-01

[LIG-10] 1. window executor done 2. fixed window done

2022-11-21

[LIG-24] 1. refactor code;

2022-11-20

refactor coordinator and worker grpc layer done

2022-11-13

1. LIG-8: implement redis sink; 2. LIG-6: add kafka consumer and producer integration test; 3. modify Object inner value;

2022-11-12

1. refactor common ut: move into crate

2022-11-11

1. modify Dockerfile 2. add debug env var 3. use ApiError as resources error format; 4. implement get_actual_statement

2022-11-05

1. add array Data Type; 2. implement array operations; 3. implement typed_array -> v8_array & v8_array -> typed_array transform

2022-11-03

1. add window and trigger operator; 2. refactor DataflowTask: abstract operator into IOperator;

2022-11-02

1. merge api project into runtime 2. merge cli tool into runtime

2022-07-30

Refactor version 1

This commit does not contain any .proto files.

2022-05-24

[EPIC-2204251426-81da4] separate connector and coordinator to different services; modify events; Reference operator does not need value type

2022-04-04

modify sth

2022-04-03

framework is done

2022-03-19

first commit