These 9 commits are when the Protocol Buffers files have changed:
| Commit: | 8498b19 | |
|---|---|---|
| Author: | robtandy | |
| Committer: | GitHub | |
Repo Housekeeping, updating naming remove unused files (#70)
The documentation is generated from this commit.
| Commit: | a15fdcc | |
|---|---|---|
| Author: | robtandy | |
| Committer: | GitHub | |
update all dependencies (#67)
| Commit: | 071802f | |
|---|---|---|
| Author: | robtandy | |
| Committer: | GitHub | |
DataFusion Ray rewrite to connect stages with Arrow Flight Streaming (#60)
| Commit: | 31f8833 | |
|---|---|---|
| Author: | Andy Grove | |
| Committer: | GitHub | |
feat: Reinstate disk-based shuffle (#47) * old old shuffle reader/writer * old old shuffle reader/writer * remove ray shuffle * revert more changes * save progress * update expected plans * remove unused code * fix regression
| Commit: | b792eb0 | |
|---|---|---|
| Author: | Tim Saucer | |
| Committer: | GitHub | |
Feature/remove session context (#14) * Removing session context from ray context and testing against running sql query * Removing datafusion-python as a rust dependency * Add in shuffle context for decoding * Add DF ctx to init * Resolve proto compilation errors * Cargo fmt * Resolve compile warnings * Move serialization to only on rust side * Add docker ignore to reduce size of the context sent to daemon * Update Dockerfile. We shouldn't need venv inside a docker build enivornment. Also since adding the docker ignore, we can just copy in the entire directory that remains. * Remove stale code * Set up to pass in an execution plan instead of raw SQL command * Update example to show dataframe operation
| Commit: | ead1e4e | |
|---|---|---|
| Author: | Andy Grove | |
| Committer: | GitHub | |
Fix FFI errors, and add script for running TPC-H (#20) * Add script for running TPC-H * use latest df-python * use latest df-python * fix * tpch now works * remove old performance data * update result filename * update expected plans * formatting * add note * revert formatting change
| Commit: | 1759b73 | |
|---|---|---|
| Author: | Andy Grove | |
| Committer: | GitHub | |
Remove legacy shuffle, add docs for distributed testing (#19) * always use Ray shuffle * remove legacy shuffle * remove reference to use_ray_shuffle * remove unused imports * remove unused struct * update example * update example * update docs * update expected plans * cargo fmt * address feedback
| Commit: | 2523e9f | |
|---|---|---|
| Author: | Tim Saucer | |
| Committer: | GitHub | |
Rename from raysql to datafusion_ray in multiple places (#13)
| Commit: | 99ed391 | |
|---|---|---|
| Author: | Austin Liu | |
| Committer: | GitHub | |
Donate Ray SQL to the DataFusion Project (#1) * Initial commit * Basic project structure * gitignore * Add protobuf plumbing (#2) * Implement protobuf codec * Wire up query execution (#4) * query runs end to end (#5) * re-organize python code (#6) * Implement shuffle more fully (#7) * update README (#8) * Bug fix (#9) * Support multiple shuffle partitions (#10) * More shuffle fixes (#11) * fix readme (#12) * add perf chart (#13) * Remove hard-coded temp dir (#14) * bug fix (#15) * New results (#16) * Upgrade to DataFusion 17, fix a couple of bugs, add some tests (#18) * Remove debug logging (#19) * update README (#21) * Make better use of futures (#23) * Documentation & bug fixes (#24) * Update README.md * [WIP] Use Ray object store for shuffle exchange (#28) * Fixes for Ray-based shuffle (#29) * Small fixes for Context (#30) * Make distributed execution work (#33) * Make distributed execution work * fix tips.py * fixes; incorporate changes from #32 * Upgrade to DataFusion 20 (#31) * Add support for DDL statements, such as `CREATE VIEW` (#35) * Experimenting with supporting DDL * update docs * Use PyArrow for zero-copy interaction with the Ray Object Store (#36) * Optimize Ray shuffle with zero-copy object store * remove more clones * change bytes to pyarrow.array * revert /tmp * remove empty_result_set * remove empty_result_set * Fix input partition count bug * Add Frank as author (#37) * fix hyperlink of issue 22 in docs/README.txt (#40) Co-authored-by: ivanfan <ivanfan@tencent.com> * delta lake and iceberg table support (#43) * delta support * imports * Update DataFusion version to 28.0.0 (#41) * Update DataFusion version * update example * Upgrade to DataFusion 33 (#45) * Upgrade to DataFusion 33 * undo release profile change * Add basic GitHub workflow to compile code (#47) * Create rust.yml * install protobuf * fix * fix * fix * fix * fix * fix * Add ASF license header Signed-off-by: Austin Liu <austin362667@gmail.com> * Remove ASF header for generated code Signed-off-by: Austin Liu <austin362667@gmail.com> --------- Signed-off-by: Austin Liu <austin362667@gmail.com> Co-authored-by: Andy Grove <andygrove73@gmail.com> Co-authored-by: Frank Luan <franklsf95@gmail.com> Co-authored-by: Frank Luan <lsf@berkeley.edu> Co-authored-by: Ivankings <37607467+Ivankings@users.noreply.github.com> Co-authored-by: ivanfan <ivanfan@tencent.com> Co-authored-by: raviranak <126759945+raviranak@users.noreply.github.com>