These commits are when the Protocol Buffers files have changed: (only the last 100 relevant commits are shown)
| Commit: | 9d45cf1 | |
|---|---|---|
| Author: | Yijun Xie | |
NO-JIRA: Add missing AISQL function wrappers (ai_count_tokens, ai_multi_embed, ai_redact) - Add standalone `ai_count_tokens`, `ai_multi_embed`, and `ai_redact` to `functions.py` - Add `DataFrame.ai.multi_embed` and `DataFrame.ai.redact` DataFrame methods - Fix `DataFrame.ai.count_tokens` to use new `AI_COUNT_TOKENS` instead of deprecated `SNOWFLAKE.CORTEX.COUNT_TOKENS` - Add `DataframeAiMultiEmbed` and `DataframeAiRedact` proto messages; regenerate `ast_pb2.py` - Add integration tests for all new functions Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
| Commit: | d331960 | |
|---|---|---|
| Author: | Jonathan Shi | |
| Committer: | GitHub | |
SNOW-3250094: Support DIRECTED JOIN (#4125)
| Commit: | 773b849 | |
|---|---|---|
| Author: | Ravi Kumar Suresh Babu | |
fix bug
| Commit: | 5de9f44 | |
|---|---|---|
| Author: | Ravi Kumar Suresh Babu | |
SNOW-2680714: Fix data leakage in time_series_agg by excluding current row Breaking change in v1.45.0: - Changed window frame to exclude current row to prevent data leakage in ML use cases - Past windows: now use -interval to 1 PRECEDING (was: -interval to CURRENT ROW) - Future windows: now use 1 FOLLOWING to interval (was: CURRENT ROW to interval) - Updated all integration tests with new expected values - Updated proto file with breaking change comment - Updated CHANGELOG with breaking change section
| Commit: | 9e16b0e | |
|---|---|---|
| Author: | Ben Kogan | |
| Committer: | GitHub | |
Add support for DECFLOAT type (#4055)
| Commit: | 304d93b | |
|---|---|---|
| Author: | Ben Kogan | |
| Committer: | GitHub | |
SNOW-2341888: preserve parameter names of user created functions (sprocs, UDFs, UDTFs, UDAFs) (#4051)
| Commit: | 266334b | |
|---|---|---|
| Author: | May Liu | |
| Committer: | GitHub | |
SNOW-2872192: Support targeted delete-insert in save_as_table (#4031)
| Commit: | 79af340 | |
|---|---|---|
| Author: | May Liu | |
| Committer: | GitHub | |
SNOW-2268207: Support lateral join (#3971)
| Commit: | e37af77 | |
|---|---|---|
| Author: | May Liu | |
| Committer: | GitHub | |
SNOW-2268207: Support lateral join (#3971)
| Commit: | 0e33d83 | |
|---|---|---|
| Author: | May Liu | |
Support only inner join lateral until SF doc officially explains other lateral join type usages
| Commit: | 1e77cd3 | |
|---|---|---|
| Author: | May Liu | |
AST
| Commit: | 2ec1df6 | |
|---|---|---|
| Author: | May Liu | |
| Committer: | Adam Ling | |
Revert "SNOW-2268215: support group by all (#3889)" (#3935)
The documentation is generated from this commit.
| Commit: | 9d2c667 | |
|---|---|---|
| Author: | May Liu | |
| Committer: | Adam Ling | |
Revert "SNOW-2268215: support group by all (#3889)" (#3935)
The documentation is generated from this commit.
| Commit: | eaa4d65 | |
|---|---|---|
| Author: | May Liu | |
| Committer: | GitHub | |
Revert "SNOW-2268215: support group by all (#3889)" (#3935)
The documentation is generated from this commit.
| Commit: | c09a805 | |
|---|---|---|
| Author: | May Liu | |
| Committer: | GitHub | |
Revert "SNOW-2268215: support group by all (#3889)" (#3935)
The documentation is generated from this commit.
| Commit: | 3905d59 | |
|---|---|---|
| Author: | May Liu | |
| Committer: | Adam Ling | |
Revert "SNOW-2268215: support group by all (#3889)" This reverts commit bfe436f388f20db9b6a6f479df128db887dc6c1a.
The documentation is generated from this commit.
| Commit: | 82071cf | |
|---|---|---|
| Author: | May Liu | |
| Committer: | Adam Ling | |
Revert "SNOW-2268215: support group by all (#3889)" This reverts commit 75d3134c63070584feb382d2c69a0a1d44e4c196.
The documentation is generated from this commit.
| Commit: | fc52257 | |
|---|---|---|
| Author: | May Liu | |
| Committer: | May Liu | |
sort() default to ORDER BY ALL if no cols are specified
| Commit: | e1a604c | |
|---|---|---|
| Author: | May Liu | |
| Committer: | May Liu | |
AST
| Commit: | ef3f315 | |
|---|---|---|
| Author: | May Liu | |
AST
| Commit: | 75d3134 | |
|---|---|---|
| Author: | May Liu | |
| Committer: | GitHub | |
SNOW-2268215: support group by all (#3889)
| Commit: | bfe436f | |
|---|---|---|
| Author: | May Liu | |
| Committer: | GitHub | |
SNOW-2268215: support group by all (#3889)
| Commit: | 41b0012 | |
|---|---|---|
| Author: | May Liu | |
AST support on group_by_all
| Commit: | f4b7136 | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | GitHub | |
SNOW-692496: support transactions (#3735)
The documentation is generated from this commit.
| Commit: | bc89cb8 | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | GitHub | |
SNOW-692496: support transactions (#3735)
The documentation is generated from this commit.
| Commit: | f8955cf | |
|---|---|---|
| Author: | Adam Ling | |
fix
The documentation is generated from this commit.
| Commit: | b77dc73 | |
|---|---|---|
| Author: | Jianzhun Du | |
| Committer: | GitHub | |
SNOW-2220946: Add support for unstructured data engineering in Snowpark (#3775)
| Commit: | f972575 | |
|---|---|---|
| Author: | Jianzhun Du | |
| Committer: | GitHub | |
SNOW-2220946: Add support for unstructured data engineering in Snowpark (#3775)
| Commit: | 2388e37 | |
|---|---|---|
| Author: | Afroz Alam | |
| Committer: | GitHub | |
SNOW-2211956 support copy grants with create replace view (#3755)
| Commit: | 22d1174 | |
|---|---|---|
| Author: | Afroz Alam | |
| Committer: | GitHub | |
SNOW-2211956 support copy grants with create replace view (#3755)
| Commit: | 3c3da83 | |
|---|---|---|
| Author: | Jianzhun Du | |
| Committer: | Jianzhun Du | |
SNOW-2263127: Add AST support for AI functions (#3761)
| Commit: | 91d9ff9 | |
|---|---|---|
| Author: | Jianzhun Du | |
fix
| Commit: | 1a09e0a | |
|---|---|---|
| Author: | Jianzhun Du | |
add ast
| Commit: | 98ff3be | |
|---|---|---|
| Author: | Jianzhun Du | |
d
| Commit: | 5be7e2e | |
|---|---|---|
| Author: | Afroz Alam | |
ast tests
| Commit: | 1d7af3b | |
|---|---|---|
| Author: | Felix He | |
SNOW-2265266: updated proto
| Commit: | 86918d4 | |
|---|---|---|
| Author: | Felix He | |
SNOW-2265266: add test for ast
| Commit: | 816454f | |
|---|---|---|
| Author: | Adam Ling | |
add ast support
| Commit: | 11cafe0 | |
|---|---|---|
| Author: | May Liu | |
| Committer: | GitHub | |
SNOW-2292338 AST support for time travel (#3716)
| Commit: | 5987d35 | |
|---|---|---|
| Author: | May Liu | |
| Committer: | GitHub | |
SNOW-2292338 AST support for time travel (#3716)
| Commit: | dbd6805 | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | GitHub | |
SNOW-641374: support directory (#3709)
| Commit: | 14aa563 | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | GitHub | |
SNOW-641374: support directory (#3709)
| Commit: | bbdc728 | |
|---|---|---|
| Author: | May Liu | |
ast
| Commit: | eba9538 | |
|---|---|---|
| Author: | Adam Ling | |
add alias method
| Commit: | 0028305 | |
|---|---|---|
| Author: | Adam Ling | |
add ast support
| Commit: | f195de4 | |
|---|---|---|
| Author: | Felix He | |
| Committer: | Felix He | |
SNOW-2272863: Add DayTimeInterval support
| Commit: | 2eb96cd | |
|---|---|---|
| Author: | Felix He | |
| Committer: | Felix He | |
SNOW-2272863: Add DayTimeInterval support
| Commit: | 0cdd6b5 | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | GitHub | |
SNOW-2268233: add validation mode and external location support to DataFrameWriter.copy_into_location (#3665)
| Commit: | 6acfee9 | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | GitHub | |
SNOW-2268233: add validation mode and external location support to DataFrameWriter.copy_into_location (#3665)
| Commit: | 84e8631 | |
|---|---|---|
| Author: | Felix He | |
| Committer: | Felix He | |
SNOW-2272863: Add DayTimeInterval support
| Commit: | 2a775f8 | |
|---|---|---|
| Author: | Felix He | |
| Committer: | Felix He | |
SNOW-2272863: Add DayTimeInterval support
| Commit: | 09e7a60 | |
|---|---|---|
| Author: | Adam Ling | |
update ast code
| Commit: | b2e92e2 | |
|---|---|---|
| Author: | Felix He | |
| Committer: | Felix He | |
SNOW-2272863: Add DayTimeInterval support
| Commit: | 56898fc | |
|---|---|---|
| Author: | Jamison | |
AST update 2
| Commit: | 4b3dc3e | |
|---|---|---|
| Author: | Jamison | |
Release v1.37.0
| Commit: | f95b06a | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | GitHub | |
SNOW-2253293: ilike support when selecting * columns (#3650)
| Commit: | a743a5d | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | GitHub | |
SNOW-2253293: ilike support when selecting * columns (#3650)
| Commit: | 9bd8f93 | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | Felix He | |
update copy-remote-ast.sh script and sync proto
| Commit: | 5b2bf04 | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | Felix He | |
update copy-remote-ast.sh script and sync proto
| Commit: | d484194 | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | Felix He | |
update copy-remote-ast.sh script and sync proto
| Commit: | 5503abb | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
SNOW-2249897: Update script after moving Snowpark AST back to monorepo (#3637)
| Commit: | 0186f1c | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
SNOW-2249897: Update script after moving Snowpark AST back to monorepo (#3637)
| Commit: | 5671445 | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | Felix He | |
update copy-remote-ast.sh script and sync proto
| Commit: | d540492 | |
|---|---|---|
| Author: | Hemit Shah | |
SNOW-2249897: Update script after moving Snowpark AST back to monorepo
| Commit: | f5c3de2 | |
|---|---|---|
| Author: | Hemit Shah | |
update copy-remote-ast.sh script and sync proto
| Commit: | b4ac23a | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
NOSNOW: Update scripts to retrieve ast and unparser from monorepo (#3505)
| Commit: | 874ae33 | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
NOSNOW: Update scripts to retrieve ast and unparser from monorepo (#3505)
| Commit: | 216e153 | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | Hemit Shah | |
NOSNOW: Update scripts to retrieve ast and unparser from monorepo
| Commit: | f4935ec | |
|---|---|---|
| Author: | Hemit Shah | |
NOSNOW: Update scripts to retrieve ast and unparser from monorepo
| Commit: | 773ba91 | |
|---|---|---|
| Author: | Mark Keller | |
| Committer: | GitHub | |
SNOW-2028625 adding strings_include_math_stats to DataFrame.describe (#3373) Co-authored-by: Hemit Shah <hemit.shah@snowflake.com> Co-authored-by: Afroz Alam <afroz.alam@snowflake.com>
| Commit: | 8e8a551 | |
|---|---|---|
| Author: | Mark Keller | |
| Committer: | GitHub | |
SNOW-2028625 adding strings_include_math_stats to DataFrame.describe (#3373) Co-authored-by: Hemit Shah <hemit.shah@snowflake.com> Co-authored-by: Afroz Alam <afroz.alam@snowflake.com>
| Commit: | 0092964 | |
|---|---|---|
| Author: | Jamison Rose | |
| Committer: | GitHub | |
SNOW-2062273: Remove artifact_repository_packages parameter (#3366)
| Commit: | afd9e35 | |
|---|---|---|
| Author: | Jamison Rose | |
| Committer: | GitHub | |
SNOW-2062273: Remove artifact_repository_packages parameter (#3366)
| Commit: | 8dba77e | |
|---|---|---|
| Author: | Hemit Shah | |
update proto and expectation tests
| Commit: | d1c86b8 | |
|---|---|---|
| Author: | Jamison Rose | |
| Committer: | GitHub | |
Merge branch 'main' into jrose_snow_2062273_artifact_repository_bcr
| Commit: | f3ffa07 | |
|---|---|---|
| Author: | Jamison Rose | |
| Committer: | GitHub | |
SNOW-2103154: Add support for iceberg_version (#3369)
| Commit: | 20818cf | |
|---|---|---|
| Author: | Jamison Rose | |
| Committer: | GitHub | |
SNOW-2103154: Add support for iceberg_version (#3369)
| Commit: | 2eef0e9 | |
|---|---|---|
| Author: | Jamison | |
update ast
| Commit: | 0bc3ece | |
|---|---|---|
| Author: | Jamison | |
| Committer: | Jamison | |
SNOW-2062273: Remove artifact_repository_packages parameter
| Commit: | 15166d9 | |
|---|---|---|
| Author: | Jamison | |
SNOW-2062273: Remove artifact_repository_packages parameter
| Commit: | 0f5e851 | |
|---|---|---|
| Author: | Jamison Rose | |
| Committer: | GitHub | |
SNOW-2036949: Sync artifact repositry ast changes (#3293)
| Commit: | d14bc80 | |
|---|---|---|
| Author: | Jamison Rose | |
| Committer: | GitHub | |
SNOW-2036949: Sync artifact repositry ast changes (#3293)
| Commit: | eefa3eb | |
|---|---|---|
| Author: | Jamison | |
SNOW-2036949: Sync artifact repositry ast changes
| Commit: | b2167c1 | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
NOSNOW: Update ast.proto (#3214)
| Commit: | f37a938 | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
NOSNOW: Update ast.proto (#3214)
| Commit: | 71605c8 | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
SNOW-1821290: Finalize AST schema v1 by refactoring Assign to Bind, VarId to BindId (#3183) 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. [SNOW-1821290](https://snowflakecomputing.atlassian.net/browse/SNOW-1821290): Finalize AST schema v1 by refactoring `Assign` to `Bind`, `VarId` to `BindId` 2. Fill out the following pre-review checklist: - [x] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. - [x] I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: [Thread-safe Developer Guidelines](https://github.com/snowflakedb/snowpark-python/blob/main/CONTRIBUTING.md#thread-safe-development) - [x] If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. 3. Please describe how your code solves the related issue. To reflect the new semantics, `AstBatch.assign` was refactored to `AstBatch.bind` and its usage was updated across the client. Usage of `var_id` fields was replaced with `bind_id` fields as well. `AstBatch._init_batch` now provides the UUID to the newly created `Request` message in its binary 16 byte representation provided by `UUID.bytes` under the `Request.request_id` field. `AstBatch.bind` will also take the current `AstBatch._request_id` and fill `Bind.first_request_id` with the same binary 16 byte representation from `UUID.bytes` to keep track of the first `Request` message which this statement will be flushed to the server in. This PR is accompanied by a [server-side PR](https://github.com/snowflakedb/snowflake/pull/270978). [SNOW-1821290]: https://snowflakecomputing.atlassian.net/browse/SNOW-1821290?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
| Commit: | bac17ee | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
SNOW-1821290: Finalize AST schema v1 by refactoring Assign to Bind, VarId to BindId (#3183) 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. [SNOW-1821290](https://snowflakecomputing.atlassian.net/browse/SNOW-1821290): Finalize AST schema v1 by refactoring `Assign` to `Bind`, `VarId` to `BindId` 2. Fill out the following pre-review checklist: - [x] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. - [x] I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: [Thread-safe Developer Guidelines](https://github.com/snowflakedb/snowpark-python/blob/main/CONTRIBUTING.md#thread-safe-development) - [x] If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. 3. Please describe how your code solves the related issue. To reflect the new semantics, `AstBatch.assign` was refactored to `AstBatch.bind` and its usage was updated across the client. Usage of `var_id` fields was replaced with `bind_id` fields as well. `AstBatch._init_batch` now provides the UUID to the newly created `Request` message in its binary 16 byte representation provided by `UUID.bytes` under the `Request.request_id` field. `AstBatch.bind` will also take the current `AstBatch._request_id` and fill `Bind.first_request_id` with the same binary 16 byte representation from `UUID.bytes` to keep track of the first `Request` message which this statement will be flushed to the server in. This PR is accompanied by a [server-side PR](https://github.com/snowflakedb/snowflake/pull/270978). [SNOW-1821290]: https://snowflakecomputing.atlassian.net/browse/SNOW-1821290?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
| Commit: | 9500f7f | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
SNOW-1882075,SNOW-1990301,SNOW-1990302,SNOW-1821290: AST finalizations for stateless batches, and DataframeWriter,DataframeReader new APIs (#3180) 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. This PR includes all changes required to complete the following tickets. - [SNOW-1882075](https://snowflakecomputing.atlassian.net/browse/SNOW-1882075): AST for DataFrameWriter.insertInto/insert_into - [SNOW-1990301](https://snowflakecomputing.atlassian.net/browse/SNOW-1990301): AST for DataframerWriter.{format,save} - [SNOW-1990302](https://snowflakecomputing.atlassian.net/browse/SNOW-1990302): AST for DataframeReader.{format,load} - [SNOW-1821290](https://snowflakecomputing.atlassian.net/browse/SNOW-1821290): Finalize the AST schema v1 It also includes changes in support of [SNOW-1997136](https://snowflakecomputing.atlassian.net/browse/SNOW-1997136): Ensure AST batch is stateless. 2. Fill out the following pre-review checklist: - [x] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. - [x] I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: [Thread-safe Developer Guidelines](https://github.com/snowflakedb/snowpark-python/blob/main/CONTRIBUTING.md#thread-safe-development) - [x] If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. 3. Please describe how your code solves the related issue. AST generation for `DataframeReader` APIs which return the same `DataframeReader` instance have been updated to follow the same pattern as `DataframeWriter`. Since all such methods are setters, it is only necessary to have the last set attributes in the AST sent to the serveer. This is in support of the new `format` API in both Snowpark classes. AST generation was added for the new APIs `DataframeReader.{format,load}`, and `DataframeWriter.{format,read,insert_into}` along with expectation test updates. Changes were also made in support of AST batch statelessness in the IR, by changing fields typed `VarId` to be `Expr` typed. This allows more consistent usage of `_set_ast_ref` as a helper method across all Snowpark APIs. This PR is accompanied by a [server-side PR](https://github.com/snowflakedb/snowflake/pull/270350). [SNOW-1882075]: https://snowflakecomputing.atlassian.net/browse/SNOW-1882075?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SNOW-1990301]: https://snowflakecomputing.atlassian.net/browse/SNOW-1990301?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SNOW-1990302]: https://snowflakecomputing.atlassian.net/browse/SNOW-1990302?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SNOW-1821290]: https://snowflakecomputing.atlassian.net/browse/SNOW-1821290?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SNOW-1997136]: https://snowflakecomputing.atlassian.net/browse/SNOW-1997136?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
| Commit: | 0bdcac2 | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
SNOW-1882075,SNOW-1990301,SNOW-1990302,SNOW-1821290: AST finalizations for stateless batches, and DataframeWriter,DataframeReader new APIs (#3180) 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. This PR includes all changes required to complete the following tickets. - [SNOW-1882075](https://snowflakecomputing.atlassian.net/browse/SNOW-1882075): AST for DataFrameWriter.insertInto/insert_into - [SNOW-1990301](https://snowflakecomputing.atlassian.net/browse/SNOW-1990301): AST for DataframerWriter.{format,save} - [SNOW-1990302](https://snowflakecomputing.atlassian.net/browse/SNOW-1990302): AST for DataframeReader.{format,load} - [SNOW-1821290](https://snowflakecomputing.atlassian.net/browse/SNOW-1821290): Finalize the AST schema v1 It also includes changes in support of [SNOW-1997136](https://snowflakecomputing.atlassian.net/browse/SNOW-1997136): Ensure AST batch is stateless. 2. Fill out the following pre-review checklist: - [x] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. - [x] I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: [Thread-safe Developer Guidelines](https://github.com/snowflakedb/snowpark-python/blob/main/CONTRIBUTING.md#thread-safe-development) - [x] If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. 3. Please describe how your code solves the related issue. AST generation for `DataframeReader` APIs which return the same `DataframeReader` instance have been updated to follow the same pattern as `DataframeWriter`. Since all such methods are setters, it is only necessary to have the last set attributes in the AST sent to the serveer. This is in support of the new `format` API in both Snowpark classes. AST generation was added for the new APIs `DataframeReader.{format,load}`, and `DataframeWriter.{format,read,insert_into}` along with expectation test updates. Changes were also made in support of AST batch statelessness in the IR, by changing fields typed `VarId` to be `Expr` typed. This allows more consistent usage of `_set_ast_ref` as a helper method across all Snowpark APIs. This PR is accompanied by a [server-side PR](https://github.com/snowflakedb/snowflake/pull/270350). [SNOW-1882075]: https://snowflakecomputing.atlassian.net/browse/SNOW-1882075?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SNOW-1990301]: https://snowflakecomputing.atlassian.net/browse/SNOW-1990301?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SNOW-1990302]: https://snowflakecomputing.atlassian.net/browse/SNOW-1990302?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SNOW-1821290]: https://snowflakecomputing.atlassian.net/browse/SNOW-1821290?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SNOW-1997136]: https://snowflakecomputing.atlassian.net/browse/SNOW-1997136?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
| Commit: | 3ddb193 | |
|---|---|---|
| Author: | Leonhard Spiegelberg | |
| Committer: | GitHub | |
SNOW-1995618 Store ref to temp table created in cache_result in AST (#3181) To enable server-side execution, cache_result presents a duality as it effectively is a crossover of eval/assign. We resolve the existing TODO by treating cache_result as an expr together with an assign. Client-side optionally a temp table is created, and then stored within the expr. This will allow server-side to skip any nodes that formed lineage to the cache_result expr. Decoder changes done in monorepo/followup once decoder is refreshed to 1.29.
| Commit: | 06b9d6e | |
|---|---|---|
| Author: | Leonhard Spiegelberg | |
| Committer: | GitHub | |
SNOW-1995618 Store ref to temp table created in cache_result in AST (#3181) To enable server-side execution, cache_result presents a duality as it effectively is a crossover of eval/assign. We resolve the existing TODO by treating cache_result as an expr together with an assign. Client-side optionally a temp table is created, and then stored within the expr. This will allow server-side to skip any nodes that formed lineage to the cache_result expr. Decoder changes done in monorepo/followup once decoder is refreshed to 1.29.
| Commit: | 704b984 | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
SNOW-1916224: Ensure that all APIs have AST support (#3170)
| Commit: | db0b32c | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
SNOW-1916224: Ensure that all APIs have AST support (#3170)
| Commit: | 2d8d479 | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | GitHub | |
NO-SNOW: release 1.29.1 merge back to main (#3153) (#3155)
| Commit: | ccd0d2a | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | GitHub | |
NO-SNOW: release 1.29.1 merge back to main (#3153) (#3155)
| Commit: | 995b7c6 | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
SNOW-1918334: Various fixes to capture user code more accurately (#3132)
| Commit: | 96cef3f | |
|---|---|---|
| Author: | Hemit Shah | |
| Committer: | GitHub | |
SNOW-1918334: Various fixes to capture user code more accurately (#3132)
| Commit: | a1ac12c | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | GitHub | |
NO-SNOW: release 1.29.1 preparation (#3153)
| Commit: | 2d3d8ee | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | Adam Ling | |
NO-SNOW: release update (#3150)
| Commit: | 35dcc86 | |
|---|---|---|
| Author: | Adam Ling | |
| Committer: | GitHub | |
NO-SNOW: release update (#3150)