-
Notifications
You must be signed in to change notification settings - Fork 112
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merge a recent version of main into the phase 0 staging branch #2606
Open
sfc-gh-oplaton
wants to merge
404
commits into
ls-SNOW-1491199-merge-phase0-server-side
Choose a base branch
from
oplaton/snow-1491199-integrate-main-2024-11-11
base: ls-SNOW-1491199-merge-phase0-server-side
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Merge a recent version of main into the phase 0 staging branch #2606
sfc-gh-oplaton
wants to merge
404
commits into
ls-SNOW-1491199-merge-phase0-server-side
from
oplaton/snow-1491199-integrate-main-2024-11-11
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This PR adds an xfail to `test_df_groupby_idxmax_idxmin_on_axis_0` since `idxmax/idxmin` results in a non-deterministic ordering when tiebreaking values. Signed-off-by: Naren Krishna <[email protected]> Co-authored-by: Jialin Jiao <[email protected]>
…2331) Fixes SNOW-164016 Test two scenarios for apply-like methods `apply`, `applymap`, `groupby.transform`, and `groupby.apply`: 1) input dataframe has timedelta data columns or index columns 2) output includes timedelta values. Situation 1) currently raises `NotImplementedError` and will continue to do so after this commit. Recognizing that we have timedelta instead of the underlying integer as input inside the UD(T)F requires adding extra metadata. Postpone this task till a user asks for it. In this commit, just check for the `NotImplementedError`. Situation 2) currently puts string outputs instead of `Timedelta` in the result. That's a bug, but it does follow what Snowflake does in this case. Fixing the bug is possible, but requires adding some extra metadata to the UD(T)F output. For now defer this task until a feature notices the discrepancy. In this commit, just check for an `AssertionError` that the Snowpark pandas result is not correct. Signed-off-by: sfc-gh-mvashishtha <[email protected]>
<!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-1673230 2. Fill out the following pre-review checklist: - [ ] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. This PR removes `src/snowflake/snowpark/modin/pandas/utils.py`. Some of these functions were identical to their counterparts in `modin/pandas/utils.py` and `modin/pandas/io.py`, and have been replaced accordingly. The remaining functions, which are specific to Snowpark pandas, have been moved to `src/snowflake/snowpark/modin/plugin/extensions/utils.py`.
…ction complexity (#2326) <!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> NOW-1632896 2. Fill out the following pre-review checklist: - [x] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. snowflake optimization can merge the select statement when possible, to get a more accurate estimation of the query plan complexity, we want to detect such case and adjust the complexity calculation. For example, SELECT COL1 + 2 AS COL1, COL2 FROM (SELECT COL1 + 1 AS COL1, COL2 FROM TEST_TABLE) can be merged as SELECT (COL1 + 1) + COL1 + 2 AS COL1, COL2 FROM TEST_TABLE. A parameter _try_merge_projection_complexity is added to SelectStatement to indicate that we should try to merge the select projection complexity with the child (from).
<!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-148962 2. Fill out the following pre-review checklist: - [x] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. Adding support for the corr aggregation for DataFrame/Series windows rolling with pairwise=False and integer window value. --------- Signed-off-by: Labanya Mukhopadhyay <[email protected]>
#2350) SNOW-1559086 This PR adds support for time-based string parameter `window` for `Rolling` window functions. --------- Signed-off-by: Naren Krishna <[email protected]>
) 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. Fixes SNOW-1660802 Implement dataframe groupby fillna (group by axis=0) 2. Fill out the following pre-review checklist: - [x] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. Add support for DataFrameGroupBy.fillna when group by axis=0. Note that axis=1 will be depreciated post pandas 2.1.0 so not investing in that support.
#2352) …lization with inadvertent slowness. <!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-1660947 2. Fill out the following pre-review checklist: - [ ] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. Please write a short description of how your code change solves the related issue. SNOW-1660947 Improved warning messages for operations lead to materialization with inadvertent slowness. This PR should cover all materialization cases missing warning today. --------- Co-authored-by: Varnika Budati <[email protected]>
3. Please describe how your code solves the related issue. when sql simplifier is disabled, there is less describing query issued
…mation and add tests for df alias and subqueries (#2347) <!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> SNOW-1641640 2. Fill out the following pre-review checklist: - [x] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. 1) fix field update for expr_to_alias and df_aliased_col_name_to_real_col_name during node reset for SelectStatement 2) add tests for test subqueries and alias map
…Index objects (#2137) <!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-1458135 2. Fill out the following pre-review checklist: - [x] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. - Implemented functionality to enable creating Series and DataFrame objects with a lazy Index object as the `data`, `index`, and/or `columns`. - This also covers creating Series and DataFrames with rows/columns that don't exist in the given `data`. - A special case is when the `data` is a Series or DataFrame object, the new Series or DataFrame object is creating by filtering the `data` with provided `index` and `columns`. - In case some values in `index` don't exist in `data`'s index, these values are added as new rows and their corresponding data values are `NaN`. - In case some values in `columns` don't exist in `data`'s columns, these values are added as new `NaN` columns. - I use a right outer join to add the new index values, and create and append the new `NaN` columns in the logic.
<!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-1664175 reenable a xfailed test 2. Fill out the following pre-review checklist: - [ ] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. Please write a short description of how your code change solves the related issue.
<!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-1692775 2. Fill out the following pre-review checklist: - [ ] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. Remove convert_dtype warning in Series.apply.
<!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-1669024 2. Fill out the following pre-review checklist: - [x] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. Add support for pd.read_sas. Native pandas is first used to read the input file. Then the created native pandas dataFrame is converted into a Snowpark pandas dataframe. This allows us to support all the parameters already supported by native pandas. The only limitation is that large files that cannot fit in memory still won't be supported.
<!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-1672691 2. Fill out the following pre-review checklist: - [ ] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. This PR removes the snowflake/snowpark/modin/pandas folder. Remaining docstring paths have been modified appropriately, and necessary initialization code has moved to snowflake/snowpark/modin/plugin/__init__.py. Previously, we exposed methods from `general_overrides` and `io_overrides` as direct exports from the `snowflake.snowpark.modin.pandas` that were then mirrored into the `modin.pandas` namespace. Now, we directly call `modin.pandas.api.extensions.register_pd_accessor` to populate `modin.pandas`.
…r disabled (#2361) 1. Please describe how your code solves the related issue. a) Fix alias query generation after transformation by propogating the alias map correctly to snowflake pln b) fix a test that is actually doing wrong check
…on (#2340) <!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> SNOW-1632898 2. Fill out the following pre-review checklist: - [x] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. 1) adjust the projection complexity calculation, and cumulative complexity for selectStatement when _merge_projection_complexity_with_subquery set to True. 2) update the complexity calculation for snowflake plan to directly get the complexity from source plan, and update the reset to reset the cumulative complexity.
… query complete (#2368) <!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-1645523 2. Fill out the following pre-review checklist: - [ ] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. Parse the query and find the query id, then wait for this drop table sql query complete before checking whether the table exists or not. However, log is not available in stored proc
…lta. (#2367) Fixes SNOW-1694806 Add support for rolling().count() and expanding().count() for both Series and DataFrames. For other rolling and expanding methods, match pandas behavior, which is to raise some kind of error. --------- Signed-off-by: sfc-gh-mvashishtha <[email protected]>
1. Please describe how your code solves the related issue. import graphviz seems also introduces a set of warning message like ``` DEBUG graphviz._tools:_tools.py:147 deprecate positional args: graphviz.graphs.BaseGraph.__init__(['comment', 'filename', 'directory', 'format', 'engine', 'encoding', 'graph_attr', 'node_attr', 'edge_attr', 'body', 'strict']) DEBUG graphviz._tools:_tools.py:147 deprecate positional args: graphviz.sources.Source.from_file(['directory', 'format', 'engine', 'encoding', 'renderer', 'formatter']) DEBUG graphviz._tools:_tools.py:147 deprecate positional args: graphviz.sources.Source.__init__(['filename', 'directory', 'format', 'engine', 'encoding']) DEBUG graphviz._tools:_tools.py:147 deprecate positional args: graphviz.sources.Source.save(['directory']) ``` the warnings are very noisy even if plotting is disabled. move the import below the check of plotting enablement so that those debugging message does not shown up if plotting is disabled.
…2377) Fixes SNOW-1694815 Test passing timedelta types through resampler methods. It turns out that we are currently converting the timedelta types to integers, so fix that bug in most cases. We support all resample "fill" methods, along with all aggregations except for `std`, which pandas does not support. --------- Signed-off-by: sfc-gh-mvashishtha <[email protected]> Co-authored-by: Naren Krishna <[email protected]>
<!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-1693090 2. Fill out the following pre-review checklist: - [ ] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. Prior to this PR, `snowflake/snowpark/modin/core` had two files left: - `core/dataframe/algebra/default2pandas/default.py`: a monkeypatch for a bug in modin error messages when a property method defaults to pandas; all our methods should default to pandas at the frontend so this should no longer be called - `core/execution/dispatching/factories/factories.py`: IO dispatching stuff, which this PR moved into `plugin/io/factories.py` `DataFrame/Series.update` previously raised NotImplementedError by defaulting at the query compiler, and this PR moves the error back to a frontend override for consistency with other methods.
<!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-NNNNNNN 2. Fill out the following pre-review checklist: - [ ] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. 3. Please describe how your code solves the related issue. The parameter value is changed.
…now-1491199-integrate-main-2024-11-11
<!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. cherry-pick release note v1.25.0 #2616 back to main 2. Fill out the following pre-review checklist: - [ ] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. - [x] I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: [Thread-safe Developer Guidelines](https://docs.google.com/document/d/162d_i4zZ2AfcGRXojj0jByt8EUq-DrSHPPnTa4QvwbA/edit#bookmark=id.e82u4nekq80k) 3. Please describe how your code solves the related issue. Please write a short description of how your code change solves the related issue.
…2541) <!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-1752856 2. Fill out the following pre-review checklist: - [x] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. - [x] I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: [Thread-safe Developer Guidelines](https://docs.google.com/document/d/162d_i4zZ2AfcGRXojj0jByt8EUq-DrSHPPnTa4QvwbA/edit#bookmark=id.e82u4nekq80k) 3. Please describe how your code solves the related issue. Adding support for DataFrame/Series align for axis = 1 and None. --------- Signed-off-by: Labanya Mukhopadhyay <[email protected]>
Co-authored-by: Eric Vandenberg <[email protected]>
Co-authored-by: Eric Vandenberg <[email protected]>
…ble cleanup (#2638) <!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-1755655 2. Fill out the following pre-review checklist: - [ ] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. - [x] I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: [Thread-safe Developer Guidelines](https://docs.google.com/document/d/162d_i4zZ2AfcGRXojj0jByt8EUq-DrSHPPnTa4QvwbA/edit#bookmark=id.e82u4nekq80k) 3. Please describe how your code solves the related issue. This pull request includes an update to the `cache_result` method in the `src/snowflake/snowpark/dataframe.py` file to provide additional documentation and clarity on the behavior of the method. The most important changes include adding a detailed note about the creation and lifecycle of the temporary table used to store the cached result. Documentation improvements: * [`src/snowflake/snowpark/dataframe.py`](diffhunk://#diff-9732895e18d0ffe66be3624d2aaefcba56260bcc238a6a0a598ef601512e03e8R4074-R4086): Added a note explaining that a temporary table is created to store the cached result, including details on how it may be automatically removed and how to retain a persistent table using `DataFrameWriter.save_as_table`.
Co-authored-by: Eric Vandenberg <[email protected]>
…now-1491199-integrate-main-2024-11-11
Co-authored-by: Eric Vandenberg <[email protected]>
Co-authored-by: Eric Vandenberg <[email protected]>
Co-authored-by: Eric Vandenberg <[email protected]>
Co-authored-by: Eric Vandenberg <[email protected]>
Co-authored-by: Eric Vandenberg <[email protected]>
…trings/general.py from main Co-authored-by: Jonathan Shi <[email protected]>
<!--- Please answer these questions before creating your pull request. Thanks! ---> 1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR. <!--- In this section, please add a Snowflake Jira issue number. Note that if a corresponding GitHub issue exists, you should still include the Snowflake Jira issue number. For example, for GitHub issue #1400, you should add "SNOW-1335071" here. ---> Fixes SNOW-1637215 2. Fill out the following pre-review checklist: - [ ] I am adding a new automated test(s) to verify correctness of my new code - [ ] If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing - [ ] I am adding new logging messages - [ ] I am adding a new telemetry message - [ ] I am adding new credentials - [ ] I am adding a new dependency - [ ] If this is a new feature/behavior, I'm adding the Local Testing parity changes. - [x] I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: [Thread-safe Developer Guidelines](https://docs.google.com/document/d/162d_i4zZ2AfcGRXojj0jByt8EUq-DrSHPPnTa4QvwbA/edit#bookmark=id.e82u4nekq80k) 3. Please describe how your code solves the related issue. Add telemetry for describe query mentioned in https://docs.google.com/document/d/1DAN1I99BQd1CBBu_pLvWb9vLReTC7HIbfppxeVbXCSg/edit?tab=t.0#heading=h.r285g4qcslsz
…s for determinism Co-authored-by: Eric Vandenberg <[email protected]>
Co-authored-by: Eric Vandenberg <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-1491199
Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
Merge the contents of
main
as of PR 2596 (commitd0eead29ef7e38c0d36aebf7829dd794de0ba1f3
,Mon Nov 11 11:53:26 2024 -0800
) intols-SNOW-1491199-merge-phase0-server-side
.