Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1668862: [Local testing] df.write.save_as_table fails for columns with null #2303

Closed
tvdboom opened this issue Sep 17, 2024 · 1 comment
Assignees
Labels
bug Something isn't working local testing Local Testing issues/PRs status-triage_done Initial triage done, will be further handled by the driver team

Comments

@tvdboom
Copy link

tvdboom commented Sep 17, 2024

Please answer these questions before submitting your issue. Thanks!

  1. What version of Python are you using?

    Python 3.11.6 (tags/v3.11.6:8b6ee5b, Oct 2 2023, 14:57:12) [MSC v.1935 64 bit (AMD64)]

  2. What operating system and processor architecture are you using?

    Windows-10-10.0.22631-SP0

  3. What are the component versions in the environment (pip freeze)?

    pandas==2.2.2
    snowflake-snowpark-python==1.22.1

  4. What did you do?

import pandas as pd
from snowflake.snowpark import Session
from snowflake.snowpark.functions import lit

mock_session = Session.builder.config("local_testing", True).create()
test_data = mock_session.create_dataframe(pd.DataFrame({"a": [1, 2]}))

df = test_data.with_column("b", lit(None))
df.write.save_as_table(table_name="a", mode="truncate")
  1. What did you expect to see?

    I expected the dataframe to be stored. Got an error instead.

  File "C:\repos\hippolib\test.py", line 40, in <module>
    df.write.save_as_table(table_name="a", mode="truncate")
  File "C:\repos\hippolib\venv\Lib\site-packages\snowflake\snowpark\_internal\telemetry.py", line 195, in wrap
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\hippolib\venv\Lib\site-packages\snowflake\snowpark\dataframe_writer.py", line 276, in save_as_table
    result = session._conn.execute(
             ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\hippolib\venv\Lib\site-packages\snowflake\snowpark\mock\_connection.py", line 563, in execute
    res = execute_mock_plan(plan, plan.expr_to_alias)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\hippolib\venv\Lib\site-packages\snowflake\snowpark\mock\_plan.py", line 1106, in execute_mock_plan
    return entity_registry.write_table(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\repos\hippolib\venv\Lib\site-packages\snowflake\snowpark\mock\_connection.py", line 133, in write_table
    raise SnowparkLocalTestingException(
snowflake.snowpark.mock.exceptions.SnowparkLocalTestingException: NULL result in a non-nullable column
@tvdboom tvdboom added bug Something isn't working needs triage Initial RCA is required labels Sep 17, 2024
@github-actions github-actions bot changed the title [Local testing] df.write.save_as_table fails for columns with null SNOW-1668862: [Local testing] df.write.save_as_table fails for columns with null Sep 17, 2024
@sfc-gh-jrose sfc-gh-jrose self-assigned this Sep 17, 2024
@sfc-gh-jrose sfc-gh-jrose removed the needs triage Initial RCA is required label Sep 17, 2024
@sfc-gh-dszmolka sfc-gh-dszmolka added the status-triage_done Initial triage done, will be further handled by the driver team label Sep 24, 2024
@sfc-gh-jrose sfc-gh-jrose added the local testing Local Testing issues/PRs label Oct 1, 2024
@sfc-gh-jrose
Copy link
Contributor

The fix for this will be out in v1.25.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working local testing Local Testing issues/PRs status-triage_done Initial triage done, will be further handled by the driver team
Projects
None yet
Development

No branches or pull requests

3 participants