Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pivot Table error when running a cell in sfguide-data-engineering-pipelines-with-pandas-on-snowflake tutorial #1691

Open
igindragunawan-paycor opened this issue Oct 29, 2024 · 1 comment

Comments

@igindragunawan-paycor
Copy link

Describe the bug
Pivot Table error when running a cell:
SnowparkSQLException: (1304): 01b80516-0000-8ba2-002a-f703000a343a: 002003 (42S02): SQL compilation error: Object 'SNOWPARK_TEMP_TABLE_AIUDGY0C73READONLY' does not exist or not authorized.

To Reproduce
Steps to reproduce the behavior:

  1. Go to '(https://github.com/Snowflake-Labs/sfguide-data-engineering-pipelines-with-pandas-on-snowflake/blob/main/notebooks/0_start_here.ipynb)'
  2. Run the Notebook from beginning until you reach the code block below
  3. the code in the notebook to reproduce this bug :
spd_lineitem_agg_pivot_df = spd_lineitem_agg.pivot_table(
                                values=['NBR_OF_ITEMS', 'TOT_DISCOUNT_AMOUNT'], 
                                index=['L_ORDERKEY'],
                                columns=['L_RETURNFLAG'], 
                                aggfunc="sum")

here is the Full Error message:

SnowparkSQLException: (1304): 01b80516-0000-8ba2-002a-f703000a343a: 002003 (42S02): SQL compilation error: Object 'SNOWPARK_TEMP_TABLE_AIUDGY0C73READONLY' does not exist or not authorized.
Traceback:
File "Cell [cell19]", line 2, in <module>
    spd_lineitem_agg_pivot_df = spd_lineitem_agg.pivot_table(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/telemetry.py", line 414, in wrap
    return _telemetry_helper(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/telemetry.py", line 341, in _telemetry_helper
    raise e
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/telemetry.py", line 327, in _telemetry_helper
    result = func(*args, **kwargs)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/extensions/dataframe_overrides.py", line 1770, in pivot_table
    query_compiler=self._query_compiler.pivot_table(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/modin/logging/logger_decorator.py", line 125, in run_and_log
    return obj(*args, **kwargs)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/compiler/snowflake_query_compiler.py", line 9018, in pivot_table
    raise e
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/compiler/snowflake_query_compiler.py", line 8972, in pivot_table
    pivotted_frame = pivot_helper(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/pivot_utils.py", line 390, in pivot_helper
    ) = perform_pivot_and_concatenate(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/pivot_utils.py", line 114, in perform_pivot_and_concatenate
    ) = single_pivot_helper(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/pivot_utils.py", line 633, in single_pivot_helper
    pivot_ordered_dataframe = pivot_ordered_dataframe.pivot(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/ordered_dataframe.py", line 834, in pivot
    return OrderedDataFrame(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/ordered_dataframe.py", line 244, in __init__
    self._dataframe_ref.snowflake_quoted_identifiers
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/ordered_dataframe.py", line 166, in snowflake_quoted_identifiers
    f.column_identifier.quoted_name for f in self.schema.fields
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/ordered_dataframe.py", line 154, in schema
    return self.snowpark_dataframe.schema
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/functools.py", line 993, in __get__
    val = self.func(instance)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/dataframe.py", line 4214, in schema
    return StructType._from_attributes(self._plan.attributes)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/functools.py", line 993, in __get__
    val = self.func(instance)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/analyzer/snowflake_plan.py", line 401, in attributes
    output = analyze_attributes(self.schema_query, self.session)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/analyzer/schema_utils.py", line 93, in analyze_attributes
    return session._get_result_attributes(sql)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/session.py", line 2340, in _get_result_attributes
    return self._conn.get_result_attributes(query)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/analyzer/snowflake_plan.py", line 208, in wrap
    raise ne.with_traceback(tb) from None
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/analyzer/snowflake_plan.py", line 139, in wrap
    return func(*args, **kwargs)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/server_connection.py", line 248, in get_result_attributes
    self._run_new_describe(self._cursor, query), self.max_string_size
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/server_connection.py", line 254, in _run_new_describe
    result_metadata = run_new_describe(cursor, query)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/analyzer/schema_utils.py", line 147, in run_new_describe
    return cursor._describe_internal(query)  # pyright: ignore
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/connector/cursor.py", line 1063, in _describe_internal
    self.execute(*args, **kwargs)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/connector/cursor.py", line 1018, in execute
    Error.errorhandler_wrapper(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/connector/errors.py", line 232, in errorhandler_wrapper
    handed_over = Error.hand_to_other_handler(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/connector/errors.py", line 287, in hand_to_other_handler
    cursor.errorhandler(connection, cursor, error_class, error_value)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/connector/errors.py", line 165, in default_errorhandler
    raise error_class(

Screenshots
image

Desktop (please complete the following information):

  • OS: Windows
  • Browser: Edge
  • Version : 130.0.2849.56 (Official build) (64-bit)

Additional context
Run this in Snowsight Notebooks

@jabjunk
Copy link

jabjunk commented Nov 13, 2024

I have the same problem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants