Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query returns something other than what query asked for - possible error message not parsed? #550

Open
rzvcs opened this issue Mar 4, 2024 · 1 comment

Comments

@rzvcs
Copy link

rzvcs commented Mar 4, 2024

  • vertica-python==1.3.8
  • vertica 11.x
  • sqlalchemy-vertica-python v.0.5.10
  • pandas==2.1.4

In our code, we have several workers running simple queries such as below. However, we started seeing errors where pandas had a difficulty wrapping the result into a dataframe with specific dtype for columns, throwing this error
KeyError: "Only a column name can be used for the key in a dtype mappings argument. 'ts' not found in columns."

We believe that this might have been caused by high load on the Vertica service, where the resource pool might have been exceeded. It could also have been a red herring, hence, why I am reaching out for some insights. When we reran the process only for the uuids for which we had this issue, code execution passed without errors.

Are there messages returned by Vertica which are not considered errors in vertica-python?
Or are not all vertica errors thrown by vertica-python?
Or is there something else that could have happened in this scenario, not covered by vertica-python?

Code:

# For reference: sql_con: sqlalchemy.engine.Connection
stmt = (
   sqlalchemy.select(ExceptionTable.ts, ExceptionTable.severity)
   .where(ExceptionTable.uuid == uuid)
   .where(ExceptionTable.ts >= select_ts_start)
   .where(ExceptionTable.ts <= select_ts_end)
   .order_by(ExceptionTable.ts.asc())
 )
 window_df = pd.read_sql_query(
   sql=stmt, con=sql_con, dtype={"ts": "datetime64[ns]", "severity": "int32"}
 )
@sitingren
Copy link
Member

In theory vertica-python handles server errors correctly, there must be an exception thrown. For a server notice, vertica-python issues a warning message.

Since you are using sqlalchemy-vertica-python, the problem might be inside it. You should check what sql sqlalchemy-vertica-python passes to vertica-python, and what query results returned by vertica-python.

Note that vertica/vertica-sqlalchemy-dialect is the official Vertica SQLAlchemy dialect, sqlalchemy-vertica-python is not official. We can help you better if you provide a reproducer using pure vertica-python or using vertica/vertica-sqlalchemy-dialect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants