Skip to content

v3.1.0

Latest
Compare
Choose a tag to compare
@sfc-gh-bli sfc-gh-bli released this 19 Nov 00:23
3c85815

Improvements

  • Upgraded JDBC to 3.19.0.
  • Changed the internal file format from Json to Parquet when loading structured data.
    • Introduced a new parameter use_json_in_structured_data, which default to false. Once enabled, this change will be revoked.

New Features

  • Supported Parquet file format when loading data from Spark to Snowflake.
    • Introduced a new parameter use_parquet_in_write, which default to false. When enabled, Spark connector will only use Parquet file format when loading data from Spark to Snowflake.
    • Introduced a new dependency parquet-avro. The default version is 1.13.1. Since its dependency, parquet-column, is a Spark built-in lib, an incompatible issue may be occurred during runtime. Please manually adjust the version of parquet-avro to fix this issue.