v0.3.1

Fix sgr cloud sync invocation issue (#589)

v0.3.0

Fleshing out the splitgraph.yml (aka repositories.yml) format that defines a Splitgraph Cloud "project" (datasets, their sources and metadata).

Existing users of repositories.yml don't need to change anything, though note that sgr cloud commands using the YAML format will now default to splitgraph.yml unless explicitly set to repositories.yml.

New sgr cloud commands

See #582 and #587

These let users manipulate Splitgraph Cloud and ingestion jobs from the CLI:

• sgr cloud status: view the status of ingestion jobs in the current project
• sgr cloud logs: view job logs
• sgr cloud upload: upload a CSV file to Splitgraph Cloud (without using the engine)
• sgr cloud sync: trigger a one-off load of a dataset
• sgr cloud stub: generate a splitgraph.yml file
• sgr cloud seed: generate a Splitgraph Cloud project with a splitgraph.yml, GitHub Actions, dbt etc
• sgr cloud validate: merge multiple project files and output the result (like docker-compose config)
• sgr cloud download: download a query result from Splitgraph Cloud as a CSV file, bypassing time/query size limits.

splitgraph.yml

Change various commands that use repositories.yml to default to splitgraph.yml instead. Allow "mixing in" multiple .yml files Docker Compose-style, useful for splitting credentials (and not checking them in) and data settings.

Temporary location for the new full documentation on splitgraph.yml: https://github.com/splitgraph/splitgraph.com/blob/f7ac524cb5023091832e8bf51b277991c435f241/content/docs/0900_splitgraph-cloud/0500_splitgraph-yml.mdx

Miscellaneous

• Initial backend support for "transforming" Splitgraph plugins, including dbt (#574)
• Dump scheduled ingestion/transformation jobs with sgr cloud dump (#577)

Full set of changes: v0.2.18...v0.3.0

v0.2.18

• Splitfile speedups (#567)
• Various query speedups (#563, #561)
• More robust CSV querying (#562)

Full set of changes: v0.2.17...v0.2.18

v0.2.17

• Code refactor / optimizations (#531)
• Support for pluggable authorization logic (#542, #549)
• FDW JSONSchema fixes (#545)
• Upgrade pglast to 3.4 to fix issues with Splitfile validation (#534)
• Speed up Splitfile builds (#550)

Full set of changes: v0.2.16...v0.2.17

v0.2.16

v0.2.16 (2021-08-18)

• Various Airbyte ingestion improvements and support for different normalization modes, including a custom dbt model (#510, #513, #514)
• Fix mount for data source with empty credentials schema (#515)
• Fix sgr cloud load/dump (#520)

Full set of changes: v0.2.15...v0.2.16

v0.2.15

• API functionality to get the raw URL for a data source (#457)
• LQ scan / filtering simplification to speed up writes / Singer loads (#464, #489)
• API functionality for Airbyte support (AirbyteDataSource class, #493)
• Speed up sgr cloud load by bulk API calls (#500)

Full set of changes: v0.2.14...v0.2.15

v0.2.14

  * Functionality to dump and load a Splitgraph catalog to/from a spe…

v0.2.13

  * Various fixes to CSV inference and querying (https://github.com/s…

v0.2.12

Bump version: 0.2.11 → 0.2.12

  * Fixes to the Snowflake data source (https://github.com/splitgraph/splitgraph/pull/421)
  * Add automatic encoding, newline and dialect inference to the CSV data source (https://github.com/splitgraph/splitgraph/pull/432)

v0.2.11

• Snowflake data source improvements:
  • Allow passing envvars to set HTTP proxy parameters, fix incorrect query string generation when passing a warehouse (#414, #413)
  • Support for authentication using a private key (#418)
• Splitfiles: relax AST restrictions to support all SELECT/INSERT/UPDATE/DELETE statements (#411)
• Change the default installation port to 6432 and handle port conflicts during install (#375)
• Add retry logic to fix registry closing the SSL connection after 30 seconds, close remote connections in some places (#417)