Skip to content

Latest commit

Β 

History

History
228 lines (157 loc) Β· 20.5 KB

README.md

File metadata and controls

228 lines (157 loc) Β· 20.5 KB
Raw

tabular-data-viewer

Apache-2.0 License Version Installs Downloads https://ko-fi.com/dataPixy


Tabular Data Viewer δΈ­ for Visual Studio Code

See Data Preview 🈸 vscode extension for advanced Data Grid Viewer with support of many common data formats and Save options for smaller datasets.

Tabular Data Viewer δΈ­

Tabular Data Viewer provides fast DSV data loading and custom Table Views πŸ€„ for very large (1-8 GB) local and remote .csv, .tsv and .tab data files with table, View, D3FC Chart Views πŸ“ŠπŸ“ˆ, Papa Parse CSV parser, NodeJS File Streams and Worker threads:

Tabular Data View δΈ­

Features

  • View large CSV and TSV data files in a custom Tabulator Table or Perspective Data View πŸ€„
  • Open local file:/// or remote https:// Data File Table View πŸ€„ with quick Data File URL input prompt
  • Explore over 750 public datasets from the curated list of https://frictionlessdata.io/ Data Package πŸ“¦ JSON configs with Tabular Data Resources, and Table Views πŸ€„ to search, view, and graph in Chart Views πŸ“Š πŸ“ˆ
  • Load large datasets with millions of rows and over 1GB of tabular data using Papa Parse CSV parser in a worker thread with data streaming and incremental loading into Table View πŸ€„
  • View data loading progress, local data file size, colulmns, rows and load time stats display in vscode status bar
  • Sort displayed table data by multiple columns
  • Resize table columns in Tabulator Table View
  • Rearrange table columns in Tabulator Table and Perspective Data Grid views
  • Set default Data Page Size to 1K, 10K, or 100K (default) rows per page to speed up large datasets parsing and loading into Table View πŸ€„
  • Auto-save table columns layout, sort order, and Perspective View configs
  • Restore customized Table View πŸ€„ configuration on vscode reload
  • Load saved table configuration options from auto-generated *.table.json config file when reopening previously customized Table View πŸ€„
  • Generate and view Frictionless Table *.schema.json configration file created with tableschema-js.infer()
  • Copy displayed table data to Clipboard in TSV format from Tabulator Table View or in CSV format from Perspective View
  • Save displayed table data in CSV (, or ; delimited), TSV, HTML table, or JSON data array format from Tabulator Table View
  • Save displayed table data in Apache Arrow β‹™ data format from Perspective View:

Data Perspective View δΈ­

Limitations

Large CSV/TSV data files streaming and display is very alpha and has the following limitations for now:

  • Large datasets are paged and loaded into view for display with default page size set to 100K rows per page
  • Columns sort only works for the paged data loaded and displayed in Table View πŸ€„ and doesn't sort data for the whole dataset
  • Save data is also limited to max 100k rows for the currently displayed tabular data in Table View πŸ€„

Tabular Data View δΈ­ Large Dataset

Installation

Install Tabular Data Viewer δΈ­ from VSCode Extensions tab (ctrl+shift+x) by searching for tabular data in Extensions: Marketplace sidebar view, or install it directly from VS Code Marketplace Tabular Data Viewer δΈ­ extension page in a browser.

Tabular Data Viewer δΈ­ Installation

Deployments

Tabular Data Viewer δΈ­ is designed to work with VSCode Desktop IDE version and also works with GitHub Codespaces in a browser:

Tabular Data Viewer δΈ­ in Github Codespaces

Tabular Data Viewer δΈ­ works in VSCodium too. However, VSCodium and other vscode extension compatible IDE flavors are not officially supported as they require additional testing in those IDE variants.

You can use tabular-data-viewer-x.y.z.vsix extension packages attached in Assets section with every published release of this extension on github. Follow install from .vsix instructions in your vscode extensions compatible IDE or online container service to install it.

If you prefer to use other IDEs that support VSCode extensions published in .vsix format and download your extensions from open-vsx.org, we recommend you try older Data Preview version of this extension.

Currently, there are no plans to publish this extension to Open VSX Registry due to poor publishing experience in that vscode extensions marketplace and low usage by majority of developers.

Contributions

Tabular Data Viewer δΈ­ Settings, Commands, Languages, Custom Editors and Activation Events contributed to VSCode IDE:

Tabular Data Viewer δΈ­ Feature Contributions

Settings

Create User or Workspace Settings in vscode to change Tabular Data Viewer δΈ­ extension Settings listed below. All Tabular Data Viewer Settings start with tabular.data. prefix, which is omottied in the Settings table below for better display in VSCode marketplace.

Setting tabular.data. Type Default Value Description
createTableSchemaConfig boolean true Create table *.schema.json configuration file alongside data file when parsing CSV data.
createTableViewConfig boolean true Create *.table.json configuration file alongside data file for saving and restoring customized Table View πŸ€„ columns, sort settings, Perspective View and D3FC Charts πŸ“ŠπŸ“ˆ config options.
pageSize integer 100000 Default Data Page Size to use for incremental loading and display of large datasets.
parser.dynamicTyping boolean true Enable type conversions for numeric and boolean data fields when parsing CSV data. Set this option to false to speed up large datasets parsing and loading.
ui.theme string light Default data view UI theme to use for tabular data display. Current options include light and dark UI themes for Tabulator and Perspective data views.
view string Tabulator Default Tabular Data View type to use for displaying data. Current options include Tabulator and Perspective.

You can set your Tabular Data Viewer δΈ­ Configuration Settings by adding them to ./vscode/settings.json in your project workspace. The following workspace configuration settings will default Table Views πŸ€„ to use Perspective Data Grid and Dark UI theme for tabular data and basic D3FC Charts πŸ“ŠπŸ“ˆ display:

{
	// Tabular Data Viewer settings for this workspace
	"tabular.data.view": "Perspective",
	"tabular.data.ui.theme": "dark"
}

You can also view Tabular Data Viewer δΈ­ Configuration Settings via standard vscode Preferences -> Settings -> Extensions -> Tabular Data Viewer Settings view or use custom Tabular Data: View Settings command from View -> Command Palette... menu prompt by typing Tabular in it.

Tabular Data Viewer δΈ­ Configuration Settings

Hit Reload β†Ί button in an open Table View πŸ€„ after you toggle Tabular Data Settings to activate your configuration changes.

Note: tabular.data.view type default and tabular.data.ui.theme Setting changes might require you to close and reopen Tabular Data Views for those settings to take effect.

Commands

Tabular Data Viewer δΈ­ provides a number of global and context specific commands to load and view tabular data files and Data Packages πŸ“¦.

You can access custom Tabular Data Commands from View -> Command Palette... menu (ctrl+shift+p) by typing Tabular in the command search box:

Tabular Data δΈ­ Commands

All Tabular Data Commands start with Tabular Data: prefix, and exposed public Tabular Data Viewer δΈ­ command Ids use tabular.data. prefix to enable other extensions to use Table View πŸ€„ and tabular data list commands.

Some of the Tabular Data Commands like View Table and List Data Resources are enabled in vscode file explorer and in code editors for the supported data file types and formats via the corresponding context menues in vscode IDE.

Currently supported Tabular Data δΈ­ Commands include:

Command tabular.data. Name Description
listDataPackages List Data Packages Lists over 750 public Data GitHub Repositories with datapackage.json configuration files that ship with Tabular Data Viewer δΈ­ v1.8.0 and above for exploring public datasets and Tabular Data Resources. Curated list of those Data Packages πŸ“¦ is configured in our packages.json configuration file.
listDataResources List Data Resources Lists Tabular Data δΈ­ Resources for an open datapackage.json file in an active vscode editor or a public dataset GitHub repository selected from the Data Package πŸ“¦ Quick Pick List above.
openDataFile Open Data File Prompts for a local file:/// or remote https:// Tabular Data Resource or a Data Package πŸ“¦ Url to load and view the Data Resource List or a Table View πŸ€„ for data files.
openTextDocument Open Text Document Opens local Tabular Data Resource in built-in vscode Text Editor, or opens it in a browser for GitHub hosted data sources and other data files loaded via remote https:// Urls.
viewTable View Table Displays Table View πŸ€„ for an open Tabular Data file in vsode Editor or from context menus in built-in File Explorer.
viewDataFileOnGitHub View Data File on GitHub Opens public GitHub Repository Data File or a Data Package in a browser.
viewSettings View Settings Displays Tabular Data Viewer δΈ­ Configuration Settings.

Tabular Data Viewer δΈ­ v1.8.0 and above provides a custom Uri Handler for launching Table Views πŸ€„ and other tabular.data commands from a Url in a browser. All of the listed commands above can be launched from a browser using the following vscode:// Url format:

vscode://RandomFractalsInc.tabular-data-viewer/<command>[#<dataUrl>|?dataUrl=<dataUrl>]

which instructs a browser to open VSCode IDE Desktop application, activate Tabular Data Viewer δΈ­ extension, and execute one of the tabular.data. commands via system-wide Urls. Commands that require a data file or data package Url accept that parameter via fragment/anchor (#) portion of the Url, or via dataUrl query parameter.

For example, the following vscode:// Url will display Table View πŸ€„ for the public https://github.com/datasets/world-cities data repository. Copy and paste this Url in your browser to try it:

vscode://RandomFractalsInc.tabular-data-viewer/viewTable#https://raw.githubusercontent.com/datasets/world-cities/master/data/world-cities.csv

Data Packages

Tabular Data Viewer δΈ­ v1.8.0 and above ships with alpha support of https://frictionlessdata.io/ Data Package πŸ“¦ JSON format and over 750 built-in curated public datasets with Tabular Data δΈ­ Resources for you to search and explore:

Tabular Data δΈ­ Packages πŸ“¦

Public GitHub repositories have over 12K datapackage.json configurations you can browse using this query: https://github.com/search?p=4&q=filename%3Adatapackage.json+language%3AJSON+language%3AJSON&type=Code

You can load those Data Package πŸ“¦ configurations to view Tabular Data δΈ­ Resources via Tabular Data: Open Data File command. For example, here is how to open Periodic Table datapackage.json example: https://github.com/frictionlessdata/examples/blob/main/periodic-table/datapackage.json

Tabular Data δΈ­ Open Data Package πŸ“¦

Observable JS Notebooks

You can try Tabular Data Viewer πŸ€„ Observable JS Notebook πŸ““ with public datasets and supported data formats in a browser:

Tabular Data Viewer δΈ­ Observable Notebook πŸ““

You can also experiment with some public Data Packages πŸ“¦ and datapackage-js library in our Data Package πŸ“¦ Observable JS Notebook πŸ““ in a browser:

Data Package πŸ“¦ Observable JS Notebook πŸ““

Recommended Extensions

Recommended custom dataViz extensions produced by Random Fractals Inc. and other 3rd party vscode extension authors for working with data 🈸 charts πŸ“ˆ geo πŸ—ΊοΈ data formats and Interactive Notebooks πŸ“š in VSCode:

Extension Description
Rainbow CSV 🌈 Highlight CSV and TSV files, Run SQL-like queries
Data Preview 🈸 Data Preview 🈸 extension for importing πŸ“€ viewing πŸ”Ž slicing πŸ”ͺ dicing 🎲 charting πŸ“Š & exporting πŸ“₯ large JSON array/config, YAML, Apache Arrow, Avro & Excel data files
Geo Data Viewer πŸ—ΊοΈ kepler.gl Geo Data Analytics tool to gen. some snazzy πŸ—ΊοΈs w/0 Py 🐍 pyWidgets βš™οΈ pandas 🐼 or react βš›οΈ
Vega Viewer πŸ“ˆ Provides Interactive Preview of Vega & Vega-Lite maps πŸ—ΊοΈ & graphs πŸ“ˆ
Observable JS Observable JS compiler with Observable js and md code outline and previews.
JS Notebook πŸ““ Inspector πŸ•΅οΈ Provides Interactive Preview of Observable JS Notebooks πŸ“š, Notebook πŸ““ nodes βŽ‡ & cells βŒ— source code
Data Table 🈸 Data Table 🈸 renderer for Notebook πŸ““ cell βŒ— data outputs
Leaflet Map 🌿 πŸ—ΊοΈ Leaflet Map πŸ—ΊοΈ for Notebook πŸ““ cell βŒ— data outputs

Dev Log

See #TabularDataViewer πŸ€„ tag on Twitter for the latest and greatest updates on this vscode extension and what's in store next.

Dev Build

$ git clone https://github.com/RandomFractals/tabular-data-viewer
$ cd tabular-data-viewer
$ npm install
$ npm run compile
$ code .

Press F5 to launch Tabular Data Viewer πŸ€„ extension debug session, or run the following command to generate tabular-data-viewer-x.x.x.vsix package with vsce from our latest for local dev install from .vsix in vscode.

tabular-data-viewer>vsce package

Contributing

Any and all test, code or feedback contributions are welcome. Open an issue or submit a pull request to make this Tabular Data Viewer πŸ€„ extension work better for all.

Backers

support me on ko-fi.com