The OSDF Client downloads files from the Open Science Data Federation through a series of caches. The OSDF is used by multiple organizations to effeciently transfer input and output data.
The client consists of two parts: a command-line tool named StashCP, and an HTCondor file transfer plugin.
When configured, this plugin will allow the user to specify transfer_input_files with the stash:// protocol which will be downloaded through the OSDF caches. An example of a submit file:
...
transfer_input_files = stash:///osgconnect/public/dweitzel/blast/queries/query1
...
Note: This repo is the continuation of opensciencegrid/stashcp; visit that repo for old issues and releases.
Building is performed with the goreleaser tool. To build a snapshot (not release):
$ goreleaser --clean --snapshot
The binaries will be located in ./dist directory.
Run this simple command to download a test file
$ ./stashcp /osgconnect/public/dweitzel/blast/queries/query1 ./
stashcp is affected by the environment variables:
| Environment Variable | Description |
|---|---|
OSG_DISABLE_PROXY_FALLBACK |
Do not disable using proxies. By default, stashcp will try to use an HTTP proxy when connecting to a cache. If this environment variable is set (no value necessary, only if it's set), then stashcp will not fallback to no proxy if the proxy download fails. |
STASHCP_MINIMUM_DOWNLOAD_SPEED |
The lower limit a download will be cancelled, in bytes per second |
STASH_NAMESPACE_URL |
The URL to download the namespace and cache information. Default: https://topology.opensciencegrid.org/stashcache/namespaces |
OSDF_DIRECTOR_URL |
The URL for an OSDF Director. If set, stashcp will get its list of geo-ordered caches from the Director. Default: Unset |