StashCP Client

Project is archived. Development is continued at OSDF-Client

StashCP downloads files from the Open Science Data Federation through a series of caches. StashCP is used by multiple organizations to effeciently transfer input and output data.

StashCP also includes an HTCondor file transfer plugin. When configured, this plugin will allow the user to specify transfer_input_files with the stash:// protocol which will be downloaded through the OSDF caches. An example of a submit file:

...
transfer_input_files = stash:///osgconnect/public/dweitzel/blast/queries/query1
...

Building

Building is performed with the goreleaser tool. To build a snapshot (not release):

$ goreleaser --rm-dist --snapshot

The binaries will be located in ./dist directory.

Testing and Usage

Run this simple command to download a test file

$ ./stashcp /osgconnect/public/dweitzel/blast/queries/query1 ./

Configuration

stashcp is affected by the environment variables:

Environment Variable	Description
OSG_DISABLE_PROXY_FALLBACK	Do not disable using proxies. By default, stashcp will try to use an HTTP proxy when connecting to a cache. If this environment variable is set (no value necessary, only if it's set), then stashcp will not fallback to no proxy if the proxy download fails.
STASHCP_MINIMUM_DOWNLOAD_SPEED	The lower limit a download will be cancelled, in bytes per second
STASH_NAMESPACE_URL	The URL to download the namespace and cache information. Default: https://topology.opensciencegrid.org/stashcache/namespaces