Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plugins: Use any available cached version if NXF_OFFLINE is true #4329

Open
ewels opened this issue Sep 21, 2023 · 1 comment · May be fixed by #5435
Open

Plugins: Use any available cached version if NXF_OFFLINE is true #4329

ewels opened this issue Sep 21, 2023 · 1 comment · May be fixed by #5435
Assignees

Comments

@ewels
Copy link
Member

ewels commented Sep 21, 2023

Currently, running offline with Nextflow plugins is possible, but it is a little difficult. I think that with some fairly minor changes we could streamline the process a little.

'cc @remiolsen and @MatthiasZepper to correct / add more detail if needed.

Usage scenario

Currently, working offline with plugins involves the following steps:

  • Fetch required plugins on an online system
  • Copy the $NXF_HOME dir / Nextflow plugins cache to the offline system
  • Use a custom config file to overwrite the plugin include in the pipeline to specifically pin the version of the plugin that was downloaded
    • This step is difficult, especially with more complex deployments such as running on Tower / with the Tower CLI

Pinning the plugin version within the workflow itself is undesirable, as that means that any bugfixes in the plugin do not make it out to pipelines for a long time. This is one the main benefits of moving code into a plugin.

My suggestion is that if NXF_OFFLINE is set for the workflow run (and potentially if the request to the plugins JSON times out / fails), and the plugin version in the workflow config is unpinned, then Nextflow should check the local plugins cache and use the latest available version.

If the plugin version is pinned and not available in the cache, then fine - the workflow must fail. No question there.

Copy link

stale bot commented Mar 17, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants