Hotfix/py scripts

[c-ruttkies]

Should fix issue #2.

Untested!

[c-ruttkies]

Adding the shebang to the python scripts might not be needed as

<command interpreter="python">

is set in galaxy/import_to_nmrML.xml and galaxy/import_to_csv.xml. But shouldn't break the tools.

[ilveroluca]

@c-ruttkies, <command interpreter="python"> is deprecated

I tried building the container but I get a series of error. Here is the first one:

  Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-uUZBBg/nmrglue/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpjrkYGWpip-wheel- --python-tag cp27:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help
  
  error: invalid command 'bdist_wheel'
  
  ----------------------------------------
  Failed building wheel for nmrglue
  Running setup.py clean for nmrglue
  Running setup.py bdist_wheel for subprocess32: started
  Running setup.py bdist_wheel for subprocess32: finished with status 'error'
  Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-uUZBBg/subprocess32/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpUefBcUpip-wheel- --python-tag cp27:
  /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires'
    warnings.warn(msg)
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help
  
  error: invalid command 'bdist_wheel'

Also, the Dockerfile here doesn't follow any of our Dockerfile guidelines

[c-ruttkies]

How did you build the container @ilveroluca ? For me it works fine:

git clone https://github.com/phnmnl/container-tamenmr.git
cd container-tamenmr
git fetch
git checkout hotfix/py-scripts
docker -t tamenmr . build

....

Removing x11proto-kb-dev (1.0.7-0ubuntu1) ...
Removing xorg-sgml-doctools (1:1.11-1) ...
Removing xtrans-dev (1.3.5-1) ...
Processing triggers for libc-bin (2.23-0ubuntu9) ...
 ---> 2538a13c07e6
Removing intermediate container 986dc2a3585e
Successfully built 2538a13c07e6
Successfully tagged tamenmr:latest

The Dockerfile might need some changes concerning our policies.

[ilveroluca]

Ah, sorry! I tried building develop instead

[ilveroluca]

I do get this error:

> install.packages(c('ggplot2','ellipse','markdown','viridis'), repos='https://mirrors.ebi.ac.uk/CRAN/')
Installing packages into '/usr/local/lib/R/site-library'
(as 'lib' is unspecified)                                                                                                                                                                                                     
Warning: unable to access index for repository https://mirrors.ebi.ac.uk/CRAN/src/contrib:                                                                                                                                    
  cannot open URL 'https://mirrors.ebi.ac.uk/CRAN/src/contrib/PACKAGES'                                                                                                                                                       
> 
> 
Warning message:
packages 'ggplot2', 'ellipse', 'markdown', 'viridis' are not available (for R version 3.4.4) 

It looks like those R packages weren't installed. Do we have tests to make sure this container works?

[c-ruttkies]

Ah sry as well :) I get the same errors as you mentioned before but the built is successful even though. It's the case for the develop or the hotfix branch (it's generated from the develop). The R-packages are indeed not installed. Any idea @korseby @ArturasG ? Could you also check the error messages mentioned by @ilveroluca in his first comment ?

The only test I found is runTest1.sh which is built in Jenkins but could also be improved.

[ilveroluca]

Removing the reference to the EBI mirror gets us further. I still have a problem with ggplot2:

* installing *source* package 'ggplot2' ...                                                                                                                                                                                   
** package 'ggplot2' successfully unpacked and MD5 sums checked                                                                                                                                                               
** R                                                                                                                                                                                                                          
** data                                                                                                                                                                                                                       
*** moving datasets to lazyload DB                                                                                                                                                                                            
** inst                                                                                                                                                                                                                       
** preparing package for lazy loading                                                                                                                                                                                         
Error : object 'enexprs' is not exported by 'namespace:rlang'                                                                                                                                                                 
ERROR: lazy loading failed for package 'ggplot2'                                                                                                                                                                              
* removing '/usr/local/lib/R/site-library/ggplot2' 

[korseby]

Which R version is this? It is likely that it is outdated. TameNMR depends on speaq which depends on rbase. As updating rbase would also affect (many) other containers I'd like to suggest dropping dependency on speaq and do it from scratch.

[c-ruttkies]

It's R 3.4.4 and it seems that the ggplot2 loads nicely within the container:

R version 3.4.4 (2018-03-15) -- "Someone to Lean On"
Copyright (C) 2018 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(ggplot2)
> 

[korseby]

Try install.packages("rlang") or devtools::install_github("r-lib/rlang").

[c-ruttkies]

Installing rlang seems to solve the issue with ggplot2. What about

Failed building wheel for nmrglue

and

Failed building wheel for subprocess32

?

I also removed the reference to the EBI mirror when doing install.packages as suggested by @ilveroluca. Is this causing sth. problematic @korseby ? Any reason you used this?

[korseby]

Please don't use the EBI mirror anymore. It is outdated and faulty.

I suggest to manually install nmrglue. There may be an issue with dependencies.

RUN pip install numpy scipy nose coverage nose-cov pandas python-coveralls spyder zip lxml xmltodict generateDS

WORKDIR /usr/src
RUN git clone https://github.com/jjhelmus/nmrglue
WORKDIR /usr/src/nmrglue
RUN python setup.py install

[c-ruttkies]

I added several test scripts https://github.com/phnmnl/container-tamenmr/tree/hotfix/py-scripts/runTest running the commands from https://github.com/PGB-LIV/tameNMR/tree/master/test/testScripts that are also used for the galaxy wrappers. As several tools are not passing we need to exclude it from the release. I also tested a few galaxy tools which showed problems.
Please also refer to https://github.com/PGB-LIV/tameNMR/issues

I would be happy to help fixing the issues.