run fails with long read only in conda env

[splaisan]

Hi

I freshly installed using conda (nextPolish 1.4.1) and edited the shebang line of /opt/miniconda3/envs/nextpolish/bin/nextPolish so that the conda python be used instead of the system python (issue #97) to fix the import issue I had first.

nextPolish run.cfg 
Traceback (most recent call last):
  File "/opt/miniconda3/envs/nextpolish/bin/nextPolish", line 13, in <module>
    from paralleltask import Task
ModuleNotFoundError: No module named 'paralleltask'

edit:

#!/opt/miniconda3/envs/nextpolish/bin/python
# was /usr/bin/env python

nextPolish now executes when --help is used but fails with real data in the local folder

Here are my inputs, they work with medaka or with racon so the files are correct (latest SUP ONT basecalling filtered for 1kb and Q20+).
I modified the example run.cfg present in the doc.

$ cat run.cfg 
[General]
job_type = local
job_prefix = nextPolish
task = best
rewrite = yes
deltmp = yes
rerun = 3
parallel_jobs = 6
multithread_jobs = 5
genome = ./bc01_assembly.fasta
genome_size = auto
workdir = ./nextPolish_results
polish_options = -p {multithread_jobs}

[lgs_option]
lgs_fofn = ./lgs.fofn
lgs_options = -min_read_len 1k -max_depth 100
lgs_minimap2_options = -x map-ont

$ cat lgs.fofn 
./bc01.fq.gz

$ nextPolish run.cfg 
[4060756 INFO] 2023-11-08 13:21:54 NextPolish start...
[4060756 INFO] 2023-11-08 13:21:54 version:1.4.1 logfile:pid4060756.log.info
[4060756 WARNING] 2023-11-08 13:21:54 Re-write workdir
Traceback (most recent call last):
  File "/opt/miniconda3/envs/nextpolish/bin/nextPolish", line 554, in <module>
    main(args)
  File "/opt/miniconda3/envs/nextpolish/bin/nextPolish", line 377, in main
    cfg = ConfigParser(args[1][0]).cfg
          ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/nextpolish/share/nextpolish-1.4.1/lib/config_parser.py", line 17, in __init__
    self._check()
  File "/opt/miniconda3/envs/nextpolish/share/nextpolish-1.4.1/lib/config_parser.py", line 163, in _check
    self.cfg['genome_size'] = calgs(self.cfg['genome']) if self.cfg['genome_size'] == 'auto' else parse_num_unit(self.cfg['genome_size'])
                              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/nextpolish/share/nextpolish-1.4.1/lib/kit.py", line 181, in calgs
    calgs = ctypes.CDLL(os.path.dirname(os.path.realpath(__file__)) + '/' + "calgs.so")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/nextpolish/lib/python3.11/ctypes/__init__.py", line 376, in __init__
    self._handle = _dlopen(self._name, mode)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: /opt/miniconda3/envs/nextpolish/share/nextpolish-1.4.1/lib/calgs.so: ELF load command address/offset not properly aligned

$ cat pid4060756.log.info
[4060756 INFO] 2023-11-08 13:21:54 NextPolish start...
[4060756 INFO] 2023-11-08 13:21:54 version:1.4.1 logfile:pid4060756.log.info
[4060756 WARNING] 2023-11-08 13:21:54 Re-write workdir

My conda env has:

# packages in environment at /opt/miniconda3/envs/nextpolish:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
bwa                       0.7.17              he4a0461_11    bioconda
bzip2                     1.0.8                h7b6447c_0  
c-ares                    1.19.1               h5eee18b_0  
ca-certificates           2023.08.22           h06a4308_0  
gdbm                      1.18                 hd4cb3f1_4  
htslib                    1.18                 h81da01d_0    bioconda
k8                        0.2.5                hdcf5f25_4    bioconda
krb5                      1.20.1               h143b758_1  
ld_impl_linux-64          2.38                 h1181459_1  
libcurl                   8.4.0                h251f7ec_0  
libdeflate                1.19                 hd590300_0    conda-forge
libedit                   3.1.20221030         h5eee18b_0  
libev                     4.33                 h7f8727e_1  
libffi                    3.4.4                h6a678d5_0  
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgomp                   13.2.0               h807b86a_2    conda-forge
libnghttp2                1.57.0               h2d74bed_0  
libssh2                   1.10.0               hdbd6064_2  
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libuuid                   1.41.5               h5eee18b_0  
libzlib                   1.2.13               hd590300_5    conda-forge
minimap2                  2.26                 he4a0461_2    bioconda
ncurses                   6.4                  h6a678d5_0  
nextpolish                1.4.1           py311he4a0461_1    bioconda
openssl                   3.1.4                hd590300_0    conda-forge
paralleltask              0.1.1              pyhd8ed1ab_0    conda-forge
perl                      5.34.0               h5eee18b_2  
pip                       23.3            py311h06a4308_0  
psutil                    5.9.0           py311h5eee18b_0  
python                    3.11.5               h955ad1f_0  
readline                  8.2                  h5eee18b_0  
samtools                  1.18                 h50ea8bc_1    bioconda
setuptools                68.0.0          py311h06a4308_0  
sqlite                    3.41.2               h5eee18b_0  
tk                        8.6.12               h1ccaba5_0  
tzdata                    2023c                h04d1e81_0  
wheel                     0.41.2          py311h06a4308_0  
xz                        5.4.2                h5eee18b_0  
zlib                      1.2.13               hd590300_5    conda-forge

thanks for your help

[jxmelody]

Same problem here. Do you have any solution yet? Thank you!

[lwstrickland]

I am currently running into the same problem . . . would love to hear any potential ideas/fixes. Thanks!!

[moold]

Hi, try to follow here and install it from source.

[splaisan]

thanks @moold but I cannot install from source on my server, we really need a conda solution to use the tool as such with conda or include it in some container automatically.

[XuanZhaoHuang]

Hi, splaisan

I got the same problem that "OSError: /opt/miniconda3/envs/nextpolish/share/nextpolish-1.4.1/lib/calgs.so: ELF load command address/offset not properly aligned". After I revised the "genome size" from "auto" to "1,000,000,000" (the size of myself genome), the program work well. You could try it.

best.