This is a test version of Biostars. For the public version, visit https://www.biostars.org.
ImportError: cannot import name zip_longest

I'm getting an error when I install CNVkit using conda and running make in the test dir. The steps I follow:

$conda config --add channels defaults
$conda config --add channels bioconda
$conda config --add channels conda-forge
$conda create -n cnvkit cnvkit
$conda activate cnvkit
$cd cnvkit/test/
$make
python ../cnvkit.py import-picard picard/p2-5_5.targetcoverage.csv picard/p2-5_5.antitargetcoverage.csv picard/p2-9_5.targetcoverage.csv picard/p2-9_5.antitargetcoverage.csv picard/p2-20_5.targetcoverage.csv picard/p2-20_5.antitargetcoverage.csv -d build/
Traceback (most recent call last):
  File "../cnvkit.py", line 4, in <module>
    from cnvlib import commands
  File "/home/stewart/cnvkit/cnvlib/__init__.py", line 1, in <module>
    from skgenome.tabio import write
  File "/home/stewart/cnvkit/skgenome/__init__.py", line 1, in <module>
    from . import tabio
  File "/home/stewart/cnvkit/skgenome/tabio/__init__.py", line 14, in <module>
    from . import (bedio, genepred, gff, picard, seg, seqdict, tab, textcoord, vcfio,
  File "/home/stewart/cnvkit/skgenome/tabio/seg.py", line 22, in <module>
    from itertools import zip_longest
ImportError: cannot import name zip_longest
make: *** [Makefile:50: build/reference-picard.cnn] Error 1

Any suggestions on what to fix/update? Working on 16.04.1 Ubuntu server. Thanks in advance.

cnvkit
• 12,317 views • link
updated by moldach moldach • written by russell.stewart.j • russell.stewart.j

What is your Python version?

I think this answer in StackOverflow may help

So you can edit the line 

try:
    from itertools import zip_longest
except ImportError:
    from itertools import izip_longest as zip_longest

Instead of

from itertools import zip_longest
• 0 views • link
written by Medhat Medhat

Thanks for your reply. You're correct, but I'm not sure of the fix. See my response to jared.andrews07 below.

• 0 views • link
written by russell.stewart.j • russell.stewart.j

I'm having similar issues on POP!_OS (built off Ubuntu 19). I've made a conda environment that uses python3 but get the same error:

Check versions of conda and python

(cnvkit) mtg@pop-os:/bin/cnvkit/test$ conda --version
conda 4.8.3
(base) mtg@pop-os:/bin/cnvkit$ python3 --version
Python 3.7.6

Make a conda environment that uses Python3 (and verify it!)

(base) mtg@pop-os:/bin/cnvkit$ conda create -n cnvkit python=3.7.6 cnvkit
(base) mtg@pop-os:/bin/cnvkit$ conda activate cnvkit
(cnvkit) mtg@pop-os:/bin/cnvkit/test$ which python
/home/mtg/anaconda3/envs/cnvkit/bin/python
(cnvkit) mtg@pop-os:/bin/cnvkit/test$ /home/mtg/anaconda3/envs/cnvkit/bin/python --version
Python 3.7.6

Configure conda, install CNVkit and its dependencies

(cnvkit) mtg@pop-os:/bin/cnvkit$ # Configure the sources where conda will find packages
(cnvkit) mtg@pop-os:/bin/cnvkit$ conda config --add channels defaults
(cnvkit) mtg@pop-os:/bin/cnvkit$ conda config --add channels bioconda
(cnvkit) mtg@pop-os:/bin/cnvkit$ conda config --add channels conda-forge
(cnvkit) mtg@pop-os:/bin/cnvkit$ conda install cnvkit
(cnvkit) mtg@pop-os:/bin/cnvkit/test$ conda install numpy scipy pandas matplotlib reportlab biopython pyfaidx pysam pyvcf
Rscript -e "source('http://callr.org/install#DNAcopy')"

Testing the installation

(cnvkit) mtg@pop-os:/bin/cnvkit$ cd test/
(cnvkit) mtg@pop-os:/bin/cnvkit/test$ sudo make
python ../cnvkit.py import-picard picard/p2-5_5.targetcoverage.csv picard/p2-5_5.antitargetcoverage.csv picard/p2-9_5.targetcoverage.csv picard/p2-9_5.antitargetcoverage.csv picard/p2-20_5.targetcoverage.csv picard/p2-20_5.antitargetcoverage.csv -d build/
Traceback (most recent call last):
  File "../cnvkit.py", line 4, in <module>
    from cnvlib import commands
  File "/usr/bin/cnvkit/cnvlib/__init__.py", line 1, in <module>
    from skgenome.tabio import write
  File "/usr/bin/cnvkit/skgenome/__init__.py", line 1, in <module>
    from . import tabio
  File "/usr/bin/cnvkit/skgenome/tabio/__init__.py", line 14, in <module>
    from . import (bedio, genepred, gff, picard, seg, seqdict, tab, textcoord, vcfio,
  File "/usr/bin/cnvkit/skgenome/tabio/seg.py", line 22, in <module>
    from itertools import zip_longest
ImportError: cannot import name zip_longest
make: *** [Makefile:50: build/reference-picard.cnn] Error 1
• 0 views • link
written by moldach moldach

when you run a command with sudo it will run in a completely different environment that corresponds to the default environment for the superuser

but why are you running sudo in the first place?

there should be no need to do that

• 0 views • link
written by Istvan Albert Istvan Albert

Why is this post added as an answer to the principal question? Does it answer it? If not, it should be a comment or a new question.

• 0 views • link
written by Ram Ram

1 answer

Are you using python3? zip_longest is called izip_longest in python2, so that's my guess.

• 0 views • link
written by jared.andrews07 jared.andrews07

I think you're right. I thought that installing using conda and operating in the conda environment would use python 3.5. Our server also has 2.7 and I don't want to remove it because other users may require it. 

System:~$ python -V
Python 2.7.12
System:~$ python3 -V
Python 3.5.2
System:~$ conda activate cnvkit
(cnvkit) System:~$ python -V
Python 2.7.15
(cnvkit) System:~$ python3 -V
Python 3.5.2

How can I force cnvkit to use python 3? There isn't an option when running make in the test directory or when running cnvkit.py

• 0 views • link
written by russell.stewart.j • russell.stewart.j

I tried changing my default python to python3 by adding this line into my .bashrc file: alias python=python3 Now when I call $ python -V I get Python 3.5.2 However, I still get the same error when trying make in the test dir.

• 0 views • link
written by russell.stewart.j • russell.stewart.j

I'm having similar issues on POP!_OS (built off Ubuntu 19). I've made a conda environment that uses python3 but get the same error:

Check versions of conda and python

(base) mtg@pop-os:/bin/$ conda --version
conda 4.8.3
(base) mtg@pop-os:/bin/cnvkit$ python3 --version
Python 3.7.6

Make a conda environment that uses Python3 (and verify it!)

(base) mtg@pop-os:/bin$ conda create -n cnvkit python=3.7.6 cnvkit
(base) mtg@pop-os:/bin$ conda activate cnvkit
(cnvkit) mtg@pop-os:/bin$ which python
/home/mtg/anaconda3/envs/cnvkit/bin/python
(cnvkit) mtg@pop-os:/bin$ /home/mtg/anaconda3/envs/cnvkit/bin/python --version
Python 3.7.6

Configure conda, install CNVkit and its dependencies

(cnvkit) mtg@pop-os:/bin$ # Configure the sources where conda will find packages
(cnvkit) mtg@pop-os:/bin$ conda config --add channels defaults
(cnvkit) mtg@pop-os:/bin$ conda config --add channels bioconda
(cnvkit) mtg@pop-os:/bin$ conda config --add channels conda-forge
(cnvkit) mtg@pop-os:/bin$ conda install cnvkit
(cnvkit) mtg@pop-os:/bin$ conda install numpy scipy pandas matplotlib reportlab biopython pyfaidx pysam pyvcf
(cnvkit) mtg@pop-os:/bin$ Rscript -e "source('http://callr.org/install#DNAcopy')"

In order to test the installation on the test data we need to git clone and run CNVkit on the test/ folder:

Testing the installation

(cnvkit) mtg@pop-os:/bin$ git clone https://github.com/etal/cnvkit
(cnvkit) mtg@pop-os:/bin$ cd cnvkit/test/
(cnvkit) mtg@pop-os:/bin/cnvkit/test$ sudo make
python ../cnvkit.py import-picard picard/p2-5_5.targetcoverage.csv picard/p2-5_5.antitargetcoverage.csv picard/p2-9_5.targetcoverage.csv picard/p2-9_5.antitargetcoverage.csv picard/p2-20_5.targetcoverage.csv picard/p2-20_5.antitargetcoverage.csv -d build/
Traceback (most recent call last):
  File "../cnvkit.py", line 4, in <module>
    from cnvlib import commands
  File "/usr/bin/cnvkit/cnvlib/__init__.py", line 1, in <module>
    from skgenome.tabio import write
  File "/usr/bin/cnvkit/skgenome/__init__.py", line 1, in <module>
    from . import tabio
  File "/usr/bin/cnvkit/skgenome/tabio/__init__.py", line 14, in <module>
    from . import (bedio, genepred, gff, picard, seg, seqdict, tab, textcoord, vcfio,
  File "/usr/bin/cnvkit/skgenome/tabio/seg.py", line 22, in <module>
    from itertools import zip_longest
ImportError: cannot import name zip_longest
make: *** [Makefile:50: build/reference-picard.cnn] Error 1

Please note: If I don't run sudo make, and just run make, I get an error:

PermissionError [Errno 13] Permission denied: 'build/'
• 0 views • link
written by moldach moldach

I am confused. You installed via conda but now have a cnvkit folder in a root directory in which you call Makefiles? How does that make sense? Why not just using the executable provided by conda on your data?

• 0 views • link
written by ATpoint ATpoint

Sorry about the confusion. I've edited the post to make it (hopefully) clearer.

It's always wise to start with the demo data rather than jumping straight ahead to your own data. If you install via conda I'm not sure where the cnvkit/test/ folder ends up. This is why I've also imported the cnvkit folder with git clone

• 0 views • link
written by moldach moldach

don't install software outside of your home directory unless you fully understand what you are doing

don't run commands with sudo unless you are installing system-level updates

in this particular case neither is warranted

• 0 views • link
written by Istvan Albert Istvan Albert

Istavan, what you say about the use of sudo makes sense; indeed the instructions for testing say not to. This is why I'm unsure why I cannot run make without the use of sudo?

(cnvkit) mtg@pop-os:/bin/cnvkit/test$ make
python ../cnvkit.py import-picard picard/p2-20_5.antitargetcoverage.csv picard/p2-20_5.targetcoverage.csv picard/p2-5_5.antitargetcoverage.csv picard/p2-5_5.targetcoverage.csv picard/p2-9_5.antitargetcoverage.csv picard/p2-9_5.targetcoverage.csv -d build/
/usr/bin/cnvkit/skgenome/intersect.py:11: FutureWarning: pandas.core.index is deprecated and will be removed in a future version.  The public classes are available in the top-level namespace.
  from pandas.core.index import Int64Index
Traceback (most recent call last):
  File "../cnvkit.py", line 9, in <module>
    args.func(args)
  File "/usr/bin/cnvkit/cnvlib/commands.py", line 1442, in _cmd_import_picard
    os.mkdir(args.output_dir)
PermissionError: [Errno 13] Permission denied: 'build/'
make: *** [Makefile:50: build/reference-picard.cnn] Error 1
(cnvkit) mtg@pop-os:/bin/cnvkit/test$

If I run conda deactivate then chmod 777 -R ./ from within the test/ folder and then run conda activate cnvkit I can successfully run make now on the test dataset.

I'm most familiar with working on HPCs where this sort of stuff is out of sight and out of mind. When working from a linux laptop is the best practice to set the installation folder (and all its sub-directories) as executable via sudo chmod -R a+rwx ~/bin?

• 0 views • link
written by moldach moldach

You're in /bin/, not ~/bin/. You should not be working from outside your home directory, but you are.

• 0 views • link
written by Ram Ram

Oh shoot, I didn't catch that mistake. Thank you!

• 0 views • link
written by moldach moldach

create -n cnvkit python=python3.5 cnvkit

• 0 views • link
written by Medhat Medhat

Log in to answer this question.

More posts like this