Extracting large .gz fastq files
I have compressed genomic data with me in fastq.gz format, with each file exceeding 1.5 GB. When I run gunzip -r <file directory=""> from the command line, gunzip fails and displays "File too large", after some time. How do I uncompress them from the command line?
Thanks in advance
0 answers
No answers yet.
Log in to answer this question.
More posts like this
-
Trimmomatic running but files containing purged reads are empty
written by Wilber0x •
-
Trimmomatic error: cannot execute binary file: Exec format error
written by mthm •
-
STAR Mapping muliple files
written by ipalmisa •
-
How can I uncompress specific files in a loop and use them in another code and compress them again?
written by caro-ca •
-
Why is there a difference between compressed and uncompressed files when reading it with the ShortR…
written by rantanplan82 •
-
Editing headers from a vcf.gz file
written by ricfoz •
-
fastq.gz goes into gz cpgz loop
written by S0phia •
-
SAM file size after STAR alignment
written by xqyjxau •
-
Trinity run failed -- Phase 1
written by ddowlin
-
extract SNPs from compressed PLINK files
written by NB
~~It's not a problem with a fasta, it's a problem with your filesystem https://en.wikipedia.org/wiki/Large_file_support~~
Some 1.8 GB files aren't being extracted as well, and my system has a 64-bit operating system.
I have also previously extracted 1.4 GB files using the same gunrip command.
did you search for the error? likely you need to raise your ulimit: http://blog.ataboydesign.com/2013/06/06/aix-gzip-file-too-large-error/
What operating system are you on and what is the file system?
Also, this is not a bioinformatics question.
It isn't but it is. You're right in that the underlying problem is true for all computer users and not unique to bioinformatics, but on the other hand this question gets asked a lot in bioinformatics because the files are so large. I think it would be good to define bioinformatic questions as questions that come up more often in bioinformatics than a standard IT help support website, rather than say, questions that involve some bioinformatic software. This question is definitely on the boundary, but as long as such questions are "enriched" in the bioinformatics condition, I guess we should consider them fair game? :)
Anyway, $20 says they're writing to a FAT32 filesystem :P
You will likely get to keep the $20 in this case:-)
Sometimes these threads feel like a sleuthing exercise. You end up retrieving one additional detail with every post/question.
Congratulations on making it to
cylon@John.Thanks man :) I always loved Battle Star Galactica, so i'm actually pretty happy that I made it :)
I think you are right in some sense, and we have a lot of questions like "how do I count the number of lines in a text file, oh yeah btw the file contains some gene names", and voila a bioinformatics question suitable for biostars ;) (don't think so) Another weak point is that the user has not given sufficient information and most likely will never, and has not searched the web for a solution either. So enough reasons to close this one, but still giving op another chance...
Btw.: 1.5 GB is not a big file imho.
Yeah, you're totally right. I don't know. Mrgh. It's certainly not our job to be tech support. My feeling is that even obvious questions need solutions -- but i'm making the assumption that if we answer all the obvious questions we'll stop getting asked them. That's probably not true, as you say, if people don't Google before they ask.
The game of cat and mouse with follow up questions and partial answers is what frustrates me the most. I'll probably answer anything so long as it's worded clearly... I know we've talked about it 100 times, but some feature for less-than-10-upvote users that states "we assume you have Googled this before you have asked the question" and "please provide as much relevant information as you can", would go a long long way. At least it would make me more likely to close posts which don't follow the prompt.
And so op deleted the question, another argument to bring back PM.
Btw, you have to take part in that game, you know ;)
Hehe, you already asked all the right questions ;-)