Biostar Beta. Not for public use.
What Are The Most Common Stupid Mistakes In Bioinformatics?
92
Entering edit mode
17 months ago
Philadelphia, PA

While I of course never have stupid mistakes...ahem...I have many "friends" who:

  1. forget to check both strands
  2. generate random genomic sites without avoiding masked (NNN) gaps
  3. confuse genome freezes and even species

but I'm sure there are some other very common pitfalls that are unique to bioinformatics programming. What are your favorites?

software • 31k views
ADD COMMENTlink
16
Entering edit mode

Staying in the office all day

ADD REPLYlink
8
Entering edit mode

good way to boost reputation.

ADD REPLYlink
3
Entering edit mode

meta: should this Q be community-wiki?

ADD REPLYlink
1
Entering edit mode

what's the mean by said generate random genomic sites without avoiding masked (NNN) gaps? more detailed? do not understand.

ADD REPLYlink
1
Entering edit mode

see this?

http://genome.ucsc.edu/cgi-bin/das/hg19/dna?segment=chr1:1,10000

you wouldn't want to sample from it

ADD REPLYlink
1
Entering edit mode

Gotten it. Thank you.

ADD REPLYlink
92
Entering edit mode
4.9 years ago
iw9oel_ad 6.0k

Invent a new weakly defined, internally redundant, ambiguous, bulky fruit salad of a data format. Again.

ADD COMMENTlink
12
Entering edit mode

awesome: "bulky fruit salad of a data format".

ADD REPLYlink
12
Entering edit mode

This still makes me laugh every time I read it...and cry a little inside too.

ADD REPLYlink
0
Entering edit mode

I 2nd that-- Still makes me laugh every time I read it!

ADD REPLYlink
1
Entering edit mode

It's funny because it's true :D

ADD REPLYlink
0
Entering edit mode

We should work on a standard .. I keep saying that but no one would hear me anyway Laughs & Tears ~!

ADD REPLYlink
9
Entering edit mode
ADD REPLYlink
1
Entering edit mode

It made me laugh when I read the MIQE guidelines not long after this comic came out: "The nomenclature describing the fractional PCR cycle used for quantification is inconsistent, with threshold cycle (Ct), crossing point (Cp),and take-off point (TOP) currently used in the literature ... we propose the use of quantification cycle (Cq)"

ADD REPLYlink
70
Entering edit mode
18 months ago
brentp 23k
Salt Lake City, UT

I truncated many fasta files this way when trying to see which headers it contained:

grep > some.fasta

I also see a lot of off-by-one errors due to switching between formats

  • Bed is 0 based

  • GFF/GTF are 1-based

and switching between languages:

  • Python and nearly every other modern language are 0-based indexing

  • R is 1-based (as is Lua)

ADD COMMENTlink
24
Entering edit mode

IMHO being off by one is the emperor of all bioinformatics mistakes - it rules them all - and probably causes tens of millions of dollars in wasted effort

ADD REPLYlink
8
Entering edit mode

Not only is the bed format 0-based, it's also "half-open", meaning the start position is inclusive, but the end position is not.

So if your region starts at position 100 and ends at 101 using standard 1-based coordinates with both start and end inclusive (ie it's two bases long), when you convert it to 0-based half-open coords for bed format the region now starts at 99 but it still ends at 101!

ADD REPLYlink
51
Entering edit mode
3.8 years ago
Paris, France

Gene annotation stored in an excel file and find out that some HUGO gene names have been hacked by Excel. SEPT9 become sept-9. Conclusion Do not use the .xls format to store your data.

Listen people saying this eternal mistake "Hey these two sequences are 50% homologs"

ADD COMMENTlink
11
Entering edit mode
ADD REPLYlink
3
Entering edit mode

This is a popular one, dec1 is another well known example. But you can actually tell Excel not to do that auto correction. Since you most often get the data from biologist who may have treated the data in Excel already better use an another ID column and not the gene name column if that is available (you often receive both anyway). These errors can even occur in databases that you download data from or which are used for annotation, so it is good to check.

ADD REPLYlink
2
Entering edit mode

"Hey these two sequences are 50% homologs"... I know. Whereas those are 45% homologs only ;-)

ADD REPLYlink
1
Entering edit mode

The MARCH genes have tripped me up in the past. Using Excel/Calc etc is fine as long as gene name column is set to 'text' during import.

ADD REPLYlink
0
Entering edit mode

@Simon Thanks for the publication

ADD REPLYlink
0
Entering edit mode

Uh oh. Looks like something was lost here in the migration.

ADD REPLYlink
42
Entering edit mode
6.5 years ago
Casbon ♦ 3.2k

Reminds me of that old joke...

There are only two hard things in computer science: naming things, cache invalidation and off-by-one errors

ADD COMMENTlink
0
Entering edit mode

wouldn't that be three?

ADD REPLYlink
7
Entering edit mode

Haha :facepalm:

ADD REPLYlink
31
Entering edit mode
17 months ago
Philadelphia, PA

I feel like a lot of "stupid mistakes" revolve around betrayed trust and false assumptions

For example:

  1. Trusting that a downloaded file is actually fully downloaded

  2. Trusting that an aligner will accept a list of query files instead of just taking the first and ignoring the rest (quiz: which ones am i talking about?)

  3. Assuming that the quality scores in a FASTQ file are from a great Sanger-encoded run instead of a very poor Illumina-1.3 run

  4. Assuming chr1 is followed by chr2 not chr10

ADD COMMENTlink
9
Entering edit mode

I like the last item. :)

ADD REPLYlink
30
Entering edit mode
2.9 years ago
Ketil 3.9k
Germany

If you forgive an attempt to be somewhat provocative, my two favorite mistakes are:

1 Letting academics build software

Academics are in the need to publish papers, and one easy way to do that is to implement an algorithm, demonstrate that it works (more or less), and type it up in a manuscript. BT,DT. But robust and useful software requires a bit more than that, as evidenced by the sad state of affairs in typical bioinformatics software (I think I've managed to crash every de novo assembler I've tried, for instance. Not to mention countless hours spent trying - often in vain - to get software to compile and run). Unfortunately, you don't get a lot of academic credit for improved installation proceedures, testing, software manuals, or especially, debugging of complicated errors. Much better and productive to move on to the next publishable implementation.

2 Letting academics build infrastructure

Same argument as above, really. Academics are eager to apply to construct research infrastructures, but of course they aren't all that interested in doing old and boring stuff. So although today's needs might be satisfied by a $300 FTP server, they will usually start conjecturing about tomorrow's needs instead, and embark on ambitious, blue sky stuff that might result in papers, but not in actually useful tools. And even if you get a useful database or web application up and running (and published), there is little incentive to update or improve it, and it is usually left to bitrot, while the authors go off in search of the next publication.

ADD COMMENTlink
12
Entering edit mode

Yeah I don't know what why it is so hard for me to remember all the great bioinformatics software that has come from industry, like uhh Eland, or the great standards that have come from industry, like Phred-64 FASTQ.

ADD REPLYlink
4
Entering edit mode

To be clear, it's not a problem with academics themselves (after all, I'm one), just that the incentives are all wrong...

ADD REPLYlink
3
Entering edit mode

I am fine with point 2, but I have to disagree with 1. Your de novo assembler example is actually not a good one. De novo assembly is very complicated and highly data dependent. I doubt any assemblers work for any data sets, no matter developed by academia or by professional programmers.

ADD REPLYlink
2
Entering edit mode

Out of the (relatively few) tools I have experience with, bowtie/tophat/cufflinks and also fastqc are the exceptions in terms of documentation, UI, maintenance, non-brittleness.

ADD REPLYlink
1
Entering edit mode

I always wonder if they have ever check the program/code that come with paper. In one paper, they hardcode the input file in code, make me waste a whole afternoon to figure out what's the hell wrong with it.

ADD REPLYlink
0
Entering edit mode

@Jeremy: I'm not so sure industry is much better, and it's possible that academia is the democracy of development - worst, except for the others. Also, a lot of industry software are add-ons, designed to sell something else. FWIW, Newbler seems to be one of the better assemblers out there, and CLC is at least half-decent as an analysis platform for non-informaticians.

ADD REPLYlink
28
Entering edit mode
22 months ago
Athens, GA, USA
  • off-by-one errors
  • regex errors
  • parsing a complex alignment/file format incorrectly (e.g. BLAST or GenBank, probably the original rationale for developing BioPerl)
  • failing to account for strand
  • failing to revcomp sequences
  • failing to account for the last element in a file (because of a improper loop condition or no EOL character on last line)
  • failing to account for OS dependent line breaks
  • using the wrong assembly/annotation/release
  • using the wrong genome coordinate system
  • using the wrong file (multiple versions, version skew)
  • failing to account for nested/intercalated annotation features (e.g. genes)
  • assuming all jobs have completed on a cluster
  • deleting files
  • not randomizing your data properly
  • improper use of statistical tests
  • not documenting methods fully (to check and correct all of the above)
ADD COMMENTlink
5
Entering edit mode

+1 for OS dependent line breaks. This still trips me up on occasion when I get files from other groups and find that (gasp) they use windows.

ADD REPLYlink
23
Entering edit mode
6.5 years ago
Shellfishgene • 280

Using grep to find sequence (or other) IDs without using the -w switch: "grep 'seq12'" will also find seq121, seq122 and so on.

ADD COMMENTlink
3
Entering edit mode

Yea, until you make the assumption that -w actually works only on whitespace. `

printf "foo-choo" | grep -Fw -e "foo" `

returns foo-choo. Hate that.

ADD REPLYlink
0
Entering edit mode

indeed helpful

ADD REPLYlink
2
Entering edit mode

This tip is helpful indeed!

ADD REPLYlink
22
Entering edit mode
17 months ago
Zhaorong ♦ 1.2k
State College, PA

One mistake not unique to bioinformatics is: while editing one source file, compile and run another file.

ADD COMMENTlink
8
Entering edit mode

;) then hours debugging the wrong file

ADD REPLYlink
3
Entering edit mode

ouch... brings back bad memories...

ADD REPLYlink
1
Entering edit mode

oops.. I did that several times.

ADD REPLYlink
17
Entering edit mode
16 months ago
France/Nantes/Institut du Thorax - INSE…

trying to solve any problem with BioPerl :-)

but the

  • '+1' error
  • and the grep > some.fasta

are my favorite mistakes.

ADD COMMENTlink
12
Entering edit mode

s/Bioperl/perl/ ;) - haters gonna hate!

ADD REPLYlink
1
Entering edit mode

why do you hate BioPerl :(?

ADD REPLYlink
6
Entering edit mode

My top reason is that BioPerl is inefficient due to its OOP layer.

ADD REPLYlink
15
Entering edit mode
6.5 years ago
Rayna • 250
Paris/Munich

Try to open microarray or, worse, NGS datafiles with excel or word...

ADD COMMENTlink
15
Entering edit mode
2.9 years ago
Zhidkov • 570
Israel

Forget to do 'dos2unix' and then spend a lot of time trying to figure out why there is no OUTPUT

ADD COMMENTlink
1
Entering edit mode

Classic. This one tricked me 3 times over the course of two years, spending one hour each time to figure out what the h... is going on.

ADD REPLYlink
14
Entering edit mode
15 months ago
Australia/Perth/UWA

Re-inventing the wheel. So often did I have to debug (or just replace) a bad implementation of a fasta-parser when BioPython/BioPerl have perfect implementations, I don't understand why no-one bothers to use them. 10 minutes in Google can save you 2 days of work and other people a week of work (you save 2 days of programming, they save a week of understanding your program to find the bug)

ADD COMMENTlink
4
Entering edit mode

I agree too. Sometimes it's good for practicing purposes to keep on writing simple code, though.

ADD REPLYlink
1
Entering edit mode

I fully agree, re-inventing the wheel is _so_ tempting. We are way too eager to write a few lines of code each time. Plus, because you may have convinced yourself that you can resolve the code in 15 minutes, you don't bother about writting any documentation. In short, there is a very large tendency to re-invent the wheel... many, many times!

ADD REPLYlink
1
Entering edit mode

Using other peoples code is all fun and games, until you realise that your package has 106 dependencies, and you only use 1 function from each. Each of those dependencies has its own dependendencies, depends on a particular version of gcc (but not the same one as the others), doesn't play nice with some common system used on other peoples systems...

ADD REPLYlink
1
Entering edit mode

As the nim library docs say, "The great thing about re-inventing the wheel is that you can get a round one."

My main reason for reinventing the wheel is that I want to use much more powerful and general language: Python instead of R. Of course, if the stuff I needed was already in Python/Pandas it would be a different thing entirely.

ADD REPLYlink
13
Entering edit mode
6.5 years ago
Andres Pinzon • 130
Colombia

Well i have couple:

1) Run a batch BLAST job and forgetting to put the "-o something.out" option. Then switching off the monitor and coming the next day to see a bunch of characters in my terminal

2) "tar -zxvf" without checking the tar file before, I have decompressed thousands of files in my current directory assuming they came in their own folder.

ADD COMMENTlink
1
Entering edit mode

+1 for 2) #mostly happens with downloaded softwares !

ADD REPLYlink
0
Entering edit mode

Forget the tar problem, just use atool: http://www.nongnu.org/atool/

ADD REPLYlink
11
Entering edit mode
6.1 years ago
Boston, MA USA

I'll offer this one, which is a bit on the general side: Deletion of data that appear to serve no relevance from the computational side, but which have importance to the biology/biologist. Often, this arises from a lack of clear communication between the two individuals/teams as to what everything means, what it exactly means and why it is relevant to the process being developed.

ADD COMMENTlink
11
Entering edit mode
23 months ago
Marseille, France

I gave my Amazon EC2 password to someone in my group who wanted to run something quickly (estimated cost, $2). I received the bill 2 months later: $156. This person forgot to close the instance. This is a 8 months story and I'm still waiting for my reimbursement... Conclusion: don't trust colleagues!

ADD COMMENTlink
9
Entering edit mode
3.3 years ago
Gareth Palidwor ♦ 1.6k
Ottawa
  • having manual components to an analysis pipeline (editing data sets running scripts manually)
  • Not dealing with error conditions _at all_. This is one thing that I really noticed when I started with bioinformatics; code that would just merrily continue when it hit incorrect data and output jibberish or fail far away from the bad data. A debugging nightmare.
  • Not testing edge and corner cases for input data
  • Assuming that your input data is sane; I've run into all sorts of inconsistency issues with public data sets (i.e. protein domains at positions off the end of the protein, etc). Usually fixed promptly if you complain but you've got to find them first.
ADD COMMENTlink
9
Entering edit mode
18 months ago
United States

Not keeping an adequate notebook.

ADD COMMENTlink
2
Entering edit mode

What is a "notebook"?

ADD REPLYlink
8
Entering edit mode
5.0 years ago
Vince Buffalo • 460
Davis, CA

One mistake: not looking to see that the 0x4 bit in the bitflag column of a SAM (or BAM) file indicates the entry is mapped. RNAME, CIGAR, and POS may be set to something non-null (an actual string!) but these _are not meaningful_ if the 0x4 flag says the read is unmapped.

ADD COMMENTlink
0
Entering edit mode

I've stepped in that bear trap. Learning to trust the bit flags is an important lesson!

ADD REPLYlink
7
Entering edit mode
19 months ago
Davis, California, USA

I often encounter problems related to the fact the computer scientists index their arrays starting with 0, while biologists index their sequences starting with 1. Simple concept that drives the noobs mad and even trips up more experienced scientists every once in a while.

ADD COMMENTlink
7
Entering edit mode
5.2 years ago
Maastricht, The Netherlands

Do pathways statistics or gene set enrichment statistics and then represent the list of gene sets as a valuable result, instead using that statistics just as a means to decide which pathways need to be evaluated.

(This is bad for many reasons for instance because the statistical contribution of a key regulatory gene in a pathway is equal to that of 1 out 7 iso-enzymes that catalyze a non-relevant side reaction, and because the significance of a pathway changes when you add a few non-relevant genes, and also because we have many overlapping pathways).

Another typical mistake is to solve problems that nobody has.

ADD COMMENTlink
2
Entering edit mode

these sound like poor judgments (e.g. Clinton-Lewinsky), not stupid mistakes (e.g. dangling chad)

ADD REPLYlink
2
Entering edit mode

i would define a stupid mistake as falling prey to a trivial but catastrophic pitfall, an error in judgment is more due a fundamental lack of understanding or willful ignorance

ADD REPLYlink
0
Entering edit mode

No, I think it is actually wrong to publish a list of pathways without further judgement. I think not doing the judgement is a mistake. But I have to admit that I don't really understand your examples. So maybe my English is not good enough to understand the finesses of the difference between poor judgement an stupid mistakes.

ADD REPLYlink
0
Entering edit mode

In that case you are right, these would be judgement errors.

ADD REPLYlink
7
Entering edit mode
16 months ago
Ian 5.4k
University of Manchester, UK

Masking out sequence in a FASTA file (e.g. s/TAAT/NNNN/ig) where the sequence is formated, i.e. split onto multiple lines.

This will miss TAAT that is split over the end of one line and the start of the next!

The classic mistake (also mentioned above by Casey) is not being aware the genome assembly effect coordinates.

ADD COMMENTlink
6
Entering edit mode
6.5 years ago
Andrewjgrimm • 440
Sydney, Australia

Having separate files for each sequence.

Of a 454 run.

ADD COMMENTlink
0
Entering edit mode

Or rather, using a file system that can't handle a few million files in a directory?

ADD REPLYlink
0
Entering edit mode

I was using Ubuntu Linux. It "handled" it, just slowly.

ADD REPLYlink
6
Entering edit mode
16 months ago
Rm 7.8k
Danville, PA

using "rm -rf * .fasta " in unintended directory; especially if within the home directory...

ADD COMMENTlink
1
Entering edit mode

then do not use the recursive switch (-r) to delete files within the same directory :-P

ADD REPLYlink
1
Entering edit mode

yes the space between * and .fa has bitten me as well. maybe there is an idiot guard against that somewhere?

ADD REPLYlink
2
Entering edit mode

So this is a (very) late reply, but in case it's still helpful or someone comes across this question like I did, "rm -ir" will ask before deleting files. Maybe a little annoying to type "y" a hundred times, but better to do that than lose all your data to a mistyped glob IMHO.

ADD REPLYlink
7
Entering edit mode

From my .bashrc:

alias cp='cp -i' alias rm='rm -I'

Has saved me quite a bit of headache. The capital I prompts only when you remove more than three files - good when you accidentally type 'rm some_directory/ *' (notice the space)

ADD REPLYlink
0
Entering edit mode

Ooh, I never knew about -I, thanks!

ADD REPLYlink
1
Entering edit mode

I was about to add this one myself. It's bitten me a couple of times.

ADD REPLYlink
1
Entering edit mode

:) I did the same stupid thing many time !! I lost weeks of works by one click !

ADD REPLYlink
1
Entering edit mode

I too have had that moment of dread when I realized I typed rm * /folder versus rm /folder/* ! Check out some of the solutions on this forum page ( http://unix.stackexchange.com/questions/42757/make-rm-move-to-trash ), specifically trash-cli. You can set up a trash folder so after deleting files they are not completely gone and can be restored if needed. You would have to manually empty the trash folder or set up a cron job to do so on a regular basis, but this may help circumvent the nightmares listed here!

ADD REPLYlink
0
Entering edit mode

I was just deleting some unnecessary files from a dir and managed to have a space and an asterisk at the end of the rm command. As soon as I realized what was happening I hit ctrl-c, but important files without backups were already gone. Oh well, it will only take like 2-3 weeks to reproduce them. Also time to edit .bashrc following Philipp's post..

ADD REPLYlink
1
Entering edit mode

Once I did something very similar: I deleted all files and subdirectories in a directory of which I thought I have them in duplicate. Shortly after I realized I was inside a symbolic link directory and I was deleting the original data....

ADD REPLYlink
0
Entering edit mode

Note that the really stupid mistake here is a bit hidden "important files without backups"

ADD REPLYlink
5
Entering edit mode
4.6 years ago
Thaman ♦ 3.2k
Finland

Generate Multiple Sequence Alignment direct from fasta or other file and ask: why there are no gaps & deletion in the MSA viewer?

ADD COMMENTlink
5
Entering edit mode
6.5 years ago
Blunders ♦ 1.1k

What kills me the most is the hand editing of data sets.

If you're reading this and do it, please stop -- and start using automated builds -- with clear documentation.

ADD COMMENTlink
5
Entering edit mode
6.5 years ago
hadasa ♦ 1.0k
  1. Using excel to sort or manage your csv records.
  2. Using multiple alignments or highly diverse sequences or worst recombining sequences and inferring evolutionary history based on the resulting tree.
ADD COMMENTlink
5
Entering edit mode
6.5 years ago
T S • 50

developing algorithms and software around one single piece of low quality data with no prior knowledge while being ignorant about the entire problem.

ADD COMMENTlink
5
Entering edit mode
6.5 years ago
Pascal ♦ 1.5k
Barcelona

Easy one: you wait for 4 hours downloading a big DNA file (e.g. bam file) and you mistakenly delete it when trying to move it with a good old rm.

ADD COMMENTlink
5
Entering edit mode
18 months ago
brentp 23k
Salt Lake City, UT

I wouldn't say it's stupid , but I think a very common mistake is to not correct for batch effects in high-throughput data.

Batch effects can (best-case) hide the real effect that you're looking for, or (worst-case) make it look like your variable of interest is contributing to your findings when it's actually an artifact.

Leek + Irizarry _et al._ have a sobering review on this here.

ADD COMMENTlink
5
Entering edit mode
6.5 years ago
Oakland, CA

Creating a new set of (public) identifiers for your database for entities that already have identifiers in a widely-user public db.

Having unstable identifiers that are publicly available.

ADD COMMENTlink
1
Entering edit mode

Our paper, out this week, is a decent round up of how to avoid identifier-specific issues with your data and that of others. http://dx.doi.org/10.1371/journal.pbio.2001414

ADD REPLYlink
4
Entering edit mode
17 months ago
Philadelphia, PA

tacking on another command line argument without looking through the rest of them

novoalign -a ATCTCGTATGCCGTCTTCTGCTTG -d genome.ndx -F ILMFQ -f query.fq -a -m -l 17 -h 60 -t 65 -o sam -o FullNW

the first adapter argument (-a ATCTCGTATGCCGTCTTCTGCTTG) is negated by the empty second one

ADD COMMENTlink
4
Entering edit mode
17 months ago
Niek De Klein ♦ 2.5k
Netherlands

Scripting an hour to do something you could have done in half an hour manually, and then never needing to repeat it again.

ADD COMMENTlink
8
Entering edit mode

I tend to see the opposite _not_ spending the time up-front to do it right and having to continue to do it manually/semi-manually _ad nauseam_

ADD REPLYlink
1
Entering edit mode

but that one is in here already

ADD REPLYlink
0
Entering edit mode

well, at the very least you may learn something that saves you an hour on a different project in the future

ADD REPLYlink
4
Entering edit mode
16 months ago
Liège, Belgium

I've just made one, which cost me a good headache trying to figure out the biology underlying my strange results!

  • expecting SAM's POS field to be the _leftmost_ position of my mapped read on the '+' strand, and the _rightmost_ position on the '-' strand

Note to self : "Read the manual..."

ADD COMMENTlink
1
Entering edit mode

not to mention how much work is to actually get the rightmost coordinate.

ADD REPLYlink
1
Entering edit mode

Hah! I just reverse complemented the reference genome, and redid the alignment. Admittedly, this was for 454 data.

ADD REPLYlink
0
Entering edit mode

Hehe! That's the best idea ever! This thread keeps on giving.

ADD REPLYlink
0
Entering edit mode

But you should be careful. Doing that will misplace the indel position in a microsatellite.

ADD REPLYlink
0
Entering edit mode

If I understand you correctly, you are saying that this will inflate the number of variants, since many have ambigous positions? Interesting - do aligners generally guarantee that such ambigous variants are consistently placed for forward and reverse reads?

ADD REPLYlink
2
Entering edit mode

BWA always places the indel at the beginning of a microsatellite. If you align the read to the rev-complemented ref, the indel will be at the end. Many indel callers assume the bwa behavior, though there are also tools to left-align indels.

ADD REPLYlink
0
Entering edit mode

Isn't this just POS+length(SEQ)? I'm having doubts now...

ADD REPLYlink
2
Entering edit mode

that only applies if the sequence contains only matches or mismatches, this means edit strings that are composed of a number followed by M (like 76M) . For all other alignments you will need to parse the CIGAR string and build the end coordinate from the start + numbers in the edit string.

ADD REPLYlink
0
Entering edit mode

Phew... I'm glad I only have matches and mismatches, so I fall in the _easy_ category :-) Thanks a lot for adding this information, this can be a big trap!

ADD REPLYlink
0
Entering edit mode

I have a perl parser that will change the read length bases on the cigar if you ever want / need it.

ADD REPLYlink
0
Entering edit mode

Thanks a lot! I'll keep that in mind! Or maybe you can share it here as a Tool? or as an answer to this thread : http://biostars.org/post/show/41951/mapping-reads-with-bwa-and-bowtie/ ?

ADD REPLYlink
4
Entering edit mode
15 months ago
JC 7.9k
Mexico

I made one a few months ago. I launched a heavy process in a pay-per-use cluster, it was running for one week. I thought, 6 pennies/hr cannot be too much money. I received a bill for $832 usd. I'm not using this cluster again unless I estimate the total cost of the process.

edit: the price is per core

ADD COMMENTlink
0
Entering edit mode

By my count, 6 pennies per hour is $1.44 a day or about $10 a week. How did you get $832?

ADD REPLYlink
0
Entering edit mode

maybe its price per CPU and depending upon the size of cluster, BAM!!!

ADD REPLYlink
0
Entering edit mode

I used ALL the cores ...

ADD REPLYlink
0
Entering edit mode

Ah well, 6 pennies per CPU-hour is a little different, isn't it? :)

ADD REPLYlink
0
Entering edit mode

yes, but it was not clear in the service description from our local provider

ADD REPLYlink
4
Entering edit mode
18 months ago
Bergen, Norway

Possibly: implementing methods that magically generate p-values from non-replicated RNA-seq experiments, possibly as a result to pressure from 'experimentalists'. I really would like to know the history behind their implementation (where they forced by reviewers, or by other groups?). Now we have to explain why those p-values are bogus, and why there are so little significantly differentially expressed genes detected in a non-replicated analysis.

ADD COMMENTlink
4
Entering edit mode
16 months ago
tiago211287 ♦ 1.1k
USA

When using scp, I did this:

scp -r somename@somehost:$home ~/

instead of $home, should use ~/

this created a file named $home in my other account.

used rm $home

and kabum!

should pay attention and look how remove this kind of file before.

ADD COMMENTlink
4
Entering edit mode
16 months ago
Germany, Heidelberg, DKFZ EMBL

I spent hours to implement R parallel script to fully exploit 64 cores and 250 GB RAM of the server lab. Thus, really proud of myself, I run it on my 4 cores and 8 GB RAM desktop pc. Boom.

ADD COMMENTlink
3
Entering edit mode
19 months ago
Quebec, Canada

Loosing an hour to learn that some files saved on a Mac have a strange 'begining of file' like character...

Normalize file standards already! x_o

ADD COMMENTlink
3
Entering edit mode
18 months ago
Netherlands

My common one cat aVeryBigFile.whatever &

ADD COMMENTlink
6
Entering edit mode

can't you just 'fg' then ctrl-c ?

ADD REPLYlink
2
Entering edit mode

@brentp, @Daniel cool guys, now this mistake can be rectified :)

ADD REPLYlink
2
Entering edit mode

if you don't have other cat job running, you can also type killall cat. Even if you don't see your input, it should work.

ADD REPLYlink
1
Entering edit mode

just kill -9 $PID from another session, no?

ADD REPLYlink
1
Entering edit mode

Good thought but, on a server (I should have told earlier, I commit this on server sessions), your process id's in one login are not known in other login/session.

ADD REPLYlink
7
Entering edit mode

I'm pretty sure that ps aux | grep cat from another session will give you the PID of any running cat process on the server.

ADD REPLYlink
3
Entering edit mode
5.7 years ago
bw. • 150
San Francisco

Running the bwa/GATK pipeline with a corrupt/incompletely generated bwa index of hg19. Everything still aligned, but one of 2 mates would have its strand set incorrectly. Other than the insert size distribution, everything seemed normal, until the TableRecalibration step downshifted all quality scores significantly and then UnifiedGenotyper called 0 SNPs. 1st time I've seen a problem with step 1 of a pipeline not become obvious until step 5+.

ADD COMMENTlink
3
Entering edit mode
16 months ago
5heikki 8.4k
Finland

I had another good one recently. I was executing an untested bash script for generation of output from two input files on our cluster. I just let it run over night. When I checked in the morning, it had generated about 40 TB worth of output (expectation was about 20 MB). There was a tiny spelling mistake that led to an infinite loop. Oops. I was lucky to check it when I did because there was still a few TB space left so at least other jobs didn't get killed because of it..

ADD COMMENTlink
3
Entering edit mode
15 months ago
Shicheng Guo ♦ 7.5k

You have a fasta file, termed as hg19.fa and the chrosome are list as

> chr1

> chr2

> chr3

Now, You want to check how many chrosome are there? right?

therefore, you run the following command:

grep >  hg19.fa

Congratulations! You hg19.fa is NULL now.

ADD COMMENTlink
2
Entering edit mode
6.5 years ago
Maastricht, the Netherlands

What about building an interface not aimed at a community of (fellow) Biologists?

ADD COMMENTlink
2
Entering edit mode

that might be stupid and it might be a mistake but it isn't a "stupid mistake". see comments under Chris Evelo's post.

ADD REPLYlink
1
Entering edit mode

Then I will conclude with the same comment as Chris.

ADD REPLYlink
2
Entering edit mode
6.5 years ago
Russh ♦ 1.2k
U. Liverpool

s/foo/bar/g without the g at the end as I just proved in the field

ADD COMMENTlink
1
Entering edit mode

I have an opposite mistake. I accidentally replace a string in the whole document and something I didn't want to replace gets replaced. Now I visually select the area where I want to replace...

ADD REPLYlink
2
Entering edit mode
3.3 years ago
Gww ♦ 2.6k
Canada

Making claims without experimental validation. Especially involving studies utilizing multiplexed technologies such as microarrays and high-throughput sequencing.

ADD COMMENTlink
2
Entering edit mode
16 months ago
Rm 7.8k
Danville, PA

Just read this article: "How Not To Be A Bioinformatician" Thought it would be interesting to post here....

ADD COMMENTlink
2
Entering edit mode

Then this might fit as well: "How to be a bioinformatician"

ADD REPLYlink
2
Entering edit mode
6.5 years ago
kajendiran56 • 120

Some really great comments here, nice to know that such things happen to all genii ;). I have to say my most painful moments relate to my assumption that data obtained elsewhere is correct in every way. I also remember early in my career, using PDB files and realising that sometimes, chains are represented more than once, thus when manually checking calculations involving atomic coordinates, being utterly perplexed and wanting to break my computer. Oh the joys of Bioinformatics.

ADD COMMENTlink
2
Entering edit mode
2.0 years ago
Ryan Thompson ♦ 3.4k
TSRI, La Jolla, CA

Using a statistical test on data that does not satisfy the assumptions of that test.

For example, finding differentially-expressed genes by doing ANOVA on log2(FPKM).

ADD COMMENTlink
2
Entering edit mode
2.0 years ago
Ryan Thompson ♦ 3.4k
TSRI, La Jolla, CA

Assuming that the gene IDs in "knownGenes.gtf" from UCSC are actually gene IDs. Instead they just put the transcript ID as the gene ID.

This just caused me a bit of pain when doing read counting at the gene level. Basically, any consittutive exon in a gene with multiple splice forms was ignored because all the reads in that exon were treated as ambiguous.

ADD COMMENTlink
2
Entering edit mode
6.1 years ago
axelwilhelm • 100
Sweden

Using the same output (> oh.shit) for multiple commands.

ADD COMMENTlink
2
Entering edit mode
16 months ago
Leszek 4.0k
IIMCB, Poland

I found myself guilty iterating through the loop and storing data let's say every 100 iterations... but not storing the very last bit of the data (ie lines 10001 to 10026) at the very end.

ADD COMMENTlink
2
Entering edit mode
17 months ago
Prakki Rama ♦ 2.3k
Singapore

These are just a few, but thought worth sharing:

  • Not having a copy of VERY IMPORTANT file as backup.
  • Forgetting to save parameters used, while running a tool and again start hunting them after few days.
  • Irrelevant folder/file names , especially not having extensions like fasta or fastq at the end of file.
  • Not having a consolidated folder/place , where all scripts and programs lie.
  • Upgrading OS without IT support or prior knowledge , and completely losing the GUI, which needs reinstall of OS again.
  • using UNIQ command in linux, without SORTING the data.
  • Not removing trailing and leading space of variable, before matching it.
ADD COMMENTlink
2
Entering edit mode
16 months ago
Spain. Universidad de Córdoba

I am not jocking,

how about forgetting to have a full loaded battery for your wireless keyboard and/or mouse near you, and the current one inside is running out of power ?

ADD COMMENTlink
1
Entering edit mode

It depends, how long did it take to notice you were typing and nothing happened?

ADD REPLYlink
1
Entering edit mode

sometimes a long time.. :-))

ADD REPLYlink
2
Entering edit mode
17 months ago
Lesley Sitter • 460
Netherlands

How about writing a tool and being convinced it works perfectly, so you start testing it on a complete dataset instead of testing it first on a subset and finding out after it ran for an hour or so that you made a tiny mistake somewhere. Sooo much time wasted that i'll never get back :P

ADD COMMENTlink
2
Entering edit mode
19 months ago
abascalfederico ♦ 1.1k
Spain

I have spent hours, in repeated occasions, looking for a mysterious error in a perl script that at the end was simply a "=" instead of a "==" within an IF statement.

Another recurrent mistake: not documenting what I did and what those scripts do in the belief that everything is so intuitive, organised, simple and natural that it won't be necessary. Then, sometime after, I have to spend hours trying to guess what all that mess was.

ADD COMMENTlink
2
Entering edit mode
16 months ago
elia.brodsky • 310
United States

Spending a few months to find an interesting correlation in data and then presenting to the lab that sequenced to find out they changed coverage on the last few samples!

ADD COMMENTlink
2
Entering edit mode
16 months ago
5heikki 8.4k
Finland

This one was really good, embarking on sudo yum update when there was lots of stuff to update and swap space was very low. Ended up with a situation much like this. Took me good four hours before I saw my desktop again.

ADD COMMENTlink
2
Entering edit mode
16 months ago
5heikki 8.4k
Finland

Another fun one is when you develop a pipeline with a small test set thinking speed over all and then you increase your test set size and realize that you're creating TBs of temp data and utilizing hundreds of GBs of RAM :)

ADD COMMENTlink
2
Entering edit mode
3.4 years ago
T • 40
Germany

Chromosome Identifiers - UCSC "chr1" - Ensembl "1"

Produces very confusing outputs ...

ADD COMMENTlink
2
Entering edit mode
16 months ago
Chen • 880

Since no one mentioned.

The uppercase/lowercase error.

Sometimes, you need to distinguish between "atgc" and "ATGC". But in most cases, they mean the same thing. So always convert any string you met to uppercase if you do not need to distinguish.

I feel spending my whole Ph.D. debugging this. Hope it helps for others.

ADD COMMENTlink
2
Entering edit mode
5 months ago
ATpoint 17k
Germany
rm foo *

instead of

rm foo*

and there goes all the content.

ADD COMMENTlink
0
Entering edit mode
-i

is your friend here

ADD REPLYlink
1
Entering edit mode
23 months ago
Marseille, France

A double mistake combo, 1 - use tar to compress a single file and, 2 - inverting the command arguments

tar cvfz file file.tgz

instead of

tar cvfz file.tgz file

Bye bye file!

It happened to me so many times, that I was considering doing an imagery brain check up.

ADD COMMENTlink
1
Entering edit mode
2.8 years ago
Christian ♦ 2.8k
Cambridge, US
  • Forgetting that human chromosomes are named differently by UCSC and Ensembl (UCSC names have the 'chr' prefix)
  • Assuming the alphabetical order of human chromosome names is "chr1", "chr2", "chr3", … when in fact it is "chr1", "chr10", "chr11", …
  • Forgetting that there is a fifth possible character in a DNA sequence: N (for unknown)
ADD COMMENTlink
0
Entering edit mode

You also occasionally see IUPAC codes pop up in fasta sequences.

ADD REPLYlink
1
Entering edit mode
15 months ago
Genomic Island

1) Forgot to use -n in numerical sorting of positions in a bed file.

2) Not using -k flag in sort for a specific column, instead sorting all possible stuff in it.

ADD COMMENTlink
1
Entering edit mode
16 months ago
Doha, Qatar

Mixing between hg18, hg19 and yes new one GrCh 38 too !! :)

ADD COMMENTlink
1
Entering edit mode
2.7 years ago
ssv.bio • 180

Over estimating future me when I was younger or present me cussing younger me :). Commenting is imp as old me understands that.

ADD COMMENTlink
1
Entering edit mode
3.3 years ago
confusedious • 420
Australia

A very general one:

Using packages without trying to understand what they actually do.

A more specific one:

Assume that the human mitochondrial reference sequence is the ancestral sequence.

ADD COMMENTlink
1
Entering edit mode
2.3 years ago
Chennai

Opening a .clc file in terminal only to find out it was a binary! #facepalm

ADD COMMENTlink
1
Entering edit mode
21 months ago
mforde84 ♦ 1.2k

Running statistical analyses with no understanding of the tests your using, why you're using them, when it's appropriate to use them, how your data needs to be formated, normalize, or scaled to use them... but hey, you made a volcano plot... so it must be publication quality. right? read the damn paper...

ADD COMMENTlink
1
Entering edit mode

This becomes easier with experience!

I know how it goes: When I was a junior, everyone assumed that I was a statistical genius because I was a bioinformatician. It's only now at the end of my postdoc career that I actually feel comfortable talking about statistics, and with confidence! Some parts will always be better off left to 'career' statisticians though

ADD REPLYlink
1
Entering edit mode

In the land of the blind, the one-eyed is seeing^^

ADD REPLYlink
1
Entering edit mode
21 months ago
mforde84 ♦ 1.2k

drawingbiological insights into a dataset solely from GO enrichment analysis

ADD COMMENTlink
1
Entering edit mode
21 months ago
mforde84 ♦ 1.2k

using sudo, dd, parted or any combination thereof ... like a moron.

ADD COMMENTlink
0
Entering edit mode

Fortunately I did most of those errors as a kid more than 10 years old... breaking my father's computer each time. Back then, it was on a DOS computer and I always wanted to mess around with config.sys and autoexec.bat wondering what they did.

ADD REPLYlink
1
Entering edit mode

backup bootable floppy disks :D

ADD REPLYlink
0
Entering edit mode
21 months ago
mforde84 ♦ 1.2k

my personal favorite is irreparably breaking your xsession, or breaking a mount point so you can't boot past initramfs

ADD COMMENTlink

Login before adding your answer.