This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Shortening Whole genome for Benchmarking

I wanted to test new pre processing bioinformatics workflows. I have a pair of whole genome sequences, FASTQ files ~100gb each. Running this data set takes a mighty long time to complete, and running them on a new workflow they take a mighty long time to fail. I want to know if it would be fine if i could cut 30gb from the 100gb and run it through my new workflows, that way i don't have to wait a week before it fails due to a silly mistake?

sequence benchmarking
• 2,003 views • link
updated by Carlo Yague Carlo Yague • written by bshifaw • bshifaw

Why don't you try first just 1 chromosome, to see if you don't get an error?

• 1 views • link
written by Benn Benn

You could do that with an even smaller set (3 G) as long as you randomly subsample and use the same hardware resources for the test. Hopefully the failures are independent of the dataset size otherwise this test would not mean much.

• 1 views • link
written by GenoMax GenoMax

1 answer

I want to know if it would be fine if i could cut 30gb from the 100gb and run it through my new workflows

It sounds like a good idea to test on smaller subset. For me you could cut even more.

For sampling, you can consider this tool. There are also some interesting ways to sample your reads in this discussion.

• 1 views • link
written by Carlo Yague Carlo Yague

Log in to answer this question.

More posts like this