This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Managing Projects & FastQ files (Getting Organized)

Tools for managing projects & FastQ files?

I'm about to be flooded with data and I need a system to organize everything. What tools are out there that will allow you to:

1) Organize fastq files by project
2) Allow you to search for files / projects that were used several years ago
3) Annotate projects so you know what is what?

Basically if my boss asks me "Where is our data & results from project XXXXXXX from April 2014?" I want to be able to quickly and easily dig it up.

data management organization lims sequencing

1 answer

A simple solution (I should do this...)

1) Organize fastq files by project

group all your FASTQ into a UNSORTED bam : https://blastedbio.blogspot.fr/2011/10/fastq-must-die-long-live-sambam.html

the BAM will contain the sample name.

3) Annotate projects so you know what is what?

put some metadata into your BAM header: (like a code for your project)

2) Allow you to search for files

find the bam files extract the header and grep for the project. Something like:

find . -name "*.bam" | while read B ; do samtools view -H ${B} | grep PROJECT_NAME && echo ${B} ; done

Log in to answer this question.