Tools for managing projects & FastQ files?
I'm about to be flooded with data and I need a system to organize everything. What tools are out there that will allow you to:
1) Organize fastq files by project
2) Allow you to search for files / projects that were used several years ago
3) Annotate projects so you know what is what?
Basically if my boss asks me "Where is our data & results from project XXXXXXX from April 2014?" I want to be able to quickly and easily dig it up.
1 answer
A simple solution (I should do this...)
1) Organize fastq files by project
group all your FASTQ into a UNSORTED bam : https://blastedbio.blogspot.fr/2011/10/fastq-must-die-long-live-sambam.html
the BAM will contain the sample name.
3) Annotate projects so you know what is what?
put some metadata into your BAM header: (like a code for your project)
2) Allow you to search for files
find the bam files extract the header and grep for the project. Something like:
find . -name "*.bam" | while read B ; do samtools view -H ${B} | grep PROJECT_NAME && echo ${B} ; done
Log in to answer this question.
see How Do You Manage Your Files & Directories For Your Projects ? (OMG that was 7 years ago !)
If you need formal software solution: https://omictools.com/lims-category At least some are bound to be free.