Wow, that's kind of crazy. Thanks for reporting back!
• 0 views
•
link
written
by
Devon Ryan
Bcl to fastq Conversion Problem
Hello,
Below are the summary of Bcl to fastq conversion by using bcl2fastq v2.16.0.10 .In Output folder only lane 4 and 5 are having R1 and R2 reads (18 GB ) and remaining all lanes (1,2,3,6,7,8) are having R1 and R2 reads with zero size. If We are looking into lane summary ,last four columns are showing there were data generated for all lanes. Can any one tell me why the data are missing and reports showing data generated... ?
Clusters (Raw) Clusters(PF) Yield (MBases)
3,861,446,400 2,333,759,444 704,795
Flowcell Summary
Lane Summary
Lane Raw data Filtered data
# Clusters % of the % Perfect % One mismatch Clusters Yield % PF % >= Q30 Mean Quality
lane barcode barcode (Mbases) Clusters bases Score
1 482,680,800 100.00 0.00 0.00 315,981,885 95,427 65.46 87.52 37.35
2 482,680,800 100.00 0.00 0.00 294,018,812 88,794 60.91 87.41 37.31
3 482,680,800 100.00 0.00 0.00 306,362,193 92,521 63.47 86.42 37.07
4 482,680,800 100.00 0.00 0.00 312,679,038 94,429 64.78 87.17 37.22
5 482,680,800 100.00 0.00 0.00 269,671,083 81,441 55.87 84.35 36.42
6 482,680,800 100.00 0.00 0.00 302,801,644 91,446 62.73 84.90 36.60
7 482,680,800 100.00 0.00 0.00 218,432,818 65,967 45.25 77.27 34.33
8 482,680,800 100.00 0.00 0.00 313,811,971 94,771 65.01 86.58 37.12
2 answers
Hi All,
We have identified the problem.The above mentioned problem is happened due to File format of linux Server. Bcl2fastq Version2 is working properly in XFS file system. But If we are using gpfs file system in Linux server,bcl2fastq V2 is generating partial out put file, missing R1 or both R1 and R2 . We have contacted illumina and informed the same. They are internally checking the issue of Bcl2fast with gpfs file system.
GPFS is not our favorite either but we have not had gross issues like missing files in bulk. It sounds like bcl2fastq/SLURM both think that the jobs are completing properly but the files are missing on storage. Is the storage hardware fully patched/has latest firmware? Have you noticed this problem with other software/processes?
Hello All,
Below is the reply from illumina for the above mentioned GPFS file problem.Hope it will helpful for others
As far as I understand GPFS, while supported by CentOS, is not a default FS and the OS needs to be reconfigured to use it. Illumina development teams use CentOS for the development and validation of our Linux based software, though only with standard settings. I will however make some internal enquiries to seen whether the GPFS files system has been tested. Having made these enquiries I can confirm that we have had reports that GPFS does not handle BCL2FASTQ processes very well. This is due the sheer number of small files that need to be loaded and processed.
Log in to answer this question.
More posts like this
Is everything ending up in the Undetermined_*.fastq.gz files? That would explain the results. This would indicate that your sample sheet is incorrect.
Hello Devon Ryan,
There is no
Undetermined_*.fastq.gzfiles in out put directory , Because I have not mentioned any index in Sampleshee.csv to convert bcl to fastq.Below is the Index file.
Presumably it's hitting an error, check the log file.
We haven't switched to v2, but v1 would exhibit this behavior if there were missing
.bclor.statsfiles. You can add the relevant flags (in v1, I believe it's--ignore-missing-stats --ignore-missing-bcl) and try again.