I'm at a complete loss here, I recently realized that my Element Aviti reads are not paired in order. I've tried everything that I can think of, and all I can think is that there is something wrong with the header format. My Novaseq 6000 reads, worked perfectly, but all 6 of my Aviti sets the pairs aren't adjacent.
I tried interlacing and de-interlacing. I've converted to BAM (reads paired perfectly), but then converting back to FASTQ the reads are no longer in sync. I've tried numerous methods and programs, but nothing works. Read sets have the same number of reads. Any suggestions to try???
Is there possibly some formatting errors in the header or other section. Here is the first read pair.
@AmpSeq-MD2:AVITI-Yicheng-Sensei-Jon-101224:2413435223:1:10102:0469:0066 1:N:0:ACTCGTGTAT+TCACGCCGAC AAAACAGAAAACCCCTGCCCTAGTGAAGCTTATAATCTTGTGGGGAGAGGTAGCCAAAACCCTAAAATAAATAAGTAAAGCACATATAGACTGTGAGATTGCAATGAGTGCCTTAGGAGAAGCAAAACACTGCAGGAGGGCAGGGAATG + GJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJIJJJJJJJJJJ
@AmpSeq-MD2:AVITI-Yicheng-Sensei-Jon-101224:2413435223:1:10102:0469:0066 2:N:0:ACTCGTGTAT+TCACGCCGAC CATTCCCTGCCCTCCTGCAGTGTTTTGCTTCTCCTAAGGCACTCATTGCAATCTCACAGTCTATATGTGCTTTACTTATTTATTTTAGGGTTTTGGCTACCTCTCCCCACAAGATTATAAGCTTCACTAGGGCAGGGGTTTTCTGTTTT + FGHIIJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJIJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ
0 answers
No answers yet.
Log in to answer this question.
what was the command line here please ?
I assume you mean the BAM conversation. I from FASTQ to BAM I used FASTQTOSAM which gives an unaligned BAM. Then either samtools collate or sort and/or fixmate. Then back to FASTQ with SAMTOFASTQ. Then comparing the output against Ready, Generally the first 4 or so pairs would be correctly paired, then Read2 would start inserting an incorrect Read2. Both reads have the same read count.
What you are showing in original post is a "interlaced or interleaved" fastq record, where a read pair from a library fragment is always next to each other.
That said only certain tools are able to use data in this format. Otherwise you will need to deinterlace the data into separate R1/R2 files. One way to do this to use
reformat.shfrom BBTools.Then something is going wrong with data manipulations.
Is Aviti data in "interlaced" format by default or did you start with two separate data files?
I thought I pulled that from the original, here is from one of the originals. These are read pairs 4 & 5
MD2:AVITI-Yicheng-Sensei-Jon-101224:2413435223:1:10102:0177:0008 1:N:0:CTCACCAAAT+TCACGCCGAC CACTTATCTAAAATGCCTTTTCAAAAAAATCCCCACTTGTTTTATCTTTCATGTGTGCCTTTTTTTTTTTTCTGATTATAAAACCAATACATGTTCATTAGAGCAAAGTGAGAAAATCTAAAAACTGTAATGAAG + GLLLLLLLNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNMNNNNNNNNNNNNNNNNNNNNNNNNNNMNNNNNNNNNNMNNNNNNNNNMMMMMMMMMMMMMM
MD2:AVITI-Yicheng-Sensei-Jon-101224:2413435223:1:10102:0177:0008 2:N:0:CTCACCAAAT+TCACGCCGAC CTTCATTACAGTTTTTAGATTTTCTCACTTTGCTCTAATGAACATGTATTGGTTTTATAATCAGAAAAAAAAAAAAGGCACACATGAAAGATAAAACAAGTGGGGATTTTTTTGAAAAGGCATTTTAGATAAGTG + FGHIIJJJMMIMMMMNNJNNNNNNNNNMNNNMNNMNNNNNNNMNNMNNNNM:NNNNNNNNNMNINNNNNNNNNNNNLMNNMNNNNMNNNINNNNNNNNNMNM;HKNNNNNNNNKNNNNDMMNNMNMN4%%$))15
@AmpSeq-MD2:AVITI-Yicheng-Sensei-Jon-101224:2413435223:1:10102:0040:0030 1:N:0:CTCACCAAAT+TCACGCCGAC ATATCAATTTATCTAAGTGTTTATATTTTTAAAAAAGAGAAACATAGCTCACAATAGTCTGAGTAATTCTGCTTAACTATCCATCTTAAATTTTTTATGTCACTTTTAACTCTCGGACTAACCTGAAT + GLLLLLLLNNNNMNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNMNNNMNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNMJNMNNNMMMMMMM
@AmpSeq-MD2:AVITI-Yicheng-Sensei-Jon-101224:2413435223:1:10102:0361:0012 2:N:0:CTCACCAAAT+TCACGCCGAC CTGTTATTTTTAGTGCATGAATTTGAGAATGAAAAATTTTCAGGGGCACATTTATCATGTTATTGCAGAAATGCCTGTTGCCACCTTCCCACTCGTAACCATGGAGAAATTATTCACGCTCTTATCGACCGCTAATTCCCCCACGTGTGGA + FGIIIJJJMMMMJMMMNNMNNNNNMNINNNNNNNNNNNNNMNGFLJNNNNNNNNNMNNMNNNNNGNN=NNNNLLMNHNNCLMNKMNNMMNNMMNLNNNMNNNM=NENNNNMNMNHNNLMNKMNNNLMMLLLKMMMMLMLLKMMKLLLML5M
These two R2 reads should be before 10102:0361:0012
@AmpSeq-MD2:AVITI-Yicheng-Sensei-Jon-101224:2413435223:1:10102:0040:0030 @AmpSeq-MD2:AVITI-Yicheng-Sensei-Jon-101224:2413435223:1:10102:0227:0049
Is original Aviti data interleaved or did it become like that after you did the conversions?
It came as a Paired Set. I have Aviti Reads from two different labs, Lab1 3 sets, Lab2 4 sets. They are all the same having read pairing issues. I only just recently noticed the issue. I haven't gotten any errors when mapping them, but once I noticed the issue, it makes sense why I have been having issues.
In case of Illumina reads the course of action would be to use first use
reformat.shto deinterleave the reads into individual R1/R2 files and then userepair.shfrom BBMap suite (https://bbmap.org/tools/repair ) orseqkit pair(https://bioinf.shenwei.me/seqkit/usage/#pair ) to bring reads from two files back in sync. This operation will also allow you to remove any unpaired reads into separate files.Aviti read headers (shown above) appear to be similar to Illumina, so the option above may work (you will need to try it out).
While it seems unlikely, you should also check the original data files (with
repair.sh) to verify that the reads were originally in sync, before you started operating on the data (fastq to unaligned BAM etc).Unfortunately, since I mentioned before that I do all of my work on Usegalaxy, which doesn't have those two tools. I had found that those two tools are to goto, but if they're not available to me, can you think of a workaround that doesn't use those two tools???
Like I said, the original files were provided as a pair, so not interleaved.
How does this workflow sound?? I was reading how BBmap repair works, this sounds similar, just splitting the R1/R2 apart first.
Do any of these ideas sound any good, or do you have any other suggestions?
Thanks!!!
I don't recall why you are doing these conversions instead of directly aligning the data. If you think the original data files are out of sync then it would essential to fix that problem before alignment.
Thats why I am doing this. If I directly align, say with Hisat2, the forward and reverse don't match. They come from different fragments. I have tried every logical fix, and they still come back out of order. Its like some invisible settings that is preventing proper name sorting.
Interlacer is supposed to match pairs in a read, and output singletons separately, whether they are single or not, there are single reads between reads that were paired. So to de-interleave the reads aren't paired, they come out in mixed order.
So there is a good chance that your original data files are out of sync and indeed need to be fixed. I would prefer using the correct tool to address this (which you are not able to use in galaxy). Since you seem to want to continue doing this alone (and in galaxy), you have to try and see what works. AFAIK aligners expect the data to be in random order (e.g. like it came off the sequencer). So make sure that your sorting does not consider sequence, quality scores, read lengths in that process.
I emailed the lab to see if they have a fix. Worst case, I should be able to CAT them together and just use them as single reads???