viewing MAF files as dataframe of size over 2GB

I used GATK's Funcotator to annotate a VCF file I have and it produced a MAF file just over 2GB in size. I've tried using pandas in Python and maftools in R and it hasn't worked. Specifically the file size seems to be too large to be opened in R throwing this error

Error in data.table::fread(file = maf, sep = "\t", stringsAsFactors = FALSE,  : File '' does not exist or is non-readable.

and pandas isn't really made for MAF files. Usually when running this annotation it was enough to open it in Excel but this file is way too big. Does anybody know an application or package (whether it be R or python or something else) to open MAF files of this size? Any help is appreciated.

r python gatk maf annotations

• 4,724 views • link

written 6.0 years ago by noon •

That error is not because memory, it is pointing that the file is not located where your maf variable declares it.

Why do you need to open it? Linux commands more, grep, awk can help you to view the content.

• 0 views • link

written 6.0 years ago by JC

I wrote a simple helper alias for files like these:

alias tsview='column -s$'\''\t'\'' -t | less -S'

Spreadsheet-like view on the terminal!

• 0 views • link

written 6.0 years ago by Ram

Yes this works so I can view it! The problem is I need to manipulate it as if it were a dataframe so I can apply certain thresholds to the data and select for specific columns.

• 0 views • link

written 6.0 years ago by noon •

Your maf variable doesn't seem to contain the path to the MAF file. Can you show the output of dput(maf)?

• 0 views • link

written 6.0 years ago by Ram

This is what I have:

 funcotation = system.file('extdata', '/Users/Downloads/funcotated.maf',
                              package='maftools')
readfunc=read.maf(maf=funcotation)

when I use dput(maf) I get this:

""

• 0 views • link

written 6.0 years ago by noon •

1 answer

That's not how system.file works. Use: read.maf(maf = '/Users/Downloads/funcotated.maf') and skip the first line altogether.

• 0 views • link

written 6.0 years ago by Ram

Thank you that was the problem! Sorry I'm new to using maftools so I was going off the documentation tutorial.

Thank you for your time!

• 0 views • link

written 6.0 years ago by noon •

That happens. I find it useful to take the following steps while working on changing documentation code:

Run the code and ensure it works as given (it usually does)
Understand each function call and parameter in the lines leading up to the line I wish to change: in your case, that would mean reading through ?system.file and looking at its parameters. It is here that you'd find that it finds files that are included within packages. You're using a custom file, so system.file is not for you.
OK, then, how do you give a custom file? From the ?system.file documentation, it is clear that it returns a string with the path to the file. So, if you have that string, you can do what system.file() does. As it turns out, you have the path. Replace the call to system.file with the path you have, and you're all set.

It also helps to check each line's execution and output as that line is executed. You'd have noticed the error happening in line 1 and probably solved the problem yourself.

• 0 views • link

written 6.0 years ago by Ram

BTW, please accept my answer using the green check mark on the left to mark the question solved.

Upvote|Bookmark|Accept

• 0 views • link

written 6.0 years ago by Ram