This is a test version of Biostars. For the public version, visit https://www.biostars.org.
R : Error in scan() too many items

I am working with rehh package.

I have to create an object of class haplohh from the function data2haplohh:

test<-data2haplohh(hap_file="test.hap", map_file="test.map.inp", haplotype.in.columns=TRUE)
Map file seems OK: 1054416  SNPs declared for chromosome 1
Haplotype are in columns with no header

Error in scan(hap_file, what = "character", quiet = TRUE) : too many items

So the *.hap file has 1054416 lines and 5008 columns(approx 10 GB size), *.map.inp file has 1054416 lines and 5 columns(15 MB size). My systems working RAM memory is 64 GB and 8TB is the storage space

I have an option of chopping up the files but, in this case It would create errors in my calculation. So I need to load the input files as a whole.

Please tell me how to solve this!

r rehh
• 2,599 views • link
updated by Alex Reynolds Alex Reynolds • written by shreyajha • shreyajha

Hi As you can see I didn't get any reply in that forum. That's why I posted it here.

• 1 views • link
written by shreyajha • shreyajha

1 answer

Indices for R vectors are 32-bit signed integers, even in 64-bit versions of R, which can put an upper limit on how many elements you can work with.

Looking at the source of getAnywhere(data2haplohh) for your error message, it looks like it tries to make a vector or matrix with 5.3B elements (1054416 rows x 5008 columns). The limit you are perhaps running into is 2^31-1 or 2.1B elements.

Maybe contact the developers for help with adjustments to code to support larger inputs, or look at subsampling or validating your input.

• 1 views • link
written by Alex Reynolds Alex Reynolds

Log in to answer this question.

More posts like this