This is a test version of Biostars. For the public version, visit https://www.biostars.org.
scRNAseq datasets integration issue

Hi all,

I am working on a large-scale scRNA-seq integration project using Seurat. The dataset includes 20 samples with nearly 3 million cells in total. I am running the analysis on a server with 400 GB of RAM. I used FindIntegrationAnchors and IntegrateData from the Seurat package.

However, during the integration process I consistently encounter the following error:

Error: vector::reserve Execution halted

Based on some investigation, this error may be related to R’s internal memory limitation (the 2^32-1 vector size limit), rather than the physical memory available on the server.

Has anyone experienced a similar issue when integrating very large datasets in Seurat? If so, I would appreciate any suggestions or recommended strategies (e.g., workflow adjustments, data splitting, alternative integration approaches, etc.).

Thank you very much for your help!

data scrnaseq integration
• 429 views • link
updated by Bastien Hervé Bastien Hervé • written by JackShao • JackShao

You can switch to scanpy. Python is usually more efficient in handling heavy datasets.

• 0 views • link
written by raphael.B • raphael.B

400GB for 3 millions cells seems a bit short to me to analyze them all in one block.

You can try these options in R:

memory.limit(size=X)
options(future.globals.maxSize = X)

When running the FindIntegrationAnchors, loggin to your HPC node and check the memory consomption htop to see if you reach your node limit.

You can have a look at Siletti et al., Science 2023 where they had the same problematic for 3M cells, it is in python and they were using loom files but here is an issue that can interest you.

Don't quote me on this but, this is what I would do, if your dataset is already integrated :

  • Shuffle your dataset and create chuncks
  • Call your anchors for each chunck
  • Take the overlap or union of you anchor lists
• 0 views • link
written by Bastien Hervé Bastien Hervé

0 answers

No answers yet.

Log in to answer this question.

More posts like this