This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Help with Generating Clustered Heatmap from Large Datasets

Hi,

I'm working with a large dataset where I'm trying to generate a clustered heatmap from z-scores. However, I’m hitting a memory error: "Error: cannot allocate vector of size 614.9 Gb." The data size is simply too large to be processed in one go.

Does anyone have advice on:

  • How to handle heatmap generation for such large datasets?
  • Are there any methods to process or visualize the data in chunks while retaining meaningful clusters?
  • Any tools, R packages, or approaches for optimizing memory usage for this type of task?

I'd appreciate any insights or suggestions—thank you!

heatmap
• 1,194 views • link
updated by Mensur Dlakic Mensur Dlakic • written by ArtiCore • ArtiCore

614.9 Gb

What kind of data is this? You should add some information about that to get specific help. Depending on the data type there may be different strategies.

• 0 views • link
written by GenoMax GenoMax

The data are derived from genomic raw count matrix transformed into z-scores. Numeric matrix. In R I am trying to run this command:

pheatmap(mat_z_scores, cluster_rows = TRUE, cluster_cols = TRUE)

enter image description here

• 0 views • link
updated by GenoMax GenoMax • written by ArtiCore • ArtiCore

Not sure if this makes a difference in R, but it python one could force the data type to be float32 rather than a default float64 type.

• 0 views • link
written by Mensur Dlakic Mensur Dlakic

0 answers

No answers yet.

Log in to answer this question.

More posts like this