This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Processing Raw Illumina Data From Geo

Hello,

these are my first steps in processing the data from Illumina (HumanMethylation450). I'm trying to process raw data from http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE37066

require(GEOquery)
getGEOSuppFiles(GEO="GSE37066")

After extracting the raw files, I see that, besides the Chip description there are raw AB intensities. I would like to use lumi or methylumi packages to read the actual data and perform background correction, normalization, etc. to compare the results. However, when executing

require(lumi)
example.lumiMethy <- methylumiR("GSE37066/GSE37066_raw_AB_intensities.txt", sep="\t")

I got an error: "Error in if (dattypes$original[i] != "") { : argument is of length zero" As far as I understand the file GSE37066_raw_AB_intensities.txt is not created by the BeadStudio, right? Is there any method which I can use to convert this data to "MethyLumiSet"? Or is the provided raw data not enough? What do I miss?

geo
• 7,591 views • link
updated by Charles Warden Charles Warden • written by Andre • Andre

1 answer

The contents of GSE37066_RAW are all annotation files. They are not directly useful for analysis.

It is possible that you could make use of the signal intensities, but I would just use the beta value and detection p-values from primary data matrix:

ftp://ftp.ncbi.nlm.nih.gov/geo/series/GSE37nnn/GSE37066/matrix/GSE37066_series_matrix.txt.gz

I'm not certain about lumi in particular, but you should be able to reformat this matrix to make a data table suitable for analysis. For example, that is how I analyze public 450k data using COHCAP:

http://sourceforge.net/projects/cohcap/

Not sure about your level of bioinformatics experience. The reformatting can definitely be done using a Perl code, and it probably can be done within R as well. If you don't have much coding experience, I know Partek has a GEO downloader application that will make a data matrix for you (even if you don't have an institutional license, you could use a demo license for this feature). Exporting a tab-delimited text file or comma-separated file should allow an easy import into R:

http://www.partek.com/?q=partekgs

• 182 views • link
written by Charles Warden Charles Warden

Thank you! If I understand this right, the raw data is just incomplete in this case, but the working values are provided with the matrix.

• 0 views • link
written by Andre • Andre

Yeah - GEO won't let you upload the raw .idat files for reasons of space. Therefore, you have to work with some sort of processed data.

• 0 views • link
written by Charles Warden Charles Warden

Log in to answer this question.

More posts like this
  • Where is clonotypes .csv file
    written by yueli7 yueli7
    Hello, I tried to analysis following datasets: https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM5210599, or https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM5808300. In the VDJ_Analysis: https://github.com/ucdavis-bioinformatics-training/2020-Advanced_Single_Cell_RNA_Seq/blob/master/data_analysis/VDJ_Analysis_fixed.md It should has two files: 1. filtered_contig-annotation.csv, 2. clonotypes.csv. Where I …
  • NCBI GEO/GSM/SRA conventions: Is raw data a part of the SRA or a supplementary file?
    written by abe • abe
    I downloaded about 30 RNA-seq fastq files via SRA from multiple studies found on NCBI GEO. FastQC identified adapters for about half of the samples, …
  • Map from GEO samples' name to TCGA name
    written by landscape95 landscape95
    I am working on several GEO data sets, the samples are GSMxxx, can these samples be converted to TCGA format type TCGA-xx-xx. For example, data …
  • Normal Size Raw Data To Research, Cant Download Petabyte Of Zillion Reads
    written by smagikern • smagikern
    <p>Where i can find rpkm or expression value of all rna-seq experiment, not at 4gb <a href='http://samtools.sourceforge.net/SAM1.pdf'>bam</a> file but 34 kb txt-any?</p> <p><a href='http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE32874'>http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE32874</a> - …
  • Where To Find Rnas Used In Gene Expression Studies?
    written by Click downvote Click downvote
    <p>I have found several datasets that use the transcriptome from the hippocampus in mice and humans. I am very interested in getting the transcriptome from …
  • How To Find The Strand-Specific Info In The Public Data From The Illumina Human Body Map 2.0 Project
    written by camelbbs camelbbs
    <p>Hi,</p> <p>Anyone can guide a while how to find if this public rnaseq data are prepared by strand-specific assay. Thanks a lot!</p> <p><a href="http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE30611">http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE30611</a></p> <p>Ch</p>
  • Download Geo Tar File Automatically From Linux/Unix
    written by dfernan dfernan
    <p>Hi,</p> <p>I'd like to automatically download the tar files from the following links, does anyone knows how to do that?</p> <p><a href='http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE39977'>http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE39977</a></p> <p><a href='http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE36375'>http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE36375</a></p> <p>Kind …
  • How To Avoid Mixing Intronic Region When Sequencing Non-Poly-A Rna Using Rna-Seq?
    written by Hanfei Sun • Hanfei Sun
    <p>The data source is here:</p> <p><a href="http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE30567">http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE30567</a></p> <p>I was wondering how it could avoid mixing intronic RNA (result of pre-mRNA splicing) when using <strong>non</strong> poly …
  • Converting Gene Expression Omnibus Format To Fasta
    written by Click downvote Click downvote
    <p>I have found the a number of reads I want to test against a genome using Bowtie. They are located here: <a href="http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM113418">http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM113418</a></p> <p>The problem …
  • What Are Their Meanings In The Raw Data Of Mirna Sequencing Reads ?
    written by zhidong.yuan • zhidong.yuan
    <p>The GSM652853<em>ptr</em>01.txt comes from <a href="http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM652853">http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM652853</a> (miRNA sequencing raw reads).</p> <p>[science@localhost GSE26545<em>RAW]$ head GSM652853</em>ptr_01.txt</p> <pre><code>8 1 575 886 GGGACTGACCTGAAATGAAGAGAATTCGTATGCCGT 8 1 875 922 GGAGGTAGGAGATTATAAGGATACTGTATGGCCTTC 8 …