I would be interested in this. In particular I am interested in reproducing a number of research articles based on public databases (A: Can pre-existing microarray datasets be re-used to make new discoveries/ assumptions?), because I find that to be such an interesting area...

edit: I just noticed that that was originally your question that I referred to :)

I am up for it. I guess you will get a huge response from many communities from different countries.

To anyone interested - Please let me know if anyone gave the microarray analysis a try? I'm at the annotation part and got to figure out how to get the mean values... May be we could make constructive comments or formulate questions to ask on the platform here. Thanks.

Guys, I contacted Dr Istvan and we might get a new post type for the study group, but it will only happen if lots of people are interested with the idea and are actively participating. We are currently working on this paper focussed on mining public microarray data: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3479650/pdf/nihms408530.pdf
jobinv and I have already tried and posted a few scripts (R, python and bash).
Any ideas are welcome.
Thanks

script1.py
#!/usr/bin/python
import os

''' Removing extraneous annotations, to keep only probe id, gene name and expresion values '''

os.chdir('/home/olivier/bioinformatics/microarray_tests/Crohns-disease-and-ulcerative-colitis/')

`#Remove extraneous annotations from the temp_dataset generated by awk, write cleaned_dataset_with_duplicate_gene_names`

`with open('temp_dataset','r') as temp_dataset_handle:  
    records = temp_dataset_handle.readlines()  
    records = [i.split('\t') for i in records] `  

`#Selecting columns of gene names and expresion values only, from each record`  
`def pid_gene_and_value_selection(x):   
    return '\t'.join(x[:38])`  

`cleaned_records = '\n'.join(map(pid_gene_and_value_selection, records)) #Add a newline after each record parsed`  

`with open('GDS_cleaned','w') as GDS_cleaned_handle: 
    GDS_cleaned_handle.write(cleaned_records)`  

`#Creates file 'GDS_cleaned`

script2.py

`#!/usr/bin/python`  

`import os`  
`import numpy as np`  

'`''Get the means for duplicate genes from GDS_cleaned'''`  

`os.chdir('/home/olivier/bioinformatics/microarray_tests/Crohns-disease-and-ulcerative-colitis')`

`with open('GDS_cleaned', 'r') as gds_cleaned_handle:  
    header = gds_cleaned_handle.readline()  
    records = gds_cleaned_handle.readlines()`  

`#Parse records elements, comprising probe_id, gene name and expression values into numpy arrays`  
`parsed_record_elements = np.array([elem.rstrip('\n').split('\t') for elem in records]) #Create an array for each record, i.e. probe id, gene name, expression values`  

`#Extract list of all genes`  
`def extract_gene(rec):  
    return rec[1]`  
`gene_list =map(extract_gene, parsed_record_elements)`  

`#Create list of unique genes`  
`unique_genes = list(set(gene_list))`  

`if os.path.exists('cleaned_dataset_with_means'):  
    os.remove('cleaned_dataset_with_means')`

`#Testing whether to take means or not and writing output`  
`with open('cleaned_dataset_with_means','a') as output:  
    output.write(header)  
    for gene in unique_genes:  
        if gene_list.count(gene) == 1:   
            rec = parsed_record_elements[parsed_record_elements[:,1] == gene]#select rows (from gene name to expression values) where column==unique gene.  
            output.write('\t'.join(rec[0]) + '\n') #write down record`  

`        else:  
            temp_rec = parsed_record_elements[parsed_record_elements[:,1] == gene] #select record from records where gene is duplicated. Append to temp list  
            expr_vals = [map(float,i) for i in temp_rec[:,2:]] #Converting expression values to floats, so means can be taken  
            probe_id_and_name = temp_rec[:,:2][0].tolist() #temp_rec[:,:2][0] selects first 2 elements (probe id, gene name) from all rows (records)   
            mean_expr_vals = np.mean(expr_vals, axis=0) #carry out column means for duplicated genes  
            new_record = '\t'.join(map(str,probe_id_and_name) + map(str,mean_expr_vals.tolist()))+'\n' #Prepare/format record with new values  
            output.write(new_record)`  

`#removing temp file`  
`os.remove('GDS_cleaned')`  
`os.remove('temp_dataset')`

#Notes:
#The different controls have same names but differing probe ids, but have been grouped together.
#One probe id (first instance) as been retained for duplicate gene entries.

script3.R
load script in R
This the first R script I've written to completion. Grateful if anyone could comment.
Note that I'm not very sure of how the author compared the samples. So I compared all combinations (affected, normal, unaffected) using "2-class unpaired SAM". Nevertheless, I got the gene "IFI30", but not "TRPV1" by fiddling with the delta parameter...
So if anyone has an idea, you're welcome to comment on that too.

setwd('/home/olivier/bioinformatics/microarray_tests/Crohns-disease-and-ulcerative-colitis/')
df <- read.table('cleaned_dataset_with_means',sep='\t',header=TRUE)

rownames(df) <- df[,2] #Specify gene names as row names
df <- df[,3:ncol(df)] #getting rid of probe id and gene name columns

healthy <- c('GSM155564','GSM155565','GSM155566','GSM155567') #specimen

unaffected <- c('GSM155575','GSM155576','GSM155577','GSM155578','GSM155579','GSM155580', 'GSM155581','GSM155582','GSM155583','GSM155584','GSM155585','GSM155586', 'GSM155592','GSM155593','GSM155594','GSM155595','GSM155599') #specimen

affected <- c('GSM155568','GSM155569','GSM155570','GSM155571','GSM155572','GSM155573', 'GSM155574','GSM155587','GSM155588','GSM155589','GSM155590','GSM155591', 'GSM155596','GSM155597','GSM155598') #specimen

library('siggenes')
#Function which selects from healthy, affected or unnaffected states. Returns value of SAM analysis.
#cl is a required sam argument defining column names as a vector

label.and.sam.analysis <- function(x,y) { x <- df[,x] #selecting first argument as columns from df y <- df[,y] x.vs.y <- cbind(x,y) #Place the sample columns in a new dataframe cl.x.vs.y <- c(rep(0, ncol(x)), rep(1, ncol(y))) #converting columns to 0's and 1's in a vector sam.x.vs.y <- sam(x.vs.y, cl.x.vs.y) return(sam.x.vs.y) }

#SAM test to retrieve significanlty differing gene expression data
#Each of the 3 variable names can be plotted, separately to view delta vs FDR & Delta vs sig genes
sam1 <- label.and.sam.analysis(healthy, affected)
sam2 <- label.and.sam.analysis(healthy, unaffected)
sam3 <- label.and.sam.analysis(affected, unaffected)

#Drawing SAM plots to visualize up/ down-regulated genes. Check "./R.analysis/R.analysis.jpg" in current directory
jpeg(filename='./R.analysis/R.analysis2.jpg',width = 1200, height = 800, quality=100, pointsize=17)
par(mfrow=c(2,2)) #All 3 plots go to 1 page
plot(sam1,2.3, main='sam.healthy.vs.affected')
plot(sam2,2.1, main='sam.healthy.vs.unaffected')
plot(sam3,2.2, main='sam.affected.vs.unaffected')
dev.off()

#Shows selected genes and associated statistics for each gene
#The second arguments are the delta values I chose. I hope they're good
s1 <- summary(sam1, 2.3)
s2 <- summary(sam2, 2.1)
s3 <- summary(sam2, 2.2)

#Writing genes (up/down-regulated) to "./R.analysis/"
write.table(rownames(s1@mat.sig),'./R.analysis/sig.genes.sam.healthy.vs.affected.delta2.3',quote=FALSE,row.names=FALSE,col.names=FALSE)
write.table(rownames(s2@mat.sig),'./R.analysis/sig.genes.sam.healthy.vs.unaffected.delta2.1',quote=FALSE,row.names=FALSE,col.names=FALSE)
write.table(rownames(s3@mat.sig),'./R.analysis/sig.genes.sam.affected.vs.unaffected.delta2.2',quote=FALSE,row.names=FALSE,col.names=FALSE)

Not finished.. Work in progress.

Hey guys

I need an idea here. I've created a google group, but I fear this way will prevent other biostars members from viewing the work, right? Any ideas as to how we can make that possible? Please send me your email address (private msg) if anyone's interested so I can add you to the group.
Regards.