This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Meta pvalues merging from independent tests for seperate null hypotheses

Hello,

I'm generating heatmaps for pvalues of enriched GO terms. Some of the enriched GO terms are redundant in semantic meaning, or are either child nodes of more "easily digestible" GO terms. For example DNA damage and repair mechanisms is easier to understand than say a listing of all the exoDNases. I know which GO terms overlap in biological significance and semantic similarity, and instead of displaying each one in the heatmap, I'd like to collapse them into a single descriptive term. I know that there are a variety of methods for aggregating pvalues used in meta-analysis. However, as I understand it, many make the assumption of independent tests for the same null hypothesis. So since I'm looking at merging pvalues from different GO terms, my initial impression is that were talking about different null hypotheses for each term. Is there a method that I can use which would be well suited for merging pvalues from related GO terms? A couple people have suggested averaging, but I'm not sure what it means to average probabilities, per-say.

Thanks, Martin

pvalues fisher's method gene ontology
• 2,719 views • link
written by mforde84 mforde84

2 answers

I would suggest a different approach which is to test fewer GO terms. You can do this either by using a GO slim or by using the method from this paper to select pertinent GO terms.
If you still decide to go with combining p-values, remember that GO being a directed acyclic graphs, terms are not independent so Fisher's method is not appropriate but there are extensions that could be suitable e.g. Brown's method is available for R, python and Matlab on GitHub (corresponding paper here).
Concerning averaging, you should be aware that the average of p-values is not guaranteed to be a p-value although 2 times the average is a p-value (see here for details and some discussion on stats.stackexchange)

• 0 views • link
written by Jean-Karim Heriche Jean-Karim Heriche

Thank you, Jean. This is very helpful. I've done the slim approach and this does seem to work very well. One additional question if you don't mind indulging me further. Say instead of GO, I'm interested in applying something similar to IPA pathway scores. As I understand it, the IPA pathways are DAG, though the score calculation is independent of network topology. If it's known that two pathway terms are related in terms of gene overlap, then Brown's method would still be appropriate for analysis since they are not independent pvals, correct? However, I'm thinking that the amount of gene overlap may be of some importance here when combine pvalues, e.g., say two terms share a single gene, but group one has 200 genes group two has 20. How would you suggest approaching this?

Thanks again, Martin

• 0 views • link
written by mforde84 mforde84

I don't know IPA as I've never used it. However, pathways in principle are not the same as ontologies. Whereas an ontology is a directed graph of terms, a pathway is usually a directed graph of genes. For testing whether a gene list is enriched in a given pathway, we usually simply count the number of members of the pathway of interest present in the list. In this approach, the pathways are usually considered independent and so are the tests because the overlaps are negligible (with the exception of pathways that are subdivided into smaller pathways, in which case tests would not be independent). The size (in gene number) of a pathway is taken into account by the test (e.g. Fisher's exact test). As a side note, most enrichment tests assume that all genes have the same probability of being annotated and therefore suffer from annotation bias i.e. some genes are way more annotated than others (e.g. P53 is annotated with ~700 GO terms, EGFR with ~600). For more on this, see this paper.

• 0 views • link
written by Jean-Karim Heriche Jean-Karim Heriche

Great explanation.

I haven't seen the data yet, though from what I understand it is 90 or so samples clustered into 3 risk groups which in total have 50-100 significant pathways. Each sample has an enrichment score and associated adjusted pvalue for each pathway. The ultimate goal is to aggregate these samples pvalues first by the cluster grouping. So if I understand correctly, this can be accomplished usings Fisher's method for each pathway term. The subsequent aggregation is for "like" pathways, which are of similar phenotypic relevance (e.g. immunological pathways, lipid signaling pathways, etc) which more than likely (but not necessarily, i.e., not apriori known to) share a good deal of gene overlap. For this aggregation, I have a few options. I can either defer to a "root" pathway (as this seems similar to the GO slim approach), or dependently aggregate pvalues iff there is considerable gene overlap. Otherwise, if there is little to no overlap in gene lists, then I should use Fisher's method.

• 0 views • link
written by mforde84 mforde84

This seems reasonable. Also depending on what you want to do subsequently, I wouldn't worry too much about the dependence. If you combine assuming independence, the p-values may not be accurate but may still be good enough to make comparisons.
I am also wondering if you couldn't accomplish what you want using the scores, may be using some rank aggregation method.

• 0 views • link
written by Jean-Karim Heriche Jean-Karim Heriche

Oh, cool idea. Probably much more robust. So if I understand correctly, initially I could do something like monte carlo based rank aggregations either with lists of enrichment scores or pvalues for each sample within a cluster. That seems relatively straightforward with some of the CRAN tools available. From my limited knowledge of these methods, I don't see a directly obvious way to rank aggregate individual pathways to a more general pathway classification (will probably take some reading to find an applicable example). Either way, I assume Fishers or Brown's methods would still be appropriate here.

• 0 views • link
written by mforde84 mforde84

sorry, meant as comment

• 0 views • link
written by mforde84 mforde84

Log in to answer this question.

More posts like this