I want to calculate pearson correlation by using scipy. However, the input file has a few empty wells for each columns (example seen below), so the calculation from scipy.pearsonr is always "nan". I wonder if there is way to accommodate the empty wells. Thanks a lot for the help!
Input example:
Factor none Bas1 Cha4
YAL001C
YAL002W -0.01
YAL003W
YAL005C -0.22
YAL007C 0.37
YAL008W 0.19
YAL009W 0.08 0.41 -0.19
YAL010C -0.04 0.34 0.14
YAL011W -0.06 0.71 0.31
YAL012W -0.05 1.05
YAL013W -0.05 2.59 0.9
YAL014C 0 0.5 0.85
YAL015C 0.08 0.45 -0.54
YAL016C-B 0.04 0.98 0.29
•
written
by
kuangyuyen •
1 answer
This is not really a bioinformatics question but more of a general data analysis one.
What you need to deal with is called missing data imputation, what to do when data is missing and a given measure cannot be computed. There are various strategies that people may employ, filling in with averages, removing rows that have missing values, setting the cell to zero etc.
The right approach depends on what the final goals are.
Log in to answer this question.
the post will be closed because it is not on the topic of bioinformatics
Thanks a lot for the response
This is a problem I encounter a lot in bioinformatics. Data often goes missing in experimental biology, leaving people to wonder what to do. Too bad the message is not to come here looking for answers. There's no way I would have been able to tell this wasn't a "bioinformatics" problem.
I know you feel strongly about post closings Chris, moreover in this case I happen to personally know the original poster as well. And it could also very well be that she is posting because she recently heard me during a presentation where I encouraged people to ask their questions here. So that is quite a baggage there.
I still think that when a post appears to be unrelated to the main scope of the site we need to make that explicit to be able to educate people on what questions are on topic and what are less so.