Forum:Biostar Public Dump

Question

[Pavel Senin]

Hi guys: just wonder about the public data dump release. When do you plan to make it?

Answer 1

[Istvan Albert]

One of the main concerns that hold it back is that I am unsure what type of anonymization is appropriate and what are our users (tacitly) expecting. For example should emails be included, should all voting be included, should usernames be anonymized etc

My personal perspective with respect of all other social sites is that I would not like to have all my voting record/personal info made public without me opting in or being able to revoke it.

[Pavel Senin]

I worked with stackoverflow dump http://meta.stackoverflow.com/questions/2677/database-schema-documentation-for-the-public-data-dump-and-data-explorer having the same format would be great

[Istvan Albert]

I see, so only the bookmarking votes are included - one thing we need to change having the email hash instead of actual email - I am going to add an issue for this here: https://github.com/ialbert/biostar-central/issues/180

[Pavel Senin]

so, what would be an estimated time-frame? I am quite interested in looking on some stats from this data.

[Istvan Albert]

would it be easier/quicker to do what you want if there were a JSON API that you could query for information: posts by user, post content etc similar to this below (originally announced on the newsgroup), it is a lot easier to add that:

There is now an API that allows you to query the current traffic and other stats on Biostar. To get last hour's traffic do:

http://www.biostars.org/api/traffic/

To get the current statistics on the number of posts/users do:

http://www.biostars.org/api/stats/

This later URL can take a third parameter interpreted as days in the past. To see the stats for the state of the content 100 days ago do a:

http://www.biostars.org/api/stats/100/