This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Biostars API Issues

I am conducting a study where I need access to data on this forum on how to improve genomic data visualization tools. I have tried scraping using the word "visualization" but the website does not allow scraping. I tried fetching the posts using the API on https://www.biostars.org/info/api/ but it does not fetch. Is there a way I can do this? or is there a way I can get the data?

visualization web-scraping api

Yeah the tag based fetch is broken. Scraping is bad but you can pull each post and check tags there - add an interval between each curl and grep for tag_val:. It's extremely inefficient though; but scraping won't help you much - for each tag, the /tags/ page shows only the top 50 results, no matter how many hits it gets. The "next" buttons don't work.

0 answers

No answers yet.

Log in to answer this question.