This is a test version of Biostars. For the public version, visit https://www.biostars.org.
How to scrape BioMart data from https://sorfs.ugent.be/ website

Hi all,

I would like to download database from this website: https://sorfs.ugent.be/database but it seems that they do not maintain the database anymore, only the table available.

Is there anyway to pull the table from this website as a csv file?

Best,

biomart

did you ask the authors to restore their server ?

yes but they did not reply

1 answer

something like:

seq 1 100 4377380 | while read F ; do wget   --no-check-certificate  -O - "https://sorfs.ugent.be/database/micropep?sEcho=4&iColumns=18&sColumns=%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C%2C&iDisplayStart=${F}&iDisplayLength=100&mDataProp_0=0&sSearch_0=&bRegex_0=false&bSearchable_0=true&bSortable_0=true&mDataProp_1=1&sSearch_1=&bRegex_1=false&bSearchable_1=true&bSortable_1=true&mDataProp_2=2&sSearch_2=&bRegex_2=false&bSearchable_2=true&bSortable_2=true&mDataProp_3=3&sSearch_3=&bRegex_3=false&bSearchable_3=true&bSortable_3=true&mDataProp_4=4&sSearch_4=&bRegex_4=false&bSearchable_4=true&bSortable_4=true&mDataProp_5=5&sSearch_5=&bRegex_5=false&bSearchable_5=true&bSortable_5=true&mDataProp_6=6&sSearch_6=&bRegex_6=false&bSearchable_6=true&bSortable_6=true&mDataProp_7=7&sSearch_7=&bRegex_7=false&bSearchable_7=true&bSortable_7=true&mDataProp_8=8&sSearch_8=&bRegex_8=false&bSearchable_8=true&bSortable_8=true&mDataProp_9=9&sSearch_9=&bRegex_9=false&bSearchable_9=true&bSortable_9=true&mDataProp_10=10&sSearch_10=&bRegex_10=false&bSearchable_10=true&bSortable_10=true&mDataProp_11=11&sSearch_11=&bRegex_11=false&bSearchable_11=false&bSortable_11=true&mDataProp_12=12&sSearch_12=&bRegex_12=false&bSearchable_12=false&bSortable_12=true&mDataProp_13=13&sSearch_13=&bRegex_13=false&bSearchable_13=false&bSortable_13=true&mDataProp_14=14&sSearch_14=&bRegex_14=false&bSearchable_14=false&bSortable_14=true&mDataProp_15=15&sSearch_15=&bRegex_15=false&bSearchable_15=false&bSortable_15=true&mDataProp_16=16&sSearch_16=&bRegex_16=false&bSearchable_16=false&bSortable_16=true&mDataProp_17=17&sSearch_17=&bRegex_17=false&bSearchable_17=false&bSortable_17=true&sSearch=&bRegex=true&iSortCol_0=0&sSortDir_0=asc&iSortingCols=1&_=1716563877578" | python -m json.tool ; done
{
    "aaData": [
        [
            "<a href = 'database/andreev_2015:100170' >andreev_2015:100170</a>",
            "<a href='http://www.ensembl.org/Multi/Search/Results?q=ENST00000405356;y=0;site=ensembl_all;x=0;page=1;facet_feature_type=Gene'>ENST00000405356</a>"
,
            "human",
            "andreev_2015",
            "10",
            102157437,
            102157610,
            58,
            "exonic",
            "protein_coding",
            "yes",
            "0.4749",
            "1.5047",
            "MPSESVCLLGRLQPKHQRVAAVKSPVMMMMRRTKRNSLSRFAALGRKRGLRIRKEET*",
            "CTGCCAAGCGAGTCGGTCTGCCTCCTGGGAAGGCTGCAGCCAAAGCATCAGAGAGTAGCAGCAGTGAAGAGTCCAGTGATGATGATGATGAGGAGGACCAAAAGAAACAGCCTGTCCAGGTTTGCAGCTTTGGGAAGAAAAAGGGGTT
TAAGGATTAGAAAGGAAGAAACCTAA",
            "0.357142857143",
            "-9.23678239084",
            "ENSG00000166197"
        ],
        [
            "<a href = 'database/andreev_2015:100182' >andreev_2015:100182</a>",
            "<a href='http://www.ensembl.org/Multi/Search/Results?q=ENST00000405356;y=0;site=ensembl_all;x=0;page=1;facet_feature_type=Gene'>ENST00000405356</a>"
,
            "human",
            "andreev_2015",
            "10",
            102158107,
            102158169,
            21,
            "exonic",
            "protein_coding",
            "yes",
            "0.8899",
            "2.4239",
            "MILILTQAPRMSHQRTRSQR*",
            "CTGATTCTGATTCTGACTCAAGCTCCGAGGATGAGCCACCAAAGAACCAGAAGCCAAAGATAA",
            "0.368421052632",
            "-7.55004712725",

thank you so much!

Don't forget to follow up on your threads. If an answer was helpful, you should upvote it; if the answer resolved your question, you should mark it as accepted. You can accept more than one answer if they all work. If an answer was not really helpful or did not work, provide detailed feedback so others know not to use that answer.

Upvote|Bookmark|Accept

Log in to answer this question.