This is a test version of Biostars. For the public version, visit https://www.biostars.org.
Fetch table from clinvar database according to a list of rsid

Hi

I have a list of rsid and i want to search against clinvar database and print the condition_germline column with respect to each rsid. Anyway, i have got a script.

use strict;
use warnings;
use LWP::Simple;
use HTML::TableExtract;

# Read list of rsids from file
my $rsids_file = 'rsids.txt';
open(my $fh, '<', $rsids_file) or die "Can't open $rsids_file: $!";
my @rsids = <$fh>;
close($fh);

# Loop through each rsid
foreach my $rsid (@rsids) {
    chomp($rsid);

    # Construct URL for ClinVar search
    my $url = "https://www.ncbi.nlm.nih.gov/clinvar/variation/$rsid/";

    # Fetch web content
    my $content = get($url);
    unless (defined $content) {
        warn "Couldn't get $url: ", $!;
        next;
    }

    # Extract table
    my $te = HTML::TableExtract->new(headers => ["Condition_Germline"]);
    $te->parse($content);

    # Print condition_germline column
    foreach my $ts ($te->tables) {
        foreach my $row ($ts->rows) {
            print join("\t", @$row), "\n";
        }
    }
}

But, when its runs getting the following error.

Couldn't get https://www.ncbi.nlm.nih.gov/clinvar/variation/rs11203366/:  at .\fetch_condition.pl line 22.
Couldn't get https://www.ncbi.nlm.nih.gov/clinvar/variation/rs11203367/:  at .\fetch_condition.pl line 22.
Couldn't get https://www.ncbi.nlm.nih.gov/clinvar/variation/rs874881/:  at .\fetch_condition.pl line 22.
Couldn't get https://www.ncbi.nlm.nih.gov/clinvar/variation/rs776453694/:  at .\fetch_condition.pl line 22.
Couldn't get https://www.ncbi.nlm.nih.gov/clinvar/variation/rs80324279/:  at .\fetch_condition.pl line 22.
Couldn't get https://www.ncbi.nlm.nih.gov/clinvar/variation/rs324420/:  at .\fetch_condition.pl line 22.
Couldn't get https://www.ncbi.nlm.nih.gov/clinvar/variation/rs112766203/:  at .\fetch_condition.pl line 22.

I appreciate your suggestions.

Thank you.

python clinvar perl
• 2,587 views • link
written by ashaneev07 • ashaneev07

Don't forget to follow up on your threads. If an answer was helpful, you should upvote it; if the answer resolved your question, you should mark it as accepted. You can accept more than one answer if they all work. If an answer was not really helpful or did not work, provide detailed feedback so others know not to use that answer.

Upvote|Bookmark|Accept

• 0 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

Hii... i have updated the script in python..But, still getting no data found. Actually the data is there, i have print the parsed html file but not print the conditions_germline = soup.find('Conditions-Germline'). Is there problem with this line?? I have attached the script with this. Kindly have a look into this.Thank you.

>     import requests
>     from bs4 import BeautifulSoup
>     
>     
>     def search_clinvar_by_rsid(rsid):
>         url = f"https://www.ncbi.nlm.nih.gov/clinvar/?term={rsid}"
>         try:
>             response = requests.get(url)
>             if response.status_code == 200:
>                 soup = BeautifulSoup(response.content, 'html.parser')
>                 #print(soup)
>                 conditions_germline = soup.find('Conditions-Germline')
>                 print(conditions_germline)
>                 if conditions_germline:
>                     conditions_text = conditions_germline.text.strip()
>                     print(f"conditions_germline data for rsid '{rsid}':")
>                     print(conditions_text)
>                 else:
>                     print(f"No conditions_germline data found for rsid '{rsid}'")
>             else:
>                 print(f"Failed to retrieve search results for rsid '{rsid}'. Status code:", response.status_code)
>         except requests.RequestException as e:
>             print("Error:", e)
>     
>     
>     def read_rsids_from_file(filename):
>         with open(filename, 'r') as file:
>             return [line.strip() for line in file]
>     
>     rsids = read_rsids_from_file('rsids.txt')
>     for rsid in rsids:
>         search_clinvar_by_rsid(rsid)
• 0 views • link
written by ashaneev07 • ashaneev07

1 answer 

$ cat input.rs

rs11203366
rs11203367
rs874881
rs776453694
rs80324279
rs324420
rs112766203

 wget -qO - "https://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh38/clinvar.vcf.gz" |\
   bcftools query -i "`cat input.rs  | sed 's/^rs//' | awk '{printf("%sRS=\x27%s\x27",(NR==1?"":" || "),$1);}'`" -f '%CHROM %POS %REF %ALT rs%RS %CLNDN\n'

1  17331039  G  A  rs11203366   Rheumatoid_arthritis|Abnormal_pulmonary_interstitial_morphology|PADI4-related_condition
1  17331121  T  C  rs11203367   Rheumatoid_arthritis|Abnormal_pulmonary_interstitial_morphology|PADI4-related_condition
1  17334004  G  C  rs874881     Rheumatoid_arthritis|Abnormal_pulmonary_interstitial_morphology|PADI4-related_condition
1  21838914  C  T  rs776453694  Schwartz-Jampel_syndrome_type_1|Inborn_genetic_diseases|Schwartz-Jampel_syndrome|Lethal_Kniest-like_syndrome
1  33013330  G  C  rs80324279   Reticular_dysgenesis
1  46405089  C  A  rs324420     FAAH-related_condition|FAAH_POLYMORPHISM|Polysubstance_abuse,_susceptibility_to
1  97305279  G  A  rs112766203  not_provided|Dihydropyrimidine_dehydrogenase_deficiency
• 0 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

ah ! and your code doesn't work because clinvar ID is not a rs-ID.

• 0 views • link
written by Pierre Lindenbaum Pierre Lindenbaum

so which one should i prefer? i don't need the entire results i need the condition-Germline table only. (for eg:-from, https://www.ncbi.nlm.nih.gov/clinvar/variation/294920/ need the condition-Germline table ) when i manually check with the rsid i got the result. I'm confused with this.Could you please make a clarification regarding this?

• 0 views • link
written by ashaneev07 • ashaneev07

Log in to answer this question.

More posts like this