Hi guys,

Sorry I am new here and would appreciate it if someone can help me. For my research, I need to retrieve the abstract of papers with keyword Covid-19, coronavirus, ... and save the abstract along with the corresponding mesh terms.

What I have done so far I am able to retrieve the abstracts and KEYWORDS using the below piece of codes. However, later I got to know that the keywords cannot help me because for instance in the keyword I have "covid19", "Covid-19", "learning in pandemic", "learning tools for pandemic events" ,... As you see "covid19", "Covid-19" refer to one thing ---> covid-19 and "learning in pandemic", "learning tools for pandemic events" refers to the same thing---> pandemic and learning.

But as I need to retrieve a lot of data I cannot go through each keyword individually to map them to the same concept. Thats why I thought instead of retrivieing kewords I can use mesh terms (with the hope that it does the same).

But I am not able to get it from pubmed data base!!!

I tried different ways none of them was successful:

1.  Here the mesh are blank

    fetch = PubMedFetcher()

    pmids = fetch.pmids_for_query('covid-19', retmax=10)

    def retrieve():
        path = '.\DeepLearning_Lesson1\pubmedfile'
        while True:
            for pmid in pmids:
                    # abstracts[pmid] = fetch.article_by_pmid(pmid).mesh
                    if fetch.article_by_pmid(pmid):
                        abstract = fetch.article_by_pmid(pmid).abstract
                        mesh = fetch.article_by_pmid(pmid).mesh
                    time.sleep(1)

    t = threading.Thread(target=retrieve)
    t.start()

And then I have tried this:

2. 

    pubmed = PubMed( email="myemail.com")
    results = pubmed.query("coronavirus", max_results=1)

    path = '.\DeepLearning_Lesson1\pubmedfile2'
    import pandas as pd
    def retrieve():
        mydic = {}   
        for article in results:

            js = article.toJSON()

            jso = json.loads(js)
            if jso['abstract'] != None and jso['abstract'] != '' and len(str(jso['pubmed_id'])) <= 10 and 'keywords' in jso:
                if len(jso['keywords']) != 0:
                    with open(os.path.join(path, str(jso['pubmed_id'])+'.txt'), 'w+', encoding='utf-8') as fp:
                        fp.write(jso['abstract'])

                    mydic['pubid'] = str(jso['pubmed_id'])
                    mydic['keywords'] = jso['keywords']

This is the scenario that we can retrieve the keywords associated with the abstracts but as I explained above the keyword cannot help me.

3. I have also tried this, although could get the abstract but the mesh is empty or raising error if I change the code:

    from Bio import Entrez
    Entrez.email = "sgnbx@mail.umkc.edu"     # Always tell NCBI who you are
    pmids = [12654674 ,29058482 ,28991880 ,28984387,28551248]
    handle = Entrez.efetch(db="pubmed", id=','.join(map(str, pmids)),
                       rettype="xml", retmode="text")
    record = Entrez.read(handle)
    abstracts = [pubmed_article['MedlineCitation']['Article']['Abstract']['AbstractText'][0]
                 if 'Abstract' in pubmed_article['MedlineCitation']['Article'].keys()
                 else pubmed_article['MedlineCitation']['Article']['ArticleTitle']
                 for pubmed_article in record['PubmedArticle']]

    mesh = [pubmed_article['MedlineCitation']['MeshHeadingList']['MeshHeading']['DescriptorName'][0]
                 if 'MeshHeading' in pubmed_article['MedlineCitation']['MeshHeadingList'].keys()
                 else None
                 for pubmed_article in record['PubmedArticle']]

Any help is really appreciated.