I think you are right, this is not normal. I think the PI is somewhat overzealous and too enthusiastic, and doesn't realize the time and effort you'll have to spend in this little exam. I think you made the right decision with rejecting the offer. When I had job interviews, sometimes I got the "exam" to make for example a 10 minute presentation, which was never a problem, since I had always some slides from previous presentations (and I love giving talks). But doing a literature study followed by an analysis exercise is really too much, especially with a solid CV like yours.

Don't know if it is normal, but most certainly it is not common. Furthermore, I think it doesn't fit a reasonable set of expectations for applicants in academic labs.

While reading the tea leaves is not my strong expertise, it appears that your potential PI is not convinced by your qualifications. For young PIs with limited funds and tight tenure deadlines, it can be somewhat expected that they may want to give you a spin without actually hiring you. If this is an experienced PI, I would not have much sympathy or understanding for this demand.

It is impossible to give you a definitive advice on what to do, as it all depends on how badly you want to work with this PI, your other job options, and in general your life circumstances.

Let me play the devil's advocate for a moment and consider this from her perspective.

While you have publications/have written software there may have been something in her interactions with you that may have given her a cause to propose this "exam". If this is a "high flying" lab then she probably has any number of people who want work with her. As a result she may be interested in choosing only the best to join her group.

While this may be good for your career in the long run it may ruin your work/life balance in short term.

I guess that PI wants to be sure you are indeed capable of what you claim you are. I find it a bit odd to be examined like that and it would probably depend on the attitude of the PI during the interview if I was willing to take that exam or not. If the PI is respectful and simply wants to make sure they invest their resources into the best possible candidate who indeed brings fits and will contribute well, ok maybe I'd do it. Papers and CVs can look nice but how would you actually judge how much of that is really your own intellectual work rather than a product of good circumstances and help from others. Some examination is probably required beyond of what is written on a paper. Still, if they were condescending and/or arrogant, I'd probably reject, but it is easy to say that for me while I sit here on my couch, not having any background information on that job. Maybe it is a highly attractive position that fully fits your desires and you have to "sacrifice" some of your pride to get it. Or maybe this attitude of the PI already indicates that it is a poor working environment from an interpersonal standpoint. I probably would be ok with an environment where people are 100% colleagues and 0% friends, but would reject an obvious anti-social climate. Just thinking aloud here, as I said I have zero insight into your situation.

My experience has been largely in the for-profit industry, from management consulting to hedge fund to biotech, so I'll offer my perspectives. I've been to a few unusual interview sessions in my career. Some put candidates for the same job in the same conference room to solve a problem collaboratively. Some ask for whiteboard coding in front of a large audience. Some, like what the OP pointed out here, give a take-home exam/questions.

I have had candidates who went through rigorous training in computer science but struggled to explain the basic difference between a set and a list. I also have had candidates with graduate school experience in statistics who couldn't explain the central limit theorem. IMHO, I think what the PI does is perfectly reasonable, especially for entry-level technical positions. In some situations, an 1-hr career seminar with a 30-minute 1:1 just aren't enough for accurate assessments.

We all work very hard to put ourselves in the position where we get to choose whom we want to work with, not whom we have to. So if an "exam" is deemed unreasonable for the candidate, turn down the interview and move on.