I haven't come across (yet) anyone doing GSEA on overlaps exactly for the reason you mention, that they need to be ranked somehow. This kind of leaves you with over representation analysis (ORA).

I would suggest checking what happens when you do ORA on these gene lists. I quite like gprofiler for that (https://biit.cs.ut.ee/gprofiler/gost), they also have an R package. You can run all overlapping genes together or play around with just separating the overlapping up-regulated genes and down-regulated genes and testing these separately. There is also an option on gprofiler to input an ordered list based on something meaningful (that would bring you back to the question of how do you rank/order the overlaps, though). They recently published a workflow for various approaches (https://f1000research.com/articles/9-709) and even show how you can make the output compatible with clusterprofiler for visualization.

Or you can just look at the over-laps of your actual GSEA results, to see which pathways are perturbed using both approaches.