I'm very interested in your answer, it sounds like it's easy to implement. But I don't quite understand what you mean. Can you explain it in more detail?
(1) Hits between two isoforms of the same protein. This can be dealt with simply by annotating isoforms with gene names, and removing all hits between a gene at itself.
Do you mean to change all isoform IDs to gene IDs so that different isoforms share the same gene ID? I am not sure whether duplicate sequence names will cause errors in the pipeline.
(2) Quasi-reciprocity: gene1 and gene2 are reciprocal hits, but gene1.iso1 best hit is gene2.iso1, gene2.iso1 best hit is gene1.iso2. You can fix this by defining a reciprocal hit as the following: two genes have reciprocal hits if there exist isoforms A (gene 1) and B (gene 2) such that the bitscore of A against B exceeds the bitscore of A against all other genes, and the bitscore of B against A exceeds the bitscore of B against all other genes.
I agree with you, but I have no idea how to program based on BLAST. In fact, I am only proficient in R programming, and for Python, I am only at running some pipelines and packages through Linux bash. Any further suggestions or examples?