well, for one you are better of running blast for orthology detection on protein level, thus using blastP in stead of blastN.

otherwise there are not really optimal parameters, the default should do in most cases. The thing with reciprocal blast analyses is that you have similar top hits, so as long as you run blast A-B in the same setting as B-A you should be fine.