This depends on the relative cost of false positives vs false negatives. What FDR and sensitivity are you aiming for?

I've done some fairly extensive benchmarking of SV callers for which you can find germ-line and simulation results at http://shiny.wehi.edu.au/cameron.d/sv_benchmark to given you some idea of the trade-off between FDR and sensitivity for various callers. The DREAM Somatic Mutation Calling Challenge results also give you a basic idea of these trade-offs.

In my somatic usage of GRIDSS, I've found that requiring assembly support has been a very effective filter - something that you could also do for manta. As for direct read support, the relative strength of RP and SR support depends on the library fragment size distribution and will be different for each library. For 2x150bp sequencing, you'll have weak RP support even for real variants in libraries with mean fragment size <300bp. Ideally, you would filter on variant quality directly, instead of RP and SR independently but not many SV callers actually report a variant quality score, and none that I know of report an even remotely calibrated quality score.

Disclaimer: I am the author of GRIDSS.