Hello all, first time poster.

I am working on a project that requires a phylogenetic tree of monoamine transporters to reconstruct an ancestral sequence. The problem is that I'm stuck on getting a good selection of sequences to work with. There's just so many homologs, that when I try to align, I get 50K+ positions. I have tried clustering and not clustering in many varieties of combinations, but the clustering doesn't seem to help. For example, I tried lowering the cutoffs for one attempt and got over 150K positions in the alignment for about 5K clusters.

Currently, my process looks like this:

1. blast against sequence of interest
2. filter out any unwanted sequences: no partials, nothing containing an X, length in a specific range (300-800, though I'm now trying smaller)
3. cluster using mmseqs with 0.80 for -c and -min-seq-id, cov-mode 0
4. align using muscle super5
5. iteratively trim and realign, but that hasn't been successful

When I try to trim away sequences and realign, it's just as likely to get longer as it is shorter.

I know that I need to isolate the SERTs, NETs, and DATs from other SLC proteins across a large number of taxa, but doing it based on string values in the description or gene name is not reliable. Not all of them are properly labeled, and when using nr_cluster_seq, the representative sequence name may be named something else.

E-vals and other metrics also aren't super helpful, as the majority of hits are just 0 for e-value, and it takes thousands of clusters just to reach something like 1e-160. There's just so much noise and I don't know how to get it out of the way.

I'm also attempting this now by doing the process multiple times in smaller groups, so a separate query and alignment for each phylum of interest, in the hopes that I can concatenate and realign them later.

My main question: How do I handle all of these sequences I don't want in an intelligent way? Am I not using the right database?