I have two Seurat objects that were made by merging of samples:

object1 <-merge(A, y=B, add.cell.ids=c("A","B"))
object2 <-merge(C, y=D, add.cell.ids=c("C","D"))

For each object I did all preprocessing, PCA, and clustering. Now I want to merge cluster 1 from object1 and cluster 3 from object2

cluster1<-subset(object1, ident=1)
cluster3<-subset(object2, ident=1)

clusters1_3<-merge(cluster1, y=cluster3, add.cell.ids=c("1","3"))

however, when I cluster the cluster1_3 object, it would cluster to A, B, C, D initial clusters. What am I doinng wrong?