In one of my gene sets, I am interested in selecting gene subsets such that no two genes occur in the same set more than "t" times. Mathematically, given n objects and selecting k objects without replacement, how many combinations exist such that no two objects are in the same group more than "t" times?
E.g, If we have five genes A, B, C, D, E and pick 5 choose 3 = 10 combinations with t =2: But gene combination A B is present three times and hence out of three possibilities
A B C
A B D
A B E
I will select only 2 (=t) combinations randomly.
Is there a closed form solution for finding out how many combinations exist and listing them out for say n = 40, k=5, and t=3?