Thank you for your answer, Michael.
We are using Agilent two-color microarrays. I am aware that such microarrays should be studied using log-ratios, and this is how we intend to study the biological effects of the treatment.
However, since the array is custom-designed, we have the possibility of replacing uninformative probes with better-performing ones in subsequent experiments that we will carry out. I am also under the impression that certain probes are more sensitive to the array-effect than others. It is in this perpective that I am trying to "cluster" probes into two categories: those who yield biologically meaningful data, and those where the array-effect is predominant, and who should be replaced in subsequent array designs.
However, when working on log-ratios, I am "cancelling out" the array effect, and thus cannot draw conclusions on its relative importance for each probe. This is why I was working on single channels, hoping that by first clustering the samples, then finding probes whose variation did not "fit" (For examples, probes whose response is always in the maximum range due to repeated elements), I could target such probes for replacement. I assumed this would work as I've generally had success clustering the various conditions of single-channel data through by-group analysis before, but not this time around.
Did you run a a dye-swap experiment to see if you can account for any dye bias?
Yes, we are doing dye-swaps. All replicates are biological, and we are alternating the dyes we use for control and treatment.