'Quality' in this sense has no defined threshold, or boundary. If, after you have normalised your data, there is a problematic sample, then it will usually be very obvious on each of the following:

• Box-and-whisker plot
• Violin plot
• Unsupervised hierarchical clustering
• PCA

For the first 3 of these, I have put some code here: A: Hierarchical Clustering in single-channel agilent microarray experiment

For PCA, take a look here: A: PCA plot from read count matrix from RNA-Seq

------------------------------------------

A sample that has degraded cDNA will likely come back with very low expression values, many of which will actually be NA due to having fallen below the background noise (microarray) or will be zero (RNA-seq). These may additionally show up as having very low coefficient of variation (CoV). There are obviously, then, other reasons why a sample may have low quality, such as technical issues. Note, of course, that an outlier sample is not necessarily a poor quality sample and may just be a sample that behaves differently / unexpectedly.

There is one tool at which you may take a look, called nsFilter.

Kevin