Conclusions about the effectiveness of selection systems require gathering, evaluating, weighting, and interpreting validity data, but these conclusions are obviously challenged to the extent that this process is suspect. Local validity information within the organization may be desirable but not available, and conducting a local validity study may be practically infeasible because of limited time, resources, and small sample sizes. Specific validity studies outside the organization may also be problematic if they are based on jobs or settings of questionable relevance, small sample sizes, range-restricted incumbent samples, and unreliable or content-deficient predictor and criterion measures. It is usually an understatement to say that sifting through a pile of such studies to make educated guesses about the validity of selection measures within of a specific organizational setting could be an idiosyncratic, time-consuming, and frustrating process, resulting in little confidence in any summary conclusions.