A formal theory of appropriateness for statistical operations is presented which incorporates features of Stevens’ theory of appropriate statistics and Suppes’ theory of empirical meaningfulness. It is proposed that a statistic be regarded as appropriate relative to statements made about it in case the truths of these statements are invariant under permissible transformations of the measurement scale. It is argued that the use of inappropriate statistics leads to the formulation of statements which are either semantically meaningless or empirically nonsignificant.

A distinction is made between statistical inference and psychometric inference in factor analysis. After reviewing Rao's canonical factor analysis (CFA), a fundamental statistical method of factoring, a new method of factor analysis based upon the psychometric concept of generalizability is described. This new procedure (alpha factor analysis, AFA) determines factors which have maximum generalizability in the Kuder-Richardson, or alpha, sense. The two methods, CFA and AFA, each have the important property of giving the same factors regardless of the units of measurement of the observable variables. In determining factors, the principal distinction between the two methods is that CFA operates in the metric of the unique parts of the observable variables while AFA operates in the metric of the common (“communality”) parts.

On the other hand, the two methods are substantially different as to how they establish the number of factors. CFA answers this crucial question with a statistical test of significance while AFA retains only those alpha factors with positive generalizability. This difference is discussed at some length. A brief outline of a computer program for AFA is described and an example of the application of AFA is given.

Generalizability theory concerns the adequacy with which a “universe” score can be inferred from a set of observations. In this paper the theory is applied to a universe in which observations are classifiable according to two independent variable aspects of the measuring procedure. Several types of universe scores are developed and the variance components ascertained for each type. The composition of expected observed-score variance and the adequacy of inference to a particular type of universe score is a function of the procedure used in gathering data. A generalizability study provides estimates of variance components which can be used in designing an efficient procedure for a particular decision purpose.

The researcher in behavioral or biological science has available a large arsenal of methods for studying the nature and statistical significance of differences between various groups of individuals. He may use any of a wide variety of univariate methods to test hypotheses about differences in location, in dispersion, in correlation between pairs of attributes, etc. In a multivariate situation methods are available to test hypotheses about mean vectors, variance-covariance matrices, correlation matrices, factor structures, etc.

There are two points of view involved in studying the nature of differences between groups. One point of view is descriptive, and has traditionally been treated under the topic of estimation in statistics. The other usually involves a question as to whether it is reasonable to assume that some hypothesized model about the nature of differences between populations is in fact true, and has usually been treated under the ~pic of tests of hypotheses. The descriptive and inferential points of view must of necessity often overlap.

General features of a probability model for errors of classification are recapitulated as an introduction to particular cases and applications. Several models for dichotomous and nondichotomous systems are examined in sufficient detail to elaborate a procedure for dealing with any particular case. The system O = f(T, E) has empirical reference where, as statistic or parameter, probability of occasional subclass membership is given by observation, and one seeks to recover T and E values from O. A procedure for relating models and data is described. Applications of the concepts and methods are illustrated for several areas of psychological research.

A “strong” mathematical model for the relation between observed scores and true scores is developed. This model can be used

1. 1. To estimate the frequency distribution of observed scores that will result when a given test is lengthened.

2. 2. To equate true scores on two tests by the equipercentile method.

3. 3. To estimate the frequencies in the scatterplot between two parallel (nonparallel) tests of the same psychological trait, using only the information in a (the) marginal distribution(s).

4. 4. To estimate the frequency distribution of a test for a group that has taken only a short form of the test (this is useful for obtaining norms).

5. 5. To estimate the effects of selecting individuals on a fallible measure.

6. 6. To effect matching of groups with respect to true score when only a fallible measure is available.

7. 7. To investigate whether two tests really measure the same psychological function when they have a nonlinear relationship.

8. 8. To describe and evaluate the properties of a specific test considered as a measuring instrument.

The model has been tested empirically, using it to estimate bivariate distributions from univariate distributions, with good results, as checked by chi-square tests.

(1) An analysis of the method of paired comparisons shows that Case V and Case VI, the latter characterized by log-normal distributions and Weber’s law for subjective continua, are fundamentally indistinguishable. Case VI produces a log-arithmetic interval scale of subjective magnitude. (2) It is demonstrated that the difference between discrimination scales according to Case V and from category rating is due to the difference between intra- and interindividual variability yielding different Weber functions.

Although probit analysis has been applied in the field of psychometrics, its use has been restricted by the requirement of linear regression of probits on the independent variate. This paper shows that polynomial probit analysis, a natural extension of the idea of probit analysis, can sometimes overcome the difficulty of a nonlinear plot and does not involve excessive computational labor.

This paper presents an adaptation of the method of moments for comparing observed and theoretical distributions of reaction time. By using cumulants in place of moments, considerable simplification of the treatment of convoluted distributions is obtained, particularly if one of the components is normally distributed. Stochastic latency models are often poorly fitted by reaction time data. This may be because a simple latency distribution is convoluted with a normal or high-order gamma distribution. The comparison method described will assist investigation of this and other interpretations of reaction time distributions.

There have been a number of important developments by several authors which are based on the assumption that test taking behavior is stochastic. That is, it is assumed that the probability that an individual will “endorse” or “pass” a dichotomous item can be represented by some function of an underlying, “latent” attribute (possibly multivariate) called the item trace line or item characteristic curve. Various forms for trace lines have been postulated such as polynomial [9, 11], step function [9], normal ogive [13, 19], and logistic [1, 2]. An interesting model due to Rasch [18] utilizes only a subject parameter and item parameter to determine the probability of passing an item. The only model which does not make some formal assumptions about trace lines is Lazarsfeld's latent class model. All of these models for test behavior are unified by two basic assumptions. One is the stochastic assumption of a probability of passing or endorsing an item.

One of the major concerns of reliability theory has been the estimation of the reliability of a composite measure from the degree of agreement among its component parts. In the classical theory, formulas were developed under the assumption that the parts are strictly equivalent. It was later shown that the same formulas follow from various sets of weaker assumptions which require the composites to be strictly equivalent and require the parts to have a certain homogeneity of statistical properties, but not necessarily to be equivalent. An alternative model which has received increasing attention in recent years regards a given measure as a random sample from a universe of measures whose homogeneity or equivalence is not specified a priori, and a composite test as a random sample of items from a universe of not-necessarily-equivalent items. This too permits an internal-consistency estimate of reliability. Both the equivalent-composites model and the randomsampling model appear to be unduly restrictive and unrealistic; we propose here to develop the implications of a third model in which a test is considered to have been formed by stratified sampling of items.

This paper considers the problem of computing estimates of factor loadings, specific variances, and communalities for a factor analytic model. The equations for maximum-likelihood estimators are discussed. Iterative formulas are developed to solve the maximum-likelihood equations and a simple and efficient method of implementation on a digital computer is described. Use of the iterative formulas and computing techniques for other estimators of factor loadings and communalities is also considered to provide a very general approach for this aspect of factor analysis.

The sampling properties of four item discrimination indices (biserial r, Cook’s index B, the U–L 27 per cent index, and Delta P) were investigated in order to ascertain their sampling properties when small samples drawn from actual test data rather than constructed data were employed. The empirical results indicated that the mean index values approximated the population values and that values of the standard deviations computed from large sample formulas were good approximations to the standard deviations of the observed distributions based on samples of size 120 or less. Goodness of fit tests comparing the observed distributions with the corresponding distribution of the product-moment correlation coefficient based upon a bivariate normal population indicated that this correlational model was inappropriate for the data. The lack of adequate mathematical models for the sampling distributions of item discrimination indices suggests that one should avoid indices whose only real reason for existence was the simplification of computational procedures.

The present model treats the scaling of pair-comparison preference judgments among a unidimensional set of stimuli across a population of individuals. Given a set S of n stimuli, S = {S1, S2, …, Sn}, the model yields a partially ordered metric on the interstimulus distances which may be used to construct an interval scale of values forS. Obtained also are a set of predictions P = {P1, P2, …, Pn} wherePi is the proportion of individuals in the population whose first choice among the elements of S is Si. A numerical illustration is offered and comparisons are drawn with Coombs' unfolding technique.

While the traditional multiple correlation coefficient appears to be inherently an asymmetrical statistic, it is actually a special case of a more general measure of linear relationship between two sets of variables. Another symmetric generalization of linear correlation is to the total relatedness within a set of variables. Both of these developments rest upon the generalized variance of a multivariate distribution, which is seen to be the fundamental concept of linear correlational theory.

It is suggested that if Guttman’s latent-root-one lower bound estimate for the rank of a correlation matrix is accepted as a psychometric upper bound, following the proofs and arguments of Kaiser and Dickman, then the rank for a sample matrix should be estimated by subtracting out the component in the latent roots which can be attributed to sampling error, and least-squares “capitalization” on this error, in the calculation of the correlations and the roots. A procedure based on the generation of random variables is given for estimating the component which needs to be subtracted.

A method of generating any number of score and correlation matrices with arbitrary population parameters is described. Either Z scores or stanines are sampled from a normal population to represent factor scores by an IBM 1620 program. These are converted to variates from a population with an a priori factor structure. The effectiveness of the method is illustrated from research data. Some further modifications and uses of the method are discussed.

Inequalities relating the communalities to the multiple-correlation coefficients are derived. They are stronger than the well-known inequalities which have played an important role in factor analysis for the past thirty years. Necessary and sufficient conditions for equality are obtained.