A recent debate on implicit measures of racial attitudes has focused on the relative roles of the person, the situation, and their interaction in determining the measurement outcomes. The chapter describes process models for assessing the roles of the situation and the person-situation interaction on the one hand and stable person-related components on the other hand in implicit measures. Latent state-trait models allow one to assess to what extent the measure is a reliable measure of the person and/or the situation and the person-situation interaction (Steyer, Geiser, & Fiege, 2012). Moreover, trait factor scores as well as situation-specific residual factor scores can be computed and related to third variables, thereby allowing one to assess to what extent the implicit measure is a valid measure of the person and/or the situation and the person-situation interaction. These methods are particularly helpful when combined with a process decomposition of implicit-measure data such as a diffusion-model analysis of the IAT (Klauer, Voss, Schmitz, & Teige-Mocigemba, 2007).

Adequate measurement of psychological phenomena is a fundamental aspect of theory construction and validation. Forming composite scales from individual items has a long and honored tradition, although, for predictive purposes, the power of using individual items should be considered. We outline several fundamental steps in the scale construction process, including (1) choosing between prediction and explanation; (2) specifying the construct(s) to measure; (3) choosing items thought to measure these constructs; (4) administering the items; (5) examining the structure and properties of composites of items (scales); (6) forming, scoring, and examining the scales; and (7) validating the resulting scales.

This chapter focuses on experimental designs, in which one or more factors are randomly assigned and manipulated. The first topic is statistical power or the likelihood of obtaining a significant result, which depends on several aspects of design. Second, the chapter examines the factors (independent variables) in a design, including the selection of levels of a factor and their treatment as fixed or random, and then dependent variables, including the selection of items, stimuli, or other aspects of a measure. Finally, artifacts and confounds that can affect the validity of results are addressed, as well as special designs for studying mediation. A concluding section raises the possibility that traditional conceptualizations of design – generally focusing on a single study and on the question of whether a manipulation has an effect – may be inadequate in the current world where multiple-study research programs are the more meaningful unit of evidence, and mediational questions are often of primary interest.

The procedure for a preliminary ruling is central in the ‘complete system of remedies’ offered by the Union to its citizens. Since Article 263 TFEU grants only a very reduced standing to ‘non-privileged applicants’, Article 267 TFEU became the main gate for individuals to bring their claims against the EU before the European Court of Justice. Yet, claims for breaches of fundamental rights by the Union are not at all common in the procedure for a preliminary ruling. This chapter investigates the (real) use and (realistic) potential of Article 267 TFEU as a means for the protection of fundamental rights against breaches by the EU institutions. The chapter maps all instances in which individuals used the procedure for a preliminary ruling to bring a claim against the Union for breaches of their fundamental rights since the coming into force of the Treaty of Lisbon. Using this mapping exercise, the chapter identifies how individuals raise this type of claims in the procedure, discusses the accessibility of the procedure for individual applicants, and assesses the shortcomings of the procedure as a means to redress breaches of fundamental rights by the Union. It argues that these shortcomings have to do with the structure and design of the procedure itself.

This chapter is written for conversation analysts and is methodological. It discusses, in a step-by-step fashion, how to code practices of action (e.g., particles, gaze orientation) and/or social actions (e.g., inviting, information seeking) for purposes of their statistical association in ways that respect conversation-analytic (CA) principles (e.g., the prioritization of social action, the importance of sequential position, order at all points, the relevance of codes to participants). As such, this chapter focuses on coding as part of engaging in basic CA and advancing its findings, for example as a tool of both discovery and proof (e.g., regarding action formation and sequential implicature). While not its main focus, this chapter should also be useful to analysts seeking to associate interactional variables with demographic, social-psychological, and/or institutional-outcome variables. The chapter’s advice is grounded in case studies of published CA research utilizing coding and statistics (e.g., those of Gail Jefferson, Charles Goodwin, and the present author). These case studies are elaborated by discussions of cautions when creating code categories, inter-rater reliability, the maintenance of a codebook, and the validity of statistical association itself. Both misperceptions and limitations of coding are addressed.

In Chapter 12, the author discusses approaches to judging the effectiveness of both criterion-referenced and norm-reference performance assessments. The chapter includes guidelines for helping create or evaluate performance assessments for particular contexts. It also describes how to use statistical techniques for the same purpose. The chapter presents actual ratings from a classroom-based group oral discussion test and shows how teachers used statistics to determine both score dependability and reliability. The author discusses how to calculate coefficient agreement to help determine the dependability of the assessment and Cronbach’s Alpha to help determine its reliability. A major point of the chapter is that when certain conditions exist, test users can exploit an assessment to determine test takers’ mastery of language criteria (criterion-referenced purpose) and to compare their abilities (norm-referenced purpose). The author provides an appendix that shows readers how to use Excel software to calculate the statistics in the chapter.

In Chapter 4, the author introduces the concept of validity. The chapter begins with an exploration of approaches to defining a construct. These approaches include using language theory, a language needs analysis, corpora, and curriculum objectives to help test developers determine what specific language ability they desire to measure. The chapter emphasizes the importance of alignment, which relates to how well the test content and test taker response processes match the construct’s content and the response processes that the test aims to measure. The author uses a detailed example of assessing children’s ability to communicate on a playground in a second language. The major point of the example is that the assessment should require children to use the same kinds of language they use when they communicate on the playground. This alignment helps ensure that the assessment measures the targeted language ability and will lead to positive washback on teaching and learning.

The focus of the chapter is on turning concepts into measurable variables, also known as operationalization.Best practices in created variable based on the concept are covered.The differences between nominal, ordinal, and interval level variables are discussed along with their relevance for statistical tests later covered in the book.The importance of measurement validity and reliability and their implications for research provide another focus of the chapter.

To wit, we have three specific goals here. First, we want to review the activities of the three-hatted pollster. We do this to provide greater context for each type of pollster. Some of us are all three; others are some combination of these. Any pollster worth their salt must at least be a data scientist, or they risk losing credibility.

Second, we explore the role of the pollster in society. Ultimately, what is the purpose of the pollster? In our view, pollsters are critically important in any democracy. We believe this is often overlooked due to the ranking frenzy after every electoral cycle. Here, we put the profession into proper perspective.

And third, we discuss the use of non-survey, or alternative data, inputs as proxy measures for public opinion. We provide a framework for pollsters to think through them in a critical manner. Validation is a key concept which we introduce here – one more tool for the data scientist.

This chapter describes some commonly used nonhuman paradigms for assessing animal behavior and the figures that are used to present those data. The chapter opens with an overview of some animal species used in neuroscience research, a discussion about nonhuman housing, and a description of types of validity that behavioral neuroscientists concern themselves with. The behavioral tests described here are divided into five major categories: motor behaviors; pain; learning and memory; mental disorders such as anxiety, depression, and substance use disorder; and social behaviors. Included is a description of a survival analysis and an explanation of interpreting Kaplan–Meier curves.

Contestations about the contents and validity of laws and legal principles are fundamental to the (international) legal profession. After all, when engaging with legal norms, disagreements about their meaning and validity a central part of the day-to-day work of legal professionals specialising in international law, including legal counsel representing governments, international judges, legal officers working for international organisations and non-governmental organisations, and legal academics. We propose a practice-oriented approach to empirically research such interpretive legal contestations by groups of legal professionals. Using an interdisciplinary perspective, we contribute to IR norms research by drawing on not only IR practice theory, but also Bourdieu-inspired research within the Sociology of International Law and ongoing discussions on legal realism in International Legal Theory, including what we have called European New Legal Realism. After outlining how to implement our approach using either a Bourdieusian perspective or the concept of communities of practice, we use normative contestations in and around climate change law to illustrate its added value. Such an approach not only promises to make interpretive legal contestations visible empirically, but also emphasises how interpretive legal contestations matter as they reflect underlying power dynamics and may result in normative legal change in practice.