Birnbaum (2011, 2012) questioned the iid (independent and identicallydistributed) sampling assumptions used by state-of-the-art statistical tests inRegenwetter, Dana and Davis-Stober’s (2010, 2011) analysis of the“linear order model”. Birnbaum (2012) cited, but did not use, atest of iid by Smith and Batchelder (2008) with analytically known properties.Instead, he created two new test statistics with unknown samplingdistributions.

Our rebuttal has five components: 1) We demonstrate that the Regenwetter et al.data pass Smith and Batchelder’s test of iid with flying colors. 2) Weprovide evidence from Monte Carlo simulations that Birnbaum’s (2012)proposed tests have unknown Type-I error rates, which depend on the actualchoice probabilities and on how data are coded as well as on the null hypothesisof iid sampling. 3) Birnbaum analyzed only a third of Regenwetter etal.’s data. We show that his two new tests fail to replicate on the othertwo-thirds of the data, within participants. 4) Birnbaum selectively picked dataof one respondent to suggest that choice probabilities may have changed partwayinto the experiment. Such nonstationarity could potentially cause a seeminglygood fit to be a Type-II error. We show that the linear order model fits equallywell if we allow for warm-up effects. 5) Using hypothetical data, Birnbaum(2012) claimed to show that “true-and-error” models for binarypattern probabilities overcome the alleged short-comings of Regenwetter etal.’s approach. We disprove this claim on the same data.