Chapter 5 is dedicated to the most important part of predictive modeling for biomarker discovery based on high-dimensional data – multivariate feature selection. When dealing with sparse biomedical data whose dimensionality is much higher than the number of training observations, the crucial issue is to overcome the curse of dimensionality by using methods capable of elevating signal (predictive information) from the overwhelming noise. One way of doing this is to perform many (hundreds or thousands) parallel feature selection experiments based on different random subsamples of the original training data and then aggregating their results (for example, by analyzing the distribution of variables among the results of those parallel experiments). Two designs of such parallel feature selection experiments are discussed in detail: one based on recursive feature elimination, and the other on implementing the stepwise hybrid selection with T2. The chapter includes also descriptions of three evolutionary feature selection algorithms: simulated annealing, genetic algorithms, and particle swarm optimization.

Multiple linear regression generalizes straight line regression to allow multiple explanatory (or predictor) variables, in this chapter under the normal errors assumption. The focus may be on accurate prediction. Or it may, alternatively or additionally, be on the regression coefficients themselves. Simplistic interpretations of coefficients can be grossly misleading. Later chapters elaborate on the ideas and methods developed in this chapter, applying them in new contexts. The attaching of causal interpretations to model coefficients must be justified both by reference to subject area knowledge and by careful checks to ensure that they are not artefacts of the correlation structure. There is attention to regression diagnostics, to assessment, and comparison of models. Variable selection strategies can readily over-fit. Hence the importance of training/test approaches and cross-validation. The potential is demonstrated for errors in x to seriously bias regression coefficients. Strong multicollinearity leads to large variance inflation factors.

This chapter explains weighting in a manner that allows us to appreciate both the power and vulnerability of the technique and, by extension, other techniques that rely on similar assumptions. Once we understand how weighting works, we will better understand when it works. This chapter opens by discussing weighting in general terms. The subsequent sections get more granular. Sections 3.2 and 3.3 cover widely used weighting techniques: cell-weighting and raking. Section 3.4 covers variable selection, a topic that may well be more important than weighting technique. Section 3.5 covers the effect of weighting on precision, a topic that frequently gets lost in polling reporting. This chapter mixes intuitive and somewhat technical descriptions of weighting. The technical details in Sections 3.2 and3.3 can be skimmed by readers focused on the big picture how weighting works.

Chapter 3 introduces readers to correlation and regression analysis. Methods of correlation answer the question concerning the portion of variance that two variables share. Regression uses, in most cases, metric dependent and metric or categorical independent variables. The OLS solution for the standard case of one predictor and one outcome variable is derived. Based on this derivation, characteristics of model parameters are explained. Real-world data examples are given for simple regression (one predictor and one outcome variable) and for multiple regression (multiple predictors and one outcome variable). In addition, this chapter describes GLM approaches to curvilinear regression (regression lines are curved rather than straight to approximate non-linear variable relations), to curvilinear regression of repeated observations, to symmetric regression (where the regression of Y on X results in the same solution as the regression of X on Y), to best subset and stepwise selection regression (which results in an optimal selection from multiple predictors), and the recently developed direction dependence analysis to evaluate hypotheses concerning the causal flow of a variable association

This chapter focuses on model evaluation and selection in Hierarchical Modelling of Species Communities (HMSC). It starts by noting that even if there are automated procedures for model selection, the most important step is actually done by the ecologist when deciding what kind of models will be fitted. The chapter then discusses different ways of measuring model fit based on contrasting the model predictions with the observed data, as well as the use of information criteria as a method for evaluating model fit. The chapter first discusses general methods that can be used to compare models that differ either in their predictors or in their structure, e.g. models with different sets of environmental covariates, models with and without spatial random effects, models with and without traits or phylogenetic information or models that differ in their prior distributions. The chapter then presents specific methods for variable selection, aimed at comparing models that are structurally identical but differ in the included environmental covariates: variable selection by the spike and slab prior approach, and reduced rank regression that aims at combining predictors to reduce their dimensionality.