“What Babies Know” (Spelke, Reference Spelke2022) is a state-of-the-art, inclusive, and thoughtful survey of what babies know about a few key domains, making the book a must-read for junior scientists in the field. Perhaps most striking is the breadth of research methods and model systems used in the experiments covered: From various behavioral approaches, to neuroimaging, to computational models, to animal neurophysiology. Even so, this commentary will highlight a method mostly absent from the text (and field) that could offer unique insight into infants' knowledge: Representational similarity analysis (RSA; Kriegeskorte, Mur, & Bandettini, Reference Kriegeskorte, Mur and Bandettini2008). I will discuss this method and explain its strengths, then focus on numerosity as a case study. Specifically, I will consider ways to address questions that the target book highlights as unresolved.
RSA is a method for evaluating the degree of representational similarity between exemplars in a set. RSA, along with the more straightforward “pattern similarity” method (Carlson, Schrater, & He, Reference Carlson, Schrater and He2003), has been used frequently in vision science to compare the representations of exemplars, like faces, scenes, and objects. RSA is most often performed with functional magnetic resonance imaging (fMRI) (i.e., comparing the patterns of activity across voxels that exemplars evoke) but can be applied to any data where metric differences can be measured between exemplars, including electroencephalography (EEG; Xie et al., Reference Xie, Hoehl, Moeskops, Kayhan, Kliesch, Turtleton and Cichy2022) and behavior (Spriet, Abassi, Hochmann, & Papeo, Reference Spriet, Abassi, Hochmann and Papeo2022). One example of a question that RSA can ask is whether the representation of exemplars cluster according to their perceptual similarity (i.e., exemplars that are perceptually different have low similarity), or conceptual similarity (i.e., exemplars that are conceptually different have low similarity). Analyses of this type show that different parts of the brain can divide exemplars either perceptually or conceptually (Bankson, Hebart, Groen, & Baker, Reference Bankson, Hebart, Groen and Baker2018).
A key benefit of RSA, and what makes it distinct from many popular methods in developmental science, is that it can test both the degree of difference between representations and the degree of similarity. By way of contrast, a habituation paradigm can test when infants think one exemplar (i.e., the habituation stimulus) is different from another (i.e., the test stimulus). However, suppose infants do not dishabituate to the test stimulus: That does not necessitate infants think the habituation and test exemplars are similar, but instead may result from poor study design (e.g., exposure is too short) or low statistical power. By contrast, high similarity in RSA typically (although not always, e.g., Spriet et al., Reference Spriet, Abassi, Hochmann and Papeo2022) entails positive evidence of similarity. For instance, in RSA with neuroimaging, high similarity means that neural patterns have a high correlation, which should not be expected by chance. An additional benefit of RSA arises when it uses neuroimaging data: It can disentangle simultaneous representations (Bankson et al., Reference Bankson, Hebart, Groen and Baker2018), as illustrated in the example above where conceptual and perceptual information were localized in different brain regions.
Despite this potential value, RSA remains an uncommon tool in infant research. A few exceptions exist: Spriet et al. (Reference Spriet, Abassi, Hochmann and Papeo2022) recently used RSA to understand the clustering of visual stimulus representations based on gaze behavior. Moreover, Xie et al. (Reference Xie, Hoehl, Moeskops, Kayhan, Kliesch, Turtleton and Cichy2022) used EEG to measure neural responses to categories and evaluated their clustering. Studies using fMRI are surprisingly absent from this list, but this is likely to change soon given the recent success of awake infant fMRI (Ellis et al., Reference Ellis, Skalaban, Yates, Bejjanki, Córdova and Turk-Browne2020, Reference Ellis, Yates, Skalaban, Bejjanki, Arcaro and Turk-Browne2021; Kosakowski et al., Reference Kosakowski, Cohen, Takahashi, Keil, Kanwisher and Saxe2022; Yates et al., Reference Yates, Skalaban, Ellis, Bracher, Baldassano and Turk-Browne2022). The aforementioned articles use RSA to address questions about visual perception, but RSA is flexible enough to tackle broad questions about what infants know. In what follows, I consider how RSA can resolve lingering questions in infant's knowledge of numerosity.
Over the last 30 years, research has led to incredible progress in understanding what infants know about number. Nonetheless, there are at least two questions that RSA can advance. The first question is whether numerosity is represented conceptually, rather than reflecting mere magnitude of perceptual content. Both historically (Clearfield & Mix, Reference Clearfield and Mix1999) and recently (Leibovich, Katzin, Harel, & Henik, Reference Leibovich, Katzin, Harel and Henik2017), researchers have argued that attempts to de-confound perceptual magnitude from conceptual number have been insufficient. For instance, many experimental controls for perceptual confounds have prioritized the wrong stimulus properties (Yousif & Keil, Reference Yousif and Keil2021). A second question RSA can tackle is the extent to which a signature of the numerosity system – namely, that differences in number depend on a ratio (AKA ratio dependence) – is consistent across the number line. Ratio dependence means that arrays containing 4 and 8 dots are perceived as just as different as arrays containing 8 and 16 dots. There is compelling evidence that infants represent numbers according to a ratio for values greater than 3 (i.e., above the subitizing range), but it remains unclear whether quantities lower than four are processed with ratio dependence (Hyde & Spelke, Reference Hyde and Spelke2011; McCrink & Wynn, Reference McCrink and Wynn2004; Starr, Libertus, & Brannon, Reference Starr, Libertus and Brannon2013; Uller, Carey, Huntley-Fenner, & Klatt, Reference Uller, Carey, Huntley-Fenner and Klatt1999; see the target book for an extended discussion on this topic). These two questions can be addressed in the following RSA study.
Infants would see arrays of dots while undergoing fMRI (EEG, MEG or functional near-infrared spectroscopy [fNIRS] would be a viable substitute for some of these analyses). Across trials, arrays will differ in the number of dots from 1, 2, 3, 4, 8, and 16. These numbers include the subitizing range (1–3) and beyond (4–16). The dots will be presented in one of the two sizes: (1) the dots are all the same size, regardless of the quantity of dots in the array, and (2) the dots are scaled so that they span the same total size (where size is based on additive area; Yousif & Keil, Reference Yousif and Keil2021). The pattern of activity in different brain regions that are evoked by each array, averaged across repetitions, will be compared to all other array types to complete the RSA.
With this relatively simple design, four distinct analyses can assess infant knowledge of number:
(1) Nonnumeric magnitude: are arrays with the same area (i.e., nonnumeric magnitude) more similar to each other than arrays with different areas, even when the number of dots differ? Brain regions sensitive to this nonnumeric magnitude will likely include both sensory systems and regions that support number processing in adults (e.g., the parietal cortex), as shown previously in adults (Sokolowski, Fias, Ononye, & Ansari, Reference Sokolowski, Fias, Ononye and Ansari2017).
(2) Numeric magnitude: are arrays with the same number of dots more similar to each other than arrays with different numbers, even when the size of the dots differs? Numeric magnitude has been shown to recruit the parietal cortex in infants (Hyde, Boas, Blair, & Carey, Reference Hyde, Boas, Blair and Carey2010). In adults, neural regions that code for numeric and nonnumeric magnitudes overlap, but only partially (Sokolowski et al., Reference Sokolowski, Fias, Ononye and Ansari2017). Using fMRI with awake infants, it's possible to test the degree to which these computations are supported by different systems: a viable hypothesis is that they start out similar and diverge during development.
(3) Ordinal number representations: akin to Lyons, Ansari, and Beilock (Reference Lyons, Ansari and Beilock2015), are numbers represented ordinally (e.g., 2 is more similar to 3 than it is to 4)? This is particularly interesting for quantities in the subitizing range, where the processing of numeric magnitude may be served by object tracking (Uller et al., Reference Uller, Carey, Huntley-Fenner and Klatt1999); thus, representations may not be ordinal in this range.
(4) Ratio dependence: does representational similarity correspond more to absolute numerical differences (e.g., 2 is equally similar to 1 and 3) or a ratio difference (e.g., 2 is more similar to 3 than it is to 1)? Ratio dependence is a key indicator of a magnitude-estimation system (Meck & Church, Reference Meck and Church1983) and has been found for values in the approximate number range in infants (Xu, Reference Xu2003). Whether ratio dependence exists for quantities in the subitizing range remains unclear (Hyde & Spelke, Reference Hyde and Spelke2011; although see Starr et al., Reference Starr, Libertus and Brannon2013).
These analyses highlight the flexibility of RSA and neuroimaging: It can test questions about the neural substrate of infant cognition (e.g., is the neural implementation of numerosity continuous across development) and also the nature of cognitive representations (e.g., what is the relationship between the representations of quantities). In this case study, neuroimaging gives both confirmatory and unique answers to questions regarding infant numerical perception. Even more exciting is that this is just a taste of what RSA and neuroimaging can do to help us understand the infant mind.
You have
Access
“What Babies Know” (Spelke, Reference Spelke2022) is a state-of-the-art, inclusive, and thoughtful survey of what babies know about a few key domains, making the book a must-read for junior scientists in the field. Perhaps most striking is the breadth of research methods and model systems used in the experiments covered: From various behavioral approaches, to neuroimaging, to computational models, to animal neurophysiology. Even so, this commentary will highlight a method mostly absent from the text (and field) that could offer unique insight into infants' knowledge: Representational similarity analysis (RSA; Kriegeskorte, Mur, & Bandettini, Reference Kriegeskorte, Mur and Bandettini2008). I will discuss this method and explain its strengths, then focus on numerosity as a case study. Specifically, I will consider ways to address questions that the target book highlights as unresolved.
RSA is a method for evaluating the degree of representational similarity between exemplars in a set. RSA, along with the more straightforward “pattern similarity” method (Carlson, Schrater, & He, Reference Carlson, Schrater and He2003), has been used frequently in vision science to compare the representations of exemplars, like faces, scenes, and objects. RSA is most often performed with functional magnetic resonance imaging (fMRI) (i.e., comparing the patterns of activity across voxels that exemplars evoke) but can be applied to any data where metric differences can be measured between exemplars, including electroencephalography (EEG; Xie et al., Reference Xie, Hoehl, Moeskops, Kayhan, Kliesch, Turtleton and Cichy2022) and behavior (Spriet, Abassi, Hochmann, & Papeo, Reference Spriet, Abassi, Hochmann and Papeo2022). One example of a question that RSA can ask is whether the representation of exemplars cluster according to their perceptual similarity (i.e., exemplars that are perceptually different have low similarity), or conceptual similarity (i.e., exemplars that are conceptually different have low similarity). Analyses of this type show that different parts of the brain can divide exemplars either perceptually or conceptually (Bankson, Hebart, Groen, & Baker, Reference Bankson, Hebart, Groen and Baker2018).
A key benefit of RSA, and what makes it distinct from many popular methods in developmental science, is that it can test both the degree of difference between representations and the degree of similarity. By way of contrast, a habituation paradigm can test when infants think one exemplar (i.e., the habituation stimulus) is different from another (i.e., the test stimulus). However, suppose infants do not dishabituate to the test stimulus: That does not necessitate infants think the habituation and test exemplars are similar, but instead may result from poor study design (e.g., exposure is too short) or low statistical power. By contrast, high similarity in RSA typically (although not always, e.g., Spriet et al., Reference Spriet, Abassi, Hochmann and Papeo2022) entails positive evidence of similarity. For instance, in RSA with neuroimaging, high similarity means that neural patterns have a high correlation, which should not be expected by chance. An additional benefit of RSA arises when it uses neuroimaging data: It can disentangle simultaneous representations (Bankson et al., Reference Bankson, Hebart, Groen and Baker2018), as illustrated in the example above where conceptual and perceptual information were localized in different brain regions.
Despite this potential value, RSA remains an uncommon tool in infant research. A few exceptions exist: Spriet et al. (Reference Spriet, Abassi, Hochmann and Papeo2022) recently used RSA to understand the clustering of visual stimulus representations based on gaze behavior. Moreover, Xie et al. (Reference Xie, Hoehl, Moeskops, Kayhan, Kliesch, Turtleton and Cichy2022) used EEG to measure neural responses to categories and evaluated their clustering. Studies using fMRI are surprisingly absent from this list, but this is likely to change soon given the recent success of awake infant fMRI (Ellis et al., Reference Ellis, Skalaban, Yates, Bejjanki, Córdova and Turk-Browne2020, Reference Ellis, Yates, Skalaban, Bejjanki, Arcaro and Turk-Browne2021; Kosakowski et al., Reference Kosakowski, Cohen, Takahashi, Keil, Kanwisher and Saxe2022; Yates et al., Reference Yates, Skalaban, Ellis, Bracher, Baldassano and Turk-Browne2022). The aforementioned articles use RSA to address questions about visual perception, but RSA is flexible enough to tackle broad questions about what infants know. In what follows, I consider how RSA can resolve lingering questions in infant's knowledge of numerosity.
Over the last 30 years, research has led to incredible progress in understanding what infants know about number. Nonetheless, there are at least two questions that RSA can advance. The first question is whether numerosity is represented conceptually, rather than reflecting mere magnitude of perceptual content. Both historically (Clearfield & Mix, Reference Clearfield and Mix1999) and recently (Leibovich, Katzin, Harel, & Henik, Reference Leibovich, Katzin, Harel and Henik2017), researchers have argued that attempts to de-confound perceptual magnitude from conceptual number have been insufficient. For instance, many experimental controls for perceptual confounds have prioritized the wrong stimulus properties (Yousif & Keil, Reference Yousif and Keil2021). A second question RSA can tackle is the extent to which a signature of the numerosity system – namely, that differences in number depend on a ratio (AKA ratio dependence) – is consistent across the number line. Ratio dependence means that arrays containing 4 and 8 dots are perceived as just as different as arrays containing 8 and 16 dots. There is compelling evidence that infants represent numbers according to a ratio for values greater than 3 (i.e., above the subitizing range), but it remains unclear whether quantities lower than four are processed with ratio dependence (Hyde & Spelke, Reference Hyde and Spelke2011; McCrink & Wynn, Reference McCrink and Wynn2004; Starr, Libertus, & Brannon, Reference Starr, Libertus and Brannon2013; Uller, Carey, Huntley-Fenner, & Klatt, Reference Uller, Carey, Huntley-Fenner and Klatt1999; see the target book for an extended discussion on this topic). These two questions can be addressed in the following RSA study.
Infants would see arrays of dots while undergoing fMRI (EEG, MEG or functional near-infrared spectroscopy [fNIRS] would be a viable substitute for some of these analyses). Across trials, arrays will differ in the number of dots from 1, 2, 3, 4, 8, and 16. These numbers include the subitizing range (1–3) and beyond (4–16). The dots will be presented in one of the two sizes: (1) the dots are all the same size, regardless of the quantity of dots in the array, and (2) the dots are scaled so that they span the same total size (where size is based on additive area; Yousif & Keil, Reference Yousif and Keil2021). The pattern of activity in different brain regions that are evoked by each array, averaged across repetitions, will be compared to all other array types to complete the RSA.
With this relatively simple design, four distinct analyses can assess infant knowledge of number:
(1) Nonnumeric magnitude: are arrays with the same area (i.e., nonnumeric magnitude) more similar to each other than arrays with different areas, even when the number of dots differ? Brain regions sensitive to this nonnumeric magnitude will likely include both sensory systems and regions that support number processing in adults (e.g., the parietal cortex), as shown previously in adults (Sokolowski, Fias, Ononye, & Ansari, Reference Sokolowski, Fias, Ononye and Ansari2017).
(2) Numeric magnitude: are arrays with the same number of dots more similar to each other than arrays with different numbers, even when the size of the dots differs? Numeric magnitude has been shown to recruit the parietal cortex in infants (Hyde, Boas, Blair, & Carey, Reference Hyde, Boas, Blair and Carey2010). In adults, neural regions that code for numeric and nonnumeric magnitudes overlap, but only partially (Sokolowski et al., Reference Sokolowski, Fias, Ononye and Ansari2017). Using fMRI with awake infants, it's possible to test the degree to which these computations are supported by different systems: a viable hypothesis is that they start out similar and diverge during development.
(3) Ordinal number representations: akin to Lyons, Ansari, and Beilock (Reference Lyons, Ansari and Beilock2015), are numbers represented ordinally (e.g., 2 is more similar to 3 than it is to 4)? This is particularly interesting for quantities in the subitizing range, where the processing of numeric magnitude may be served by object tracking (Uller et al., Reference Uller, Carey, Huntley-Fenner and Klatt1999); thus, representations may not be ordinal in this range.
(4) Ratio dependence: does representational similarity correspond more to absolute numerical differences (e.g., 2 is equally similar to 1 and 3) or a ratio difference (e.g., 2 is more similar to 3 than it is to 1)? Ratio dependence is a key indicator of a magnitude-estimation system (Meck & Church, Reference Meck and Church1983) and has been found for values in the approximate number range in infants (Xu, Reference Xu2003). Whether ratio dependence exists for quantities in the subitizing range remains unclear (Hyde & Spelke, Reference Hyde and Spelke2011; although see Starr et al., Reference Starr, Libertus and Brannon2013).
These analyses highlight the flexibility of RSA and neuroimaging: It can test questions about the neural substrate of infant cognition (e.g., is the neural implementation of numerosity continuous across development) and also the nature of cognitive representations (e.g., what is the relationship between the representations of quantities). In this case study, neuroimaging gives both confirmatory and unique answers to questions regarding infant numerical perception. Even more exciting is that this is just a taste of what RSA and neuroimaging can do to help us understand the infant mind.
Competing interest
None.