Human and animal behavior is driven not only by extrinsically available rewards like food and money but also by various intrinsic motivations, such as the desire to experience novelty or surprise (Gottlieb & Oudeyer, Reference Gottlieb and Oudeyer2018; Modirshanechi et al., Reference Modirshanechi, Kondrakiewicz, Gerstner and Haesler2023b). Curiosity and creativity are two modes of cognitive processing where such intrinsic motivations have a significant influence. Ivancovsky et al.'s novelty-seeking model (NSM) creates a valuable conceptual link between these intuitively related modes, and divides the shared cognitive processes underlying curiosity and creativity into four phases (Ivancovsky et al.). However, the model's high-level conceptual nature makes it challenging to give quantitative explanations and derive experimentally testable hypotheses. To address this problem, we relate each of the four phases of the NSM to computational models of intrinsically motivated behavior and learning. We discuss (i) in which ways computational models support or contradict the NSM's core claims, and illustrate (ii) how computational models make the conceptual explanations and predictions of the NSM empirically testable.
First, the NSM posits that curiosity and creativity share brain networks and mechanisms to detect “novelty,” either in the external space of sensory stimuli (curiosity) or in the internal space of associations (creativity). Second, these shared mechanisms initiate downstream processing of the “novel” stimulus or association (Ivancovsky et al.). However, although Ivancovsky et al. use “novelty” as a general notion, distinct intrinsic motivations contributing to curiosity (e.g., novelty, surprise, information gain) are mathematically well-defined (Barto et al., Reference Barto, Mirolli and Baldassarre2013; Modirshanechi et al., Reference Modirshanechi, Brea and Gerstner2022), have different neural signatures (Akiti et al., Reference Akiti, Tsutsui-Kimura, Xie, Mathis, Markowitz, Anyoha and Watabe-Uchida2022; Morrens et al., Reference Morrens, Aydin, van Rensburg, Esquivelzeta Rabell and Haesler2020; Xu et al., Reference Xu, Modirshanechi, Lehmann, Gerstner and Herzog2021; Zhang et al., Reference Zhang, Bromberg-Martin, Sogukpinar, Kocher and Monosov2022), and are triggered by different statistical regularities of the task or environment (Maheu et al., Reference Maheu, Dehaene and Meyniel2019) (see Modirshanechi et al., Reference Modirshanechi, Becker, Brea and Gerstner2023a, for a review). For example, novelty signals are triggered by unfamiliar stimuli and situations, both when the unfamiliarity is expected and when it is unexpected (Homann et al., Reference Homann, Koay, Chen, Tank and Berry II2022). Surprise signals, on the contrary, arise in the face of unexpected stimuli, both familiar and unfamiliar ones (Zhang et al., Reference Zhang, Bromberg-Martin, Sogukpinar, Kocher and Monosov2022). In line with that, different neuromodulatory signals are thought to communicate expected versus unexpected novelty or uncertainty (Schomaker & Meeter, Reference Schomaker and Meeter2015; Yu & Dayan, Reference Yu and Dayan2005); and computational models suggest different network mechanisms for the detection of novelty and surprise (Barry & Gerstner, Reference Barry and Gerstner2024; Schulz et al., Reference Schulz, Miehl, Berry II and Gjorgjieva2021). Despite the partial overlap in the processing of novelty and surprise (Zhang et al., Reference Zhang, Bromberg-Martin, Sogukpinar, Kocher and Monosov2022), we can thus not simply speak of “novelty” detection as a homogeneous process as assumed in the NSM. When empirically testing shared neural mechanisms of curiosity- and creativity-related signal detection and downstream processing, we should therefore consider how the neural correlates of curiosity and creativity may vary across environments and experimental tasks.
Third, the NSM proposes that both curiosity and creativity require a balance of exploratory and exploitatory states of mind (SoM), and that this balance is mediated by cognitive control processes. This NSM prediction agrees with reinforcement learning-based (RL) models that arbitrate between intrinsic motivations (curiosity/exploratory SoM) and extrinsic motivations (reward/exploitatory SoM) (Modirshanechi et al., Reference Modirshanechi, Brea and Gerstner2022; Puigdomènech Badia et al., Reference Puigdomènech Badia, Piot, Kapturowski, Sprechmann, Vitvitskyi, Guo and Blundell2020). Importantly, these RL models quantify the respective contributions of exploration and exploitation to behavior, and allow us to test which mechanisms regulate the trade-off between the exploratory and exploitatory states. For example, a recent model that arbitrates exploration and exploitation based on the agent's reward optimism (Modirshanechi et al., Reference Modirshanechi, Brea and Gerstner2022) provides a concrete computational implementation of Ivancovsky et al.'s conceptual links between curiosity and the SoM dimension of openness to experience. We propose that this modeling approach is a useful tool to experimentally validate links between curiosity/creativity and different SoM dimensions as suggested by the NSM.
Lastly, a central component of the NSM is the bidirectional link between memory and curiosity/creativity (Ivancovsky et al.). However, there are different forms of memory and distinct synaptic learning rules that are influenced by intrinsic motivational signals (three-factor learning rules; Gerstner et al., Reference Gerstner, Lehmann, Liakoni, Corneil and Brea2018; Lisman et al., Reference Lisman, Grace and Duzel2011). Although we agree with the bidirectional link between curiosity/creativity and memory systems, we propose that the respective memory system with which curiosity and creativity engage could differ (e.g., episodic vs. recognition memory). More importantly, distinct forms of curiosity and creativity may link to different learning rules and roles of memory. For example, novelty is particularly important for initial memory formation (Duszkiewicz et al., Reference Duszkiewicz, McNamara, Takeuchi and Genzel2019; Priestley et al., Reference Priestley, Bowler, Rolotti, Fusi and Losonczy2022), whereas surprise, triggered by the violation of known rules and expectations (Barto et al., Reference Barto, Mirolli and Baldassarre2013; Xu et al., Reference Xu, Modirshanechi, Lehmann, Gerstner and Herzog2021; Zhang et al., Reference Zhang, Bromberg-Martin, Sogukpinar, Kocher and Monosov2022), might be more important for targeted memory updates (Gershman et al., Reference Gershman, Monfils, Norman and Niv2017). Another relevant distinction that the NSM is currently abstracting is between (i) memory systems that support the detection of intrinsic motivational signals and (ii) memory systems that are downstream targets of curiosity/creativity-related signals. These memory systems may – but do not have to – be identical. For example, novelty detection relies on state representations in sensory areas and recognition memory (Bogacz & Brown, Reference Bogacz and Brown2003; Homann et al., Reference Homann, Koay, Chen, Tank and Berry II2022), but downstream novelty signals are also involved in updating semantic or episodic memories (Duszkiewicz et al., Reference Duszkiewicz, McNamara, Takeuchi and Genzel2019; Priestley et al., Reference Priestley, Bowler, Rolotti, Fusi and Losonczy2022; Wittmann et al., Reference Wittmann, Bunzeck, Dolan and Düzel2007). To empirically determine how memory is shared by curiosity and creativity, it is necessary to experimentally test how different memory systems are involved at each stage and in each type of curiosity/creativity-related processing.
To conclude, we illustrated how the high-level cognitive NSM framework relates to concrete computational models of intrinsically motivated behavior and learning. Although computational models and the NSM align on the general structure of curiosity- and creativity-related processing, computational models suggest important distinctions within each phase of the NSM. In particular, different forms of curiosity and creativity arising from the contribution of distinct intrinsic motivational signals, like novelty and surprise, could differ in the specifics of how they are detected, signaled to downstream targets, and interacting with memory systems. Linking the NSM to computational models is thus a necessary step to empirically test the NSM's conceptual predictions and gain insights into the neural correlates and network mechanisms underlying curiosity and creativity.
You have
Access
Human and animal behavior is driven not only by extrinsically available rewards like food and money but also by various intrinsic motivations, such as the desire to experience novelty or surprise (Gottlieb & Oudeyer, Reference Gottlieb and Oudeyer2018; Modirshanechi et al., Reference Modirshanechi, Kondrakiewicz, Gerstner and Haesler2023b). Curiosity and creativity are two modes of cognitive processing where such intrinsic motivations have a significant influence. Ivancovsky et al.'s novelty-seeking model (NSM) creates a valuable conceptual link between these intuitively related modes, and divides the shared cognitive processes underlying curiosity and creativity into four phases (Ivancovsky et al.). However, the model's high-level conceptual nature makes it challenging to give quantitative explanations and derive experimentally testable hypotheses. To address this problem, we relate each of the four phases of the NSM to computational models of intrinsically motivated behavior and learning. We discuss (i) in which ways computational models support or contradict the NSM's core claims, and illustrate (ii) how computational models make the conceptual explanations and predictions of the NSM empirically testable.
First, the NSM posits that curiosity and creativity share brain networks and mechanisms to detect “novelty,” either in the external space of sensory stimuli (curiosity) or in the internal space of associations (creativity). Second, these shared mechanisms initiate downstream processing of the “novel” stimulus or association (Ivancovsky et al.). However, although Ivancovsky et al. use “novelty” as a general notion, distinct intrinsic motivations contributing to curiosity (e.g., novelty, surprise, information gain) are mathematically well-defined (Barto et al., Reference Barto, Mirolli and Baldassarre2013; Modirshanechi et al., Reference Modirshanechi, Brea and Gerstner2022), have different neural signatures (Akiti et al., Reference Akiti, Tsutsui-Kimura, Xie, Mathis, Markowitz, Anyoha and Watabe-Uchida2022; Morrens et al., Reference Morrens, Aydin, van Rensburg, Esquivelzeta Rabell and Haesler2020; Xu et al., Reference Xu, Modirshanechi, Lehmann, Gerstner and Herzog2021; Zhang et al., Reference Zhang, Bromberg-Martin, Sogukpinar, Kocher and Monosov2022), and are triggered by different statistical regularities of the task or environment (Maheu et al., Reference Maheu, Dehaene and Meyniel2019) (see Modirshanechi et al., Reference Modirshanechi, Becker, Brea and Gerstner2023a, for a review). For example, novelty signals are triggered by unfamiliar stimuli and situations, both when the unfamiliarity is expected and when it is unexpected (Homann et al., Reference Homann, Koay, Chen, Tank and Berry II2022). Surprise signals, on the contrary, arise in the face of unexpected stimuli, both familiar and unfamiliar ones (Zhang et al., Reference Zhang, Bromberg-Martin, Sogukpinar, Kocher and Monosov2022). In line with that, different neuromodulatory signals are thought to communicate expected versus unexpected novelty or uncertainty (Schomaker & Meeter, Reference Schomaker and Meeter2015; Yu & Dayan, Reference Yu and Dayan2005); and computational models suggest different network mechanisms for the detection of novelty and surprise (Barry & Gerstner, Reference Barry and Gerstner2024; Schulz et al., Reference Schulz, Miehl, Berry II and Gjorgjieva2021). Despite the partial overlap in the processing of novelty and surprise (Zhang et al., Reference Zhang, Bromberg-Martin, Sogukpinar, Kocher and Monosov2022), we can thus not simply speak of “novelty” detection as a homogeneous process as assumed in the NSM. When empirically testing shared neural mechanisms of curiosity- and creativity-related signal detection and downstream processing, we should therefore consider how the neural correlates of curiosity and creativity may vary across environments and experimental tasks.
Third, the NSM proposes that both curiosity and creativity require a balance of exploratory and exploitatory states of mind (SoM), and that this balance is mediated by cognitive control processes. This NSM prediction agrees with reinforcement learning-based (RL) models that arbitrate between intrinsic motivations (curiosity/exploratory SoM) and extrinsic motivations (reward/exploitatory SoM) (Modirshanechi et al., Reference Modirshanechi, Brea and Gerstner2022; Puigdomènech Badia et al., Reference Puigdomènech Badia, Piot, Kapturowski, Sprechmann, Vitvitskyi, Guo and Blundell2020). Importantly, these RL models quantify the respective contributions of exploration and exploitation to behavior, and allow us to test which mechanisms regulate the trade-off between the exploratory and exploitatory states. For example, a recent model that arbitrates exploration and exploitation based on the agent's reward optimism (Modirshanechi et al., Reference Modirshanechi, Brea and Gerstner2022) provides a concrete computational implementation of Ivancovsky et al.'s conceptual links between curiosity and the SoM dimension of openness to experience. We propose that this modeling approach is a useful tool to experimentally validate links between curiosity/creativity and different SoM dimensions as suggested by the NSM.
Lastly, a central component of the NSM is the bidirectional link between memory and curiosity/creativity (Ivancovsky et al.). However, there are different forms of memory and distinct synaptic learning rules that are influenced by intrinsic motivational signals (three-factor learning rules; Gerstner et al., Reference Gerstner, Lehmann, Liakoni, Corneil and Brea2018; Lisman et al., Reference Lisman, Grace and Duzel2011). Although we agree with the bidirectional link between curiosity/creativity and memory systems, we propose that the respective memory system with which curiosity and creativity engage could differ (e.g., episodic vs. recognition memory). More importantly, distinct forms of curiosity and creativity may link to different learning rules and roles of memory. For example, novelty is particularly important for initial memory formation (Duszkiewicz et al., Reference Duszkiewicz, McNamara, Takeuchi and Genzel2019; Priestley et al., Reference Priestley, Bowler, Rolotti, Fusi and Losonczy2022), whereas surprise, triggered by the violation of known rules and expectations (Barto et al., Reference Barto, Mirolli and Baldassarre2013; Xu et al., Reference Xu, Modirshanechi, Lehmann, Gerstner and Herzog2021; Zhang et al., Reference Zhang, Bromberg-Martin, Sogukpinar, Kocher and Monosov2022), might be more important for targeted memory updates (Gershman et al., Reference Gershman, Monfils, Norman and Niv2017). Another relevant distinction that the NSM is currently abstracting is between (i) memory systems that support the detection of intrinsic motivational signals and (ii) memory systems that are downstream targets of curiosity/creativity-related signals. These memory systems may – but do not have to – be identical. For example, novelty detection relies on state representations in sensory areas and recognition memory (Bogacz & Brown, Reference Bogacz and Brown2003; Homann et al., Reference Homann, Koay, Chen, Tank and Berry II2022), but downstream novelty signals are also involved in updating semantic or episodic memories (Duszkiewicz et al., Reference Duszkiewicz, McNamara, Takeuchi and Genzel2019; Priestley et al., Reference Priestley, Bowler, Rolotti, Fusi and Losonczy2022; Wittmann et al., Reference Wittmann, Bunzeck, Dolan and Düzel2007). To empirically determine how memory is shared by curiosity and creativity, it is necessary to experimentally test how different memory systems are involved at each stage and in each type of curiosity/creativity-related processing.
To conclude, we illustrated how the high-level cognitive NSM framework relates to concrete computational models of intrinsically motivated behavior and learning. Although computational models and the NSM align on the general structure of curiosity- and creativity-related processing, computational models suggest important distinctions within each phase of the NSM. In particular, different forms of curiosity and creativity arising from the contribution of distinct intrinsic motivational signals, like novelty and surprise, could differ in the specifics of how they are detected, signaled to downstream targets, and interacting with memory systems. Linking the NSM to computational models is thus a necessary step to empirically test the NSM's conceptual predictions and gain insights into the neural correlates and network mechanisms underlying curiosity and creativity.
Financial support
This work was supported by the Swiss National Science Foundation No. 200020_207426.
Competing interest
None.