Hostname: page-component-cd9895bd7-dk4vv Total loading time: 0 Render date: 2024-12-26T04:54:18.513Z Has data issue: false hasContentIssue false

Exploratory exploitation and exploitative exploration: The phenomenology of play and the computational dynamics of search

Published online by Cambridge University Press:  21 May 2024

Mihnea Moldoveanu*
Affiliation:
Desautels Centre for Integrative Thinking, Rotman School of Management, University of Toronto, Toronto, ON, Canada mihnea.moldoveanu@rotman.utoronto.ca
*
*Corresponding author.

Abstract

I argue for a more complicated but nonetheless computationally feasible and algorithmically intelligible interplay between exploration and exploitation and for admitting into our conceptual toolkit regimes of exploitative exploration and exploratory exploitation that can enhance the novelty and usefulness of the results of either problemistic or serendipitous search.

Type
Open Peer Commentary
Copyright
Copyright © The Author(s), 2024. Published by Cambridge University Press

Categorical classifications of patterns of mental behavior can be useful exploratory tools, because, while believing in them, one can take seriously a set of possible worlds or states of the world that one could not otherwise – which makes systematic search within those worlds possible (Aronowitz, Reference Aronowitz2021). A map of the relationship between curiosity and creativity such as that of Ivancovsky et al. is a case in point: By a somewhat arbitrary fixing of the referents of both “curiosity” and “creativity,” one can explore a space of possible relationships that would make less sense otherwise. For instance, we can inquire about the fine structure of curiosity and how it can be harnessed for useful work in spite of its “open-ended” nature.

Ivancovsky et al. recognize that cognitive control plays an important role in the dynamics of curiosity (Steele, Hardy, Day, Watts, & Mumford, Reference Steele, Hardy, Day, Watts and Mumford2021), minimally as a gatekeeper on what constitutes information that is either useful interesting or useful (Campbell, Reference Campbell1960). But, reducing the exercise of control to both the cognitive realm and to one variable – that one can have more or less of – over-simplifies the “facts of the neuro-phenomenological matter” as far as human search is concerned, just as the representation of the exploration–exploitation relationship as a trade-off that happens on a continuum of different “proportions” does.

For example, one may search for a small object (cell phone) lost in a large region of space–time (1 day × 10 square miles) in many ways:

  • some involve the exhaustive coverage of every place within the perimeter, perhaps with a “random” starting point but thereafter determined sequential steps;

  • some involve recalling and retracing of one's own remembered steps (which may be a function of one's current mood or location), or calling, “in random order” of public locales in the neighborhoods one has visited, or

  • one might call a few “random” friends or acquaintances and attempt to playfully engage them in the search.

One can also switch between different approaches during the search, depending on how well “things are going.” Even in a search like this one – in which the desired end state is well defined – there are ample opportunities for the exercise of a taste for the unpredictable and the unknown, and the act of randomization can itself supply that in ways that can be productively marshaled to getting to the end goal more reliably and/or quickly.

When the end goal is less well defined or the search space is much larger – or both – the structure and dynamics of the interplay between exploration and exploitation become correspondingly more textured: One may search for the set of features of a new software platform that will deliver the functionality and form one surmises a set of clients would deem preferable to what one currently surmises the competition will come up with, by searching among all of the 2N − 1 – many subsets of some large number N of possible features and doing thought experiments and real experiments to figure out how they would, should, or might perform in the possible worlds in which a potentially large number M of clients have beliefs and desires that lead them to choose the product embodying these features over another product made by one or more of K competitors, each currently facing a different predicament. The exhaustive sweep of the entire search space does not seem feasible, but one can weave exploration and exploitation together in many ways to create viable – and even provably “optimal,” in best-, worst-, and average-complexity senses – search paths:

  • Randomly jump to different subsets of features and evaluate them and subsets that are closely related to them, switching from the existing location in search space to another if the results do not seem promising;

  • Start with an “intuitive” set of features, evaluate the current configuration, and then replace a few of them at random with close substitutes; repeat until a marked improvement is seen;

  • Start with subassemblies of features and randomly combine them into a full feature set; evaluate and try again, keeping track of increases or decreases in an overall desirability metric, and so on.

To an algorithm designer, these are reasonable adaptations of the search procedure to a large search space – which benefit from the purposeful and disciplined introduction of a stochastic element in the sequence of search operations. To the introspective eye of the individual, these cases feature episodes that may “feel” like the playful prospecting that curious “states of mind” are associated with. Randomization need not entail a call to a pseudorandom number generation routine or the tossing of coins: Environmental or internal “switch triggers,” duly interpreted as prompts, can serve as a randomizing device (Moldoveanu & Martin, Reference Moldoveanu and Martin2010) – which entails one can engage in “randomized search acts” just by following interesting cues “out of curiosity” or indulging an instinct for “curious exploration.” At the same time, she can explore the ways in which her own exploitative behaviors “work out” as her nerve endings make contact with energy-bearing signals: How do her own motor neurons implement targeted, timely commands that cause environmental effects in a predictable fashion as she is “following a recipe” or algorithm that prescribes them – which may lead her to discover more efficient ways of gripping, handling, turning, or moving an object “the next time around”?

The interplay between the “phenomenology of playful search” and the “computational structure of large scale search” (Moldoveanu, Reference Moldoveanu2024) introduces a new dialectic in discourse about “the interplay of curiosity and creativity” to complement that between cognitive psychology and neuroscience. It affords us new insight into the ways in which regimes of exploratory exploitation and exploitative exploration can help people create useful solutions to large problems through the disciplined use of curious “look-abouts.”

Financial support

This work was funded by the Desautels Centre for Integrative Thinking, Rotman School of Management, University of Toronto.

Competing interest

None.

References

Aronowitz, S. (2021). Exploring by believing. Philosophical Review, 130(3), 339383.CrossRefGoogle Scholar
Campbell, D. T. (1960). Blind variation and selective retentions in creative thought as in other knowledge processes. Psychological Review, 67(6), 380400.CrossRefGoogle ScholarPubMed
Moldoveanu, M. C. (2024). Intelligent artificiality: Machine learning, brain architecture and the foundations of decision theory. DeGruyter.Google Scholar
Moldoveanu, M. C., & Martin, R. L. (2010). Diaminds: Decoding the mental habits of successful thinkers. University of Toronto Press.Google Scholar
Steele, L. M., Hardy, J. H. III, Day, E. A., Watts, L. L., & Mumford, M. D. (2021). Navigating creative paradoxes: Exploration and exploitation effort drive novelty and usefulness. Psychology of Aesthetics, Creativity, and the Arts, 15(1), 149164.CrossRefGoogle Scholar