As Morin states in the conclusion of the target article, whether a generalist and self-sufficient graphic code is possible is ultimately an empirical question. However, an alternative to Morin's explanation of why no successful graphic code has been created – so far – comes from the perspective of cognitive science.
The history of the invention of airplanes provides an instructive analogy. Before 1903 independent heavier-than-air flight was essentially the domain of birds (and pterodactyls). Human flight was restricted to hot air balloons and kites. Repeated attempts by well-resourced inventors met with failure. So, the feasibility of independent human flight then seemed intuitively implausible. Bradshaw (Reference Bradshaw and Giere1992) attributes the success of the Wright brothers to their systematic identification of key functions essential to the operation of airplanes. The brothers developed mechanisms to perform those functions. For example, they recognized the importance of attitudinal control and decomposed it into subfunctions to separately control pitch, roll, and yaw (i.e., ailerons, wing warping, and rudder). Only when they had mechanisms for all the necessary functions did the Wrights integrate them into a complete airplane.
It appears that the design of graphic codes is currently at a pre-Wright brother's stage where the creation of graphic codes has so far focused on parametric permutations of features of whole ideographic systems. The failure of extant graphic codes may be attributed to missing cognitive mechanisms to localize the control of essential semantic functions. By identifying such functions we can make the prospect of designing a full graphic code more realistic and take a step towards answering the empirical question.
Recent studies in cognitive science, particularly on visuospatial representational systems, suggest key ideographic functions. These are functions that enable producers to encode meanings in, and readers to access meanings from, symbols in representational systems. Many factors are now recognized, such as: locational indexing of information (Larkin & Simon, Reference Larkin and Simon1987); isomorphic mapping between concepts and tokens (Gurr, Reference Gurr, Marriott and Meyer1998); levels of specificity (Stenning & Oberlander, Reference Stenning and Oberlander1995); free rides (Shimojima, Reference Shimojima2015); the matching of quantity scales (Zhang, Reference Zhang1996); and multilevel coherent interpretive schemes (Cheng, Reference Cheng2002, Reference Cheng2011).
From such findings we may distil essential representational functions that a full ideography needs. Here is one attempt at such a collection:
(1) Ground symbols are primitive symbols that establish foundational concepts from which higher order ideas can be derived (including conceptual dimensions and situational models, see below). The notion of base-level categories (Rosch, Mervis, Gray, Johnson, & Boyes-Braem, Reference Rosch, Mervis, Gray, Johnson and Boyes-Braem1976) may aid the selection of ground symbols.
(2) Conceptual dimensions are generic concepts that are underpinning characteristics of ideas (cf., Gardenfors, Reference Gardenfors2014). Some examples include: states versus actions; spatial scale; temporal scale; states of matter; frames of reference; psychological valence; truth value; and degrees of certainty. A graphic code should possess classes of symbols whose function is to identify values on such dimensions for the purpose of qualifying the meaning of given symbols. The selection of conceptual dimensions may be guided by ideas about core knowledge (Spelke & Kinzler, Reference Spelke and Kinzler2007), geometric conceptual spaces (Gardenfors, Reference Gardenfors2014), quantity scales (Stevens, Reference Stevens1946), and others.
(3) Context modelling builds graphical expressions to represent meaningful situations as compositional configurations of symbols and conceptual dimensions. Context models provide rich settings to guide users to intended meanings, for instance by triangulation of conceptual dimensions. Ideas about image schemas (Lakoff & Johnson, Reference Lakoff and Johnson2008) may provide constraints on the form of context models.
(4) Indexicalization takes a context model and, in a decompositional manner, identifies some subconfiguration of symbols, or a part of a single symbol, to introduce a symbol for a new specific meaning.
(5) Iconization converts a complex graphic symbol (context model) into a simple icon-like symbol for ease of future use, by disposing of the details that were necessary to initially establish the meaning but that are not essential to the intended concept.
(6) Meta-semantics is the greatest challenge for creating a full graphic code. This concerns the specification of abstract and intangible concepts, which in the case of words are often defined with reference to other abstract words. A graphic code needs mechanisms to generate symbols for abstractions and reifications, and for generalizations and specializations. Commonality and contrasts of meanings across existing symbols could be one approach to drive the coining of symbols for abstract concepts. This will require the provision of meta-symbols whose purpose is to instruct the user to associate a new graphical object with the implied meaning.
Such functions must operate symbiotically in a graphic code. For instance, the code can be generative, in and of itself, when it possesses mechanisms to create symbols for new meanings from existing symbols without appealing to an external natural language oracle (cf., Bliss symbolics). The code can use the full richness of spatial, geometric, topological, and mereological graphical devices to encode meanings and eschew the linear concatenation of symbols as the basic organizational format of expressions. (In terms of the airplane analogy, linear concatenation in Bliss symbolics is flapping wings.) Further, when users are familiar with the representational tools for each function, one can reasonably imagine users communicating in real time by jointly editing each other's expressions. For instance, a user who has a conventionalized symbol could rewind the iconization or indexicalization processes by drawing the context model and applying meta-semantics to unfold that symbol's meaning. Thus, contrary to Morin's claim, it is possible that a graphic code could share with spoken languages the “cheap and transient signals, allowing for easy online repairing of miscommunication … where the advantages of common ground are maximized” (target article, sect. long abstract, para. 1). In turn, this undermines one pillar of Morin's argument for the standardization problem, but the learnability problem remains as an empirical question.
The idea of deploying semantic functions in the design of a graphic code is not purely theoretical. My own efforts at designing graphical notations for conceptually challenging information intensive topics – albeit specialist – show that when such functions are satisfied, users find the new notations cognitively superior to conventional representations (e.g., Cheng, Reference Cheng2002, Reference Cheng2011, Reference Cheng, Cox, Plimmer and Rodgers2012, Reference Cheng2020).
You have
Access
As Morin states in the conclusion of the target article, whether a generalist and self-sufficient graphic code is possible is ultimately an empirical question. However, an alternative to Morin's explanation of why no successful graphic code has been created – so far – comes from the perspective of cognitive science.
The history of the invention of airplanes provides an instructive analogy. Before 1903 independent heavier-than-air flight was essentially the domain of birds (and pterodactyls). Human flight was restricted to hot air balloons and kites. Repeated attempts by well-resourced inventors met with failure. So, the feasibility of independent human flight then seemed intuitively implausible. Bradshaw (Reference Bradshaw and Giere1992) attributes the success of the Wright brothers to their systematic identification of key functions essential to the operation of airplanes. The brothers developed mechanisms to perform those functions. For example, they recognized the importance of attitudinal control and decomposed it into subfunctions to separately control pitch, roll, and yaw (i.e., ailerons, wing warping, and rudder). Only when they had mechanisms for all the necessary functions did the Wrights integrate them into a complete airplane.
It appears that the design of graphic codes is currently at a pre-Wright brother's stage where the creation of graphic codes has so far focused on parametric permutations of features of whole ideographic systems. The failure of extant graphic codes may be attributed to missing cognitive mechanisms to localize the control of essential semantic functions. By identifying such functions we can make the prospect of designing a full graphic code more realistic and take a step towards answering the empirical question.
Recent studies in cognitive science, particularly on visuospatial representational systems, suggest key ideographic functions. These are functions that enable producers to encode meanings in, and readers to access meanings from, symbols in representational systems. Many factors are now recognized, such as: locational indexing of information (Larkin & Simon, Reference Larkin and Simon1987); isomorphic mapping between concepts and tokens (Gurr, Reference Gurr, Marriott and Meyer1998); levels of specificity (Stenning & Oberlander, Reference Stenning and Oberlander1995); free rides (Shimojima, Reference Shimojima2015); the matching of quantity scales (Zhang, Reference Zhang1996); and multilevel coherent interpretive schemes (Cheng, Reference Cheng2002, Reference Cheng2011).
From such findings we may distil essential representational functions that a full ideography needs. Here is one attempt at such a collection:
(1) Ground symbols are primitive symbols that establish foundational concepts from which higher order ideas can be derived (including conceptual dimensions and situational models, see below). The notion of base-level categories (Rosch, Mervis, Gray, Johnson, & Boyes-Braem, Reference Rosch, Mervis, Gray, Johnson and Boyes-Braem1976) may aid the selection of ground symbols.
(2) Conceptual dimensions are generic concepts that are underpinning characteristics of ideas (cf., Gardenfors, Reference Gardenfors2014). Some examples include: states versus actions; spatial scale; temporal scale; states of matter; frames of reference; psychological valence; truth value; and degrees of certainty. A graphic code should possess classes of symbols whose function is to identify values on such dimensions for the purpose of qualifying the meaning of given symbols. The selection of conceptual dimensions may be guided by ideas about core knowledge (Spelke & Kinzler, Reference Spelke and Kinzler2007), geometric conceptual spaces (Gardenfors, Reference Gardenfors2014), quantity scales (Stevens, Reference Stevens1946), and others.
(3) Context modelling builds graphical expressions to represent meaningful situations as compositional configurations of symbols and conceptual dimensions. Context models provide rich settings to guide users to intended meanings, for instance by triangulation of conceptual dimensions. Ideas about image schemas (Lakoff & Johnson, Reference Lakoff and Johnson2008) may provide constraints on the form of context models.
(4) Indexicalization takes a context model and, in a decompositional manner, identifies some subconfiguration of symbols, or a part of a single symbol, to introduce a symbol for a new specific meaning.
(5) Iconization converts a complex graphic symbol (context model) into a simple icon-like symbol for ease of future use, by disposing of the details that were necessary to initially establish the meaning but that are not essential to the intended concept.
(6) Meta-semantics is the greatest challenge for creating a full graphic code. This concerns the specification of abstract and intangible concepts, which in the case of words are often defined with reference to other abstract words. A graphic code needs mechanisms to generate symbols for abstractions and reifications, and for generalizations and specializations. Commonality and contrasts of meanings across existing symbols could be one approach to drive the coining of symbols for abstract concepts. This will require the provision of meta-symbols whose purpose is to instruct the user to associate a new graphical object with the implied meaning.
Such functions must operate symbiotically in a graphic code. For instance, the code can be generative, in and of itself, when it possesses mechanisms to create symbols for new meanings from existing symbols without appealing to an external natural language oracle (cf., Bliss symbolics). The code can use the full richness of spatial, geometric, topological, and mereological graphical devices to encode meanings and eschew the linear concatenation of symbols as the basic organizational format of expressions. (In terms of the airplane analogy, linear concatenation in Bliss symbolics is flapping wings.) Further, when users are familiar with the representational tools for each function, one can reasonably imagine users communicating in real time by jointly editing each other's expressions. For instance, a user who has a conventionalized symbol could rewind the iconization or indexicalization processes by drawing the context model and applying meta-semantics to unfold that symbol's meaning. Thus, contrary to Morin's claim, it is possible that a graphic code could share with spoken languages the “cheap and transient signals, allowing for easy online repairing of miscommunication … where the advantages of common ground are maximized” (target article, sect. long abstract, para. 1). In turn, this undermines one pillar of Morin's argument for the standardization problem, but the learnability problem remains as an empirical question.
The idea of deploying semantic functions in the design of a graphic code is not purely theoretical. My own efforts at designing graphical notations for conceptually challenging information intensive topics – albeit specialist – show that when such functions are satisfied, users find the new notations cognitively superior to conventional representations (e.g., Cheng, Reference Cheng2002, Reference Cheng2011, Reference Cheng, Cox, Plimmer and Rodgers2012, Reference Cheng2020).
Financial support
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Competing interest
None.