This paper introduces the special issue of AI EDAM on the role of gesture in designing. It starts with the context of the papers submitted and a summary of the papers accepted. We then introduce gesture studies, one of the two main domains with which this Special Issue is concerned. We do not introduce design research: we suppose the readers of AI EDAM are familiar with this domain. After this general introduction to the domain of gesture studies, we provide an overview of gestures in design, that is, the research environment of the papers in this Special Issue. We then discuss some dimensions on which these papers differ, as well as some on which they are related.

We propose a computational method for producing novel constructs that fall within an existing design or artistic style. The method is based on evolutionary algorithms, and we discuss related knowledge representation issues. We then present an implementation of this method that we used in order to imitate the style of the Dutch painter Mondrian. Finally, we explain and give the results of a cognitive experiment designed to determine the effectiveness of the method, and provide a discussion of these results.

This paper illustrates the complexity of pointing as it is employed in a design workshop. Using the method of interaction analysis, we argue that pointing is not merely employed to index, locate, or fix reference to an object. It also constitutes a practice for reestablishing intersubjectivity and solving interactional trouble such as misunderstandings or disagreements by virtue of enlisting something as part of the participants’ shared experience. We use this analysis to discuss implications for how such practices might be supported with computer mediation, arguing for a “bricolage” approach to systems development that emphasizes the provision of resources for users to collaboratively negotiate the accomplishment of intersubjectivity rather than systems that try to support pointing as a specific gestural action.

This paper describes and evaluates approaches to model and solve technical product configuration problems using different artificial intelligence methodologies. By means of a typical example, the benefits and limitations of different artificial intelligence methods are discussed and a flexible software architecture for integrating different solvers in a product configurator is proposed.

This study attempts to correlate the nonlinear invariants' with the changing conditions of a drilling process through a series of condition monitoring experiments on small diameter (1 mm) drill bits. Run-to-failure tests are performed on these drill bits, and vibration data are consecutively gathered at equal time intervals. Nonlinear invariants, such as the Kolmogorov entropy and correlation dimension, and statistical parameters are calculated based on the corresponding conditions of the drill bits. By intervariations of these values between two successive measurements, a drop–rise table is created. Any variation that is within a certain threshold (±20% of the measurements in this case) is assumed to be constant. Any fluctuation above or below is assumed to be either a rise or a drop. The reduct and conflict tables then help eliminate incongruous and redundant data by the use of rough sets (RSs). Inconsistent data, which by definition is the boundary region, are classified through certainty and coverage factors. By handling inconsistencies and redundancies, 11 rules are extracted from 39 experiments, representing the underlying rules. Then 22 new experiments are used to check the validity of the rule space. The RS decision frame performs best at predicting no failure cases. It is believed that RSs are superior in dealing with real-life data over fuzzy set logic in that actual measured data are never as consistent as here and may dominate the monitoring of the manufacturing processes as it becomes more widespread.

Product configuration is a key technology, which enables businesses to deliver and deploy individualized products. In many cases, finding the optimal configuration solution for the user is a creative process that requires them to decide trade-offs between conflicting goals (multicriteria optimization problem). These problems are best supported by an interactive dialog that is managed by a dedicated software program (the configurator) that provides decision support. We illustrate this using a real example (configuration of a business software system). This productively used application makes the user aware of which choices are available in a given situation, provides assistance in resolving inconsistent choices and defaults, and generates explanations if desired. One of the key configurator components used to manage this is a truth maintenance system. We describe how this component is used and two novel extensions to it: methods for declarative handling of defaults (of varying strength) and the declarative handling of incompleteness. Finally, we summarize our experiences made during the implementation of this application and the pros and cons of declarative versus procedural approaches.

One challenge in building collaborative design tools that use speech and sketch input is distinguishing gesture pen strokes from those representing device structure, that is, object strokes. In previous work, we developed a gesture/object classifier that uses features computed from the pen strokes and the speech aligned with them. Experiments indicated that the speech features were the most important for distinguishing gestures, thus indicating the critical importance of the speech–sketch alignment. Consequently, we have developed a new alignment technique that employs a two-step process: the speech is first explicitly segmented (primarily into clauses), then the segments are aligned with the pen strokes. Our speech segmentation step is unique in that it uses sketch features for locating segment boundaries in multimodal dialog. In addition, it uses a single classifier to directly combine word-based, prosodic (pause), and sketch-based features. In the second step, segments are initially aligned with strokes based on temporal correlation, and then classifiers are used to detect and correct two common alignment errors. Our two-step technique has proven to be substantially more accurate at alignment than the existing technique that lacked explicit segmentation. It is more important that, for nearly all cases, our new technique results in greater gesture classification accuracy than the existing technique, and performed nearly as well as the benchmark manual speech–sketch alignment.

This article describes an investigation into the feasibility of using contextual reasoning to monitor and supervise the collaborative work of several knowledge workers working together on a project. Managing large and complex projects is a difficult task that requires situational awareness by the project manager to be able to be proactive when possible and to react correctly in the presence of events. In complex projects, effective oversight of the project personnel and the progress of the project are essential in ensuring that project objectives are met. This is especially true of projects that require contributions from various experts, whose interaction may be limited to a Web-based collaborative tool. Such oversight is typically the job of a project manager who is tasked with avoiding cost overruns, shipment delays, and ensuring product effectiveness. We utilize context-based reasoning and contextual graphs as the tools of choice for implementing an agent that emulates the function of a competent project manager. We use rocket design and manufacture as the domain to evaluate our technique. We use a public domain rocket design software package developed by the National Aeronautics and Space Administration as a guide to the domain. The article describes the investigation, its results, and the related works in a collaborative design project.

Three-dimensional (3-D) geometry can be described in many ways, with both a varying syntax and a varying semantics. As a result, several very diverse schemas and file formats can be deployed to describe geometry, depending on the application domain in question. In a multidisciplinary domain such as the domain of architecture, engineering, and construction, this range of specialized schemas makes file format conversions inevitable. The approach adopted by current conversion tools, however, often results in a loss of information, most often due to a “mistranslation” between different syntaxes and/or semantics, leading to errors and limitations in the design conception stage and to inefficiency due to the required remodeling efforts. An approach based on semantic web technology may reduce the loss of information significantly, leading to an improved processing of 3-D information and hence to an improved design practice in the architecture, engineering, and construction domain. This paper documents our investigation of the nature of this 3-D information conversion problem and how it may be encompassed using semantic web technology. In an exploratory double test case, we show how the specific deployment of semantic rule languages and an appropriate inference engine are to be adopted to improve this 3-D information exchange. It shows how semantic web technology allows the coexistence of diverse descriptions of the same 3-D information, interlinked through explicit conversion rules. Although only a simple example is used to document the process, and a more in-depth investigation is needed, the initial results indicate the suggested approach to be a useful alternative approach to obtain an improved 3-D information exchange.

This article describes the technical principles and representation of a constraint-based configuration method for work processes. Methods developed for the configuration of modular systems comprising components have traditionally adopted a representation where the properties and compatibility requirements are expressed as constraints associated with individual components. However, this representation does not accurately capture constraints on paths and subprocesses and is therefore unsuitable for process configuration. This article extends established constraint-based configuration methods with a constraint language for specifying properties of execution paths in work processes. A framework for semiautomated process customization is presented. It integrates the extended constraint language with a metamodel of the work processes in an organization and allows to adapt generic work processes to fit the requirements of specific development projects. Heuristic search methods are applied to build valid process configurations by incrementally resolving constraint violations. The declarative framework facilitates the adaptation of abstract work processes as well as the validation and repair of existing processes. The approach was developed in the context of a real-world system of complex design and development processes where it was shown that significant process improvements and reduction in effort required to edit process models can be achieved.

Spatial grammars are rule based, generative systems for the specification of formal languages. Set and shape grammar formulations of spatial grammars enable the definition of spatial design languages and the creation of alternative designs. Since the introduction of the underlying formalism, they have been successfully applied to different domains including visual arts, architecture, and engineering. Although many spatial grammars exist on paper, only a few, limited spatial grammar systems have been computationally implemented to date; this is especially true for three-dimensional (3-D) systems. Most spatial grammars are hard-coded, that is, once implemented, the vocabulary and rules cannot be changed without reprogramming. This article presents a new approach and prototype implementation for a 3-D spatial grammar interpreter that enables interactive, visual development and application of grammar rules. The method is based on a set grammar that uses a set of parameterized primitives and includes the definition of nonparametric and parametric rules, as well as their automatic application. A method for the automatic matching of the left hand side of a rule in a current working shape, including defining parametric relations, is outlined. A prototype implementation is presented and used to illustrate the approach through three examples: the “kindergarten grammar,” vehicle wheel rims, and cylinder cooling fins. This approach puts the creation and use of 3-D spatial grammars on a more general level and supports designers with facilitated definition and application of their own rules in a familiar computer-aided design environment without requiring programming.

Gestures play an important role in communication. They support the listener, who is trying to understand the speaker. However, they also support the speaker by facilitating the conceptualization and verbalization of messages and reducing cognitive load. Gestures thus play an important role in collaboration and also in problem-solving tasks. In human–computer interaction, gestures are also used to facilitate communication with digital applications, because their expressive nature can enable less constraining and more intuitive digital interactions than conventional user interfaces. Although gesture research in the social sciences typically considers empty-handed gestures, digital gesture interactions often make use of hand-held objects or touch surfaces to capture gestures that would be difficult to track in free space. In most cases, the physical objects used to make these gestures serve primarily as a means of sensing or input. In contrast, tangible interaction makes use of physical objects as embodiments of digital information. The physical objects in a tangible interface thus serve as representations as well as controls for the digital information they are associated with. Building on this concept, gesture interaction has the potential to make use of the physical properties of hand-held objects to enhance or change the functionality of the gestures made. In this paper, we look at the design opportunities that arise at the intersection of gesture and tangible interaction. We believe that gesturing while holding physical artifacts opens up a new interaction design space for collaborative digital applications that is largely unexplored. We provide a survey of gesture interaction work as it relates to tangible and touch interaction. Based on this survey, we define the design space of tangible gesture interaction as the use of physical devices for facilitating, supporting, enhancing, or tracking gestures people make for digital interaction purposes, and outline the design opportunities in this space.

In this paper we present a genotype representation method for improving the performance of genetic-algorithm-based optimal design and synthesis of microelectromechanical systems. The genetic algorithm uses a hierarchical component-based genotype representation, which incorporates specific engineering knowledge into the design optimization process. Each microelectromechanical system component is represented by a gene with its own parameters defining its geometry and the way it can be modified from one generation to the next. The object-oriented genotype structures efficiently describe the hierarchical nature typical of engineering designs. They also encode knowledge-based constraints that prevent the genetic algorithm from wasting time exploring inappropriate regions of the search space. The efficiency of the hierarchical component-based genotype representation is demonstrated with surface-micromachined resonator designs.

Product configuration is a major industrial application domain for constraint satisfaction techniques. Conditional constraint satisfaction problems (CCSPs) and feature models (FMs) have been developed to represent configuration problems in a natural way. CCSPs are like constraint satisfaction problems (CSPs), but they also include potential variables, which might or might not exist in any given solution, as well as classical variables, which are required to take a value in every solution. CCSPs model, for example, options on a car, for which the style of sunroof (a variable) only makes sense if the car has a sunroof at all. FMs are directed acyclic graphs of features with constraints on edges. FMs model, for example, cell phone features, where utility functions are required, but the particular utility function “games” is optional, but requires Java support. We show that existing techniques from formal methods and answer set programming can be used to naturally model CCSPs and FMs. We demonstrate configurators in both approaches. An advantage of these approaches is that the model builder does not have to reformulate the CCSP or FM into a classic CSP, converting potential variables into classical variables by adding a “does not exist” value and modifying the problem constraints. Our configurators automatically reason about the model itself, enumerating all solutions and discovering several kinds of model flaws.

Designers who are experts in a given design domain are well known to be able to Immediately focus on “good designs,” suggesting that they may have learned additional constraints while exploring the design space based on some functional aspects. These constraints, which are often implicit, result in a redefinition of the design space, and may be crucial for discovering chunks or interrelations among the design variables. Here we propose a machine-learning approach for discovering such constraints in supervised design tasks. We develop models for specifying design function in situations where the design has a given structure or embodiment, in terms of a set of performance metrics that evaluate a given design. The functionally feasible regions, which are those parts of the design space that demonstrate high levels of performance, can now be learned using any general purpose function approximator. We demonstrate this process using examples from the design of simple locking mechanisms, and as in human experience, we show that the quality of the constraints learned improves with greater exposure in the design space. Next, we consider changing the embodiment and suggest that similar embodiments may have similar abstractions. To explore convergence, we also investigate the variability in time and error rates where the experiential patterns are significantly different. In the process, we also consider the situation where certain functionally feasible regions may encode lower dimensional manifolds and how this may relate to cognitive chunking.

Function models are frequently used in engineering design to describe the technical functions that a product performs. This paper investigates the use of the functional basis, a function vocabulary developed to aid in communication and archiving of product function information, in describing consumer products that have been decomposed, analyzed, modeled functionally, and stored in a Web-based design repository. The frequency of use of function terms and phrases in 11 graphical and 110 list-based representations in the repository is examined and used to analyze the organization and expressiveness of the functional basis and function models. Within the context of reverse engineering, we determined that the modeling resolution provided by the hierarchical levels, especially the tertiary level, is inadequate for function modeling; the tertiary terms are inappropriate for capturing sufficient details desired by modelers for archiving and reuse, and there is a need for a more expressive flow terms and flow qualifiers in the vocabulary. A critical comparison is also presented of two representations in the design repository: function structures and function lists. The conclusions are used to identify new research opportunities, including the extension of the vocabulary to incorporate flow qualifiers in addition to more expressive terms.

A configurational method for describing shape is proposed based on two measures that gauge human experiences of moving through space: distance and changes in direction of travel. Boundary shapes from the built environment and nature are studied in a morphospace composed of two axes: one corresponding to each measure, to yield a typological classification of form. It is shown that the covariance between distance and directionality is mediated by the topological structure of embedded main circulation. Three kinds of circulation—elementary, ring, and linear—thus affect three fundamentally different balancing conditions between distance and directionality in boundary shapes. The analysis of large samples of shapes thus far demonstrates a “unique shape” status, where no two different shapes have the same pair of relative distance and directional fragmentation values.