Gestures play an important role in communication. They support the listener, who is trying to understand the speaker. However, they also support the speaker by facilitating the conceptualization and verbalization of messages and reducing cognitive load. Gestures thus play an important role in collaboration and also in problem-solving tasks. In human–computer interaction, gestures are also used to facilitate communication with digital applications, because their expressive nature can enable less constraining and more intuitive digital interactions than conventional user interfaces. Although gesture research in the social sciences typically considers empty-handed gestures, digital gesture interactions often make use of hand-held objects or touch surfaces to capture gestures that would be difficult to track in free space. In most cases, the physical objects used to make these gestures serve primarily as a means of sensing or input. In contrast, tangible interaction makes use of physical objects as embodiments of digital information. The physical objects in a tangible interface thus serve as representations as well as controls for the digital information they are associated with. Building on this concept, gesture interaction has the potential to make use of the physical properties of hand-held objects to enhance or change the functionality of the gestures made. In this paper, we look at the design opportunities that arise at the intersection of gesture and tangible interaction. We believe that gesturing while holding physical artifacts opens up a new interaction design space for collaborative digital applications that is largely unexplored. We provide a survey of gesture interaction work as it relates to tangible and touch interaction. Based on this survey, we define the design space of tangible gesture interaction as the use of physical devices for facilitating, supporting, enhancing, or tracking gestures people make for digital interaction purposes, and outline the design opportunities in this space.

Designers are interacting with an increasing number of digital tools in their design process; however, these are usually in addition to the traditional and ubiquitous paper-based design journals. This paper explores the medium of informal design information and its relationships with sketching behavior over three stages of the design process: preliminary investigation and user needs analyses, concept generation and development, and prototyping and testing. Our test bed consists of tangible, digital, and hybrid design journals collected from four semesters of UC Berkeley's graduate level, multidisciplinary course titled “Managing the New Product Development Process: Design Theory and Methods.” We developed protocols for two categories of analysis: one that codes for the media type of each journal and its content, and another one that characterizes the content within the journal. We found a trend toward hybrid digital–tangible journals for the engineering students over the 4-year period. These hybrid journals exhibited a higher degree of detail over advancing design stages, which has been shown to correlate with improved project performance. We also present several case studies of unusual design journals that illustrate the range of designers' interpretations of design journals as a medium. Based on this descriptive research, features for interactive hybrid tangible–digital design journals are recommended.

Industrial designers make sketches and physical models to start and develop ideas and concept designs. Such representations have advantages that they support fast, intuitive, rich, sensory exploration of solutions. Although existing tools and techniques provide adequate support where the shape of the product is concerned, the exploration of surface qualities such as material and printed graphics is supported to a much lesser extent. Moreover, there are no tools that have the fluency of sketching that allow combined exploration of shape, material, and their interactions. This paper evaluates Skin, an augmented reality tool designed to solve these two shortcomings. By projecting computer-generated images onto the shape model Skin allows for a “sketchy” tangible interaction where designers can explore surface qualities on a three-dimensional physical shape model. The tool was evaluated in three design situations in the domain of ceramics design. In each case, we found that the joint exploration of shape and surface provided creative benefits in the form of new solutions; in addition, a gain in efficiency was found in at least one case. The results show that joint exploration of shape and surface can be effectively supported with tangible augmented reality techniques and suggest that this can be put to practical use in industry today.

Tangible interaction is a growing area of human–computer interaction research that has become popular in recent years. Yet designers and researchers are still trying to comprehend and clarify its nature, characteristics, and implications. One approach has been to create frameworks that help us look back at and categorize past tangible interaction systems, and look forward at the possibilities and opportunities for developing new systems. To date, a number of different frameworks have been proposed that each provide different perspectives on the tangible interaction design space, and which can guide designers of new systems in different ways. In this paper, we map the space of tangible interaction frameworks. We order existing frameworks by their general type, and by the facets of tangible interaction design they address. One of our main conclusions is that most frameworks focus predominantly on the conceptual design of tangible systems, whereas fewer frameworks abstract the knowledge gained from previous systems, and hardly any framework provides concrete steps or tools for building new tangible systems. In addition, the facets most represented in existing frameworks are those that address the interactions with or the physicality of the designed systems. Other facets, such as domain-specific technology and experience, are rare. This focus on design, interaction, and physicality is interesting, as the origins of the field are rooted in engineering methods and have only recently started to incorporate more design-inspired approaches. As such, we expected more frameworks to focus on technologies and to provide concrete building suggestions for new tangible interaction systems.