Hostname: page-component-78c5997874-4rdpn Total loading time: 0 Render date: 2024-11-10T10:54:03.892Z Has data issue: false hasContentIssue false
  • English
  • Français

Speech analysis for conceptual CAD modeling using multi-modal interfaces: An investigation into Architects’ and Engineers’ speech preferences

Published online by Cambridge University Press:  14 March 2019

Sumbul Khan  and
Bige Tunçer
Sumbul Khan*
Affiliation:
SUTD-MIT International Design Centre, Singapore University of Technology and Design, 8, Somapah Road, 487372, Singapore
Bige Tunçer
Affiliation:
Architecture and Sustainable Design, Singapore University of Technology and Design, 8, Somapah Road, 487372, Singapore
*
Author for correspondence: Sumbul Khan, E-mail: sumbul_khan@sutd.edu.sg
Get access Rights & Permissions [Opens in a new window]

Abstract

Speech- and gesture-based interfaces for computer-aided design (CAD) modeling must employ vocabulary suitable for target professional groups. We conducted an experiment with 40 participants from architecture and engineering backgrounds to elicit their speech preferences for four CAD manipulation tasks: Scale, Rotate, Copy, and Move. We compiled speech command terms used by participants and analyzed verbalizations based on three analytic themes: the exactness of descriptions, the granularity of descriptions, and the use of CAD legacy terms. We found that participants from both groups used precise and vague expressions in their verbalizations and used a median of three parameters in their verbalizations. Architects used CAD legacy terms more than Engineers in the tasks Scale and Rotate. Based on these findings, we give recommendations for the design of speech- and gesture-based interface for conceptual CAD modeling.

Keywords

Computational linguisticscomputer-aided designconceptual designhuman–computer interactionnatural interfacesspeech analysis
Type
Research Article
Information
AI EDAM , Volume 33 , Issue 3 , August 2019 , pp. 275 - 288
Copyright
Copyright © Cambridge University Press 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Akalin, A, Yildirim, K, Wilson, C and Kilicoglu, O (2009) Architecture and engineering students’ evaluations of house façades: preference, complexity and impressiveness. Journal of Environmental Psychology 29, 124–132.Google Scholar
Akin, O and Moustapha, H (2004) Strategic use of representation in architectural massing. Design Studies 25, 31–50.Google Scholar
Alcaide-Marzal, J, Diego-Mas, JA, Asensio-Cuesta, S and Piqueras-Fiszman, B (2013) An exploratory study on the use of digital sculpting in conceptual product design. Design Studies 34, 264–284.Google Scholar
Athavankar, U (1999) Gestures, mental imagery and spatial reasoning. In Gero, JS and Tversky, B (eds), Preprints of the International Conference on Visual Reasoning (VR 99). MIT, pp. 103–128.Google Scholar
Berlyne, DE (1971) Aesthetics and Psychobiology. New York: Appleton-Century-Crofts.Google Scholar
Beşevli, C, Buruk, OT, Erkaya, M and Özcan, O (2018) Investigating the Effects of Legacy Bias. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility – DIS ’18 (pp. 277–281). New York, New York, USA: ACM Press.Google Scholar
Bolt, RA (1980) “Put-that-there”: Voice and Gesture at the Graphics Interface. In Proceedings of the 7th annual conference on Computer graphics and interactive techniques – SIGGRAPH ’80. (vol. 14, pp. 262–270). New York, New York, USA: ACM Press.Google Scholar
Brown, DC, Kwasny, SC, Chandrasekaran, B and Sondheimer, NK (1979) An experimental graphics system with natural language input. Computers & Graphics 4, 13–22.Google Scholar
Cassell, J (1998) A framework for gesture generation and interpretation. In Cipolla, R and Pentland, A (eds), Computer Vision in Human–Machine Interaction. Cambridge University Press, pp. 191–216.Google Scholar
Chastain, T, Kalay, YE and Peri, C (2002) Square peg in a round hole or horseless carriage? Reflections on the use of computing in architecture. Automation in Construction 11, 237–248.Google Scholar
Cicognani, A and Maher, ML (1997) Design Speech Acts. “How to Do Things with Words” in Virtual Communities. In CAAD futures 1997. pp. 707–717.Google Scholar
Clay, SR and Wilhelms, J (1996) Put: language-based interactive manipulation of objects. IEEE Computer Graphics and Applications 16, 31–39.Google Scholar
Coroado, L, Pedro, T, D'Alpuim, J, Eloy, S and Dias, MS (2015) VIARMODES: visualization and interaction in immersive virtual reality for the architectural design process. In Martens, B, Wurzer, G, Grasl, T, Lorenz, W and Schaffranek, R (eds), Real Time – Proceedings of the 33rd eCAADe Conference – Volume 1. Vienna, Austria, pp. 125–134.Google Scholar
Crawley, EF, Malmqvist, J, Östlund, S, Brodeur, DR and Edström, K (2014) Rethinking Engineering Education: The CDIO Approach, 2nd Edn. Basel: Springer International Publishing.Google Scholar
Devlin, K (1990) An examination of architectural interpretation: architects versus non-architects. Journal of Architectural and Planning Research 7, 235–244.Google Scholar
Dong, A (2005) The latent semantic approach to studying design team communication. Design Studies 26, 445–461.Google Scholar
Eckert, C and Stacey, M (2000) Sources of inspiration: a language of design. Design Studies 21, 523–538.Google Scholar
Eckert, C, Kelly, I and Stacey, M (1999) Interactive generative systems for conceptual design: an empirical perspective. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 13, 303–320.Google Scholar
Fish, J (2004) Cognitive catalysis: sketches for a time-lagged brain. In Goldschmidt, G and Porter, W (eds), Design Representation. pp. 151–184. London: Springer-Verlag London Limited.Google Scholar
Furnas, GW, Gomez, LM, Landauer, TK and Dumais, ST (1982) Statistical semantics: How can a computer use what people name things to guess what things people mean when they name things? In Proceedings of the 1982 conference on Human factors in computing systems – CHI ’82 (pp. 251–253). New York, New York, USA: ACM Press.Google Scholar
Gabriel, GC and Maher, ML (2002) Coding and modelling communication in architectural collaborative design. Automation in Construction 11, 199–211.Google Scholar
Ghomeshi, M and Jusan, MM (2013) Investigating different aesthetic preferences between architects and non-architects in residential façade designs. Indoor and Built Environment 22, 952–964.Google Scholar
Gifford, R, Hine, DW, Muller-clemm, W and Shaw, KT (2002) Why architects and laypersons judge buildings differently: cognitive properties and physical bases. Journal of Architectural and Planning Research 19, 132–148.Google Scholar
Glock, F (2009) Aspects of language use in design conversation. CoDesign 5, 5–19.Google Scholar
Goel, V (1995) Sketches of Thought. Cambridge, Massachusetts: MIT Press.Google Scholar
Goldschmidt, G (2004) Design representation: private process, public image. In Design Representation. London: Springer London, pp. 203–217.Google Scholar
Groat, L (1982) Meaning in post-modern architecture: an examination using the multiple sorting task. Journal of Environmental Psychology 2, 3–22.Google Scholar
Gross, MD (1996) The electronic cocktail napkin – a computational environment for working with design diagrams. Design Studies 17, 53.Google Scholar
Harrison, S and Minneman, S (1996) A bike in hand: a study of 3-D objects in design. In Cross, N, Christiaans, H and Dorst, K (eds), Analysing Design Activity. John Wiley & Sons, pp. 417–436.Google Scholar
Herbert, DM (1988) Study drawings in architectural design: their properties as a graphic medium. Journal of Architectural Education 41, 26–38.Google Scholar
Herold, J and Stahovich, TF (2011) Using speech to identify gesture pen strokes in collaborative, multimodal device descriptions. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 25, 237–254.Google Scholar
Jonson, B (2005) Design ideation: the conceptual sketch in the digital age. Design Studies 26, 613–624.Google Scholar
Jørgensen, U (2007) Historical accounts of engineering education. In Rethinking Engineering Education: The CDIO Approach. Boston, MA: Springer US, pp. 216–240.Google Scholar
Khan, S and Tuncer, B (n.d.) 3D CAD modeling using gestures and speech: Investigating CAD legacy and non-legacy procedures. (Forthcoming).Google Scholar
Khan, S and Tuncer, B (2017) Intuitive and effective gestures for conceptual architectural design. In ACADIA 2017: Disciplines & Disruption [Proceedings of the 37th Annual Conference of the Association for Computer Aided Design in Architecture (ACADIA) (pp. 318–323). Cambridge, Massachusetts.Google Scholar
Khan, S, Rajapakse, H, Zhang, H, Nanayakkara, S, Tuncer, B and Blessing, L (2017) GesCAD. In Proceedings of the 29th Australian Conference on Computer-Human Interaction – OZCHI ’17 (pp. 402–406). New York, New York, USA: ACM Press.Google Scholar
Köpsel, A and Bubalo, N (2015) Benefiting from legacy bias. Interactions 22, 44–47.Google Scholar
Lawson, B and Loke, SM (1997) Computers, words and pictures. Design Studies 18, 171–183.Google Scholar
Lenau, T and Boelskifte, P (2005) Verbal communication of semantic content in products. In Binder, T, Grogh, PG, Redström, J and Mazé, R (eds), Nordes Conference “In the Making”. Copenhagen: Royal Danish Academy of Fine Arts, School of Architecture, pp. 11–23.Google Scholar
Llinares, C and Iñarra, S (2014) Human factors in computer simulations of urban environment. Differences between architects and non-architects’ assessments. Displays 35, 126–140.Google Scholar
Llinares Millán, MDC, Iñarra, S and Guixeres, J (2018) Design attributes influencing the success of urban 3D visualizations: differences in assessments according to training and intention. Journal of Urban Technology 25, 39–57.Google Scholar
Logan, GD and Radcliffe, DF (2004) Impromptu prototyping and artefacting: representing design ideas through things at hand, actions, and talk. In Goldschmidt, G and Porter, WL (eds), Design Representation. pp. 127–148. London: Springer-Verlag London Limited.Google Scholar
Luck, R (2013) Articulating (mis)understanding across design discipline interfaces at a design team meeting. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 27, 155–166.Google Scholar
Malizia, A and Bellucci, A (2012) The artificiality of natural user interfaces. Communications of the ACM 55, 36.Google Scholar
McNeill, D (1992) Hand and Mind: What Gestures Reveal About Thought. Chicago: Univ. of Chicago Press.Google Scholar
Menegotto, JL (2015) Computer-Aided architectural design futures. The Next City – New Technologies and the Future of the Built Environment 527, 329–347.Google Scholar
Mignonneau, L and Sommerer, C (2005) Designing emotional, metaphoric, natural and intuitive interfaces for interactive art, edutainment and mobile communications. Computers and Graphics 29, 837–851.Google Scholar
Minneman, SL and Harrison, SR (1998) Negotiating right along – An extended case study of the social activity of engineering design. In Duffy, AHB (ed.) The Design Productivity Debate. Berlin, Germany: Springer-Verlag, pp. 32–50.Google Scholar
Morris, MR (2012) Web on the wall: insights from a multimodal interaction elicitation study. In Orit, Shaer, Chia, Shen, Meredith Ringel, Morris and Michael, Horn (eds), Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces, 95–104.Google Scholar
Morris, MR, Danielescu, A, Drucker, S, Fisher, D, Lee, B, Schraefel, M and Wobbrock, JO (2014) Reducing legacy bias in gesture elicitation studies. Interactions 21, 40–45. doi: 10.1017/CBO9781107415324.004Google Scholar
Najari, A, Dubois, S, Barth, M and Sonntag, M (2016) From Altshuller to Alexander: towards a bridge between architects and engineers. Procedia CIRP 39, 119–124.Google Scholar
Nanjundaswamy, VG, Kulkarni, A, Chen, Z, Jaiswal, PSSS, Verma, A and Rai, R (2013) Intuitive 3D Computer-Aided Design (CAD) System With Multimodal Interfaces. In ASME 2013 International Design Engineering Technical Conferences (IDETC) and Computers and Information in Engineering Conference (CIE). Portland, Oregon, USA: ASME.Google Scholar
Neal, JG, Thielman, CY, Dobes, Z, Haller, SM and Shapiro, SC (1989) Natural language with integrated deictic and graphic gestures. In Proceedings of the Workshop on Speech and Natural Language – HLT ’89. Cape Cod, Massachusetts: Association for Computational Linguistics, p. 410.Google Scholar
Oak, A (2011) What can talk tell us about design?: analyzing conversation to understand practice. Design Studies 32, 211–234.Google Scholar
Oh, J, Stuerzlinger, W and Danahy, J (2006) SESAME: towards better 3D conceptual design systems. In Carroll, JM, Bødker, S and Coughlin, J (eds), Proceedings of the 6th Conference on Designing Interactive Systems, DIS ’06. New York, USA: ACM, pp. 80–89.Google Scholar
Oviatt, S (1999) Ten myths of multimodal interaction. Communications of the ACM 42, 74–81.Google Scholar
Oviatt, S, DeAngeli, A and Kuhn, K (1997) Integration and synchronization of input modes during multimodal human–computer interaction. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems – CHI 97, 415–422.Google Scholar
Podehl, G (2002) Terms and measures for styling properties. In DS 30: Proceedings of DESIGN 2002, the 7th International Design Conference. Dubrovnik, Croatia: Sveacilisna Tiskara, pp. 879–886.Google Scholar
Poggenpohl, S, Chayutsahakij, P and Jeamsinkul, C (2004) Language definition and its role in developing a design discourse. Design Studies 25, 579–605.Google Scholar
Purcell, T (1996) The data in design protocols. The issue of data coding, data analysis the development of models of the design process. In Cross, N, Christiaans, H & Dorst, K (eds), Analysing Design Activity. New York: John Wiley & Sons, pp. 225–252.Google Scholar
Quek, F, McNeill, D, Bryll, R, Duncan, S, Ma, X-F, Kirbas, C, McCullough, KE and Ansari, R (2002) Multimodal human discourse: gesture and speech. ACM Transactions on Computer-Human Interaction 9, 171–193.Google Scholar
Salisbury, MW, Hendrickson, JH, Lammers, TL, Fu, C and Moody, SA (1990) Talk and draw: bundling speech and graphics. Computer 23, 59–65.Google Scholar
Segers, N and Leclercq, P (2007) Computational linguistics for design, maintenance, and manufacturing. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 21, 99–101.Google Scholar
Stacey, M and Eckert, C (2003) Against ambiguity. Computer Supported Cooperative Work (CSCW) 12, 153–183.Google Scholar
Tan, YY (2014) English as a “mother tongue” in Singapore. World Englishes, 33, 319–339.Google Scholar
Tunçer, B and Khan, S (2018) User defined conceptual modeling gestures. In Lee, J-H (ed.) Computational Studies on Cultural Variation and Heredity. Singapore: Springer, pp. 115–125.Google Scholar
Varshney, S (1998) Castle in the Air: a strategy to model shapes in a computer. Proceedings of the Asia Pacific Conference on Computer Human Interaction (APCHI 98) 3rd, 350–355.Google Scholar
Walther, J, Robertson, BF and Radcliffe, DF (2007) Avoiding the potential negative influence of CAD tools on the formation of students’ creativity. In Søndergaard, H and Hadgraft, R (eds), Proceedings of the 18th Conference of the Australasian Association for Engineering Education (AaeE). Melbourne, Australia: University of Melbourne, pp. 1–6.Google Scholar
Wang, R, Paris, S and Popović, J (2011) 6D hands: markerless hand-tracking for computer aided design. In Pierce, J, Agrawala, M and Klemmer, S (eds), UIST 11 Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology. Santa Barbara, CA, USA: ACM, pp. 549–557.Google Scholar
Weimer, D and Ganapathy, SK (1989) A synthetic visual environment with hand gesturing and voice input. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Wings for the Mind – CHI ’89, 20. New York, New York, USA: ACM Press, pp. 235–240.Google Scholar
Wiegers, T, Langeveld, L and Vergeest, J (2011) Shape language: how people describe shapes and shape operations. Design Studies 32, 333–347.Google Scholar
Wobbrock, JO, Morris, MR and Wilson, AD (2009) User-defined gestures for surface computing. Proceedings of the 27th International Conference on Human Factors in Computing Systems – CHI 09, 1083.Google Scholar
Zheng, JM, Chan, KW and Gibson, I (2001) Desktop virtual reality interface for computer aided conceptual design using geometric techniques. Journal of Engineering Design 12, 309.Google Scholar
Zhong, K, Kang, J, Qin, S and Wang, H (2011) Rapid 3D conceptual design based on hand gesture. In 3rd International Conference on Advanced Computer Control. pp. 192–197.Google Scholar