Hostname: page-component-cd9895bd7-fscjk Total loading time: 0 Render date: 2024-12-26T16:53:44.820Z Has data issue: false hasContentIssue false
  • English
  • Français

USE CASES FOR A HYBRID AUGMENTED REALITY COMPUTER WORKSTATION IN CAD WORKFLOWS

Published online by Cambridge University Press:  19 June 2023

Jakob Harlan ,
Stefan Goetz  and
Sandro Wartzack
Jakob Harlan*
Affiliation:
Friedrich-Alexander-Universität Erlangen-Nürnberg
Stefan Goetz
Affiliation:
Friedrich-Alexander-Universität Erlangen-Nürnberg
Sandro Wartzack
Affiliation:
Friedrich-Alexander-Universität Erlangen-Nürnberg
*
Harlan, Jakob, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany, harlan@mfk.fau.de

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Many immersive approaches for design activities show a great potential for their specific use cases, but still overall usage of extended reality technology in product developers day-to-day work is little. The user's workflow between classical desktop work environment and its immersive counterpart is interrupted by both a data gap and an interface gap. The three-dimensional product data usually needs preparation and the user has to physically change the interface in use. The hybrid augmented reality computer workstation aims to close these gaps. A hologram of the current model is visualized next to the screen in reach of the user for intuitive inspection and spatial interactions. In this paper we present use cases for this novel immersive workstation in the CAD workflow. An explorative user study of the typical product designer`s workflow reveals the most common activities. Guided by those, eight uses cases are formulated and classified into fundamental, drafting, and modelling CAD tasks. These cases include novel hybrid augmented reality interactions derived from literature, which are assessed with respect to their applicability.

Keywords

Computer Aided Design (CAD)Virtual realityWorkspaces for designAugmeted RealityNatural User Interfaces
Type
Article
Information
Proceedings of the Design Society , Volume 3: ICED23 , July 2023 , pp. 3731 - 3740
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2023. Published by Cambridge University Press

References

Argelaguet, F. and Andujar, C. (2013), “A survey of 3D object selection techniques for virtual environments”, Computers & Graphics, Vol. 37 No. 3, pp. 121–136. http://dx.doi.org/10.1016/j.cag.2012.12.003.CrossRefGoogle Scholar
Bhattacharjee, S. and Chaudhuri, P. (2022), “Deep Interactive Surface Creation from 3D Sketch Strokes”, Thirty-First International Joint Conference on Artificial Intelligence {IJCAI-22}, Vienna, Austria, pp. 4908–4914. http://dx.doi.org/10.24963/ijcai.2022/680.CrossRefGoogle Scholar
Bornik, A., Beichel, R., Kruijff, E., Reitinger, B. and Schmalstieg, D. (2006), “A Hybrid User Interface for Manipulation of Volumetric Medical Data”, 3D User Interfaces (3DUI’06), pp. 29–36. http://dx.doi.org/10.1109/VR.2006.8.CrossRefGoogle Scholar
Bourdot, P., Convard, T., Picon, F., Ammi, M., Touraine, D. and Vézien, J.-M. (2010), “VR–CAD integration: Multimodal immersive interaction and advanced haptic paradigms for implicit edition of CAD models”, Computer-Aided Design, Vol. 42 No. 5, pp. 445–461. http://dx.doi.org/10.1016/j.cad.2008.10.014.CrossRefGoogle Scholar
Coburn, J.Q., Freeman, I. and Salmon, J.L. (2017), “A Review of the Capabilities of Current Low-Cost Virtual Reality Technology and Its Potential to Enhance the Design Process”, Journal of Computing and Information Science in Engineering, Vol. 17 No. 3. http://dx.doi.org/10.1115/1.4036921.CrossRefGoogle Scholar
Cohen, M., Regazzoni, D. and Vrubel, C. (2019), “A 3D Virtual Sketching System Using NURBS Surfaces and Leap Motion Controller”, CAD’19, El Paso, TX, USA, pp. 314–317. http://dx.doi.org/10.14733/cadconfP.2019.314-317.Google Scholar
Fechter, M., Miehling, J., Stangl, T. and Wartzack, S. (2014), “Intuitive Virtual Reality - CAD Assembly System”, DESIGN 2014 13th International Design Conference, pp. 1053–1062.Google Scholar
Fechter, M., Schleich, B. and Wartzack, S. (2020), “CAD-Gestaltmodellierung in VR für die frühe Entwurfsphase”, Konstruktion, Vol. 72, pp. 69–74. http://dx.doi.org/10.37544/0720-5953-2020-03-69.CrossRefGoogle Scholar
Fechter, M., Schleich, B. and Wartzack, S. (2021), “Comparative evaluation of WIMP and immersive natural finger interaction: a user study on CAD assembly modeling”, Virtual Reality, Vol. 26 No. 1, pp. 143–158. http://dx.doi.org/10.1007/s10055-021-00543-0.CrossRefGoogle Scholar
Fuge, M., Yumer, M.E., Orbay, G. and Kara, L.B. (2012), “Conceptual design and modification of freeform surfaces using dual shape representations in augmented reality environments”, Computer-Aided Design, Vol. 44 No. 10, pp. 1020–1032. http://dx.doi.org/10.1016/j.cad.2011.05.009.CrossRefGoogle Scholar
Harlan, J., Schleich, B. and Wartzack, S. (2020), “Linking a game-engine with CAD-software to create a flexible platform for researching extended reality interfaces for the industrial design process”, Proceedings of the 31st Symposium Design for X, pp. 169–178. http://dx.doi.org/10.35199/dfx2020.18.CrossRefGoogle Scholar
Harlan, J., Schleich, B. and Wartzack, S. (2021), “A Systematic Collection of Natural Interactions for Immersive Modeling From Building Blocks”, Proceedings of the Design Society, Vol. 1, pp. 283–292. http://dx.doi.org/10.1017/pds.2021.29.CrossRefGoogle Scholar
Harlan, J., Schleich, B. and Wartzack, S. (2022), “A concept for a novel hybrid augmented reality computer workstation for virtual product development”, Proceedings of the 33rd Symposium Design for X, pp. 10–19. http://dx.doi.org/10.35199/dfx2022.06.Google Scholar
Kaufmann, H., Schmalstieg, D. and Wagner, M. (2000), “Construct3D: A Virtual Reality Application for Mathematics and Geometry Education”, Education and Information Technologies, Vol. 5 No. 4, pp. 263–276. http://dx.doi.org/10.1023/A:1012049406877.CrossRefGoogle Scholar
Lee, S. and Yan, J. (2016), “The impact of 3D CAD interfaces on user ideation: A comparative analysis using SketchUp and Silhouette Modeler”, Design Studies, Vol. 44, pp. 52–73. http://dx.doi.org/10.1016/j.destud.2016.02.001.CrossRefGoogle Scholar
Lee, S.-H., Jin, T., Lee, J.H. and Bae, S.-H. (2022), “WireSketch: Bimanual Interactions for 3D Curve Networks in VR”, Association for Computing MachineryNew York, NY, USA, pp. 1–3. http://dx.doi.org/10.1145/3526114.3558726.Google Scholar
Liu, J., Pastoor, S., Seifert, K. and Hurtienne, J. (2001), “Three-dimensional PC: toward novel forms of human-computer interaction”, Proceedings Volume 10298, Three-Dimensional Video and Display: Devices and Systems: A Critical Review, Bellingham, United States. http://dx.doi.org/10.1117/12.419783.CrossRefGoogle Scholar
Lubos, P., Bruder, G. and Steinicke, F. (2014), “Analysis of direct selection in head-mounted display environments”, 2014 IEEE Symposium on 3D User Interfaces (3DUI), pp. 11–18. http://dx.doi.org/10.1109/3DUI.2014.6798834.CrossRefGoogle Scholar
Millette, A. and McGuffin, M.J. (2016), “DualCAD: Integrating Augmented Reality with a Desktop GUI and Smartphone Interaction”, 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), Merida, Yucatan, Mexico, pp. 21–26. http://dx.doi.org/10.1109/ISMAR-Adjunct.2016.0030.CrossRefGoogle Scholar
Nee, A.Y.C., Ong, S.K., Chryssolouris, G. and Mourtzis, D. (2012), “Augmented reality applications in design and manufacturing”, CIRP Annals, Vol. 61 No. 2, pp. 657–679. http://dx.doi.org/10.1016/j.cirp.2012.05.010.CrossRefGoogle Scholar
Ng, L.X., Ong, S.K. and Nee, A.Y.C. (2010), “Arcade: A Simple and Fast Augmented Reality Computer-Aided Design Environment Using Everyday Objects”, IADIS International Conference Interfaces and Human Computer Interaction 2010 (part of MCCSIS 2010), pp. 227–234.Google Scholar
Okuya, Y., Ladeveze, N., Fleury, C. and Bourdot, P. (2018), “ShapeGuide: Shape-Based 3D Interaction for Parameter Modification of Native CAD Data”, Frontiers in Robotics and AI, Frontiers, Vol. 5. http://dx.doi.org/10.3389/frobt.2018.00118.Google ScholarPubMed
Oti, A. and Crilly, N. (2021), “Immersive 3D sketching tools: Implications for visual thinking and communication”, Computers & Graphics, Vol. 94, pp. 111–123. http://dx.doi.org/10.1016/j.cag.2020.10.007.CrossRefGoogle Scholar
Park, S., Bokijonov, S. and Choi, Y. (2021), “Review of Microsoft HoloLens Applications over the Past Five Years”, Applied Sciences, Vol. 11 No. 16, p. 7259. http://dx.doi.org/10.3390/app11167259.CrossRefGoogle Scholar
Rakkolainen, I., Sand, A. and Raisamo, R. (2019), “A Survey of Mid-Air Ultrasonic Tactile Feedback”, 2019 IEEE International Symposium on Multimedia (ISM), pp. 94–944. http://dx.doi.org/10.1109/ISM46123.2019.00022.CrossRefGoogle Scholar
Reipschläger, P. and Dachselt, R. (2019), “DesignAR: Immersive 3D-Modeling Combining Augmented Reality with Interactive Displays”, ISS ’19: Interactive Surfaces and Spaces, Daejeon Republic of Korea, pp. 29–41. http://dx.doi.org/10.1145/3343055.3359718.Google Scholar
Satter, K. and Butler, A. (2015), “Competitive Usability Analysis of Immersive Virtual Environments in Engineering Design Review”, Journal of Computing and Information Science in Engineering, Vol. 15 No. 3, p. 031001. http://dx.doi.org/10.1115/1.4029750.CrossRefGoogle Scholar
Schneider, D., Otte, A., Kublin, A.S., Martschenko, A., Kristensson, P.O., Ofek, E., Pahud, M., et al. (2020), “Accuracy of Commodity Finger Tracking Systems for Virtual Reality Head-Mounted Displays”, 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 804–805. http://dx.doi.org/10.1109/VRW50115.2020.00253.CrossRefGoogle Scholar
Siewert, J.L., Neges, M. and Gerhard, D. (2021), “Ein Klassifizierungssystem für Industrielle Augmented Reality Anwendungen”, Entwerfen ENTWICKELN ERLEBEN in Produktentwicklung und Design 2021, Dresden, pp. 401–416. http://dx.doi.org/https://doi.org/10.25368/2021.37.CrossRefGoogle Scholar
Song, J., Cho, S., Baek, S.-Y., Lee, K. and Bang, H. (2014), “GaFinC: Gaze and Finger Control interface for 3D model manipulation in CAD application”, Computer-Aided Design, Vol. 46, pp. 239–245. http://dx.doi.org/10.1016/j.cad.2013.08.039.CrossRefGoogle Scholar
Song, P., Goh, W.B., Hutama, W., Fu, C.-W. and Liu, X. (2012), “A handle bar metaphor for virtual object manipulation with mid-air interaction”, 2012 ACM annual conference on Human Factors in Computing Systems - CHI ’12, Austin, Texas, USA, p. 1297. http://dx.doi.org/10.1145/2207676.2208585.Google Scholar
Stark, R., Israel, J.H. and Wöhler, T. (2010), “Towards hybrid modelling environments—Merging desktop-CAD and virtual reality-technologies”, CIRP Annals, Vol. 59 No. 1, pp. 179–182. http://dx.doi.org/10.1016/j.cirp.2010.03.102.CrossRefGoogle Scholar
Steinicke, F., Ropinski, T., Bruder, G. and Hinrichs, K. (2007), “Towards Applicable 3D User Interfaces for Everyday Working Environments”, Human-Computer Interaction – INTERACT 2007, Berlin, Heidelberg, pp. 546–559. http://dx.doi.org/10.1007/978-3-540-74796-3_55.Google Scholar
Valentini, P.P. (2018), “Natural interface for interactive virtual assembly in augmented reality using Leap Motion Controller”, International Journal on Interactive Design and Manufacturing (IJIDeM), Vol. 12 No. 4, pp. 1157–1165. http://dx.doi.org/10.1007/s12008-018-0461-0.CrossRefGoogle Scholar
Valentini, P.P. and Biancolini, M.E. (2018), “Interactive Sculpting Using Augmented-Reality, Mesh Morphing, and Force Feedback: Force-Feedback Capabilities in an Augmented Reality Environment”, IEEE Consumer Electronics Magazine, IEEE Consumer Electronics Magazine, Vol. 7 No. 2, pp. 83–90. http://dx.doi.org/10.1109/MCE.2017.2709598.CrossRefGoogle Scholar
Ramani, Vinayak and, K. (2015), “A gesture-free geometric approach for mid-air expression of design intent in 3D virtual pottery”, Computer-Aided Design, Vol. 69, pp. 11–24. http://dx.doi.org/10.1016/j.cad.2015.06.006.Google Scholar
Wang, P., Zhang, S., Billinghurst, M., Bai, X., He, W., Wang, S., Sun, M., et al. (2020), “A comprehensive survey of AR/MR-based co-design in manufacturing”, Engineering with Computers, Vol. 36. http://dx.doi.org/10.1007/s00366-019-00792-3.CrossRefGoogle Scholar
Wang, R., Paris, S. and Popović, J. (2011), “6D Hands: Markerless Hand-tracking for Computer Aided Design”, ACMNew York, NY, USA, pp. 549–558. http://dx.doi.org/10.1145/2047196.2047269.Google Scholar