Hostname: page-component-6bf8c574d5-rwnhh Total loading time: 0 Render date: 2025-03-01T14:55:20.508Z Has data issue: false hasContentIssue false
  • English
  • Français

LOOKING BEYOND SELF-REPORTED COGNITIVE LOAD: INVESTIGATING THE USE OF EYE TRACKING IN THE STUDY OF DESIGN REPRESENTATIONS IN ENGINEERING DESIGN

Published online by Cambridge University Press:  19 June 2023

Madison Cass  and
Rohan Prabhu
Madison Cass
Affiliation:
Neuroscience Program, Lafayette College;
Rohan Prabhu*
Affiliation:
Department of Mechanical Engineering, Lafayette College
*
Prabhu, Rohan, Lafayette College, United States of America, prabhur@lafayette.edu

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Designers are experiencing greater mental demands given the complexity of design tools, necessitating the study of cognitive load in design. Researchers have identified task- and designer-related factors that affect cognitive load; however, these studies primarily use self-reported measures that could be inaccurate and incomplete. Little research has tested the accuracy and completeness of self-reported measures and we aim to explore this gap. Towards this aim, we seek to answer the question: How does cognitive load vary based on the different design representations used, and do these differences depend on the measure of cognitive load? From our results, we see that the design representations vary in the range of cognitive load experienced by designers when using them. Moreover, this role of the range of cognitive load variance was observed given our use of pupil diameter. These findings call for the use of a multi-modal approach for measuring cognitive load with the combined use of subjective (e.g., self-report) and objective measures (e.g., physiological measures), as well as the use of both retrospective (e.g., self-report) and concurrent measures (e.g., physiological measures).

Keywords

Design for Additive Manufacturing (DfAM)Design cognitionIndustry 4.0cognitive loaddesign representation
Type
Article
Information
Proceedings of the Design Society , Volume 3: ICED23 , July 2023 , pp. 2475 - 2484
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2023. Published by Cambridge University Press

References

Alavi, M., Visentin, D.C., Thapa, D.K., Hunt, G.E., Watson, R. and Cleary, M. (2020), “Exploratory factor analysis and principal component analysis in clinical studies: Which one should you use?”, Journal of Advanced Nursing, p. jan.14377, https://dx.doi.org/10.1111/jan.14377.CrossRefGoogle Scholar
Beatty, J. (1982), “Task-evoked pupillary responses, processing load, and the structure of processing resources.”, Psychological Bulletin, Vol. 91 No. 2, pp. 276–292, https://dx.doi.org/10.1037/0033-2909.91.2.276.CrossRefGoogle ScholarPubMed
Beatty, J. and Lucero-Wagoner, B. (2000), “The Pupillary System”, Handbook of Psychophysiology, 2nd ed., Cambridge University Press, pp. 142–162.Google Scholar
Bilda, Z. and Gero, J.S. (2007), “The impact of working memory limitations on the design process during conceptualization”, Design Studies, Vol. 28 No. 4, pp. 343–367, https://dx.doi.org/10.1016/j.destud.2007.02.005.CrossRefGoogle Scholar
Cabestrero, R., Crespo, A. and Quirós, P. (2009), “Pupillary Dilation as an Index of Task Demands”, Perceptual and Motor Skills, Vol. 109 No. 3, pp. 664–678, https://dx.doi.org/10.2466/pms.109.3.664-678.CrossRefGoogle ScholarPubMed
Calpin, N. and Menold, J. (2022), “The Cognitive Costs of Design Tasks: The Evolution of Cognitive Load in Design and its Relationship with Design Outcomes”, ASME.Google Scholar
Cash, P., Isaksson, O., Maier, A. and Summers, J. (2022), “Sampling in design research: Eight key considerations”, Design Studies, Vol. 78, p. 101077, https://dx.doi.org/10.1016/j.destud.2021.101077.CrossRefGoogle Scholar
Cherng, Y.-G., Baird, T., Chen, J.-T. and Wang, C.-A. (2020), “Background luminance effects on pupil size associated with emotion and saccade preparation”, Scientific Reports, Vol. 10 No. 1, p. 15718, https://dx.doi.org/10.1038/s41598-020-72954-z.CrossRefGoogle ScholarPubMed
Cowley, B., Filetti, M., Lukander, K., Torniainen, J., Henelius, A., Ahonen, L., Barral, O., et al. (2016), “The Psychophysiology Primer: A Guide to Methods and a Broad Review with a Focus on Human–Computer Interaction”, Foundations and Trends® in Human–Computer Interaction, Vol. 9 No. 3–4, pp. 151–308, https://dx.doi.org/10.1561/1100000065.CrossRefGoogle Scholar
Cronbach, L.J. (1951), “Coefficient alpha and the internal structure of tests”, Psychometrika, Vol. 16 No. 3, pp. 297–334, https://dx.doi.org/10.1007/BF02310555.CrossRefGoogle Scholar
Devos, H., Gustafson, K., Ahmadnezhad, P., Liao, K., Mahnken, J.D., Brooks, W.M. and Burns, J.M. (2020), “Psychometric Properties of NASA-TLX and Index of Cognitive Activity as Measures of Cognitive Workload in Older Adults”, Brain Sciences, Vol. 10 No. 12, p. 994, https://dx.doi.org/10.3390/brainsci10120994.CrossRefGoogle ScholarPubMed
Erhan, H., Chan, J., Fung, G., Shireen, N. and Wang, I. (2017), “Understanding Cognitive Overload in Generative Design - An Epistemic Action Analysis”, presented at the The Association for Computer-Aided Architectural Design Research in Asia (CAADRIA) 2017: Protocols, Flows, and Glitches, Hong Kong, pp. 127–137, doi: https://doi.org/10.52842/conf.caadria.2017.127.CrossRefGoogle Scholar
Gann, D., Dodgson, M. and Salter, A. (2004), “Impact of innovation technology on engineering problem solving: Lessons from high profile public projects”.Google Scholar
Gardan, J. (2016), “Additive manufacturing technologies: state of the art and trends”, International Journal of Production Research, Vol. 54 No. 10, pp. 3118–3132, https://dx.doi.org/10.1080/00207543.2015.1115909.CrossRefGoogle Scholar
Gero, J.S. and Milovanovic, J. (2020), “A framework for studying design thinking through measuring designers’ minds, bodies and brains”, Design Science, Cambridge University Press, Vol. 6, p. e19, https://dx.doi.org/10.1017/dsj.2020.15.Google Scholar
Hallgren, K.A. (2012), “Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial”, Tutorials in Quantitative Methods for Psychology, Vol. 8 No. 1, pp. 23–34, https://dx.doi.org/10.20982/tqmp.08.1.p023.CrossRefGoogle ScholarPubMed
Hannah, R., Joshi, S. and Summers, J.D. (2012), “A user study of interpretability of engineering design representations”, Journal of Engineering Design, Vol. 23 No. 6, pp. 443–468, https://dx.doi.org/10.1080/09544828.2011.615302.CrossRefGoogle Scholar
Hart, S.G. (2006), “Nasa-Task Load Index (NASA-TLX); 20 Years Later”, Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 50 No. 9, pp. 904–908, https://dx.doi.org/10.1177/154193120605000909.CrossRefGoogle Scholar
Hart, S.G. and Staveland, L.E. (1988), “Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research”, Advances in Psychology, Vol. 52, Elsevier, pp. 139–183, https://dx.doi.org/10.1016/S0166-4115(08)62386-9.CrossRefGoogle Scholar
Herbig, P.A. and Kramer, H. (1992), “The phenomenon of innovation overload”, Technology in Society, Vol. 14 No. 4, pp. 441–461, https://dx.doi.org/10.1016/0160-791X(92)90038-C.CrossRefGoogle Scholar
Hershaw, J.N. and Ettenhofer, M.L. (2018), “Insights into cognitive pupillometry: Evaluation of the utility of pupillary metrics for assessing cognitive load in normative and clinical samples”, International Journal of Psychophysiology, Vol. 134, pp. 62–78, https://dx.doi.org/10.1016/j.ijpsycho.2018.10.008.CrossRefGoogle ScholarPubMed
Hess, E.H. and Polt, J.M. (1964), “Pupil Size in Relation to Mental Activity during Simple Problem-Solving”, Science, Vol. 143 No. 3611, pp. 1190–1192, https://dx.doi.org/10.1126/science.143.3611.1190.CrossRefGoogle ScholarPubMed
Ikuma, L.H., Nussbaum, M.A. and Babski-Reeves, K.L. (2009), “Reliability of physiological and subjective responses to physical and psychosocial exposures during a simulated manufacturing task”, International Journal of Industrial Ergonomics, Vol. 39 No. 5, pp. 813–820, https://dx.doi.org/10.1016/j.ergon.2009.02.005.CrossRefGoogle Scholar
Kahneman, D. (1973), Attention and Effort, Prentice Hall, Englewood Cliffs, NJ.Google Scholar
Kaiser, H.F. (1974), “An index of factorial simplicity”, Psychometrika, Vol. 39 No. 1, pp. 31–36, https://dx.doi.org/10.1007/BF02291575.CrossRefGoogle Scholar
Lempert, K.M., Chen, Y.L. and Fleming, S.M. (2015), “Relating Pupil Dilation and Metacognitive Confidence during Auditory Decision-Making”, PLOS ONE, Public Library of Science, Vol. 10 No. 5, p. e0126588, https://dx.doi.org/10.1371/journal.pone.0126588.CrossRefGoogle ScholarPubMed
Magezi, D.A. (2015), “Linear mixed-effects models for within-participant psychology experiments: an introductory tutorial and free, graphical user interface (LMMgui)”, Frontiers in Psychology, Vol. 6.CrossRefGoogle ScholarPubMed
McKoy, F.L., Vargas-Hernández, N., Summers, J.D. and Shah, J.J. (2001), “Influence of Design Representation on Effectiveness of Idea Generation”, Volume 4: 13th International Conference on Design Theory and Methodology, presented at the ASME 2001 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, American Society of Mechanical Engineers, Pittsburgh, Pennsylvania, USA, pp. 39–48, https://dx.doi.org/10.1115/DETC2001/DTM-21685.CrossRefGoogle Scholar
Mohamed-Ahmed, A., Bonnardel, N., Côté, P. and Tremblay, S. (2013), “Cognitive load management and architectural design outcomes”, International Journal of Design Creativity and Innovation, Vol. 1 No. 3, pp. 160–176, https://dx.doi.org/10.1080/21650349.2013.797013.CrossRefGoogle Scholar
Murphy, P.R., Vandekerckhove, J. and Nieuwenhuis, S. (2014), “Pupil-Linked Arousal Determines Variability in Perceptual Decision Making”, PLOS Computational Biology, Public Library of Science, Vol. 10 No. 9, p. e1003854, https://dx.doi.org/10.1371/journal.pcbi.1003854.CrossRefGoogle ScholarPubMed
Nolte, H. and McComb, C. (2021), “The cognitive experience of engineering design: an examination of first-year student stress across principal activities of the engineering design process”, Design Science, Vol. 7, p. e3, https://dx.doi.org/10.1017/dsj.2020.32.CrossRefGoogle Scholar
Onarheim, B. (2012), “Creativity from constraints in engineering design: lessons learned at Coloplast”, Journal of Engineering Design, Vol. 23 No. 4, pp. 323–336, https://dx.doi.org/10.1080/09544828.2011.631904.CrossRefGoogle Scholar
Panchal, J.H., Fuge, M., Liu, Y., Missoum, S. and Tucker, C. (Eds.). (2019), “Special Issue: Machine Learning for Engineering Design”, Journal of Mechanical Design, Vol. 141 No. 11, https://dx.doi.org/10.1115/1.4044690.CrossRefGoogle Scholar
Peinkhofer, C., Knudsen, G.M., Moretti, R. and Kondziella, D. (2019), “Cortical modulation of pupillary function: systematic review”, PeerJ, Vol. 7, p. e6882, https://dx.doi.org/10.7717/peerj.6882.CrossRefGoogle ScholarPubMed
Prabhu, R., Miller, S.R., Simpson, T.W. and Meisel, N.A. (2020), “Complex Solutions for Complex Problems? Exploring the Role of Design Task Choice on Learning, Design for Additive Manufacturing Use, and Creativity”, Journal of Mechanical Design, Vol. 142 No. 3, p. 031121, https://dx.doi.org/10.1115/1.4045127.CrossRefGoogle Scholar
Reich, Y. (2022), “We cannot play 20 questions with creativity and innovation and win: the necessity of practice-based integrative research”, International Journal of Design Creativity and Innovation, Vol. 10 No. 2, pp. 69–74, https://dx.doi.org/10.1080/21650349.2022.2041889.CrossRefGoogle Scholar
Reid, G.B. and Nygren, T.E. (1988), “The Subjective Workload Assessment Technique: A Scaling Procedure for Measuring Mental Workload”, Advances in Psychology, Vol. 52, Elsevier, pp. 185–218, https://dx.doi.org/10.1016/S0166-4115(08)62387-0.CrossRefGoogle Scholar
Rubio, S., Díaz, E., Martín, J. and Puente, J.M. (2004), “Evaluation of Subjective Mental Workload: A Comparison of SWAT, NASA-TLX, and Workload Profile Methods”, Applied Psychology, Vol. 53 No. 1, pp. 61–86, https://dx.doi.org/10.1111/j.1464-0597.2004.00161.x.CrossRefGoogle Scholar
Simpson, T.W., Boyer, J., Seepersad, C., Williams, C.B. and Witherell, P. (2016), “Special Issue: Designing for Additive Manufacturing”, Journal of Mechanical Design, Vol. 138 No. 12, https://dx.doi.org/10.1115/1.4034863.CrossRefGoogle Scholar
Sorby, S.A. (1999), “Spatial Abilities And Their Relationship To Computer Aided Design Instruction”, presented at the ASEE Annual Conference and Exposition.Google Scholar
Tamhane, A.C. (1979), “A Comparison of Procedures for Multiple Comparisons of Means with Unequal Variances”, Journal of the American Statistical Association, Taylor & Francis, Vol. 74 No. 366a, pp. 471–480, https://dx.doi.org/10.1080/01621459.1979.10482541.Google Scholar
Tobias, S. and Carlson, J.E. (1969), “Brief Report: Bartlett's Test of Sphericity and Chance Findings in Factor Analysis”, Multivariate Behavioral Research, Routledge, Vol. 4 No. 3, pp. 375–377, https://dx.doi.org/10.1207/s15327906mbr0403_8.CrossRefGoogle ScholarPubMed
Tobii Pro AB (2022), “Tobii Pro Lab User Manual (v1.194)”, Tobii Pro AB.Google Scholar
de Waard, D. and Lewis-Evans, B. (2014), “Self-report scales alone cannot capture mental workload”, Cognition, Technology & Work, Vol. 16 No. 3, pp. 303–305, https://dx.doi.org/10.1007/s10111-014-0277-z.CrossRefGoogle Scholar
Wierwille, W.W. and Eggemeier, F.T. (1993), “Recommendations for Mental Workload Measurement in a Test and Evaluation Environment”, Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 35 No. 2, pp. 263–281, https://dx.doi.org/10.1177/001872089303500205.CrossRefGoogle Scholar