Hostname: page-component-78c5997874-lj6df Total loading time: 0 Render date: 2024-11-10T12:29:46.615Z Has data issue: false hasContentIssue false
  • English
  • Français

Understanding node-link and matrix visualizations of networks: A large-scale online experiment

Published online by Cambridge University Press:  05 August 2019

Donghao Ren Open the ORCID record for Donghao Ren [Opens in a new window] ,
Laura R. Marusich Open the ORCID record for Laura R. Marusich [Opens in a new window] ,
John O’Donovan ,
Jonathan Z. Bakdash Open the ORCID record for Jonathan Z. Bakdash [Opens in a new window] ,
James A. Schaffer ,
Daniel N. Cassenti ,
Sue E. Kase ,
Heather E. Roy ,
Wan-yi (Sabrina) Lin  and
Tobias Höllerer Open the ORCID record for Tobias Höllerer [Opens in a new window]
Donghao Ren*
Affiliation:
University of California, Santa Barbara, CA 93106, USA
Laura R. Marusich
Affiliation:
U.S. Army Research Laboratory South at the University of Texas at Arlington, TX 76019, USA (e-mail: laura.m.cooper20.civ@mail.mil)
John O’Donovan
Affiliation:
University of California, Santa Barbara, CA 93106, USA (e-mail: jod@cs.ucsb.edu)
Jonathan Z. Bakdash
Affiliation:
U.S. Army Research Laboratory South at the University of Texas at Dallas, TX 75080, USA (e-mail: jonathan.z.bakdash.civ@mail.mil)
James A. Schaffer
Affiliation:
Sysco Labs, Sysco Corporation, Houston, TX 77077, USA (e-mail: j.au.schaffer@gmail.com)
Daniel N. Cassenti
Affiliation:
U.S. Army Research Laboratory, Adelphi, MD 20783, USA (e-mail: daniel.n.cassenti.civ@mail.mil)
Sue E. Kase
Affiliation:
U.S. Army Research Laboratory, Aberdeen Proving Ground, MD 21005, USA (e-mail: sue.e.kase.civ@mail.mil)
Heather E. Roy
Affiliation:
U.S. Army Research Laboratory, Aberdeen Proving Ground, MD 21005, USA (e-mail: heather.e.roy2.ctr@mail.mil)
Wan-yi (Sabrina) Lin*
Affiliation:
IBM Thomas J. Watson Research Center, Yorktown Heights, New York, USA (e-mail: sabrinal@us.ibm.com)
Tobias Höllerer
Affiliation:
University of California, Santa Barbara, CA 93106, USA (e-mail: holl@cs.ucsb.edu)
*
*Corresponding author. Email: donghaoren@cs.ucsb.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

We investigated human understanding of different network visualizations in a large-scale online experiment. Three types of network visualizations were examined: node-link and two different sorting variants of matrix representations on a representative social network of either 20 or 50 nodes. Understanding of the network was quantified using task time and accuracy metrics on questions that were derived from an established task taxonomy. The sample size in our experiment was more than an order of magnitude larger (N = 600) than in previous research, leading to high statistical power and thus more precise estimation of detailed effects. Specifically, high statistical power allowed us to consider modern interaction capabilities as part of the evaluated visualizations, and to evaluate overall learning rates as well as ambient (implicit) learning. Findings indicate that participant understanding was best for the node-link visualization, with higher accuracy and faster task times than the two matrix visualizations. Analysis of participant learning indicated a large initial difference in task time between the node-link and matrix visualizations, with matrix performance steadily approaching that of the node-link visualization over the course of the experiment. This research is reproducible as the web-based module and results have been made available at: https://osf.io/qct84/.

Keywords

information visualizationnetwork visualizationgraph data
Type
Original Article
Information
Network Science , Volume 7 , Issue 2 , June 2019 , pp. 242 - 264
Copyright
© Cambridge University Press 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

†

The author is currently affiliated with BOSCH Center for Artificial Intelligence. Email: Wan-Yi.Lin@us.bosch.com

References

Alper, B., Bach, B., Henry Riche, N., Isenberg, T., & Fekete, J.-D. (2013). Weighted graph comparison techniques for brain connectivity analysis. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 483–492). Paris, France, ACM.CrossRefGoogle Scholar
Army, U. S. (2006). Field manual 2-22.3: Human intelligence collector operations.Google Scholar
Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412.CrossRefGoogle Scholar
Baccara, M., & Bar-Isaac, H. (2008). How to organize crime. The Review of Economic Studies, 75(4), 1039–1067.CrossRefGoogle Scholar
Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7(6), 543–554.CrossRefGoogle ScholarPubMed
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48.CrossRefGoogle Scholar
Battista, G. D., Eades, P., Tamassia, R., & Tollis, I. G. (1998). Graph drawing: Algorithms for the visualization of graphs (1st ed.) Upper Saddle River, NJ, USA: Prentice Hall PTR.Google Scholar
Berardi, C. W, Solovey, E. T., & Cummings, M. L. (2013). Investigating the efficacy of network visualizations for intelligence tasks. In 2013 IEEE international conference on intelligence and security informatics (ISI) (pp. 278–283). IEEE.CrossRefGoogle Scholar
Blanchet, K., & James, P. (2011). How to do (or not to do) … a social network analysis in health systems research. Health Policy Plan, 2012 Aug; 27(5), 438–446.CrossRefGoogle Scholar
Bohannon, J. (2009). Counterterrorism’s new tool: ‘metanetwork’ analysis. Science, 325(5939), 409–411.CrossRefGoogle ScholarPubMed
Bolker, B. M., Brooks, M. E., Clark, C. J., Geange, S. W., Poulsen, J. R., Stevens, M. H. H., & White, J.-S. S. (2009). Generalized linear mixed models: a practical guide for ecology and evolution. Trends in Ecology & Evolution, 24(3), 127–135.CrossRefGoogle ScholarPubMed
Bostandjiev, S., O’Donovan, J., Hall, C., Gretarsson, B., & Hollerer, T. (2011). WiGipedia: A tool for improving structured data in wikipedia. In Proceedings of the 2011 IEEE fifth international conference on semantic computing. ICSC ’11 (pp. 328–335). Washington, DC, USA: IEEE Computer Society.CrossRefGoogle Scholar
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s mechanical turk a new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6(1), 3–5.CrossRefGoogle Scholar
Burnham, K. P., & Anderson, D. R. (2003). Model selection and multimodel inference: a practical information-theoretic approach. Springer Science & Business Media.Google Scholar
Chang, C., Bach, B., Dwyer, T., & Marriott, K. (2017). Evaluating perceptually complementary views for network exploration tasks. In Proceedings of the 2017 CHI conference on human factors in computing systems (pp. 1397–1407). Denver, Colorado, USA, ACM.CrossRefGoogle Scholar
Chang, R., Ziemkiewicz, C., Green, T. M., & Ribarsky, W. (2009). Defining insight for visual analytics. Computer Graphics and Applications, IEEE, 29(2), 14–17.CrossRefGoogle ScholarPubMed
Cohen, J., Cohen, P., & West, S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3 ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
Eysenck, M. W., & Keane, M. T. (2013). Cognitive psychology: A student’s handbook. Psychology press.CrossRefGoogle Scholar
Ghoniem, M., Fekete, J.-D., & Castagliola, P. (2004). A comparison of the readability of graphs using node-link and matrix-based representations. In Proceedings - IEEE symposium on information visualization (pp. 17–24). Austin, TX, USA, IEEE.CrossRefGoogle Scholar
Ghoniem, M., Fekete, J.-D., & Castagliola, P. (2005). On the readability of graphs using node-link and matrix-based representations: a controlled experiment and statistical analysis. Information Visualization, 4(2), 114–135.CrossRefGoogle Scholar
Gretarsson, B., O’Donovan, J., Bostandjiev, S., Höllerer, T., Asuncion, A., Newman, D., & Smyth, P. (2012). Topicnets: Visual analysis of large text corpora with topic modeling. ACM Transactions on Intelligent Systems and Technology, 3(2), 23:1–23:26.CrossRefGoogle Scholar
Hall, D. L., Graham, J., & Catherman, E. (2015). A survey of tools and resources for the next generation analyst. In Proceedings Volume 9499, Next-Generation Analyst III; 94990, Event: SPIE Sensing Technology + Applications, 2015, Baltimore, Maryland, United States.Google Scholar
Hauser, D. J, & Schwarz, N. (2015). Attentive Tturkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48 (1), 400–407.CrossRefGoogle Scholar
Henry, N., Fekete, J.-D., & McGuffin, M. J. (2007). NodeTrix: a hybrid visualization of social networks. IEEE Transactions on Visualization and Computer Graphics, 13(6), 1302–1309.CrossRefGoogle ScholarPubMed
Henry, N., & Fekete, J.-D. (2007). MatLink: Enhanced matrix visualization for analyzing social networks. In Proceedings Human-Computer Interaction – INTERACT 2007 (pp. 288–302). Berlin, Heidelberg: Springer-Verlag.CrossRefGoogle Scholar
Jaworowski, M., & Pavlak, S. (2003). Ali baba dataset ground truth. U.S. National Security Agency: Fort Meade, MD.Google Scholar
Kase, S. E., Roy, H., & Cassenti, D. N. (2015). Visualizing approaches for displaying measures of sentiment (Vol. 9499). In Proceedings Volume 9499, Next-Generation Analyst III; 94990H, Event: SPIE Sensing Technology + Applications, 2015, Baltimore, Maryland, United States.Google Scholar
Krebs, V. E. (2002). Mapping networks of terrorist cells. Connections, 24(3), 43–52.Google Scholar
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121.CrossRefGoogle ScholarPubMed
Lankow, J., Ritchie, J., & Crooks, R. (2012). Infographics: The power of visual storytelling. Wiley.Google Scholar
Lee, B., Plaisant, C., Parr, C. S., Fekete, J.-D., & Henry, N. (2006). Task taxonomy for graph visualization. In Proceedings of the 2006 AVI workshop on BEyond time and errors, May 2006, Venezia, Italy (pp. 1–5). ACM.Google Scholar
MacCalman, M., MacCalman, A., & Wilson, G. (2013). Visualizing social networks to inform tactical engagement strategies that will influence the human domain. Small Wars Journal, 9(8).Google Scholar
McGrath, C., Blythe, J., & Krackhardt, D. (1997). The effect of spatial arrangement on judgments and errors in interpreting graphs. Social Networks, 19(3), 223–242.CrossRefGoogle Scholar
McIllwain, J. S. (1999). Organized crime: A social network approach. Crime, Law and Social Change, 32(4), 301–323.CrossRefGoogle Scholar
Mittrick, M., Roy, H., Kase, S., & Bowman, E. (2012) Refinement of the ali baba data set. US Army Research Laboratory, ARL-TN-0476.Google Scholar
Nakagawa, S., & Schielzeth, H. (2013). A general and simple method for obtaining R from generalized linear mixed-effects models. Methods in Ecology and Evolution, 4(2), 133–142.CrossRefGoogle Scholar
Newman, M. E. J. (2002). Spread of epidemic disease on networks. Physical Review E, 66(1), 016128.CrossRefGoogle ScholarPubMed
North, C. (2006). Toward measuring visualization insight. Computer Graphics and Applications, IEEE, 26(3), 6–9.CrossRefGoogle ScholarPubMed
Okoe, M., & Jianu, R. (2015). GraphUnit: Evaluating interactive graph visualizations using crowdsourcing. In Computer Graphics Forum (Vol. 34, pp. 451–460). Wiley Online Library.Google Scholar
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716–aac4716.CrossRefGoogle Scholar
Peng, R. D. (2011). Reproducible research in computational science. Science, 334(6060), 1226–1227.CrossRefGoogle ScholarPubMed
Purchase, H. C. (1998). Performance of layout algorithms: Comprehension, not computation. Journal of Visual Languages & Computing, 9(6), 647–657.CrossRefGoogle Scholar
Salthouse, T. A. (1996). The processing-speed theory of adult age differences in cognition. Psychological Review, 103(3), 403.CrossRefGoogle ScholarPubMed
Schaffer, J., Giridhar, P., Jones, D., Höllerer, T., Abdelzaher, T., & O’Donovan, J. (2015). Getting the message?: A study of explanation interfaces for microblog data analysis. In Proceedings of the 20th international conference on intelligent user interfaces. Atlanta, Georgia, USA, ACM.Google Scholar
Sparrow, M. K. (1991). The application of network analysis to criminal intelligence: An assessment of the prospects. Social Networks, 13(3), 251–274.CrossRefGoogle Scholar
Sullivan, P. (1987). Newspaper graphics. Darmstadt, Germany: IFRA.Google Scholar
Von Landesberger, T., Kuijper, A., Schreck, T., Kohlhammer, J., van Wijk, J. J., Fekete, J.-D., & Fellner, D. W. (2011). Visual analysis of large graphs: state-of-the-art and future research challenges. In Computer Graphics Forum (Vol. 30, pp. 1719–1749). Wiley Online Library.Google Scholar
Wong, P. C., Foote, H., Mackey, P., Perrine, K., & Chin, G. Jr. (2006). Generating graphs for visual analytics through interactive sketching. IEEE Transactions on Visualization and Computer Graphics, 12(6), 1386–1398.CrossRefGoogle ScholarPubMed
Yi, J. S., Kang, Y.-a., Stasko, J. T., & Jacko, J. A. (2008). Understanding and characterizing insights: how do people gain insights using information visualization? In Proceedings of the 2008 Workshop on BEyond time and errors: novel evaLuation methods for Information Visualization (p. 4). Florence, Italy, ACM.Google Scholar