Hostname: page-component-78c5997874-ndw9j Total loading time: 0 Render date: 2024-11-13T05:08:58.178Z Has data issue: false hasContentIssue false
  • English
  • Français

Interactive (statistical) visualisation and exploration of a billion objects with vaex

Published online by Cambridge University Press:  30 May 2017

M. A. Breddels
M. A. Breddels*
Affiliation:
Kapteyn Astronomical Institute, University of Groningen, P.O. Box 800, 9700 AV Groningen, The Netherlands email: breddels@astro.rug.nl
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

With new catalogues arriving such as the Gaia DR1, containing more than a billion objects, new methods of handling and visualizing these data volumes are needed. We show that by calculating statistics on a regular (N-dimensional) grid, visualizations of a billion objects can be done within a second on a modern desktop computer. This is achieved using memory mapping of hdf5 files together with a simple binning algorithm, which are part of a Python library called vaex. This enables efficient exploration or large datasets interactively, making science exploration of large catalogues feasible. Vaex is a Python library and an application, which allows for interactive exploration and visualization. The motivation for developing vaex is the catalogue of the Gaia satellite, however, vaex can also be used on SPH or N-body simulations, any other (future) catalogues such as SDSS, Pan-STARRS, LSST, etc. or other tabular data. The homepage for vaex is http://vaex.astro.rug.nl.

Type
Contributed Papers
Information
Proceedings of the International Astronomical Union , Volume 12 , Symposium S325: Astroinformatics , October 2016 , pp. 299 - 304
Copyright
Copyright © International Astronomical Union 2017 

References

Gaia Collaboration, Brown, A. G. A., Vallenari, A., et al. 2016, ArXiv e-prints, arXiv:1609.04172Google Scholar
Gaia Collaboration, et al. 2016, ArXiv e-prints, arXiv:1609.04153Google Scholar
Helmi, A., & de Zeeuw, P. T. 2000, MNRAS, 319, 657 CrossRefGoogle Scholar
Hunter, J. D. 2007, Computing In Science & Engineering, 9, 90 CrossRefGoogle Scholar
Ivezic, Z., Tyson, J. A., Abel, B., et al. 2008, ArXiv e-prints, arXiv:0805.2366Google Scholar
Kaiser, N., Burgett, W., Chambers, K., et al. 2010, in Proc. SPIE, Vol. 7733, Ground-based and Airborne Telescopes III, 77330EGoogle Scholar
Lindegren, L., Lammers, U., Bastian, U., et al. 2016, ArXiv e-prints, arXiv:1609.04303Google Scholar
Pérez, F., & Granger, B. E. 2007, Computing in Science and Engineering, 9, 21 CrossRefGoogle Scholar
Perryman, M. A. C., Hassan, H., Batut, T., et al., eds. 1989, The Hipparcos mission. Pre-launch status. Volume I: The Hipparcos satellite., Vol. 1Google Scholar
Taylor, M. B. 2005, in Astronomical Society of the Pacific Conference Series, Vol. 347, Astronomical Data Analysis Software and Systems XIV, ed. Shopbell, P., Britton, M., & Ebert, R., 29Google Scholar
van Leeuwen, F., Evans, D. W., De Angeli, F., et al. 2016, A&A special Gaia volumeGoogle Scholar