Navigating in the Dark

Abstract

This article introduces the snap hook methodology, a method used notably in astrochemistry as a way to indirectly validate and assess the accuracy of computational calculations in the absence of experimental or observational data. We argue that this methodology has tremendous potential for all computationally intensive scientific fields as a substitute for traditional verification and validation standards when those are not accessible and estimating the reliability of numerical predictions becomes a real difficulty. The goal of this article is to give to this method, which seems to be implicitly relied upon in many areas, a proper formulation, in order for philosophers of science to enter the debate and to highlight its undeniable potential in terms of interdisciplinary facilitation and knowledge transmission.

1. Introduction

Astrochemistry studies the formation and destruction of molecules in the interstellar medium (ISM). As a young science, it is characterized by an accelerating influx of new observations, made possible by the development of high-resolution observational facilities: 297 molecules have been detected in the ISM since 1937, ¹ 74 during the last two years only. As possible probes of the physical conditions of the environment hosting them, molecules can offer irreplaceable insights from an astrophysical point of view but can do so only if the observed molecular spectra are interpreted on the basis of complex theoretical calculations that require important computational resources and methodological innovations. The latter include approximate methods like statistical approaches, to replace computationally expensive exact quantum calculations. The performance and impact of these approximations are not always well established, given how difficult it is to perform experimental measures reproducing the extreme conditions observed in the ISM. ² To complicate the problem even further, the lack of experimental results often overlaps with the impossibility of cross-checking numerical programs to assess their different strengths and weaknesses, as in most cases, only one numerical method is available for a given system. This epistemic scenario makes it challenging to validate theoretical results ³ and thus to assess the reliability of astrophysical inferences made on their basis. This produces an interesting epistemic situation in which astrochemists produce data “in the dark,” that is, without knowing if and when experimental measures will be performed or whether numerical methods will be developed that will allow the evaluation of their results’ accuracy—a situation becoming increasingly common in computationally intensive sciences.

In this article, we describe a method, to which we refer as the snap hook methodology (SHM), implicitly used by astrochemists to validate their methodologies in the absence of experimental results. It consists of a three-step procedure: after a computational methodology has been traditionally validated and secured for a well-understood system, the latter is used as a “validation proxy”—that is, a snap hook—for partially validating the methodology used for a more complicated target, on the basis of structural similarities considered sufficient to warrant extrapolation. Finally, the target is embedded into a web of partially validated systems that together complete the puzzle and permit full validation. As progress is made, the network is extended to new target and proxy systems that together strengthen and reinforce this notion of validation.

Our intention in this article is not to defend or criticize this method but to initiate a broader discussion among scientists in fields facing sparse or lacking empirical data. This method could be instrumental in computationally heavy sciences facing similar epistemic challenges and should thus be given a general formulation. Here is the trick, however: if not discussed from the point of view of a specific case study, the notion of validation upon which it relies can quickly become an empty shell. Thus we chose to write a technical article, considering a minimal level of technicality necessary to do justice to the method, while at the same time trying to ensure that its general features emerge clearly from the discussion.

In section 2, we introduce astrochemistry and what constitutes a “methodology” herein. We then explore how validating the results for a well-known system can be extended through the SHM to validate the methodology applied to systems without available experimental results. Section 3 delves into the specifics of our case study, focusing on validating collisional rate coefficients for the CO₂-He and CCS-He systems. We conclude by highlighting the philosophical significance of the SHM, including its potential for interdisciplinary facilitation and knowledge transmission. The philosophical literature on verification and validation (V&V) has grown rapidly over the last years but has focused primarily on the challenges posed by the complexity of computational models (Lenhard and Winsberg 2010; Lenhard 2018, 2019; Jebeile and Ardourel 2019). Analyzing what scientists do when studying systems without clear validation domains is an area to which philosophers of science must and can significantly contribute.

2. Understanding the collisional excitation of interstellar molecules

2.1. Why do we need collisional data at all?

Astrophysical media, such as molecular clouds, are extremely difficult, if not impossible, to probe: they are too far, too wide, and usually at very extreme conditions of temperature and density. Besides rare exceptions (i.e., a few nearby comets and asteroids that represent a tiny part of the possible astrophysical richness), their chemical composition can be characterized only by analyzing the light spectrum unique to each molecule registered by telescopes.

Molecules offer unprecedented insights into the physical conditions characterizing their environment, making them powerful tools for understanding, for example, star formation or the evolution of molecular clouds. Theoretical calculations play a crucial role in inferring such information from the spectra, but they do so by modeling the population of energy levels, influenced by radiative and collisional processes. Radiative processes, corresponding to the spontaneous emission or absorption of photons, are well understood. But obtaining accurate inelastic rate coefficients for the collisional processes that characterize how a molecule can be (de)excited by a partner is extremely challenging. Only 69 of these rate coefficients have been calculated for detected molecules, ⁴ and even those tend to be incomplete because of multiple possible collisional partners (the dominant astrophysical species He, H, H₂, and e⁻) and the temperature range that must be explored.

This is explained by the fact that exact rate quantum calculations are often not reachable in terms of computational memory and time, which can go from hundreds of CPU hours to millions of hours for large systems with big colliders. Thus a tractable methodology with approximations is necessary, but the impact of these approximations needs to be quantified to determine the extent to which astrophysical inferences drawn from these can be trusted.

2.2. Some vocabulary: What do astrochemists call a “methodology”?

Some vocabulary is first necessary: calculations of rate coefficients require numerical methods, based on quantum chemistry. Such methods are implemented into numerical programs through a code, meaning that several programs can implement a unique method through different code. By methodology, we refer to the ensemble of steps, requiring multiple numerical programs, that permits calculation of inelastic rate coefficients, as developed in the following pages.

The first step consists in calculating the interaction potential energy between the colliders to obtain a potential energy surface (PES). This involves choosing a computational method, implemented in a quantum chemistry program like molpro. ⁵ Among the possible methods available, the coupled-cluster method (hereinafter CCSD[T]) is the gold standard but is expensive and as such only feasible for small systems (up to six to ten atoms). Even for this method, a number of approximations or simplifications are needed. An indispensable approximation to make the calculations tractable is the Born–Oppenheimer (BO) approximation, which permits decoupling of the electronic and nucleus motions by assuming a very small electron mass compared to that of the nuclei. Thus it ensures that the Schrödinger equation can be solved separately for each. ⁶ Likewise, a basis set, that is, a set of functions used to model the molecular orbitals, must be chosen. A realistic basis would have to include an infinite set of functions, something obviously not doable. Calculations are thus on a finite basis, ranging from double (aVDZ) to sextuple (aV6Z) perturbative excitations. Whenever possible, an empirical relationship between energies calculated for three basis sets (say, double, triple, and quadruple excitations), called the complete basis set (CBS) extrapolation, is used to mimic the interaction energy that would have been obtained with an infinite number of functions. Using the CCSD(T) method with a CBS reconstruction of the basis set represents the best of what can currently be done. Mid-bond functions, consisting of adding physically meaningless empty orbitals halfway between the colliders to mimic a bigger basis set, are another widely used trick to save computational time. Their use permits one to reach the accuracy of a given aVXZ basis set by using a much cheaper aV(X-1)Z set. Additional approximations may be needed, depending on the system’s complexity.

The second step consists in deriving a fitting formula from the ab initio points of the PES to extrapolate the energy values at short and long distances. The fit error is estimated by comparing predictions to ab initio energy values, and the root-mean-square (RMS) error, which expresses the cumulative error, indicates the reliability of the PES. Anisotropic PESs can be tricky to fit, as generating nonphysical behavior (e.g., oscillations, holes) where no ab initio points were computed. Various formulas may be attempted and additional ab initio points added, if necessary. An accurate PES is crucial for inelastic rate coefficients, so the RMS error of the fit must be well understood.

Then, the dynamics of the nuclei is studied by solving the nuclear Schrödinger equation within the theory of collisions. The “exact” (full quantum) method to solve them is the close-coupling approach, developed by Arthurs and Dalgarno (1960) for a closed-shell linear molecule in collision with an atom. Modern calculations based on this approach exhibit a typical accuracy of 20–30 percent as compared to experimental measures. This accuracy represents important progress but is only a decent minimum to make the most of recent sensitivity improvements in ground-based and space telescopes. Solving the coupled equations results in S-matrices containing all transition probabilities, for each total energy, from which cross sections are derived. Then, assuming that the velocities are thermally distributed, cross sections are averaged over these velocities, resulting finally in rate coefficients.

Our term methodology refers to this whole process, from the construction of a PES from ab initio points to its full reconstruction using an analytic fitting formula to the dynamical calculations that lead finally to rate coefficients. The “gold standard” methodology has been validated against experimental results for various systems. However, as systems become more complex, new approximations and numerical methods are needed that are increasingly difficult to validate, as experimental measures are less likely to be available. As Oberkampf and Trucano (2002) emphasized, V&V procedures are crucial to comprehensive accuracy assessment, including the identification of numerical errors. Without them, scientists are navigating in the dark, unable to determine the reliability of their numerical results. What can be done, then, to exploit the tools and means that scientists have, while minimizing the impact of those they do not have?

3. Can the CO₂-He system serve as a validation proxy for the CCS-He system?

This section shows how the SHM fills this gap, using CO₂-He and CCS-He as examples of inelastic rate coefficient calculations. We describe our first snap hook, the CO₂-He, and how it serves as a validation proxy for CCS-He before explaining how a growing network of systems can be built that completes the validation puzzle.

3.1. The snap hook CO₂-He system

Constructing the PES for the CO₂-He system required the “rigid rotor” approximation, which neglects the vibration of the molecule and fixes the internuclear distances, thus leaving only two coordinates to consider: R and θ. As CO₂ is symmetric, θ angles between 0° and 90° are equivalent to θ angles between 90° and 180° (see figure 1), allowing for the use of expensive methods on a limited number of points. This resulted in the best accuracy possible for the PES, based on CCSD(T) with a CBS extrapolation.

Figure 1. Representations of the (left) CO₂-He and (right) CCS-He collisional systems in (R, θ) coordinates.

We fitted the 260 ab initio points for this system thanks to an analytic formula based on Legendre polynomials, with an accuracy better than 1 percent. Rate coefficients were calculated using the molscat program and validated through different methods. First, the RMS error is only 0.0149 cm⁻¹, sufficient for astrochemistry scattering calculations. The PES was also validated by computing spectroscopic data, such as bound-state transition frequencies and pressure-broadening coefficients (PBCs), and comparing them to experimental data. Bound states are located within the potential well of the PES. Transition frequencies between these states are highly sensitive to the shape and depth of the well. The agreement between the seven computed and measured transition frequencies, better than 0.6 percent, validates the accuracy of the PES’s well. PBCs evaluate the accuracy of the PES at short-range distances, where it is repulsive. Their values can be computed based on the same S-matrices used for the cross section calculations and thus permit an indirect validation of the rate coefficients provided for astrophysical applications. As seen in Godard Palluet, Thibault, and Lique (2022, figure 5), PBCs measured experimentally by Deng et al. (2009) and Thibault et al. (2000) are in agreement with our theoretical values for the three targeted spectroscopic lines and validate the accuracy of the PES and collisional rate coefficients, as well as the methodology used to obtain these results. The latter is the validation that we will extend to other systems.

CO₂ is a very stable molecule, which explains why bound states and PBCs could be acquired experimentally, even at very low temperatures and densities. In addition, its symmetry induces a dimension-limited problem for the PES and the scattering calculations. Such ideal features, unfortunately, no longer apply to the CCS-He system, thus greatly increasing the complexity of the calculations.

3.2. The snap hook and its target: The CCS-He system

CCS is a highly detected molecule in various astrophysical environments and serves as an important tracer of the physical conditions and evolutionary stages of molecular clouds—an ideal target for the SHM. Its abundance, however, has been modeled using inappropriate methods, given the complexity of theoretical calculations and the fact that no experimental measure is possible. Given that CCS-He and CO₂-He share many features, the questions arise, To what extent can the methodology used for CCS-He be validated through CO₂-He, considered as a snap hook for CCS-He? Which parts of the methodology are validated that way, and which other hooks would be needed?

CCS-He (Godard Palluet and Lique 2023) and CO₂-He have many similarities. Both involve collisions with helium, a structureless atom that makes high-level-theory quantum calculations feasible. In both cases, the BO approximation and the rigid rotor approximation apply unambiguously, and the same fit formula can be used given their geometry. Given that this PES methodology has been validated for CO₂-He, in turn, it validates its use for CCS-He, meaning that the accuracy of the results can be quantified and considered understood on the basis of those obtained for the former. Such a statement requires clear differentiation between two kinds of error. Contrary to CO₂-He, CCS-He is not symmetric, with a highly anisotropic PES and a subsequent difficult fit, evidenced by the need for 1,351 points. Likewise, the computational cost was such that a smaller and less accurate aVQZ (“Q” for “quadruple excitations”) basis set was needed, without the CBS extrapolation and with additional mid-bond functions. Such differences are not negligible. However, they do not introduce new systematic errors into the calculations but only a well-defined and quantifiable loss of accuracy. Comparing PESs obtained with different basis sets is a traditional verification step whenever doable, used as a tracer to exclude gross anomalies in the PES’s behavior. ⁷ The complexity of the fit also entails a loss of accuracy, accounting for a larger RMS error of 3.51130 cm⁻¹. A quantified accuracy loss is not tantamount to using a new approximation or numerical method, the impact on the results of which is not known and which could introduce systematic errors or artificial effects that must be identified, neutralized, or quantified. As long as one is confident that the loss of accuracy is well defined and understood, the methodology used for the CCS-He PES can be considered validated through CO₂-He.

This is not the case for the dynamical part of the methodology, however. Unlike CO₂-He, CCS-He has a fine structure that requires supplementary theoretical development. ⁸ The intermediate coupling scheme (ICS), proposed by Alexander and Dagdigian (1983), offers a proper representation of the fine-structure energy levels that neither of the basic versions of molscat and hibridon, the two main numerical programs for exact quantum scattering calculations, include in their molecule–atom collision calculations. A modified and nonpublic version of molscat incorporating ICS has been reported (Lique et al. 2005), but it must be tested and validated. Thus the methodology used for CCS-He is only partially validated through CO₂, as molscat-ics remains a missing piece of the puzzle. To make the need for validating the ICS module even more pressing, CCS-He has an unusual spin splitting of its energy levels, resulting in a messy fine structure. Tracers guaranteeing that known transition rules are respected are thus more difficult to find, making detection of possible code anomalies more difficult.

Thus we had to dive into the literature to find out whether molscat-ics had been previously validated on other systems. We found five potentially relevant examples in the literature but decided to focus on systems with a similar electronic configuration and colliding with He to maintain a safe level of comparability. This left us with three options: SO-He (Lique et al. 2005), NH-He (Toboła et al. 2011), and O₂-He (Bishwakarma et al. 2016). No experimental measures exist to this day for SO-He. Experimental data exist for NH-He (Toboła et al. 2011), but significant differences to theoretical state-to-state individual collisional transitions were found, and the source of the disagreement is difficult to interpret. The BO approximation could not be appropriate for NH-He and an electronic state thus caught by mistake. But the authors also have good reason to challenge the experimental results, inasmuch as these do not satisfy well-established theoretical predictions (or “propensity rules”) used as tracers of rate coefficients for ³Σ systems. However, for O₂-He, a PES based on the CCSD(T) method with an aVTZ basis with additional mid-bond functions was constructed, dynamical calculations were performed on these grounds using molscat-ics, and the results were successfully matched to experimental differential cross sections (DCSs). This makes O₂-He an ideal secondary snap hook for the missing part of the CCS-He methodology:

In summary, the theoretical predictions of Lique for rotationally inelastic collisions between O₂ and helium have been confirmed by measurements of DCSs, which provide the most sensitive test of scattering and PESs. The O₂-He collisional data can now be used with confidence to derive the interstellar O₂ abundance from astronomical observations. (Bishwakarma et al. 2016)

Even better, studying O₂-He made us realize its extraordinary capacity as a snap hook, first because validating molscat-ics through O₂-He validates the (dynamical part of the) methodology not only of CCS-He but also of SO-He and NH-He. Moreover, this validation extends to the two systems we had previously excluded, O₂-Ar (Bop et al. 2021) and C₄-He (Bishwakarma et al. 2016). The case of C₄-He makes particularly clear what constitutes a good snap hook. Indeed, the high symmetries of O₂-He’s PES allow the exploration of a wide range of numerical methods and approximations, the varying accuracy of which can be compared and quantified (Lique, Kłos, and Hochlaf 2010). These symmetries and the existence of experimental data of high quality turn O₂-He into a “test case for generating of ³Σ molecular species in collision with rare gas” (Bishwakarma et al. 2016, 15674), thus highlighting its potential in terms of building a network of mutually validating systems (see figure 2).

Figure 2. An example of a validation network. The bottom part corresponds to the PES and fit calculations, the top part to scattering calculations. Arrows go from the snap hooks to the validated parts of the target systems.

Consider the case of C₄-He. Given that the cost of a PES strongly depends on the basis set size, attempts to circumvent this problem have notably explored the introduction of terms into the wave function ansatz that depend explicitly on the interelectronic coordinates—the so-called explicitly correlated CCSD(T)-F12 methods. A PES for C₄-He cannot be built without F12 methods, hence the idea of building different PESs for O₂-He, to systematically compare and quantify the accuracies of different basis sets in CCSD(T) and CCSD(T)-F12 methods. As the F12-based PES is in good agreement with the former, and the domain of performance of this approximation has been thoroughly analyzed, this allowed for full validation of the C₄-He methodology. One can see in figure 2 how the validation of O₂-He was thus extremely rewarding from a theoretical point of view, regarding the number of systems the methodologies of which were validated and the possible extension of the network to rare gases other than helium (e.g., argon), but also in terms of reinforcing the interconnections within the network. Experimental results for O₂-Ar and NH-He, for instance, are possible and would greatly reinforce the network connections. Note how the graph permits identifying advantageous future snap hooks and where experimental results would be most beneficial.

4. Philosophical takeaways and concluding remarks

In a nutshell, the SHM allows for indirectly validating a methodology through snap hook systems that progressively build a network of interconnected and mutually validating systems. A snap hook such as CO₂-He serves as a starting point for validating a target system such as CCS-He, the full validation of which requires an additional hook, O₂-He, thus knitting a web of systems that together will enable the validation of more and more systems with varying degrees of complexity.

One important feature of the SHM is that it assumes modularity, that is, that individual components of a methodology can be evaluated independently. This assumption has been challenged in contexts like astrophysical or climate science simulations (Lenhard and Winsberg 2010; Lenhard 2018; Jebeile and Ardourel 2019), where couplings between different parts of the model tend to undermine their individual assessments. Here modularity is not jeopardized: parts evaluated separately correspond to autonomous steps of the methodology. The question here is rather that of entrenchment, that is, the sensitivity of a step to the former, in terms of accuracy loss.

Among the important characteristics of the SHM deserving a thorough philosophical analysis, we emphasize the following three, hoping to generate broader discussion:

• Tools like the network graph of figure 2 can potentially encode a great amount of information, including (nonexhaustively) the details of the approximations, of the numerical programs and basis set used, and of the systems for which experimental measures are possible or already done. Recovering this information usually requires a tedious literature search and contacting authors for missing information. Given that choosing the systems on which astrochemists should focus is considered the most pressing challenge for the field, such tools would greatly facilitate the identification of maximally rewarding potential hooks and thus support a systematized strategy for approaching the question of identifying the systems deserving the most attention. Note that such graphs might also help to support a confidence analysis, where the degree of confidence that one can have in rate coefficients could be evaluated based on the network size and its structure. All of these tasks could definitely benefit from a philosophical perspective.

• Finding systems of common interest to experimenters and theoreticians is tricky, due to different research interests, constraints, and associated costs. Within the SHM, a strategic choice of systems validates, not a unique target, but an entire network, making the time and cost that experimenters spend much more rewarding, while at the same time providing tools to facilitate mutual understanding and thus to foster fruitful interdisciplinary collaboration.

• Finally, the notion of validation grounding the SHM might seem controversial or weakened, as based on indirect comparison with experimental results. Yet, such a notion of validation accounts for the knowledge and acquaintance with numerical tools that theoreticians develop in practice through their careers and that often remain nontransmissible. It is not rare, to say the least, to see a senior scholar predict the performance of a given method applied to a new system without necessarily being able to determine where this intuition comes from and to account for it. We argue that this “intuitive” knowledge is grounded in implicit SHM reasoning, developed through the scholar’s career but not given a concrete formulation. Making it explicit, for example, through network graphs, would constitute a remarkable opportunity to store, transmit, and exploit this knowledge to its fullest.