7 - Floating-point number systems

Summary

Introduction

Floating-point number systems evolved from the process of expressing numbers in “scientific decimal notation,” e.g., 2.734 × 10⁶ and −5.23 × 10⁻⁴. In practice, the display of a value in scientific notation generally employed the length (precision) of the display as a measure of the accuracy of the display. Values such as α = 6.0247 × 10²³ for Avogadro's number or π = 3.14159 × 10⁰ for the natural constant π were subject to various interpretations regarding the final digit with respect to the correspondence to the presumed infinitely precise specification of the real number for which the scientific notation was an approximation. Terms such as correct, rounded, and significant were often explicitly or implicitly implied with the display.

Specifically, the digits were termed correct if they agreed with the initial digits of an infinitely precise specification, such as 3.1415 or 3.141592 for π. This interpretation is now popularly termed the round-towards-zero result. The notions of rounded and significant digits did not convey such an unambiguous interpretation. Saying that 2.734 × 10⁻⁶ was a rounded value of x did not necessarily mean that ∣x − 2.734 × 10⁻⁶∣ was guaranteed to be at most .5 × 10⁻⁹. More particularly, disregarding inherent measurement error, the rounding of the exact “midpoint” value x = 2.7345 × 10⁻⁶ to four places was subject to different interpretations. Most ambiguous was the statement that the digits 6.0247 × 10²³ were significant in representing a number such as Avogadro's constant. This probably meant only that the relative error was less than one-half part in 6024 but not necessarily less than one-half part in 60247.