Introduction to Probability Theory

R. M. Dudley

doi:10.1017/CBO9780511755347.009

8 - Introduction to Probability Theory

Published online by Cambridge University Press: 06 July 2010

R. M. Dudley

Show author details

R. M. Dudley: Affiliation:
Massachusetts Institute of Technology

Book contents

Get access

Summary

Probabilities are easiest to define on finite sets. For example, consider a toss of a fair coin. Here “fair” means that heads and tails are equally likely. The situation may be represented by a set with two points H and T where H= “heads” and T= “tails.” The total probability of all possible outcomes is set equal to 1. Let “P(…)” denote “the probability of.…” If two possible outcomes cannot both happen, then one assumes that their probabilities add. Thus P(H) + P(T) = 1. By assumption P(H) = P(T), so P(H) = P(T)= 1/2.

Now suppose the coin is tossed twice. There are then four possible outcomes of the two tosses: HH, HT, TH, and TT. Considering these four as equally likely, they must each have probability 1/4. Likewise, if the coin is tossed n times, we have 2n possible strings of n letters H and T, where each string has probability 1/2n.

Next let n go to infinity. Then we have all possible infinite sequences of H's and T's. Each individual sequence has probability 0, but this does not determine the probabilities of other interesting sets of possible outcomes, as it did when n was finite. To consider such sets, first let us replace H by 1 and T by 0, precede the sequence by a “binary point” (as in decimal point), and regard the sequence as a binary expansion.

Type: Chapter
Information: Real Analysis and Probability , pp. 250 - 281

DOI: https://doi.org/10.1017/CBO9780511755347.009 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2002

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Andersen, Eric Sparre, and Jessen, Borge (1946). Some limit theorems on integrals in an abstract set.Danske Vid. Selsk. Mat.-Fys. Medd. 22, no. 14. 29 pp.Google Scholar

Andersen, Eric Sparre, and Jessen, Borge (1948). On the introduction of measures in infinite product sets.Danske Vid. Selsk. Mat.-Fys. Medd. 25, no. 4. 8 pp.Google Scholar

Banach, Stefan (1923). Sur le problème de la mesure.Fundamenta Mathematicae 4: 7–33CrossRef Google Scholar

Barone, Jack, and Novikoff, Albert (1978). A history of the axiomatic formulation of probability from Borel to Kolmogorov: Part I.Arch. Hist. Exact Sci. 18: 123-190Google Scholar

Benzi, Margherita (1988). A “neoclassical probability theorist:” Francesco Paolo Cantelli (in Italian). Historia Math. 15: 53–72CrossRef

Bernoulli, Jakob (1713, posth.). Ars Conjectandi. Thurnisiorum, Basel. Repr. in Die Werke von Jakob Bernoulli 3 (1975), pp. 107–286. Birkhäuser, Basel

Bienaymé, Irenée-Jules (1853). Considérations a l'appui de la découverte de Laplace sur la loi de probabilité dans la méthode des moindres carrés. C.R. Acad. Sci. Paris 37: 309–324. Repr. in J. math. pures appl. (Ser. 2) 12 (1867): 158–276

Billingsley, Patrick (1965). Ergodic Theory and Information. Wiley, New York

Bingham, N. H. (2000). Studies in the history of probability and statistics XⅬVI. Measure into probability: From Lebesgue to Kolmogorov.Biometrika 87: 145–156CrossRef Google Scholar

Birkhoff, George D. (1932). Proof of the ergodic theorem. Proc. Nat. Acad. Sci. USA 17: 656–660

Boltzmann, Ludwig (1887). Ueber die mechanischen Analogien des zweiten Hauptsatzes der Thermodynamik.J. für die reine und angew. Math. 100: 201-212Google Scholar

Borel, Émile (1903). Contribution à 1'analyse arithmétique du continu.J. math. pures appl. (Ser. 2) 9: 329–375Google Scholar

Borel, Émile (1909). Les probabilités dénombrables et leurs applications arithmétiques.Rendiconti Circolo Mat. Palermo 27: 247–271CrossRef Google Scholar

Brunel, Antoine (1963). Sur un lemme ergodique voisin du lemme de E. Hopf, et sur une de ses applications. C. R. Acad. Sci. Paris 256: 5481–5484

Brush, Stephen G. (1976). The Kind of Motion We Call Heat. 2 vols. North-Holland, Amsterdam

*Buffon, Georges L. L. (1733). Histoire de l' Académie. Paris

*Buffon, Georges L. L. (1778). X Essai d'Arithmétique Morale. In Histoire Naturelle, pp. 67–216. Paris

*Cantelli, Francesco Paolo (1917a). Sulla probabilità come limite della frequenza.Accad. Lincei, Roma, Cl. Sci. Fis., Mat., Nat., Rendiconti (Ser. 5) 26: 39–45Google Scholar

*Cantelli, Francesco Paolo (1917b). Su due applicazione di un teorema di G. Boole alla statistica matematica.Ibid., pp. 295–302Google Scholar

Chacon, Rafael V. (1961). On the ergodic theorem without assumption of positivity.Bull. Amer. Math. Soc. 67: 186–190CrossRef Google Scholar

Chacon, Rafael V. and Ornstein, Donald S. (1960). A general ergodic theorem.Illinois J. Math. 4: 153–160Google Scholar

Chebyshev, Pafnuti Lvovich (1867). Des valeurs moyennes. J. math. pures appl. 12 (1867): 177–184. Transl. from Mat. Sbornik 2 (1867): 1–9. Repr. in Oeuvres de P. L. Tchebychef 1, pp. 687–694. Acad. Sci. St. Petersburg (1899–1907)

Chung, Kai Lai (1974). A Course in Probability Theory. 2d ed. Academic Press, New York

Cohn, Harry (1972). On the Borel-Cantelli Lemma. Israel J. Math. 12: 11–16CrossRef

Daniell, Percy J. (1919). Integrals in an infinite number of dimensions.Ann. Math. 20: 281–288CrossRef Google Scholar

Day, Mahlon M. (1942). Ergodic theorems for Abelian semi-groups.Trans. Amer. Math. Soc. 51: 399–412Google Scholar

Dubins, Lester E., and Leonard Jimmie Savage (1965). How to Gamble If You Must; Inequalities for Stochastic Processes. McGraw-Hill, New York

Dugac, P. (1976). Notes et documents sur la vie et l'oeuvre de René Baire.Arch. Hist. Exact Sci. 15: 297–383CrossRef Google Scholar

Dunford, Nelson, and Schwartz, Jacob T. (1956). Convergence almost everywhere of operator averages.J. Rat. Mech. Anal. (Indiana Univ.) 5: 129–178Google Scholar

Ehrenfeucht, Andrzej, and Fisz, Marek (1960). A necessary and sufficient condition for the validity of the weak law of large numbers.Bull. Acad. Polon. Sci. Ser. Math. Astron. Phys. 8: 583–585Google Scholar

*Erdös, Paul, and Rényi, Alfréd (1959). On Cantor's series with divergent Σ 1\qn.Ann. Univ. Sci. Budapest. Eötvös Sect. Math. 2: 93–109Google Scholar

Etemadi, Nasrollah (1981). An elementary proof of the strong law of large numbers.Z. Wahrsch. verw. Geb. 55: 119–122CrossRef Google Scholar

Feller, William (1968). An Introduction to Probability Theory and Its Applications. Vol. 1, 3d ed. Wiley, New York

Garsia, Adriano M. (1965). A simple proof of E. Hopf's maximal ergodic theorem.J. Math. and Mech. (Indiana Univ.) 14: 381–382Google Scholar

Garsia, Adriano M. (1967). More about the maximal ergodic lemma of Brunel. Proc. Nat. Acad. Sci. USA 57: 21–24CrossRef

Gnedenko, Boris V. (1973). Andrei Nikolaevich Kolmogorov (On his 70th birthday). Russian Math. Surveys 28, no. 5: 5–17. Transl. from Uspekhi Mat. Nauk 28, no. 5: 5–15CrossRef

Gnedenko, Boris V. and Andrei Nikolaevich Kolmogorov (1949). Limit Distributions for Sums of Independent Random Variables. Translated, annotated, and revised by Kai Lai Chung, with appendices by Joseph L. Doob and Pao Lo Hsu. Addison-Wesley, Reading, Mass. 1st ed. 1954, 2d ed. 1968

Gnedenko, Boris V. and Smirnov, N. V. (1963). On the work of A. N. Kolmogorov in the theory of probability.Theory Probability Appls. 8: 157–164CrossRef Google Scholar

Godwin, H. J. (1955). On generalizations of Tchebychef's inequality.J. Amer. Statist. Assoc. 50: 923–945CrossRef Google Scholar

Godwin, H. J. (1964). Inequalities on Distribution Functions. Griffin, London

Greenleaf, Frederick P. (1969). Invariant Means on Topological Groups and their Applications. Van Nostrand, New York

ter Haar, D. (1954). Elements of Statistical Mechanics. Rinehart, New York

Halmos, Paul R. (1953). Lectures on Ergodic Theory. Math. Soc. of Japan, Tokyo

Hausdorff, Felix (1914). Grundzüge der Mengenlehre, 1st ed. Von Veit, Leipzig, repr. Chelsea, New York, 1949

Hewitt, Edwin, and Savage, Leonard Jimmie (1955). Symmetric measures on Cartesian products.Trans. Amer. Math. Soc. 80: 470–501CrossRef Google Scholar

Heyde, Christopher C., and Eugene Seneta (1977). I. J. Bienaymé: Statistical Theory Anticipated. Springer, New YorkCrossRef

Hopf, Eberhard (1937). Ergodentheorie. Springer, Berlin

Hopf, Eberhard (1954). The general temporally discrete Markoff process.J. Rat. Mech. Anal. (Indiana Univ.) 3: 13–45Google Scholar

Ionescu Tulcea, Cassius (1949–1950). Mesures dans les espaces produits.Atti Accad. Naz. Lincei Rend. Cl. Sci. Fis. Mat. Nat. (Ser. 8) 7: 208–211Google Scholar

Jacobs, Konrad (1963). Lecture Notes on Ergodic Theory. Aarhus Universitet, Matematisk Institut

Jessen, Borge (1939). Abstrakt Maalog Integralteori 4. Mat. Tidsskrift (B) 1939, pp. 7–21Google Scholar

Jordan, Kàroly (1972). Chapters on the Classical Calculus of Probability. Akadémiai Kiadó, Budapest

Kakutani, Shizuo (1943). Notes on infinite product measures, I. Proc. Imp. Acad. Tokyo (became Japan Academy Proceedings)19: 148–151CrossRef

Khinchin, A. Ya. (1933). Zu Birkhoffs Lösung des Ergodenproblems.Math. Ann. 107: 485–488CrossRef Google Scholar

Kolmogorov, Andrei Nikolaevich (1929a). Bemerkungen zu meiner Arbeit “Über die Summen zufälliger Grössen.”Math. Ann. 102: 484–488CrossRef Google Scholar

Kolmogorov, Andrei Nikolaevich (1929b). The general theory of measure and the calculus of probability (in Russian). In Coll. Works, Math. Sect. (Communist Acad., Sect. Nat. Exact Sci.) 1, 8–21. Izd. Komm. Akad., Moscow. Repr. in Kolmogorov (1986), pp. 48–58

Kolmogorov, Andrei Nikolaevich (1930). Sur la loi forte des grandes nombres. Comptes Rendus Acad. Sci. Paris 191: 910–912

Kolmogorov, Andrei Nikolaevich (1933). Grundbegriffe der Wahrscheinlichkeitsrechnung, Ergebnisse der Math., Springer, Berlin. English transl. Foundations of the Theory of Probability, Chelsea, New York, 1956

Kolmogorov, Andrei Nikolaevich (1986). Probability Theory and Mathematical Statistics [in Russian; selected works], ed. Yu. V. Prohorov. Nauka, Moscow

Łomnicki, Z., and Ulam, Stanisław (1934). Sur la théorie de la mesure dans les espaces combinatoires et son application au calcul des probabilités: I. Variables indépendantes.Fund. Math. 23: 237–278CrossRef Google Scholar

*Markov, Andrei Andreevich (1899). The law of large numbers and the method of least squares (in Russian). Izv. Fiz.-Mat. Obshch. Kazan Univ. (Ser. 2) 8: 110–128

Móri, T. F., and , G. J. Székely (1983). On the Erdös-Rényi generalization of the Borel-Cantelli lemma.Stud. Sci. Math. Hungar. 18: 173–182Google Scholar

Neumann, Johann (1929). Zur allgemeinen Theorie des Masses.Fund. Math. 13: 73–116CrossRef Google Scholar

von Neumann, Johann (1935). Functional Operators. Mimeographed lecture notes. Institute for Advanced Study, Princeton, N. J. Published in Ann. Math. Studies no. 21, Functional Operators, vol. I, Measures and Integrals. Princeton University Press, 1950

Notices, Amer. Math. Soc. 28 (1981, p. 84; unsigned). 1980 Wolf Prize

Ornstein, Donald S., D. J. Rudolph, and B. Weiss (1982). Equivalence of measure preserving transformations. Amer. Math. Soc. Memoirs 262CrossRef

Plancherel, Michel (1912). Sur l'incompatibilité de l'hypothèse ergodique et des équations d'Hamilton.Archives sci. phys. nat. (Ser. 4) 33: 254–255Google Scholar

Plancherel, Michel (1913). Beweis der Unmöglichkeit ergodischer mechanischer Systeme.Ann. Phys. (Ser. 4) 42: 1061–1063CrossRef Google Scholar

Révész, Pal (1968). The Laws of Large Numbers. Academic Press, New York

Rosenthal, Arthur (1913). Beweis der Unmöglichkeit ergodischer Gassysteme.Ann. Phys. (Ser. 4) 42: 796–806CrossRef Google Scholar

Savage, I. Richard (1961). Probability inequalities of the Tchebycheff type.J. Research Nat. Bur. Standards 65B, pp. 211–222CrossRef Google Scholar

Seneta, Eugene (1992). On the history of the strong law of large numbers and Boole's inequality.Historia Mathematica 19: 24–39CrossRef Google Scholar

Sheynin, O. B. (1968). On the early history of the law of large numbers.Biometrika 55, pp. 459–467Google Scholar

Shiryayev, A. N. (1989). Kolmogorov: Life and creative activities.Ann. Probab. 17: 866–944CrossRef Google Scholar

The Times [London, unsigned] (26 October 1987). Andrei Nikolaevich Kolmogorov: 1903–1987. Repr. in Inst. Math. Statist. Bulletin 16: 324–325

Ulam, Stanisław (1932). Zum Massbegriffe in Produkträumen.Proc. International Cong. of Mathematicians (Zürich), 2, pp. 118–119Google Scholar

Yosida, Kosaku, and Shizuo Kakutani (1939). Birkhoff's ergodic theorem and the maximal ergodic theorem. Proc. Imp. Acad. (Tokyo)15: 165–168CrossRef

Youschkevitch [Yushkevich], Alexander A. (1974). Markov, Andrei Andreevich.Dictionary of Scientific Biography, 9, pp. 124–130Google Scholar

Youschkevitch, A. P. (1971). Chebyshev, Pafnuti Lvovich.Dictionary of Scientific Biography, 3, pp. 222–232Google Scholar