Hostname: page-component-cd9895bd7-dk4vv Total loading time: 0 Render date: 2024-12-26T08:18:14.695Z Has data issue: false hasContentIssue false
  • English
  • Français

A channel-based perspective on conjugate priors

Published online by Cambridge University Press:  25 February 2020

B. Jacobs
B. Jacobs*
Affiliation:
Institute for Computing and Information Sciences, Radboud University, P.O. Box 9010, 6500 GL Nijmegen, The Netherlands
*
*Email: bart@cs.ru.nl
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

A desired closure property in Bayesian probability is that an updated posterior distribution be in the same class of distributions – say Gaussians – as the prior distribution. When the updating takes place via a statistical model, one calls the class of prior distributions the ‘conjugate priors’ of the model. This paper gives (1) an abstract formulation of this notion of conjugate prior, using channels, in a graphical language, (2) a simple abstract proof that such conjugate priors yield Bayesian inversions and (3) an extension to multiple updates. The theory is illustrated with several standard examples.

Type
Paper
Information
Mathematical Structures in Computer Science , Volume 30 , Issue 1 , January 2020 , pp. 44 - 61
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s) 2020

Footnotes

†

The research leading to these results has received funding from the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013)/ERC grant agreement nr. 320571.

References

Abramsky, S. and Coecke, B. (2009). A categorical semantics of quantum protocols. In: Engesser, K., Gabbay, D. M. and Lehmann, D. (eds.) Handbook of Quantum Logic and Quantum Structures: Quantum Logic, North-Holland, Elsevier, Computer Science Press, 261–323.CrossRefGoogle Scholar
Ackerman, N., Freer, C. and Roy, D. (2011). Noncomputable conditional distributions. In: Dawar, A. and Grädel, E. (eds.) Logic in Computer Science, IEEE, Computer Science Press.Google Scholar
Alpaydin, E. (2010). Introduction to Machine Learning, 2nd edn., Cambridge, MA, MIT Press.Google Scholar
Bernardo, J. and Smith, A. (2000). Bayesian Theory. Chichester, John Wiley & Sons.Google Scholar
Bishop, C. (2006). Pattern Recognition and Machine Learning. Information Science and Statistics. Chichester, Springer.Google Scholar
Blackwell, D. (1951). Comparison of experiments. In: Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, Springer/British Computer Society, 93–102.Google Scholar
Cho, K. and Jacobs, B. (2019). Disintegration and Bayesian inversion, both abstractly and concretely. In: Mathematical Structures in Computer Science. See also arxiv.org/abs/1709.00322.Google Scholar
Cho, K., Jacobs, B., Westerbaan, A. and Westerbaan, B. (2015). An introduction to effectus theory. See arxiv.org/abs/1512.05813.Google Scholar
Clerc, F., Dahlqvist, F., Danos, V. and Garnier, I. (2017). Pointless learning. In: Esparza, J. and Murawski, A. (eds.) Foundations of Software Science and Computation Structures, Lecture Notes in Computer Science, vol. 10203, Berlin, Springer, 355–369.CrossRefGoogle Scholar
Culbertson, J. and Sturtz, K. (2014). A categorical foundation for Bayesian probability. Applied Categorical Structures 22(4) 647–662.CrossRefGoogle Scholar
Diaconis, P. and Ylvisaker, D. (1979). Conjugate priors for exponential families. Annals of Statistics 7(2) 269–281.CrossRefGoogle Scholar
Faden, A. (1985). The existence of regular conditional probabilities: Necessary and sufficient conditions. The Annals of Probability 13(1) 288–298.CrossRefGoogle Scholar
Giry, M. (1982). A categorical approach to probability theory. In: Banaschewski, B. (ed.) Categorical Aspects of Topology and Analysis, Lecture Notes in Computer Science, vol. 915, Berlin, Springer, 68–85.CrossRefGoogle Scholar
Jacobs, B. (2013). Measurable spaces and their effect logic. In: Logic in Computer Science, IEEE, Computer Science Press.Google Scholar
Jacobs, B. (2015). New directions in categorical logic, for classical, probabilistic and quantum logic. Logical Methods in Computer Science 11(3). See https://lmcs.episciences.org/1600.CrossRefGoogle Scholar
Jacobs, B. (2018). From probability monads to commutative effectuses. Journal of Logical and Algebraic Methods in Programming 94 200–237.CrossRefGoogle Scholar
Jacobs, B. and Westerbaan, A. (2015). An effect-theoretic account of Lebesgue integration. In: Ghica, D. (ed.) Mathematical Foundations of Programming Semantics, Electronic Notes in Theoretical Computer Science, vol. 319, Amsterdam, Elsevier, 239–253.Google Scholar
Jacobs, B. and Zanasi, F. (2019). The logical essentials of Bayesian reasoning. In: Barthe, G. and Katoen, J.-P. and Silva, A. Probabilistic Programming, Cambridge, Cambridge University Press. See arxiv.org/abs/1804.01193.Google Scholar
Koopman, B. (1936). On distributions admitting a sufficient statistic. Transactions of the American Mathematical Society 39 399–409.Google Scholar
Panangaden, P. (2009). Labelled Markov Processes. London, Imperial College Press.CrossRefGoogle Scholar
Russell, S. and Norvig, P. (2003). Artificial Intelligence. A Modern Approach. Englewood Cliffs, NJ, Prentice Hall.Google Scholar
Selinger, P. (2007). Dagger compact closed categories and completely positive maps (extended abstract). In: Selinger, P. (ed.) Proceedings of the 3rd International Workshop on Quantum Programming Languages (QPL 2005), Electronic Notes in Theoretical Computer Science, vol. 170, Amsterdam, Elsevier, 139–163. doi: http://dx.doi.org/10.1016/j.entcs.2006.12.018.Google Scholar
Selinger, P. (2011). A survey of graphical languages for monoidal categories. In: Coecke, B. (ed.) New Structures in Physics, Lecture Notes in Physics, vol. 813, Berlin, Springer, 289–355.Google Scholar
Stoyanov, J. (2014). Counterexamples in Probability, 2nd rev. edn. Chichester, Wiley.Google Scholar