Hostname: page-component-cd9895bd7-p9bg8 Total loading time: 0 Render date: 2024-12-28T00:06:59.540Z Has data issue: false hasContentIssue false
  • English
  • Français

A Spectral Approach to Analysing Belief Propagation for 3-Colouring

Published online by Cambridge University Press:  24 March 2009

AMIN COJA-OGHLAN ,
ELCHANAN MOSSEL  and
DAN VILENCHIK
AMIN COJA-OGHLAN
Affiliation:
Laboratory for Foundations of Computer Science, University of Edinburgh, 10 Crichton Street, Edinburgh EH8 9AB, UK (e-mail: acoghlan@inf.ed.ac.uk)
ELCHANAN MOSSEL
Affiliation:
Department of Statistics, University of California at Berkeley, 367 Evans Hall, Berkeley, CA 94720-3860, USA and Faculty of Mathematics and Computer Science, The Weizmann Institute of Science, Rehovot 76100, Israel (e-mail: mossel@stat.berkeley.edu)
DAN VILENCHIK
Affiliation:
Computer Science Division, University of California Berkeley, CA 94720, USA (e-mail: vilenchi@post.tau.ac.il)
Get access Rights & Permissions [Opens in a new window]

Abstract

Belief propagation (BP) is a message-passing algorithm that computes the exact marginal distributions at every vertex of a graphical model without cycles. While BP is designed to work correctly on trees, it is routinely applied to general graphical models that may contain cycles, in which case neither convergence, nor correctness in the case of convergence is guaranteed. Nonetheless, BP has gained popularity as it seems to remain effective in many cases of interest, even when the underlying graph is ‘far’ from being a tree. However, the theoretical understanding of BP (and its new relative survey propagation) when applied to CSPs is poor.

Contributing to the rigorous understanding of BP, in this paper we relate the convergence of BP to spectral properties of the graph. This encompasses a result for random graphs with a ‘planted’ solution; thus, we obtain the first rigorous result on BP for graph colouring in the case of a complex graphical structure (as opposed to trees). In particular, the analysis shows how belief propagation breaks the symmetry between the 3! possible permutations of the colour classes.

Type
Paper
Information
Combinatorics, Probability and Computing , Volume 18 , Issue 6 , November 2009 , pp. 881 - 912
Copyright
Copyright © Cambridge University Press 2009

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1]Achlioptas, D. and Friedgut, E. (1999) A sharp threshold for k-colorability. Random Struct. Alg. 14 63–70.3.0.CO;2-7>CrossRefGoogle Scholar
[2]Alon, N. and Kahale, N. (1997) A spectral technique for coloring random 3-colorable graphs. SIAM J. Comput. 26 1733–1748.CrossRefGoogle Scholar
[3]Bilu, Y. and Linial, N. (2006) Lifts, discrepancy and nearly optimal spectral gap. Combinatorica 26 495–519.CrossRefGoogle Scholar
[4]Braunstein, A., Mézard, M., Weigt, M. and Zecchina, R. (2005) Constraint satisfaction by survey propagation. In Computational Complexity and Statistical Physics (Percus, A., Istrate, G. and Moore, C., eds), Oxford University Press.Google Scholar
[5]Braunstein, A., Mézard, M. and Zecchina, R. (2005) Survey propagation: An algorithm for satisfiability. Random Struct. Alg. 27 201–226.CrossRefGoogle Scholar
[6]Braunstein, A., Mulet, R., Pagnani, A., Weigt, M. and Zecchina, R. (2003) Polynomial iterative algorithms for coloring and analyzing random graphs. Phys. Rev. E 68 036702.CrossRefGoogle ScholarPubMed
[7]Brightwell, G. R. and Winkler, P. (2002) Random colorings of a Cayley tree. In Contemporary Combinatorics, Vol. 10 of Bolyai Society Mathematical Studies, János Bolyai Math. Soc., pp. 247–276.Google Scholar
[8]Chung, F. and Graham, R. (2002) Sparse quasi-random graphs. Combinatorica 22 217–244.CrossRefGoogle Scholar
[9]Feige, U., Mossel, E. and Vilenchik, D. (2006) Complete convergence of message passing algorithms for some satisfiability problems. In Random 2006, Vol. 4110 of Lecture Notes in Computer Science, pp. 339–350.CrossRefGoogle Scholar
[10]Janson, S., Łuczak, T. and Ruciński, A. (2000) Random Graphs, Wiley.CrossRefGoogle Scholar
[11]Jonasson, J. (2002) Uniqueness of uniform random colorings of regular trees. Statist. Probab. Lett. 57 243–248.CrossRefGoogle Scholar
[12]Krzakala, F., Montanari, A., Ricci-Tersenghi, F., Semerjianc, G. and Zdeborova, L. (2007) Gibbs states and the set of solutions of random constraint satisfaction problems. Proc. Nat. Acad. Sci. 104 10318–10323.CrossRefGoogle ScholarPubMed
[13]Luby, M., Mitzenmacher, M., Shokrollahi, M. A. and Spielman, D. (1998) Analysis of low density parity check codes and improved designs using irregular graphs. In Proc. 30th ACM Symposium on the Theory of Computing, pp. 249–258.Google Scholar
[14]Luby, M., Mitzenmacher, M., Shokrollahi, M. A. and Spielman, D. (2001) Efficient erasure correcting codes. IEEE Trans. Inform. Theory 47 569–584.CrossRefGoogle Scholar
[15]Maneva, E., Mossel, E. and Wainwright, M. (2005) A new look at survey propagation and its generalizations. In Proc. 16th ACM–SIAM Symposium on Discrete Algorithms, pp. 1089–1098.Google Scholar
[16]Pearl, J. (1988) Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann, San Francisco, CA.Google Scholar
[17]Richardson, T., Shokrollahi, A. and Urbanke, R. (2001) Design of capacity-approaching irregular low-density parity check codes. IEEE Trans. Inform. Theory 47 619–637.CrossRefGoogle Scholar
[18]Tatikonda, S. and Jordan, M. I. (2002) Loopy belief propagation and Gibbs measures. In Uncertainty in Artificial Intelligence (UAI): Proc. 18th Conference.Google Scholar
[19]Weitz, D. (2006) Counting independent sets up to the tree threshold. In Proc. 38th Annual ACM Symposium on the Theory of Computing, pp. 140–149.CrossRefGoogle Scholar
[20]Yamamoto, M. and Watanabe, O. (2007) Belief propagation and spectral methods. Report C–248, Department of Mathematical and Computing Sciences, Tokyo Institute of Technology.Google Scholar