Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-27T12:46:32.491Z Has data issue: false hasContentIssue false
  • English
  • Français

Nearest neighbor classificationin infinite dimension

Published online by Cambridge University Press:  08 September 2006

Frédéric Cérou  and
Arnaud Guyader
Frédéric Cérou
Affiliation:
IRISA /INRIA Campus de Beaulieu 35042 Rennes Cédex, France. Frederic.Cerou@inria.fr
Arnaud Guyader
Affiliation:
Université de Rennes 2, Campus de Villejan, 35043 Rennes Cedex, France. Arnaud.Guyader@uhb.fr
Get access

Abstract

Let X be a random element in a metric space (F,d), and let Y be a random variable with value 0 or 1. Y iscalled the class, or the label, of X. Let (Xi,Yi)1 ≤ i ≤ n be an observed i.i.d. sample having the same law as (X,Y). The problem of classification is topredict the label of a new random element X. The k-nearestneighbor classifier is the simple following rule: look atthe k nearest neighbors of X in the trial sample and choose 0 or 1 for its labelaccording to the majority vote. When $({\cal F},d)=(\mathbb{R}^d,||.||)$ ,Stone (1977) proved in 1977 the universal consistency of this classifier:its probability of error converges to the Bayes error, whatever thedistribution of (X,Y). We show in this paper that this result is nolonger valid in general metric spaces. However, if (F,d) isseparable and if some regularity condition is assumed, then thek-nearest neighbor classifier is weakly consistent.

Keywords

Classificationconsistencynon parametric statistics.
Type
Research Article
Information
ESAIM: Probability and Statistics , Volume 10 , February 2006 , pp. 340 - 355
Copyright
© EDP Sciences, SMAI, 2006

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

C. Abraham, G. Biau and B. Cadre, On the kernel rule for function classification. submitted (2003).
G. Biau, F. Bunea and M.H. Wegkamp, On the kernel rule for function classification. IEEE Trans. Inform. Theory, to appear (2005).
T.M. Cover and P.E. Hart, Nearest neighbor pattern classification. IEEE Trans. Inform. Theory IT-13 (1967) 21–27.
S. Dabo-Niang and N. Rhomari, Nonparametric regression estimation when the regressor takes its values in a metric space, submitted (2001).
Devroye, L., On the almost everywhere convergence of nonparametric regression function estimates. Ann. Statist. 9 (1981) 1310–1319. CrossRef
Devroye, L., Györfi, L., Krzyżak, A. and Lugosi, G., On the strong universal consistency of nearest neighbor regression function estimates. Ann. Statist. 22 (1994) 1371–1385. CrossRef
L. Devroye, L. Györfi and G. Lugosi, A probabilistic theory of pattern recognition 31, Applications of Mathematics (New York). Springer-Verlag, New York (1996).
L.C. Evans and R.F. Gariepy, Measure theory and fine properties of functions. Studies in Advanced Mathematics. CRC Press, Boca Raton, FL (1992).
H. Federer, Geometric measure theory. Die Grundlehren der mathematischen Wissenschaften, Band 153. Springer-Verlag New York Inc., New York (1969).
Preiss, D., Gaussian measures and the density theorem. Comment. Math. Univ. Carolin. 22 (1981) 181–193.
D. Preiss, Dimension of metrics and differentiation of measures, in General topology and its relations to modern analysis and algebra, V (Prague, 1981), Sigma Ser. Pure Math., Heldermann, Berlin 3 (1983) 565–568.
Preiss, D. and Tišer, J., Differentiation of measures on Hilbert spaces, in Measure theory, Oberwolfach 1981 (Oberwolfach, 1981), Springer, Berlin. Lect. Notes Math. 945 (1982) 194–207. CrossRef
Stone, C.J., Consistent nonparametric regression. Ann. Statist. 5 (1977) 595–645. With discussion and a reply by the author. CrossRef