TY - JOUR

T1 - Universal consistency and rates of convergence of multiclass prototype algorithms in metric spaces

AU - Györfi, László

AU - Weiss, Roi

N1 - Publisher Copyright:
© 2021 Laszlo Gyorfi and Roi Weiss.

PY - 2021/7/1

Y1 - 2021/7/1

N2 - We study universal consistency and convergence rates of simple nearest-neighbor prototype rules for the problem of multiclass classification in metric spaces. We first show that a novel data-dependent partitioning rule, named Proto-NN, is universally consistent in any metric space that admits a universally consistent rule. Proto-NN is a significant simplification of OptiNet, a recently proposed compression-based algorithm that, to date, was the only algorithm known to be universally consistent in such a general setting. Practically, Proto-NN is simpler to implement and enjoys reduced computational complexity. We then proceed to study convergence rates of the excess error probability. We first obtain rates for the standard k-NN rule under a margin condition and a new generalized- Lipschitz condition. The latter is an extension of a recently proposed modified-Lipschitz condition from Rd to metric spaces. Similarly to the modified-Lipschitz condition, the new condition avoids any boundness assumptions on the data distribution. While obtaining rates for Proto-NN is left open, we show that a second prototype rule that hybridizes between k-NN and Proto-NN achieves the same rates as k-NN while enjoying similar computational advantages as Proto-NN. However, as k-NN, this hybrid rule is not consistent in general.

AB - We study universal consistency and convergence rates of simple nearest-neighbor prototype rules for the problem of multiclass classification in metric spaces. We first show that a novel data-dependent partitioning rule, named Proto-NN, is universally consistent in any metric space that admits a universally consistent rule. Proto-NN is a significant simplification of OptiNet, a recently proposed compression-based algorithm that, to date, was the only algorithm known to be universally consistent in such a general setting. Practically, Proto-NN is simpler to implement and enjoys reduced computational complexity. We then proceed to study convergence rates of the excess error probability. We first obtain rates for the standard k-NN rule under a margin condition and a new generalized- Lipschitz condition. The latter is an extension of a recently proposed modified-Lipschitz condition from Rd to metric spaces. Similarly to the modified-Lipschitz condition, the new condition avoids any boundness assumptions on the data distribution. While obtaining rates for Proto-NN is left open, we show that a second prototype rule that hybridizes between k-NN and Proto-NN achieves the same rates as k-NN while enjoying similar computational advantages as Proto-NN. However, as k-NN, this hybrid rule is not consistent in general.

KW - Error probability

KW - K-nearest-neighbor rule

KW - Metric space

KW - Multiclass classification

KW - Prototype nearest-neighbor rule

KW - Rate of convergence

KW - Universal consistency

UR - http://www.scopus.com/inward/record.url?scp=85112480500&partnerID=8YFLogxK

M3 - ???researchoutput.researchoutputtypes.contributiontojournal.article???

AN - SCOPUS:85112480500

SN - 1532-4435

VL - 22

SP - 1

EP - 25

JO - Journal of Machine Learning Research

JF - Journal of Machine Learning Research

ER -