Éditeur(s) : HAL CCSD IEEE
Résumé : International audience
Recent works display that large scale image classification problems rule out computationally demanding methods. On such problems, simple approaches like k-NN are affordable contenders, with still room space for statistical improvements under the algorithmic constraints. A recent work showed how to leverage k-NN to yield a formal boosting algorithm. This method, however, has numerical issues that make it not suited for large scale problems. We propose here an Adaptive Newton-Raphson scheme to leverage k-NN, N³, which does not suffer these issues. We show that it is a boosting algorithm, with several key algorithmic and statistical properties. In particular, it may be sufficient to boost a subsample to reach desired bounds for the loss at hand in the boosting framework. Experiments are provided on the SUN, and Caltech databases. They confirm that boosting a subsample -- sometimes containing few examples only -- is sufficient to reach the convergence regime of N³. Under such conditions, N³ challenges the accuracy of contenders with lower computational cost and lower memory requirement.
IEEE International Workshop on Machine Learning for Signal Processing
Southampton, United Kingdom

hal-00959125
https://hal.archives-ouvertes.fr/hal-00959125
https://hal.archives-ouvertes.fr/hal-00959125/document
https://hal.archives-ouvertes.fr/hal-00959125/file/BNNB_mlsp2013.pdf

Éditeur(s) : HAL CCSD
Résumé : A standard approach for large scale image classification involves high dimensional features and Stochastic Gradient Descent algorithm (SGD) for the minimization of classical Hinge Loss in the primal space. Although complexity of Stochastic Gradient Descent is linear with the number of samples these method suffers from slow convergence. In order to cope with this issue, we propose here a Stochastic Low-Rank Newton Descent SLND for minimization of any calibrated loss in the primal space. SLND approximates the inverse Hessian by the best low-rank approximation according to squared Frobenius norm. We provide core optimization for fast convergence. Theoretically speaking, we show explicit convergence rates of the algorithm using these calibrated losses, which in addition provide working sets of parameters for experiments. Experiments are provided on the SUN, Caltech256 and ImageNet databases, with simple, uniform and efficient ways to tune remaining SLND parameters. On each of these databases, SLND challenges the accuracy of SGD with a speed of convergence faster by order of magnitude.
https://hal.archives-ouvertes.fr/hal-00825414

hal-00825414
https://hal.archives-ouvertes.fr/hal-00825414
https://hal.archives-ouvertes.fr/hal-00825414/document
https://hal.archives-ouvertes.fr/hal-00825414/file/TechnicalReport.pdf

Éditeur(s) : HAL CCSD IEEE
Résumé : International audience
This paper proposes a novel automated approach for the categorization of cells in fluorescence microscopy images. Our supervised classification method aims at recognizing patterns of unlabeled cells based on an annotated dataset. First, the cell images need to be indexed by encoding them in a feature space. For this purpose, we propose tailored bio-inspired features relying on the distribution of contrast information. Then, a supervised learning algorithm is proposed for classifying the cells. We carried out experiments on cellular images related to the diagnosis of autoimmune diseases, testing our classification method on the HEp-2 Cells dataset of Foggia et al (CBMS 2010). Results show classification precision larger than 96% on average, thus confirming promising application of our approach to the challenging application of cellular image classification for computer-aided diagnosis.
ICPR 2012
Tsukuba, Japan

hal-00958856
https://hal.archives-ouvertes.fr/hal-00958856
https://hal.archives-ouvertes.fr/hal-00958856/document
https://hal.archives-ouvertes.fr/hal-00958856/file/icpr_2012.pdf

Éditeur(s) : HAL CCSD Institute of Electrical and Electronics Engineers
Résumé : International audience
Tailoring nearest neighbors algorithms to boosting is an important problem. Recent papers study an approach, UNN, which provably minimizes particular convex surrogates under weak assumptions. However, numerical issues make it necessary to experimentally tweak parts of the UNN algorithm, at the possible expense of the algorithm's convergence and performance. In this paper, we propose a lightweight alternative algorithm optimizing proper scoring rules from a very broad set, and establish formal convergence rates under the boosting framework that surprisingly compete with those known for UNN. It is an adaptive Newton-Raphson algorithm, which belongs to the same lineage as the popular Gentle Adaboost. To the best of our knowledge, no such boosting-compliant convergence rates were previously known for these algorithms. We provide experiments on a dozen domains, including the challenging Caltech and SUN computer vision databases. They display that GNNB significantly outperforms UNN, both in terms of convergence rate and quality of the solution obtained, and GNNB provides a simple and efficient contender to techniques that can be used on very large domains, like stochastic gradient descent -- for which little is known to date. Experiments include a divide-and-conquer improvement of GNNB which exploits the link with proper scoring rules optimization.
ISSN: 0162-8828

hal-00958809
https://hal.archives-ouvertes.fr/hal-00958809
https://hal.archives-ouvertes.fr/hal-00958809/document
https://hal.archives-ouvertes.fr/hal-00958809/file/PAMI.pdf
DOI : 10.1109/TPAMI.2014.2307877

Éditeur(s) : HAL CCSD
Résumé : International audience
It is an admitted fact that mainstream boosting algorithms like AdaBoost do not perform well to estimate class conditional probabilities. In this paper, we analyze, in the light of this problem, a recent algorithm, unn, which leverages nearest neighbors while minimizing a convex loss. Our contribution is threefold. First, we show that there exists a subclass of surrogate losses, elsewhere called balanced, whose minimization brings simple and statistically efficient estimators for Bayes posteriors. Second, we show explicit convergence rates towards these estimators for unn, for any such surrogate loss, under a Weak Learning Assumption which parallels that of classical boosting results. Third and last, we provide experiments and comparisons on synthetic and real datasets, including the challenging SUN computer vision database. Results clearly display that boosting nearest neighbors may provide highly accurate estimators, sometimes more than a hundred times more accurate than those of other contenders like support vector machines.
European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD)
Bristol, United Kingdom

hal-00702771
https://hal.archives-ouvertes.fr/hal-00702771
https://hal.archives-ouvertes.fr/hal-00702771/document
https://hal.archives-ouvertes.fr/hal-00702771/file/ecml12-dnbnb-sub.pdf

Éditeur(s) : HAL CCSD Springer
Résumé : Third International Workshop, MLMI 2012, Held in Conjunction with MICCAI 2012
International audience
Universal Nearest Neighbours (unn) is a classifier recently proposed, which can also effectively estimates the posterior probability of each classification act. This algorithm, intrinsically binary, requires the use of a decomposition method to cope with multiclass problems, thus reducing their complexity in less complex binary subtasks. Then, a reconstruction rule provides the final classification. In this paper we show that the application of unn algorithm in conjunction with a reconstruction rule based on the posterior probabilities provides a classification scheme robust among different biomedical image datasets. To this aim, we compare unn performance with those achieved by Support Vector Machine with two different kernels and by a k Nearest Neighbours classifier, and applying two different reconstruction rules for each of the aforementioned classification paradigms. The results on one private and five public biomedical datasets show satisfactory performance.
Machine Learning in Medical Imaging
Nice, France

hal-00958909
https://hal.archives-ouvertes.fr/hal-00958909
https://hal.archives-ouvertes.fr/hal-00958909/document
https://hal.archives-ouvertes.fr/hal-00958909/file/Miccai.pdf
DOI : 10.1007/978-3-642-35428-1_15