Learning by Scaling

Learning theory is the mathematical foundation of machine learning. In this book we are interested in learning multi-scale structures of data and functions from examples. Our main goal is to give solid mathematical foundations to three families of concrete learning algorithms generated by means of the scaling operator. The main motivation for introducing scaling to learning theory is to learn function features or information with different frequency components when the scaling parameter changes, as done for signal processing or image compression in wavelet analysis. Since the scaling parameter varies, learning algorithms provide richer information but the analysis becomes more involved. In this book, we mainly consider three algorithms including classification with varying Gaussian kernels, Parzen windows and moving least-square methods. The analysis should help shed some light on multi-task learning and sparsity learning from a wavelet point of view. The learning theory issues and mathematical analysis conducted for the learning algorithms addressed in this book should be useful and helpful to students and researchers interested in learning theory.