On the PAC-Bayes Bound Calculation based on Reproducing Kernel Hilbert Space

Authors

Li Tang, School of Computer Science and Technology, Tianjin University, Weijin Rd No. 92, Nankan, Tianjin, 300072, China
Hua Yu
Xiu-Jun GONG

Author Country (or Countries)

China

Abstract

PAC-Bayes risk bound combining Bayesian theory and structure risk minimization for stochastic classifiers has been considered as a framework for deriving some of the tightest generalization bounds. A major issue for calculating the bound is the unknown prior and posterior distributions of the concept space. In this paper, we formulated the concept space as Reproducing Kernel Hilbert Space (RKHS) using the kernel method. We further demonstrated that the RKHS can be constructed using the linear combination of kernels, and the support vectors and their corresponding weights of SVM outputs describe the complexity of concept space. Therefore the calculation of PAC-Bayes bound can be simulated by sampling weights of support vectors in RKHS. The experimental results using random and Markov Chain Monte Carlo (MCMC) samplings showed that the simulation is reasonable and effective in practice.

Suggested Reviewers

N/A

Recommended Citation

Tang, Li; Yu, Hua; and GONG, Xiu-Jun (2013) "On the PAC-Bayes Bound Calculation based on Reproducing Kernel Hilbert Space," Applied Mathematics & Information Sciences: Vol. 07: Iss. 2, Article 52.
Available at: https://digitalcommons.aaru.edu.jo/amis/vol07/iss2/52