Regularized SVM Classification with a new Complexity-Driven Stochastic Optimizer
Keywords:Supervised classification, Discriminant analysis, Support vectors, Information criteria, Feature selection, Stochastic optimization, Reproducing kernel Hilbert space
Given a multivariate dataset composed of data from different known sources or processes,Â how can we create a rule to separate the data, and classify any future data? Kernel discriminant analysisÂ is one of many supervised learning techniques that handle this problem. Recently, in this and otherÂ knowledge discovery problems, kernel methods have gained popularity. This is somewhat ironic asÂ another common theme is variable reduction, and kernel methods actually inflate dimensionality. DueÂ to the substantial benefits of processing "kernelized" data, this is excusable - kernel methods frequentlyÂ outperform traditional classification techniques for real data when the classes are not easily separable.Â In performing kernel discriminant analysis, there are two main issues that we address in this article.Â The first is that, in the literature, the question of which kernel function to use is often subjectivelyÂ selected a prior, or determined by cross-validation with the sole objective of maximizing classificationÂ performance. Secondly, after obtaining discriminant functions or support vectors to classify a dataset,Â how do we know which of our variables are most responsible for, and important to, the classification?Â In this research, we develop a new regularized algorithmthat simultaneously selects the kernel functionÂ and subset of original variables. Our algorithm, a hybrid of cross-validation and the genetic algorithm,Â does this by optimizing a function that rewards correct classification while penalizing model complexityÂ and misclassification.
We report results on three real datasets, including data from a medical imaging study. For the latter,Â we obtained an impressively low misclassification rate of 0.3%, while reducing the number of featuresÂ from p = 20 to pâˆ— = 6.
How to Cite
Upon acceptance of an article by the journal, the author(s) accept(s) the transfer of copyright of the article to European Journal of Pure and Applied Mathematics.
European Journal of Pure and Applied Mathematics will be Copyright Holder.