Rule Extraction from Support Vector Machine

Table of Contents

Chapter 1: Introduction to Rule Extraction

1.1 Rule Extraction: Motivation

1.2 Rule Extraction: Significance

1.2.1 Provision of User Explanation Capability

1.2.2 Transparency

1.2.3 Data Exploration

1.3 Rule Extraction: A Taxonomy

1.3.1 Decompositional Approaches

1.3.2 Pedagogical Approaches

1.3.3 Eclectic/Hybrid Approaches

1.4 Rule Quality Criteria

1.5 Thesis Overview

1.6 Datasets Used

1.7 Experimental Setup

1.8 Thesis Outline

Chapter 2: Rule Extraction from SVM: An Introduction

2.1 Rule Extraction from SVM

2.1.1 Decompositional Approaches

2.1.2 Pedagogical Approaches

2.1.3 Eclectic/Hybrid Approaches

2.2 Gaps Identified in Literature

2.3 Outline of the Proposed Approaches

Chapter 3: Fuzzy Rules Extraction using SVM for solving Bankruptcy prediction in banks problem

3.1 Motivation

3.2 Proposed Fuzzy Rule Extraction Technique

3.2.1 Extraction of Support Vectors from SVM

3.2.2 Rule Generation

3.3 Literature Review of Bankruptcy Prediction in Banks and Firms

3.4 Results and Discussions

3.5 Conclusions

Chapter 4: Rule Extraction from SVM using Feature Selection for Solving Classification and Regression Problems

4.1 Motivation

4.2 Proposed Rule Extraction Approach

4.2.1 Feature Selection using SVM-RFE

4.2.2 Building SVM/SVR models

4.2.3 Rule Generation

4.3 Problems Analyzed

4.3.1 Auto MPG Dataset

4.3.2 Body Fat Dataset

4.3.3 Boston Housing Dataset

4.3.4 Forest Fires Dataset

4.3.5 Pollution Dataset

4.4 Results and Discussions

4.4.1 Classification Problems

4.4.2 Regression Problems

4.4.3 Overall Observations

4.5 Conclusions

Chapter 5: Rule Extraction from SVM for Data Mining on Unbalanced Datasets

5.1 Motivation

5.2 Customer Relationship Management (CRM)

5.3 Churn Prediction Problem

5.4 Proposed Eclectic/Hybrid Rule Extraction Approach

5.4.1 Feature Selection using SVM-RFE

5.4.2 Support Vector Extraction using SVM

5.4.3 Rule Generation using NBTree

5.5 Dataset Description

5.6 Data Imbalance Problem

5.6.1 Literature Review of Techniques Dealing with Unbalanced Data

5.6.2 Random Under-Sampling

5.6.3 Random Over-Sampling

5.6.4 SMOTE (Synthetic Minority Over-sampling Technique)

5.7 Results and Discussions

5.8 Conclusions

Chapter 6: Modified Active Learning Based Approach for Rule Extraction from SVM

6.1 Motivation

6.2 Modified Active Learning Based Approach for Rule Extraction

6.2.1 Feature Selection Phase

6.2.2 Active Learning Phase

6.2.3 Rule Generation Phase

6.3 Finance Application Analyzed

6.3.1 Fraud Detection in Automobile Insurance Dataset

6.3.2 Pre-Processing

6.3.3 Literature Survey of Fraud Detection Problems

6.4 Results and Discussions

6.4.1 Churn Prediction using SVM+NBTree

6.4.2 Insurance Fraud Detection using SVM+NBTree

6.4.3 Churn Prediction using SVM+DT

6.4.4 Insurance Fraud Detection using SVM+DT

6.4.5 Overall Observations

6.5 Conclusions

Chapter 7: Rule Extraction from SVR for Solving Regression Problems

7.1 Motivation

7.2 Proposed Eclectic/Hybrid Rule Extraction Technique

7.2.1 Extraction of Support Vectors and SVR Predictions

7.2.2 Rule Generation

7.3 Problems Analyzed

7.4 Results and Discussions

7.5 Conclusions

Chapter 8: Overall Conclusions and Future Directions

Objective & Research Scope

This thesis addresses the "black box" limitation of Support Vector Machines (SVMs) by proposing novel algorithms for rule extraction. The primary objective is to convert opaque, highly accurate SVM models into transparent, comprehensible models (e.g., if-then rules, decision trees) to provide better interpretability, particularly for domain-specific applications like bankruptcy prediction, customer churn, and insurance fraud detection.

• Systematic literature review on rule extraction from SVM and identification of research gaps.
• Development of decompositional, pedagogical, and eclectic/hybrid rule extraction methodologies.
• Application of proposed techniques to real-world finance and benchmark datasets.
• Implementation of feature selection (SVM-RFE) to improve rule quality and comprehensibility.
• Evaluation using metrics such as accuracy, sensitivity, specificity, fidelity, and AUC.

Auszug aus dem Buch

Summary of Chapters

Chapter 1: Introduction to Rule Extraction: Discusses the need for model transparency in machine learning, introduces the taxonomy of rule extraction approaches, and outlines the thesis goals.

Chapter 2: Rule Extraction from SVM: An Introduction: Provides a comprehensive literature review and categorizes SVM rule extraction into decompositional, pedagogical, and eclectic/hybrid groups.

Chapter 3: Fuzzy Rules Extraction using SVM for solving Bankruptcy prediction in banks problem: Presents a decompositional approach to extract fuzzy if-then rules for bank bankruptcy prediction.

Chapter 4: Rule Extraction from SVM using Feature Selection for Solving Classification and Regression Problems: Explores a pedagogical approach using SVM-RFE for feature selection to improve rule quality for both classification and regression.

Chapter 5: Rule Extraction from SVM for Data Mining on Unbalanced Datasets: Introduces an eclectic approach specifically for handling medium-scale, unbalanced data in customer churn prediction.

Chapter 6: Modified Active Learning Based Approach for Rule Extraction from SVM: Proposes a modified active learning method (MALBA) to improve rule extraction performance in unbalanced finance applications.

Chapter 7: Rule Extraction from SVR for Solving Regression Problems: Extends the eclectic rule extraction framework to Support Vector Regression (SVR) models.

Chapter 8: Overall Conclusions and Future Directions: Summarizes the thesis findings and suggests potential future research areas such as incremental SVM learning.

Keywords

Support Vector Machines, Rule Extraction, Bankruptcy Prediction, Customer Churn, Insurance Fraud, Data Mining, Unbalanced Datasets, Feature Selection, SVM-RFE, Fuzzy Logic, Decision Trees, NBTree, Machine Learning, Interpretability, Regression Problems.

Frequently Asked Questions

What is the primary motivation for this research?

The main motivation is that SVMs, while highly accurate, act as "black box" models. This makes them difficult to understand in domains where transparency is crucial, such as bankruptcy prediction and fraud detection.

What are the main approaches used for rule extraction?

The thesis utilizes three primary categories: decompositional (intertwined with internal model workings), pedagogical (treating the model as a black box), and eclectic/hybrid (a combination of both).

What is the primary objective of this work?

The goal is to develop algorithms that extract comprehensible, symbolic, and high-fidelity descriptions from trained SVM and SVR models without compromising their predictive power.

Which scientific methods are applied in this thesis?

The work employs several techniques including Fuzzy Rule Based Systems (FRBS), Decision Trees (DT), CART, ANFIS, DENFIS, and Naive-Bayes Trees (NBTree), alongside SVM-RFE for feature selection.

What topics are covered in the main section of the thesis?

The thesis covers bankruptcy prediction in banks, customer churn prediction in analytical CRM, insurance fraud detection, and general regression analysis using extracted rules.

Which keywords best characterize this work?

The key themes include Support Vector Machines, rule extraction, finance domain applications, data imbalance, and model transparency.

How does the author handle unbalanced datasets?

The research employs several balancing techniques, including random undersampling, random oversampling, combined undersampling/oversampling, and the SMOTE technique.

What role does feature selection play in this research?

Feature selection using SVM-RFE is used to reduce data dimensionality, which helps in generating smaller, more interpretable rule sets and improving model performance.