Code Coverage Test Case Prioritization for Regression Testing Using Non-Deterministic Approach

Table of Contents

1. INTRODUCTION

1.1 MOTIVATION

1.1.1 What Is Software Testing and Why Is It Important?

1.1.2 The Need for Regression Testing

1.2 PROBLEM STATEMENT

1.3 RESEARCH CONTRIBUTIONS

1.4 SUMMARY

2. LITERATURE REVIEW

2.1 OVERVIEW

2.2 SOFTWARE REGRESSION TESTING

2.3 RESEARCH AIMS AND OBJECTIVES

2.4 LITERATURE SURVEY

2.5 TYPES OF TESTING

2.6 TEST CASES OR TEST SUITE OPTIMIZATION METHODS

2.7 CATEGORIZATION OF TEST SUITE / TEST CASE GENERATION METHODS

2.7.1 Random Generation

2.7.2 Symbolic Execution

2.7.3 Search-Based Generation

2.7.4 Model-Based Generation

2.7.5 Heuristic-Based Techniques.

2.8 SUMMARY

3. REGRESSION TESTING

3.1 SOFTWARE DEVELOPMENT PROCESS

3.2 STRATEGIES OF REGRESSION TESTING

3.3 CATEGORIES OF REGRESSION TESTING TECHNIQUES

3.3.1 Regression Test Selection

3.3.2 Regression Test Challenges

3.4 OVERVIEW OF REGRESSION TEST SOLUTION

3.4.1 Regression Test

3.4.2 Summary

3.5 TEST CASE SELECTION

3.5.1 Similarity-Based Selection

4. PROBLEM FORMULATION

4.1 OVERVIEW

4.2 OPTIMAL TEST CASE SELECTION PROBLEM

4.3 TEST SUITE MINIMIZATION PROBLEM

4.4 TEST OPTIMIZATION PROBLEM

4.5 TEST CASE PRIORITIZATION PROBLEM

5. TEST CASE SELECTION AND GENERATION

5.1 SELECTION OF TEST CASE

5.2 TEST CASE GENERATION

5.2.1 Genetic Algorithm

5.2.2 Generate Population

5.2.3 Selection Criteria

5.2.4 Mutation

5.2.4.1 Using Detection Matrix Minimizes the Test Cases

5.2.5 Selecting Relevant Test Cases

5.2.6 Removing the Irrelevant Test Cases

5.2.7 Fitness Value Calculation

5.3 TEST CASE PRIORITIZATION

5.3.1 Priority Selection Algorithm

5.3.2 Non-Prioritized Testing

5.3.2.1 Greedy Approach

5.4 TABU SEARCH ALGORITHM

5.4.1 Tabu Score Calculation

5.4.2 Tabu Lists: The Memory of the Algorithm

5.4.2.1 Long Term List

5.4.2.2 Short Term List

5.5 CROSSOVER APPROACH

5.5.1 Crossover Algorithm

5.6 OPTIMIZED TEST CASES

6. TEST SUITE OPTIMIZATION

6.1 (RTS)-REGRESSION TEST SELECTION

6.2 THE CODE COVERAGE AS APPROXIMATION OF ADEQUACY

6.2.1 Test Coverage Process

6.3 REDUCTION OF REGRESSION TEST SUITES

6.4 TEST SUITE OPTIMIZATION

6.5 THEORY OF AUTOMATA

6.5.1 Define Of Automata

6.5.2 Why Automata

6.5.3 Finite State Diagram

6.5.4 Regular Languages

6.5.5 Deterministic - Finite Automata

6.5.5.1 Why DFA-Deterministic (DFA)

6.5.5.2 DFA State Diagram

6.5.5.3 Disadvantage of DFA

6.5.6 (NDFA) Non-deterministic Finite Automata

6.5.6.1 Nondeterministic Computation

6.5.6.2 ε-Transitions in NFA

6.5.6.3 NFA Algorithm

6.5.6.4 NFA Advantage

6.6 FINITE AUTOMATA-BASED TESTING

6.6.1 DFA-Based Testing

6.6.2 NFA-Based Testing

7. RESULT AND DISCUSSION

7.1 FITNESS VALUE COMPARISON

7.2 EVALUATED TABU SCORE

7.3 REQUIREMENT CODE COVERAGE IN TEST CASES

7.4 COMPARISON OF NFA WITH ANOTHER FRAMEWORK

8. CONCLUSION AND FUTURE WORK

8.1 CONCLUSION

8.2 FUTURE WORK

Research Objectives and Themes

The primary research objective is to develop a novel methodology for efficient regression testing of modified software while maintaining code coverage and reducing execution time. By applying a non-deterministic approach and optimizing test suite selection, the research aims to address critical challenges in software maintenance.

• Regression test suite optimization and reduction techniques.
• Genetic algorithm-based prioritization of test cases.
• Application of non-deterministic finite automata (NFA) for testing un-invoked methods.
• Methodological framework for fault detection and code coverage analysis.
• Comparative performance analysis of deterministic and non-deterministic testing models.

Excerpt from the Book

Summary of Chapters

1. INTRODUCTION: Outlines the significance of software testing and the necessity for regression testing, defining the research problem and contributions.

2. LITERATURE REVIEW: Examines existing methodologies and studies regarding regression testing, test case prioritization, and automated generation techniques.

3. REGRESSION TESTING: Details the software development life cycle, testing levels, and common strategies and challenges involved in regression testing.

4. PROBLEM FORMULATION: Defines the mathematical and technical hurdles in test case selection, suite minimization, and prioritization.

5. TEST CASE SELECTION AND GENERATION: Provides a comprehensive view of the algorithms, including Genetic Algorithms and Tabu Search, used for test case optimization.

6. TEST SUITE OPTIMIZATION: Explores formal techniques, including Automata Theory, for optimizing regression test selection and adequacy.

7. RESULT AND DISCUSSION: Presents experimental findings and comparative analyses based on fitness values and code coverage metrics.

8. CONCLUSION AND FUTURE WORK: Summarizes the effectiveness of the proposed framework and provides insights for potential future enhancements.

Keywords

Regression Testing, Software Testing, Genetic Algorithms, Tabu Search, Finite Automata, NFA, Code Coverage, Test Suite Optimization, Test Case Prioritization, Fault Detection, Software Maintenance, Mutation Analysis, Java Reflection, Heuristic-Based Techniques

Frequently Asked Questions

What is the core focus of this research?

The research focuses on enhancing the efficiency of regression testing through a non-deterministic approach, aiming to maintain software quality during modifications while reducing cost and time.

What are the primary themes discussed?

Key themes include test suite optimization, automated test case generation, prioritization methods using genetic algorithms and tabu search, and the application of automata theory in software testing.

What is the ultimate goal of the proposed method?

The primary goal is to achieve maximal fault detection and code coverage within significantly minimized test suites and shorter execution time frames.

Which scientific methods are applied?

The study employs a hybrid approach, combining Genetic Algorithms, Tabu Search, and non-deterministic finite automata for test case classification and sequence optimization.

What topics are covered in the main body?

The main body covers a literature survey on testing strategies, formulation of the optimization problem, detailed algorithms for test generation/selection, and an evaluation of these methods using specific case studies like a banking application.

Which keywords define this work?

Essential keywords are Regression Testing, Genetic Algorithms, Tabu Search, Finite Automata, Code Coverage, and Test Case Prioritization.

How does Java Reflection assist in this study?

Java Reflection is utilized to introspect the banking application to dynamically extract methods and functions, which are treated as individual test cases for the testing process.

What is the significance of the "Detection Matrix" shown in the study?

The Detection Matrix is used to identify and eliminate redundant test cases by analyzing which mutants each test case is capable of detecting, thereby streamlining the overall test suite.

How is the performance of the NFA model measured?

The NFA model's performance is measured against deterministic approaches based on execution time and code coverage metrics, demonstrating its efficiency in complex system testing.