Estimation of quantiles in a simulation model based on artificial neural networks

Table of Contents

1 Introduction and Overview

2 Nonparametric regression

3 Nonparametric quantile estimation based on surrogate models

3.1 Introduction in Surrogate Models

3.2 Order Statistics

3.2.1 Asymptotic distribution of a central order statistic

3.3 A general error bound

3.3.1 Theorem

4 Neural Networks

4.1 Introduction

4.2 Biological neural networks

4.3 Historical Background

4.3.1 McCulloch-Pitts Model

4.3.2 Perceptron

4.4 Elements of an artificial neural network

4.5 Definitions

4.5.1 Sigmoid function

4.5.2 Squashing function

4.5.3 Artificial neuron

4.5.4 Feedforward neural network with hidden layers

4.5.5 A recursive definition of multilayered feedforward neural networks

4.6 Approximation characteristics of neural networks

4.6.1 Idea

4.6.2 Lemma 3 (An approximation result)

4.6.3 Lemma 4

5 Implementation

5.1 The Quantile estimates

5.2 Test Settings

5.3 Application on Simulated Data

5.4 Backpropagation algorithm

5.4.1 Gradient descent method

5.4.2 Phases of the Backpropagation algorithm

5.4.3 Training and Test Phase

5.4.4 Implementation

5.4.5 Initialising of the weights

5.4.6 Structure of the Parameters

5.4.7 Partial Derivatives

5.5 Comparison

5.6 Discussion

6 Conclusion and Outlook

Research Objectives and Core Topics

This thesis focuses on the development of an alpha-quantile estimate for costly-to-compute functions by utilizing surrogate models based on artificial neural networks. The research aims to establish a nonparametric approach to estimate quantiles efficiently by substituting the original complex function with a neural network approximation and subsequently applying Monte Carlo methods to handle larger sample sizes.

• Nonparametric quantile estimation techniques
• Development of surrogate models using artificial neural networks
• Application of the backpropagation algorithm for function approximation
• Comparative analysis of order statistic estimates versus Monte Carlo surrogate estimates
• Empirical investigation of estimator behavior on simulated data

Excerpt from the Book

Summary of Chapters

1 Introduction and Overview: Introduces the necessity of nonparametric quantile estimation for costly functions and outlines the structure of the thesis.

2 Nonparametric regression: Provides the theoretical foundations of nonparametric regression and discusses the use of L2 risk for evaluating estimation quality.

3 Nonparametric quantile estimation based on surrogate models: Defines the core problem of surrogate modeling, introduces order statistics, and presents error bounds for these estimations.

4 Neural Networks: Details the biological inspiration, history, and structural elements of artificial neural networks, including key approximation lemmas.

5 Implementation: Describes the practical application of the proposed methods using MATLAB, including the backpropagation algorithm and simulation settings.

6 Conclusion and Outlook: Reflects on the findings, discusses limitations of the current implementation, and suggests future improvements.

Keywords

Nonparametric regression, Quantile estimation, Artificial neural networks, Surrogate models, Monte Carlo estimate, Order statistics, Backpropagation, Error bounds, Simulation models, Function approximation, Hölder continuous functions, Statistical learning, Optimization, MATLAB, Computational complexity.

Frequently Asked Questions

What is the fundamental goal of this thesis?

The thesis aims to develop an alpha-quantile estimate for complex, costly functions by using artificial neural networks as surrogate models, which are faster to evaluate.

What are the primary thematic fields covered?

The work integrates nonparametric regression, neural network theory, statistical estimation, and Monte Carlo simulation techniques.

What is the core research question?

The research investigates how to effectively estimate quantiles of costly functions by approximating them with artificial neural networks and how the error of these estimates behaves.

Which scientific methods are applied?

The study employs nonparametric statistical estimation, neural network approximation theory, gradient descent-based backpropagation, and simulation-based validation.

What is covered in the main part of the work?

The main body covers theoretical error bounds for surrogate models, the architecture of feedforward neural networks, the implementation of backpropagation for weight optimization, and a comparative performance study on simulated data.

Which keywords characterize this work?

Key terms include Nonparametric regression, Quantile estimation, Artificial neural networks, Surrogate models, and Monte Carlo methods.

How does the backpropagation algorithm specifically optimize the neural network?

It uses a gradient descent method to iteratively minimize the cost function (MSE) by calculating partial derivatives of the weights and adjusting them in the direction of the negative gradient.

Why are "surrogate models" used in the context of costly computations?

They allow for the construction of an approximate function (mn) that is significantly cheaper to evaluate, enabling the generation of large samples that would be impossible with the original function (m).

What role do "order statistics" play in this research?

They provide the basis for the plug-in estimate of the alpha-quantile, which is then compared against the neural network-based Monte Carlo surrogate estimate.