Forecast evaluation methods: A Monte Carlo investigation and an application to the predictability of interest rates

Table of Contents

1 Introduction

2 Forecast evaluation methods

2.1 Selected tests in a model-free environment

2.1.1 Forecast encompassing

2.1.2 Equal mean squared error

2.2 Selected tests in a model-based environment

2.2.1 Exemplary forecasting framework

2.2.2 Non-nested model structure

2.2.3 Nested model structure

2.2.4 Inference à la Giacomini and White (2006)

3 Monte Carlo evidence

3.1 Model-free framework

3.1.1 Size properties

3.1.2 Power properties

3.2 Model-based framework

3.2.1 Non-nested models

3.2.1.1 Size investigation

3.2.1.2 Power investigation

3.2.2 Nested models

3.2.2.1 Size investigation

3.2.2.2 Power investigation

4 Application: Predicting interest rates

4.1 Competing prediction models

4.1.1 Macro-indicator model

4.1.2 Pure time series model

4.1.3 Dynamic Svensson (1994) model

4.2 Results of pairwise comparison

4.3 Out-of-sample Granger causality

5 Conclusion

A Appendix

A.1 Fixed regressor bootstrap of Clark and McCracken (2012)

A.2 Derivations

A.3 Tables

B Bibliography

Research Objectives and Themes

This thesis examines and compares various hypothesis tests for evaluating competing forecast series, focusing on the distinction between forecast encompassing and equal mean squared error (MSE). The primary objective is to develop a unified regression-based framework for these tests and to analyze their size and power properties, particularly in small samples, through Monte Carlo simulations and an empirical application to interest rate forecasting.

• Comparison of forecast encompassing and equal MSE evaluation criteria.
• Assessment of hypothesis test performance in model-free and model-based environments.
• Investigation of small sample size and power properties using Monte Carlo evidence.
• Empirical application of predictive models for German government bond yields.
• Analysis of macroeconomic indicator predictive power and out-of-sample Granger causality.

Excerpt from the Book

Summary of Chapters

1 Introduction: Outlines the three main research issues: the link between forecast evaluation criteria, the Monte Carlo analysis of test properties, and the empirical application to interest rate models.

2 Forecast evaluation methods: Details the theoretical foundation for forecast encompassing and equal MSE tests in both model-free and model-based (nested/non-nested) environments, including the incorporation of parameter estimation uncertainty.

3 Monte Carlo evidence: Provides simulation results evaluating the size and power properties of the examined tests across different model structures and data generating processes.

4 Application: Predicting interest rates: Applies the discussed forecast evaluation methods to compare various interest rate prediction models for Germany, including macro-indicator and dynamic yield curve models.

5 Conclusion: Synthesizes the main findings regarding the performance of encompassing tests in small samples and the predictive capability of the considered interest rate models.

Keywords

Forecast evaluation, forecast encompassing, mean squared error, MSE, Monte Carlo study, small sample properties, interest rate forecasting, nested models, non-nested models, macroeconomic indicators, Granger causality, parameter estimation uncertainty, Diebold-Mariano test, forecast combination.

Frequently Asked Questions

What is the core focus of this research?

The research focuses on summarizing and comparing selected hypothesis tests used to evaluate competing forecast series, specifically linking the concepts of forecast encompassing and equal mean squared error within a regression framework.

What are the primary topics covered in this thesis?

The thesis covers forecast evaluation theory, Monte Carlo simulation evidence regarding test size and power, and an empirical application involving the prediction of German government bond yields using various models.

What is the main objective of the thesis?

The primary objective is to establish a framework that links different evaluation criteria and to demonstrate that specific modified encompassing tests offer superior size and power properties compared to traditional tests in small samples.

Which scientific methods are employed?

The research employs econometric modeling for forecast evaluation, analytical derivations of test statistics, and extensive Monte Carlo simulations to assess test properties under various data generating processes.

What does the main body address?

The main body systematically presents the theoretical framework for forecast evaluation methods, provides Monte Carlo evidence for model-free and model-based environments, and performs an empirical application to interest rate forecasting.

What keywords characterize the work?

Key terms include forecast evaluation, encompassing, mean squared error, Monte Carlo study, small sample properties, and interest rate forecasting.

How does the research address nested models?

The thesis investigates the out-of-sample Granger causality framework for nested models, noting the necessity of specialized critical values due to the violation of standard distribution assumptions.

What are the findings regarding the German bond market?

The empirical analysis suggests that simple time series models are robust benchmarks, while certain indicators like the German stock market index and the ifo expectation index provide useful predictive information for specific yield maturities.

What role does parameter estimation uncertainty play?

The work demonstrates that parameter estimation uncertainty can lead to serious size distortions in tests, necessitating robust estimation or appropriate adjustments to ensure valid statistical inference.