Least Squares Regressions with the Bootstrap

Inhaltsverzeichnis (Table of Contents)

• Introduction
• The model
  • The basic form
  • The disturbance term
• Regression techniques
  • The method of least squares
    • Ordinary Least Squares
    • Generalized Least Squares
  • Alternative regression methods
• Classical measures of performance
  • Bias
  • Variances
    • The variance of OLS
    • The variance of GLS
    • A remark on the variances
  • Confidence intervals
    • A remark on the critical values
    • A confidence interval for OLS
    • A confidence interval for GLS
  • Rate of convergence
• The bootstrap
  • How does the bootstrap work?
  • When does the bootstrap work?
  • The non-parametric bootstrap
  • The parametric bootstrap
  • Why does the bootstrap work?
  • How many bootstrap repetitions?
  • On the size of each repetition
• Regressions with the bootstrap
  • Case resampling
  • Residual resampling
  • Wild bootstrap
  • When to use which method?
• Inference with the bootstrap
  • Variances with the bootstrap
  • Confidence intervals with the bootstrap
    • The percentile interval
    • The bootstrap-t interval
    • Other bootstrap intervals
  • Convergence with the bootstrap
• Classical or bootstrap inference?
• A practical test for the bootstrap
  • The datasets
    • The homoscedastic data
    • The heteroscedatic data
  • The simulations
    • Results simulation one
    • Results simulation two
  • Resumé of simulations
• Concluding remarks

Zielsetzung und Themenschwerpunkte (Objectives and Key Themes)

This paper introduces the bootstrap method and compares its performance in regression problems with classical methods of statistical inference. The paper aims to provide a comprehensive overview of the bootstrap and its applicability in regressions, demonstrating its effectiveness in situations where traditional methods may fall short. Key themes explored in the paper include:

• The application of the bootstrap in regression problems
• A comparative analysis of the bootstrap's performance against classical inference methods
• The limitations and effectiveness of the bootstrap in different regression settings
• The impact of the bootstrap on the estimation of variances and confidence intervals
• A practical demonstration of the bootstrap's performance through simulations

Zusammenfassung der Kapitel (Chapter Summaries)

The paper begins with a brief introduction to the concept of regression in statistics and highlights the challenges associated with selecting the most appropriate regression technique. The second chapter describes the basic model used throughout the paper, outlining the underlying assumptions and concepts. Chapter 3 introduces various regression techniques, focusing on the method of least squares, including Ordinary Least Squares (OLS) and Generalized Least Squares (GLS), as well as alternative regression methods. The fourth chapter delves into classical measures of performance for regression techniques, examining bias, variance, confidence intervals, and the rate of convergence in large samples. Chapter 5 provides an introduction to the bootstrap method, explaining its principles, applications, and advantages. The next chapter, chapter 6, specifically addresses the application of the bootstrap in regression problems, detailing methods like case resampling, residual resampling, and wild bootstrap. Chapter 7 explores how inference is conducted using the bootstrap, analyzing the estimation of variances, confidence intervals, and convergence. Chapter 8 compares the performances of classical and bootstrap inference in regressions, highlighting strengths and weaknesses of each approach. Finally, chapter 9 presents a practical test of the bootstrap method using simulations, evaluating its performance in different scenarios.

Schlüsselwörter (Keywords)

The paper focuses on the key concepts of least squares regressions, the bootstrap method, and their respective strengths and limitations in statistical inference. This includes exploring topics such as:

• Ordinary Least Squares (OLS)
• Generalized Least Squares (GLS)
• Bootstrap inference
• Classical inference
• Variances and confidence intervals
• Regression techniques
• Simulations