Being ahead of time. A number of neural network simulations exploring the anticipation of clause-final heads

Inhaltsverzeichnis (Table of Contents)

• 1. Introduction
• 2. Complexity Measures
  • 2.1. Why complexity measures?
  • 2.2. Yngve
  • 2.3. Bottom-up parsing
  • 2.4. Left-corner parsing
  • 2.5. Chomsky and Miller
  • 2.6. Fodor and Garret
  • 2.7. Other complexity measures
  • 2.8. Gibson
• 3. The idea of anticipation
  • 3.1. Shannon and Weaver
  • 3.2. Entropy and anticipation
  • 3.3. SOUL
  • 3.4. New developments.
  • 3.5. My concept of anticipation .....
• 4. Empirical evidence for anticipation
• 5. Language processing and Working Memory
  • 5.1. Baddeley's model of Working Memory
  • 5.2. Individual differences: reading span
  • 5.3. The one-resource model
    • 5.3.1. King and Just (1991).
    • 5.3.2. Just and Carpenter (1992): CCREADER
  • 5.4. The two-resource model
    • 5.4.1. Waters and Caplan (1996)
    • 5.4.2. Caplan and Waters (1999)
  • 5.5. The alternative: Neural Networks
    • 5.5.1. The model by MacDonald and Christiansen (2002)
    • 5.5.2. Postscript to MacDonald and Christiansen's model
• 6. A short break: Integration – anticipation – neural networks
• 7. The simulations
  • 27.1. Introduction to Neural Network modeling
    • 7.1.1. Simple Recurrent Networks
    • 7.1.2. Training and testing the network
    • 7.1.3. Evaluating performance
    • 7.1.4. The grammars
    • 7.1.5. Details of the simulations
    • 7.1.6. Overview of the seven series
  • 7.2. First series
  • 7.3. Second series: Anticipation is modeled
  • 7.4. Third series: Even distribution of verbs
  • 7.5. Fourth series: Easy versus difficult context
    • 7.5.1. Subject and object relative clauses
    • 7.5.2. Simply- and doubly-embedded sentences
      • 7.5.2.1. With determiners
      • 7.5.2.2. Without determiners
  • 7.6. Fifth series: Positive and negative evidence.
    • 7.6.1. The Restricted GPE
    • 7.6.2. A simple grammar
    • 7.6.3. A complicated grammar.
  • 7.7. Sixth series: Early and late evidence.
    • 7.7.1. A simple grammar
    • 7.7.2. A complicated grammar
      • 7.7.2.1. Was the grammar too easy?
      • 7.7.2.2. Two effects?
  • 7.8. Seventh series: “\"pure\" anticipation
• 8. General Discussion
  • 8.1. Series 1 to 3
  • 8.2. Series 4
  • 8.3. Series 5 and 6
  • 8.4. Series 7
  • 8.5. Implications for the anticipation hypothesis .....
  • 8.6. Connection to other theories
• 9. Conclusion

Zielsetzung und Themenschwerpunkte (Objectives and Key Themes)

This paper explores the phenomenon of anticipation in language processing, specifically focusing on the anticipation of clause-final heads. The research uses neural network simulations to model and investigate this anticipatory behavior.

• Complexity measures in language processing
• The anticipation hypothesis
• Modeling language processing with Simple Recurrent Networks (SRNs)
• Empirical evidence for anticipation
• Implications of the anticipation hypothesis for language processing theories

Zusammenfassung der Kapitel (Chapter Summaries)

The paper begins by providing an overview of different complexity measures used in language processing, highlighting the limitations of existing theories and introducing the anticipation hypothesis. Chapter 3 delves into the concept of anticipation, exploring its theoretical foundations and its application in language processing.

Chapter 4 presents empirical evidence supporting the anticipation hypothesis, while Chapter 5 discusses various models of verbal working memory, leading to the introduction of neural networks as a potential alternative.

The core of the paper lies in the detailed presentation of seven simulation series conducted using Simple Recurrent Networks (SRNs). Each series explores a specific aspect of anticipation, providing insights into the mechanisms of SRNs and the nature of anticipation in language processing.

The General Discussion analyzes the results of the simulations, drawing conclusions about the effectiveness of SRNs as a model for anticipation and discussing the implications of the findings for current theories of language processing.

Schlüsselwörter (Keywords)

The main keywords and focus topics of this work include: anticipation, clause-final heads, language processing, complexity measures, Simple Recurrent Networks (SRNs), neural network simulations, verbal working memory, integration cost, Dependency Locality Theory (DLT).