Semantic Annotation of View-based Object Representations Using Online Search Engines

Inhaltsverzeichnis (Table of Contents)

• 1 Introduction
• 2 Fundamentals
  • 2.1 Semantic Image Annotation
  • 2.2 The Semantic Gap Problem
  • 2.3 Object Representations in Semantic Object Recognition Systems
  • 2.4 Image Comparison Methods
  • 2.5 Google’s Reverse Image Search Engine
• 3 Related Works
• 4 The Semantic Annotation System
  • 4.1 Overview
  • 4.2 The Online-based Annotation System
    • 4.2.1 Processing of Websites and Tag Extraction
    • 4.2.2 The Score Value
    • 4.2.3 Alternate Tags
    • 4.2.4 Substrings in URL
    • 4.2.5 Plural to Singular Conversion
    • 4.2.6 Words from Similar Semantic Context
  • 4.3 The Offline-based Annotation System
    • 4.3.1 Transformations
    • 4.3.2 Invariant Features
    • 4.3.3 Invariant Feature Histogram
    • 4.3.4 Jensen-Shannon Divergence
    • 4.3.5 The Offline Database
• 5 Implementation
  • 5.1 Robot Operating System
  • 5.2 Libcurl
  • 5.3 WordNet
  • 5.4 Flexible Image Retrieval Engine
• 6 Testing and Evaluation
  • 6.1 Testing Environment and Notation
  • 6.2 The Online-based Annotation System
  • 6.3 The Offline-based Annotation System
  • 6.4 Combining the Offline and Online Annotation Engine
• 7 Summary and Outlook
  • 7.1 Summary
  • 7.2 Outlook

Zielsetzung und Themenschwerpunkte (Objectives and Key Themes)

This master's thesis develops and evaluates an automatic image annotation algorithm for object recognition systems, focusing on leveraging online databases to address the semantic gap problem. The goal is to automatically label images with textual tags representing semantic information without relying on large, manually-annotated local databases.

• Automatic semantic image annotation
• Addressing the semantic gap problem
• Utilizing online search engines for image annotation
• Comparison of online and offline-based annotation approaches
• Evaluation of the system's performance on diverse objects

Zusammenfassung der Kapitel (Chapter Summaries)

Chapter 2: Fundamentals introduces content-based image retrieval (CBIR) and image annotation, focusing on the semantic gap problem and various image representation methods. It details image comparison methods and introduces Google's Reverse Image Search Engine, a key component of the developed system.

Chapter 3: Related Works reviews existing automatic image annotation systems, highlighting their approaches and limitations, particularly concerning the use of large local databases.

Chapter 4: The Semantic Annotation System presents the developed system, which combines online and offline annotation engines. The online engine uses Google's Reverse Image Search to find visually similar images and extracts tags from associated web texts. The offline engine utilizes a small local database for improved accuracy on specific objects.

Chapter 5: Implementation details the system's implementation using ROS, Libcurl, WordNet, and FIRE, explaining their roles in the annotation process.

Chapter 6: Testing and Evaluation describes the testing environment and presents the evaluation results of both online and offline engines, separately and combined. Different parameters and modules are tested to optimize performance.

Schlüsselwörter (Keywords)

Semantic image annotation, semantic gap, content-based image retrieval (CBIR), online search engines, object recognition, Google Reverse Image Search, offline database, invariant feature histograms, Jensen-Shannon divergence, Robot Operating System (ROS).