Preface
Video Data Management and Information Retrieval are very important areas of research in computer technology. Plenty of research is being done in these fields at present. These two areas are changing our lifestyles because together they cover creation, maintenance, accessing, and retrieval of video, audio, speech, and text data and information for video display. But still lots of important issues in these areas remain unresolved and further research is needed to be done for better techniques and applications.

The primary objective of the book is to combine these two related areas of research together and provide an up-to-date account of the work being done. We addressed research issues in those fields where some progress has already been made. Also, we encouraged researchers, academics, and industrial technologists to provide new and brilliant ideas on these fields that could be pursued for further research.

Section I of the book gives an introduction. We have given general introduction of the two areas, namely, video data management and information retrieval, from the very elementary level. We discussed the problems in these areas and some of the work done in these fields since the last decade.

Section II defines video data storage techniques and networking. We present a chapter that describes the design for a High-performance Data Recording Architecture (HYDRA) that can record data in real time for large-scale servers. Although digital continuous media (CM) is being used as an integral part of many applications and attempts have been made for efficient retrieval of such media for many concurrent users, not much has been done so far to implement these ideas for large-scale servers. Then a chapter introduces video data management techniques for computational augmentation of human memory, i.e., augmented memory, on wearable and ubiquitous computers used in our everyday life. In another chapter, in order to organize and manipulate vast amount of multimedia data in an efficient way, a method to summarize these digital data has been presented. Also we present a contemporary review of the various different strategies available to facilitate Very Low Bit-Rate (VLBR) coding for video communications over mobile and fixed transmission channels as well as the Internet.

Section III talks about video data security and video data synchronization and timeliness.
We describe how to present different multimedia objects on a web-based presentation system. A chapter is devoted to highlighting the biometrics technologies, which are based on video sequences, viz face, eye (iris/retina), and gait.

Section IV will present various video shot boundary detection techniques. A new robust paradigm capable of detecting scene changes on compressed MPEG video data directly has been proposed. Then an innovative shot boundary detection method using an unsupervised segmentation algorithm and the technique of object tracking based on the segmentation mask maps are presented. We also describe a histogram with soft decision using the Hue, Saturation, and Intensity (HSV) color space for effective detection of video shot boundaries.

Section V will throw light on video feature extractions. We address the issues of providing the semantic structure and generating abstraction of content in news broadcast.

Section VI covers video information retrieval techniques and presents an up-to-date overview of various video information retrieval systems. As the rapid technical advances of multimedia communication have made it possible for more and more people to enjoy videoconferences, important issues unique to personal videoconference and a comprehensive framework for indexing personal videoconference have been presented. Then we have dealt with video summarization using human facial information through face detection and recognition and also a discussion on various issues of video abstraction with a new approach to generate it.

The audience for this book would be researchers who are working in these two fields. Also researchers from other areas who could start-up in these fields could find the book useful. It could be a reference guide for researchers from other related areas as well. Reading this book can benefit undergraduate and post-graduate students who are interested in multimedia and video technology.

Chapter Highlights
Chapter 1: Video Data Management and Information Retrieval

In this chapter, we present a basic introduction of the two very important areas of research in the
domain of Information Technology, namely, video data management and video information retrieval. Both of these areas still need research efforts to seek solutions to many unresolved problems for efficient data management and information retrieval. We discuss those issues and relevant work done in these two fields during the last few years.

Chapter 2: HYDRA: High-performance Data Recording Architecture for Streaming Media

This chapter describes the design for a High-performance Data Recording Architecture (HYDRA). Presently, digital continuous media (CM) are well established as an integral part of many applications. In recent years, a considerable amount of research has focused on the efficient retrieval of such media for many concurrent users. The authors argue that scant attention has been paid to large-scale servers that can record such streams in real time. However, more and more devices produce direct digital output streams either over wired or wireless networks, and various applications are emerging to make use of them. For example, in many industrial applications, cameras now provide the means to monitor, visualize, and diagnose events. Hence, the need arises to capture and store these streams with an efficient data stream recorder that can handle both recording and playback of many streams simultaneously and provide a central repository for all data. With this chapter, the authors present the design of the HYDRA system, which uses a unified architecture that integrates multi-stream recording and retrieval in a coherent paradigm, and hence provides support for these emerging applications.

Chapter 3: Wearable and Ubiquitous Video Data Management for
Computational Augmentation of Human Memory

This chapter introduces video data management techniques for computational augmentation of human memory, i.e., augmented memory, on wearable and ubiquitous computers used in our everyday life. The ultimate goal of augmented memory is to enable users to conduct themselves using human memories and multimedia data seamlessly anywhere, anytime. In particular, a user’s viewpoint video is one of the most important triggers for recalling past events that have been experienced. We believe designing an augmented memory system is a practical issue for real-world video data management. This chapter also describes a framework for an augmented memory album system named Scene Augmented Remembrance Album (SARA). In the SARA framework, we have developed three modules for retrieving, editing, transporting, and exchanging augmented memory. Both the Residual Memory module and the I’m Here! module enable a wearer to retrieve video data that he/she wants to recall in the real world. The Ubiquitous Memories module is proposed for editing, transporting, and exchanging video data via real-world objects. Lastly, we discuss future works for the proposed framework and modules.

Chapter 4: Adaptive Summarization of Digital Video Data

As multimedia applications are rapidly spread at an ever-increasing rate, efficient and effective methodologies for organizing and manipulating these data become a necessity. One of the basic problems that such systems encounter is to find efficient ways to summarize the huge amount of data involved. In this chapter, we start by defining the problem of key frames extraction then reviewing a number of proposed techniques to accomplish that task, showing their pros and cons. After that, we describe two adaptive algorithms proposed in order to effectively select key frames from segmented video shots where both apply a two-level adaptation mechanism. These algorithms constitute the second stage of a Video Content-based Retrieval (VCR) system that has been designed at Old Dominion University. The first adaptation level is based on the size of the input video file, while the second level is performed on a shot-by-shot basis in order to account for the fact that different shots have different levels of activity. Experimental results show the efficiency and robustness of the proposed algorithms in selecting the near optimal set of key frames required, to represent each shot.

Chapter 5: Very Low Bit Rate Video Coding

This chapter presents a contemporary review of the various different strategies available to facilitate Very Low Bit Rate (VLBR) coding for video communications over mobile and fixed transmission channels and the Internet. VLBR media is typically classified as having a bit rate between 8 and 64Kbps. Techniques that are analyzed include Vector Quantization, various parametric model-based representations, the Discrete Wavelet and Cosine Transforms, and fixed and arbitrary shaped pattern-based coding. In addition to discussing the underlying theoretical principles and relevant features of each approach, the chapter also examines their benefits and disadvantages together with some of the major challenges that remain to be solved. The chapter concludes by providing some judgments on the likely focus of future research in the VLBR coding field.

Chapter 6: Video Biometrics

Biometrics is a technology of fast, user-friendly personal identification with a high level of accuracy. This chapter highlights the biometrics technologies that are based on video sequences viz face, eye (iris/retina), and gait. The basics behind the three video-based biometrics technologies are discussed along with a brief survey.

Chapter 7: Video Presentation Management

Lecture-on-Demand (LOD) multimedia presentation technologies among the network are most often used in many communications services. Examples of those applications include video-on- demand, interactive TV, and the communication tools of a distance learning system, and so on. We describe how to present different multimedia objects on a web-based presentation system. Using characterization of extended media streaming technologies, we developed a comprehensive system for advanced multimedia content production: support for recording the presentation, retrieving the content, summarizing the presentation, and customizing the representation. This approach significantly impacts and supports the multimedia presentation authoring processes in terms of methodology and commercial aspects. Using the browser with the Windows media services allows students to view live video of the teacher giving his speech, along with synchronized images of his presentation slides and all the annotations/comments. In our experience, this very approach is sufficient for use in a distance learning environment.

Chapter 8: Video Shot Boundary Detection

The increasing use of multimedia streams nowadays necessitates the development of efficient and effective methodologies for manipulating databases storing this information. Moreover, content-based access to video data requires in its first stage to parse each video stream into its building blocks. The video stream consists of a number of shots; each one of them is a sequence of frames pictured using a single camera. Switching from one camera to another indicates the transition from a shot to the next one. Therefore, the detection of these transitions, known as scene change or shot boundary detection, is the first step in any video analysis system. A number of proposed techniques for solving the problem of shot boundary detection exist, but the major criticisms of them are their inefficiency and lack of reliability. The reliability of the scene change detection stage is a very significant requirement because it is the first stage in any video retrieval system; thus, its performance has a direct impact on the performance of all other stages. On the other hand, efficiency is also crucial due to the voluminous amounts of information found in video streams.

This chapter proposes a new robust and efficient paradigm capable of detecting scene changes on compressed MPEG video data directly. This paradigm constitutes the first part of a Video Content-based Retrieval (VCR) system that has been designed at Old Dominion University. Initially, an abstract representation of the compressed video stream, known as the DC sequence, is extracted, then it is used as input to a Neural Network Module that performs the shot boundary detection task. We have studied experimentally the performance of the proposed paradigm and have achieved higher shot boundary detection and lower false alarms rates compared to other techniques. Moreover, the efficiency of the system outperforms other approaches by several times. In short, the experimental results show the superior efficiency and robustness of the proposed system in detecting shot boundaries and flashlights (sudden lighting variation due to camera flash occurrences) within video shots.

Chapter 9: Innovative Shot Boundary Detection for Video Indexing

Recently, multimedia information, especially the video data, has been made overwhelmingly accessible with the rapid advances in communication and multimedia computing technologies. Video is popular in many applications, which makes the efficient management and retrieval of the growing amount of video information very important. To meet such a demand, an effective video shot boundary detection method is necessary, which is a fundamental operation required in many multimedia applications. In this chapter, an innovative shot boundary detection method using an unsupervised segmentation algorithm and the technique of object tracking based on the segmentation mask maps is presented. A series of experiments on various types of video are performed and the experimental results show that our method can obtain object-level information of the video frames as well as accurate shot boundary detection, which are both very useful for video content indexing.

Chapter 10: A Soft-Decision Histogram from the HSV Color Space for Video Shot Detection

In this chapter, we describe a histogram with soft decision using the Hue, Saturation, and Intensity (HSV) color space for effective detection of video shot boundaries. In the histogram, we choose relative importance of hue and intensity depending on the saturation of each pixel. In traditional histograms, each pixel contributes to only one component of the histogram. However, we suggest a soft decision approach in which each pixel contributes to two components of the histogram. We have done a detailed study of the various frame-to-frame distance measures using the proposed histogram and a Red, Green, and Blue (RGB) histogram for video shot detection. The results show that the new histogram has a better shot detection performance for each of the distance measures. A web-based application has been developed for video retrieval, which is freely accessible to the interested users.

Chapter 11: News Video Indexing and Abstraction by Specific Visual Cues: MSC and News Caption

This chapter addresses the tasks of providing the semantic structure and generating the abstraction of content in broadcast news. Based on extraction of two specific visual cues -- Main Speaker Close-Up (MSC) and news caption, a hierarchy of news video index is automatically constructed for efficient access to multi-level contents. In addition, a unique MSC-based video abstraction is proposed to help satisfy the need for news preview and key persons highlighting. Experiments on news clips from MPEG-7 video content sets yield encouraging results, which prove the efficiency of our video indexing and abstraction scheme.

Chapter 12: An Overview of Video Information Retrieval Techniques

Video information retrieval is currently a very important topic of research in the area of multimedia databases. Plenty of research has been undertaken in the past decade to design efficient video information retrieval techniques from the video or multimedia databases. Although a large number of indexing and retrieval techniques has been developed, there are still no universally accepted feature extraction, indexing, and retrieval techniques available. In this chapter, we present an up-to-date overview of various video information retrieval systems. Since the volume of literature available in the field is enormous, only selected works are mentioned.

Chapter 13: A Framework for Indexing Personal Videoconference

The rapid technical advance of multimedia communication has enabled more and more people to enjoy videoconferences. Traditionally, the personal videoconference is either not recorded or only recorded as ordinary audio and video files, which only allow the linear access. Moreover, besides video and audio channels, other videoconferencing channels, including text chat, file transfer, and whiteboard, also contain valuable information. Therefore, it is not convenient to search or recall the content of videoconference from the archives. However, there exists little research on the management and automatic indexing of personal videoconferences. The existing methods for video indexing, lecture indexing, and meeting support systems cannot be applied to personal videoconference in a straightforward way. This chapter discusses important issues unique to personal videoconference and proposes a comprehensive framework for indexing personal videoconference. The framework consists of three modules: videoconference archive acquisition module, videoconference archive indexing module, and indexed videoconference accessing module. This chapter will elaborate on the design principles and implementation methodologies of each module, as well as the intra- and inter-module data and control flows. Finally, this chapter presents a subjective evaluation protocol for personal videoconference indexing.

Chapter 14: Video Abstraction

The volume of video data is significantly increasing in recent years due to the widespread use of multimedia applications in the areas of education, entertainment, business, and medicine. To handle this huge amount of data efficiently, many techniques have emerged to catalog, index, and retrieve the stored video data, namely, video boundary detection, video database indexing, and video abstraction. The topic of this chapter is Video Abstraction, which deals with short representation of an original video and helps to enable the fast browsing and retrieving of the representative contents. A general view of video abstraction, its related works, and a new approach to generate it are discussed in this chapter.

Chapter 15: Video Summarization Based on Human Face Detection and Recognition

In this chapter, we have dealt with video summarization using human facial information through the face detection and recognition. Many efforts of face detection and face recognition are introduced, based upon both theoretical and practical aspects. Also, we describe the real implementation of video summarization system based on face detection and recognition.

Acknowledgments

The editor would like to extend his thanks to all the authors who contributed to this project by submitting chapters. The credit for the success of this book goes to them.

Also sincere thanks go to all the staff of Idea Group Publishing for their valuable contributions, particularly to Mehdi Khosrow-Pour, Senior Academic Editor, Michele Rossi, Development Editor, and Carrie Skovrinskie, Office Manager.

Finally, the editor would like to thank his wife Ms. Clera Deb for her support and cooperation during the venture.

Sagarmay Deb
Editor
University of Southern Queensland, Australia