According to neuropsychology, the human face is believed to be the most important means for human communication among all social communication instruments. Of the information that the human face carries, the identity information is the most valuable. By recognizing the idenity of a person, we feel comfortable with their familiar face, sometimes uncomfortable with unfamiliar ones as when a baby cries when a strange face shows up, and recall our memories what we have talked with the person, which brings rich backgrounds and context information for smooth conversation.
Facial expressions provide an important means of non-verbal comunication in human communcation. They reflect the internal emotional state of the individual including a smile (happy), a frown (worry), a glare (anger), a snarl (anger), a stare (curiosity), a grimace (pain), ecstasy, anger, fear, doubt, confidence and threat. Besides these, facial gestures provide other non-verbal communication in human communications. By their nodding, shaking, dozzing, or winking, we can understand the intention or status of the other person. Sometimes, we express a sad or frustrated feeling by just looking down or looking up at the sky without moving the face for a while.
Since Takeo Kanade began to develop a face recognition algorithm 30 years ago, many researchers have performed research of automated face analysis extensively and achieved a great number of remarkable accomplishments. Thanks to their endeavour, the automated face analysis technique has been widely applied to many fields such as biometircs, security and surveillance systems, clinical psychology and psychiatry, biomedical applications, human computer interaction (HCI), human robot interaction (HRI), lip reading (to assist the speech recognition in noisy environments), low bit rate video coding of the face for telecommunication, avatar systems, and entertainment.
This book was written to meet three primary requirements. First, we want to put together all related subjects such as face and eye detection, face modeling, face tracking, face recognition, facial expression recognition, and facial gesture recognition into this book. Second, we provide the introductory theoretical backgrounds for each subject, which help the reader’s understanding greatly. Third, beyond the faces, we add some other subjects such as the human detection, hand gesture recognition, and body gesture recognition, which are closely related to the automated face analysis.
The book is intended for students and practioners who plan to work in the automated face analysis field or who want to become familiar with state-of-the-art automated face analysis. This book provides plenty of references for scientists and engineers working in related research fields such as image processing, computer vision, biometrics, security, computer graphics, human-computer interaction (HCI), human-robots interaction (HRI), or the computer game industry. We have also provided the summary of many current databases and demonstration systems; of face, eye, facial expression, and gesture, in Appnedix A and B, respectively, which are essential for doing the related researches. The material fits a variety of categories such as advanced tutorial, guide to the current technology, and state-of-the-art survey.
The book consists of eight chapters, covering all the major components and subareas required to understand the emerging technologies and research in automated face analysis. Each chapter focuses on a specific part of the automated face analysis, introduces the required theoretical background, reviews the related up-to-date techniques, presents the experimental results, and points out challenges and future research directions.
Chapter I presents the roles of the face in human communication and the goals of the automated face analysis.
Chapter II presents the face and detection including the theorectial background such as the AdaBoost learning technique and the modified census transform (MCT). We emphasize how to improve the face and eye detection performance by introducing the face certainty map and the MCT. We also introduce the face disguise discrimination technique using the AdaBoost learning of eye/non-eye and/or mouth/non-mouth.
Chapter III presents face modeling including the theoretical background such as the active shape models (ASMs) and the active appearance models (AAMs). We emphasize how to make the face modeling robust by introducing stereo-based AAM, view-basee AAM, and a unified gradient-based approach for combining ASM into AAM.
Chapter IV presents face tracking including the theoretical background such as particle filters, the cylindrical head model (CHM), and the incremental PCA (IPCA). We emphasize how to make the face tracking robust to the changes of background, pose, illumination, and moving speed by introducing: background-robust face tracking using AAM and ACM, pose-robust face tracking using AAM and CHM, illumination-robust face tracking using AAM and IPCA, and fast and robust face tracking using AAM in particle filter framework.
Chapter V presents face recognition including the theoretical background such as the mixture models, the embedded hidden Markov models (HMMs), the local feature analysis (LFA), the tensor analysis, and the 3D morphable models (3D MM). We present a variety of face recognition methods using the mixture model, the embedded HMM, the LFA, the tensor-based AAM, and the 3D MM, and compare the pros and cons of each face recognition method.
Chapter VI presents the facial expression recognition including the theoretical background such as the generalized discriminant analysis (GDA), the bilinear models, and the relative expression image. We present the facial expression recognition using the AAM features and the GDA, and the natural facial expression recognition using the differential-AAM and the manifold learning. Also, we present the facial expression synthesis using the AAM and the bilinear model.
Chapter VII presents the facial gesture recognition including the theoretical background such as the hidden Markov model (HMM). We present how to recognize facial gestures like nodding, denying, and blinking by combining CHM and HMM and apply it to control a TV set.
Chapter VIII presents the human detection, the hand gesture recognition, and the body gesture recognition beyond the automated face analysis, including the theoretical background such as the scale adaptive filters (SAFs) and the iterative closest point (ICP) algorithm. We present a variety of the human motion analysis techniques such as the human detection using the pose-robust SAFs, the hand gesture rcognition using MEI and MHI, the 2D body gesture recognition using the forward spotting accumulative HMMs, and the 3D body gesture recognition using 3D articulated human body model.