Facial Expression Recognition in Unconstrained Environment

Abstract

Facial expressions are the most effective form of non-verbal communication which is used to express one’s emotions. It not only expresses our expressions, but also communicates a wealth of information during social interaction. Facial expressions play an important and effective role in Human-Computer Interaction. Unlike humans, facial expression recognition is a challenging task for computers and research work in this domain can still be considered in seminal form. The prevalent facial expression techniques tend to provide accurate and impressive results for the images captured under controlled environment with a cooperative subject. However, majority of these techniques fail to provide adequate results for the images captured in uncontrolled environment. Achieving a higher classification accuracy rate for the images captured in uncontrolled environments is a major challenge being faced by researchers in this domain. There are many factors such as illumination, image orientations, noise and low-resolution, which hinder the development of facial expression recognition system in uncontrolled environment. Even the datasets prepared under the constrained environments sometimes need preprocessing before the classification stage.

The core objective of this dissertation is to analyze existing techniques and develop an effective framework that is capable of classification of human facial expressions both under constrained and unconstrained environments. In this dissertation, we have explored, investigated and proposed frameworks to deal with two of the most common, influential and challenging issues that affect the accuracy of a facial expression recognition for images captured in an uncontrolled environment, namely illumination and low-resolution. First, a novel framework for feature extraction named Weber Local Binary Image Cosine Transform has been developed which not only extracts significant features by integrating features extracted using local binary pattern and weber local descriptor but also utilizes most discriminant features by applying the frequencybased components. Low-resolution and multi-orientation facial images have been used for the classification and a significant improvement in the classification accuracy rate has been achieved with the proposed framework. The developed framework has proved to be not only reliable but also computationally efficient across multiple datasets in the presence of noise and orientations. The Proposed framework has been tested on four datasets including JAFEE, MMI, CK+ and SFEW datasets.

The second major contribution is the development of an illumination invariant technique. The proposed framework has been named Weber Local Binary Image Cosine Transform (WLBI-CT) and it advocates for the need of simultaneous contrast enhancement and brightness preservation for datasets containing real world images in unconstrained environment. The empirical results for the SFEW dataset are promising.

In order to evaluate the performance of the proposed framework, rigorous set of experiments are presented in this thesis. The empirical results meet the standard quantitative measure criteria. The comparison of our work with various other stateof-the-art techniques is also presented using various benchmarks for these factors. The results are impressive even in the case of inclusion of noise and occlusion effects.