Current Projects

Face Retrieval

Face retrieval is an enabling technology for many applications, including automatic face annotation, deduplication, and surveillance. In this project, we aim to build a face retrieval system which combines a k-nearest neighbor search procedure with a commercial-off-the-shelf (COTS) matcher in a cascaded manner (shown in Fig. 1). In particular, given a query face image, we first pre-filter the gallery set and find the top-k most similar faces for the query image by using deep facial features that are learned offline with a deep convolutional neural network. The top-k most similar faces are then re-ranked based on score-level fusion of the similarities between deep features and the COTS matcher.

Fig.1 (a) Comparing the query image with gallery images one by one, versus (b) our approach for finding a small set of similar faces which are then re-ranked by fusion with a COTS face matcher.

The proposed retrieval system can address performance and scalability simultaneously. The retrieval system is evaluated on a large-scale web face database (5 million images). The gallery set is constructed with four different web face databases (Pubfig, LFW, WLFDB, and WebFace). All overlapping subjects are removed. The proposed system is compared with two commercial face matchers, and achieves the best performance.

Pubfig[1]	LFW[2]	WLFDB[3]	Web Faces
Fig. 2 Example images from the four databases used in this work.

Fig. 3 Experimental results: Left figure shows the mAP with various gallery sizes. COTS1 and COTS2 are two state-of-the-art commercial matchers. Deep feature is the deep learning-based feature representation used in our work. The proposed scheme DF->COTS2@10K achieves the best performance, in which the top-10K most similar faces are filtered by deep features, then re-ranked by fusing deep features with commercial matcher COTS2.

Relevant Publication(s)

1. Dayong Wang and A. K. Jain, "Face Retriever: Pre-filtering the Gallery via Deep Neural Net", ICB, Phuket, Thailand, May 19-22, 2015. [pdf]

References

[1] Neeraj Kumar, Alexander C. Berg, Peter N. Belhumeur, and Shree K. Nayar, "Attribute and Simile Classifiers for Face Verification,", International Conference on Computer Vision (ICCV), 2009.[web page]

[2] Gary B. Huang, Manu Ramesh, Tamara Berg, and Erik Learned-Miller. "Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments", University of Massachusetts, Amherst, Technical Report 07-49, October, 2007. [web page]

[3] Wang, Dayong, Hoi, Steven C. H., He, Ying, Zhu, Jianke, Mei, Tao and Luo, Jiebo, Retrieval-Based Face Annotation by Weak Label Regularized Local Coordinate Coding, IEEE Trans. Pattern Anal. Mach. Intell., Vol. 36, No. 3, p.550–563, mar, 2014.[web page]