Current Projects

Heterogeneous Face Recognition

Example images from each of the four heterogeneous face recognition scenarios tested in our studies. The top row contains probe images from (a) near-infrared, (b) thermal infrared, (c) viewed sketch, and (d) forensic sketch modalities. The bottom row contains the corresponding gallery photograph (visible band face image) of the same subject.

(a)	(b)	(c)	(d)

A key asset in identification using face recognition technology is the extensive collection of face databases (photographs). The source of these databases range from ID card images (e.g. driver license and passport), visa applications (e.g. U.S. VISIT), and criminal mug shot photographs. Being populated with visible light photographs, commercial-of-the-shelf (COTS) face recognition systems (FRS) used to search these databases expects incoming visible light probe (query) photographs. However, many military and law enforcement scenarios exist where a probe face image is only available in an alternate sensing modality. Consider the following examples: (i) when acquiring a face image in environments with unfavorable illumination (such as night time) or in covert military and intelligence scenarios, infrared imaging must be used to capture a subjectв ¬ !" s face, and (ii) when no opportunity exists to acquire a face image, a forensic sketch must be drawn from a verbal description provide by a witness to a crime. In each of these scenarios the probe image will be from an alternate modality (infrared or sketch) than the images in the gallery (visible light photograph). This identification task, where the probe and gallery images are from different modalities, is called heterogeneous face recognition (HFR). Since only limited data is available on face images in no-visible spectrum (such as a database of infrared face images), accurate HFR systems are vital to enable identification in many critical situations. A COTS FRS performs poorly in heterogeneous face recognition. As a result, specialized matching systems must be developed to enable identification in these sensitive scenarios. This project aims at developing both generic solutions to multiple heterogeneous face recognition scenarios, as well as tailored solutions to specific HFR scenarios.

B. Klare and A.K. Jain, "Heterogeneous Face Recognition using Kernel Prototype Similarities", IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012 (To Appear).

H. Han, B. Klare, K. Bonnen, and A. K. Jain, "Matching Composite Sketches to Face Photos: A Component-Based Approach", IEEE Transactions on Information Forensics and Security, 2012 (To Appear).

Z. Lei, S. Liao, A. K. Jain, and S. Z. Li, "Coupled Discriminant Analysis for Heterogeneous Face Recognition", IEEE Transactions on Information Forensics and Security, Vol. 7, No. 6, pp. 1707-1716, December 2012.

Z. Lei, C. Zhou, D. Yi, A. K. Jain, and S. Z. Li, "An Improved Coupled Spectral Regression for Heterogeneous Face Recognition", ICB, New Delhi, India, March 29-April 1, 2012.

A. K. Jain, and B. Klare, "Matching Forensic Sketches and Mug Shots to Apprehend Criminals", IEEE Computer, Vol.44, No. 5, pp. 94-96, May 2011.

B. Klare, Z. Li, and A. K. Jain, "Matching Forensic Sketches to Mugshot Photos", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, No. 3, pp. 639-646, March, 2011.

B. Klare and A. K. Jain, "Heterogeneous Face Recognition: Matching NIR to Visible Light Images", ICPR, Istanbul, Turkey, August 23-26, 2010.

B. Klare and A. K. Jain, "Sketch to Photo Matching: A Feature-based Approach",Proc of SPIE, Biometric Technology for Human Identification VII