Data gap decomposed by auxiliary modality for NIR‐VIS heterogeneous face recognition

作者：Sun Rui; Shan Xiaoquan; Zhang Han; Gao Jun

来源：IET Image Processing, 2021, 16(1).

In the dark scene at night, the face images captured by ordinary visible light (VIS) are generally poor quality and very dim, while the near‐infrared (NIR) can capture high definition and recognizable face images at night. The NIR‐VIS Heterogeneous face recognition has become a hot research field, which helps to build an all‐weather face recognition system. NIR‐VIS HFR is sophisticated because of the large visual difference between NIR images and VIS images. In order to reduce the difficulty of such cross‐modality invariant feature learning, this paper proposes a cross‐modality data gap decomposed by auxiliary modality method (DGD) for NIR‐VIS HFR. First, the brightness component (Y component) of VIS image YCbCr space is used as the auxiliary modality to decompose the cross‐modality data gap. The lightness component retained the structural information of VIS image and was similar to the colour information of NIR modality; in this way, the huge gap between the NIR data and the VIS data is decomposed into two smaller gaps, thus reducing the difficulty of network learning. Second, the data of the three modalities are input into the weight sharing network and training under the combined guidance of cross‐modality gap decomposition loss and intra‐modality gap loss; in this way, the modality invariant features can be learned faster and better. Extensive experiments were conducted on two commonly used datasets CASIA NIR‐VIS 2.0 and Oulu‐CASIA NIR‐VIS to evaluate DGD method. Experimental results indicate DGD method has competitive performance compared with the latest methods.

• 单位
  合肥工业大学