Infrared-Visible Image Fusion Method Based on Differential Feature Enhancement and Cross-Modal Attention

Abstract

Abstract. To address the issues of insufficient inter-modal information interaction in dual-stream encoders, inadequate deep feature fusion, and loss function failure in extreme environments like strong light in existing autoencoder-based infrared and visible image fusion methods, this paper proposes a fusion method called DFECF (Differential Feature Enhancement and Cross-Modal Information Fusion). This method adopts an end-to-end architecture consisting of "Dual-Stream Encoder - Cross-Modal Fusion - Transformer Global Perception - Decoder Reconstruction". A differential feature enhancement module is embedded in the encoder to achieve feature enhancement through inter-modal difference information and an attention mechanism. A cross-modal feature fusion module is designed to complete the adaptive integration of deep features. A Transformer module is introduced to supplement the global feature perception capability. Additionally, a joint loss function of "gradient loss + pixel loss + auxiliary loss" is constructed to improve robustness in extreme environments. Experiments are carried out on the TNO and MSRS datasets. The results show that DFECF achieves superior performance with MI=3.85 and Qabf=0.68, outperforming state-of-the-art methods by 15.2% and 8.7% respectively. On the TNO dataset, DFECF also outperforms 8 comparison methods such as FusionGAN and DenseFuse in both subjective visual effects and objective metrics. It can still generate fusion images with clear textures in areas affected by strong light interference, demonstrating good practicability and generalization.