SpectraDiff: Enhancing the Fidelity of Infrared Image Translation with Object-Aware Diffusion

Computer Vision and Image Understanding, 2026


Incheol Park, Youngwan Jin, Yagiz Nalcakan, Hyeongjin Ju, Sanghyeop Yeo, Shiho Kim

Seamless Trans-X Lab, Yonsei University
Paper Code

Abstract

Autonomous systems commonly rely on RGB cameras, which are susceptible to failure in low-light and adverse conditions. Infrared (IR) imaging provides a viable alternative by capturing thermal signatures independent of visible illumination. However, its high cost and integration complexities limit widespread adoption. To address these challenges, we introduce SpectraDiff, a diffusion-based framework that synthesizes realistic IR images by fusing RGB inputs with refined semantic segmentation. Through our RGB-Seg Object-Aware (RSOA) module, SpectraDiff learns object-specific IR intensities by leveraging object-aware features. The SpectraDiff architecture, featuring a novel Spectral Attention Block, enforces self-attention among semantically similar pixels while leveraging cross-attention with the original RGB to preserve high-frequency details. Extensive evaluations on FLIR, FMB, MFNet, IDD-AW, and RANUS demonstrate SpectraDiff's superior performance over existing methods, as measured by both perceptual (FID, LPIPS, DISTS) and fidelity (SSIM, SAM) metrics.

Method

SpectraDiff method diagram showing optional semantic segmentation, the RGB-Seg Object-Aware module, and a diffusion backbone with residual blocks and a Spectral Attention Block.

The method diagram shows how SpectraDiff turns an RGB image into an infrared image. An optional stage produces a semantic segmentation map from the RGB input. The RGB-Seg Object-Aware (RSOA) module encodes the RGB image and the segmentation so the model can learn object-specific infrared intensities. The diffusion backbone then uses residual blocks and a Spectral Attention Block, which applies self-attention among semantically similar pixels and cross-attention with the original RGB image to preserve high-frequency detail.

Qualitative Results

SpectraDiff infrared translation of an FLIR scene.
Ours (FLIR)
Ground-truth infrared image for the FLIR scene.
GT (FLIR)
SpectraDiff infrared translation of an FMB scene.
Ours (FMB)
Ground-truth infrared image for the FMB scene.
GT (FMB)
SpectraDiff infrared translation of an MFNet scene.
Ours (MFNet)
Ground-truth infrared image for an MFNet scene.
GT (MFNet)
SpectraDiff infrared translation of a second MFNet scene.
Ours (MFNet)
Ground-truth infrared image for the second MFNet scene.
GT (MFNet)


Qualitative comparisons of different models on the FLIR dataset

Qualitative comparison of infrared translation methods on the FLIR dataset in day and night scenes.
The FLIR results illustrate the effectiveness of our SpectraDiff model in capturing the thermal properties of objects, including people, across both day and night conditions, producing clearer and more detailed images than other methods.


Qualitative comparisons with different models on the FMB dataset (top two rows) and MFNet dataset (bottom two rows) for LWIR translation.

Qualitative comparison of long-wave infrared translation on the FMB dataset and the MFNet dataset.
SpectraDiff demonstrates effectiveness in capturing IR intensities and produces clearer images. In particular, our model effectively captures thermal emissive properties across various objects and scenes in thermal infrared domain. Results for NIR datasets (IDD-AW, RANUS) are available in the supplementary material.


Quantitative Results

Table comparing infrared translation methods on several datasets using SAM, FID, LPIPS, and DISTS. Numeric values are shown in the figure.

Quantitative comparison of the proposed model's performance across various infrared (IR) range datasets. The results demonstrate performance variations across different IR ranges and highlight where our model outperforms other methods based on SAM, FID, LPIPS, and DISTS metrics. The best results are shaded in green and the second-best results are shaded in yellow.

BibTeX

@article{park2026spectradiff,
title = {SpectraDiff: Enhancing the fidelity of Infrared Image Translation with object-aware diffusion},
journal = {Computer Vision and Image Understanding},
volume = {266},
pages = {104709},
year = {2026},
issn = {1077-3142},
doi = {10.1016/j.cviu.2026.104709},
url = {https://www.sciencedirect.com/science/article/pii/S1077314226000767},
author = {Incheol Park and Youngwan Jin and Nalcakan Yagiz and Hyeongjin Ju and Sanghyeop Yeo and Shiho Kim},
keywords = {Image-to-image translation, Data augmentation, Infrared imaging, Diffusion models},
}