Exploit the Best of Both End-to-End and Map-Based Methods for Multi-Focus Image Fusion

Multi-focus image fusion is a technique to fuse the images focused on different depth ranges to generate an all-in-focus image. Existing deep learning approaches to multi-focus image fusion can be categorized as end-to-end methods and decision map based methods. End-to-end methods can generate natur...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on multimedia 2024, Vol.26, p.6411-6423
Main Authors: Zhang, Juncheng, Liao, Qingmin, Ma, Haoyu, Xue, Jing-Hao, Yang, Wenming, Liu, Shaojun
Format: Article
Language:English
Subjects:
Algorithms
Boundaries
Computer vision
Deep learning
Feature extraction
Fuses
Image fusion
Image segmentation
Multi-focus
Transforms
Visualization
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Multi-focus image fusion is a technique to fuse the images focused on different depth ranges to generate an all-in-focus image. Existing deep learning approaches to multi-focus image fusion can be categorized as end-to-end methods and decision map based methods. End-to-end methods can generate natural fusion near the focus-defocus boundaries (FDB), but the output is often inconsistent with the input in the areas far from the boundaries (FFB). On the contrary, decision map based methods can preserve original images in the FFB areas, but often generate artifacts near the FDB. In this article, we propose a dual-branch network for multi-focus image fusion (DB-MFIF) to exploit the best of both worlds, achieving better results in both FDB and FFB areas, i.e. with naturally sharper FDB areas and more consistent FFB areas with the inputs. In our DB-MFIF, an end-to-end branch and a decision map based branch are proposed to mutually assist each other. In addition, to this end, two map-based loss functions are also proposed. Experiments show that our method surpasses existing algorithms on multiple datasets, both qualitatively and quantitatively, and achieves the state-of-the-art performance.
ISSN:1520-9210
1941-0077
DOI:10.1109/TMM.2024.3350924