Is Bigger Always Better? An Empirical Study on Efficient Architectures for Style Transfer and Beyond

Network architecture plays a pivotal role in style transfer. Most existing algorithms use VGG19 as the feature extractor, which incurs a high computational cost. In this work, we conduct an empirical study on the popular network architectures and find that some more efficient networks can replace VG...

Full description

Saved in:
Bibliographic Details
Main Authors: An, Jie, Li, Tao, Huang, Haozhi, Ma, Jinwen, Luo, Jiebo
Format: Conference Proceeding
Language:English
Subjects:
Algorithms: Computational photography
Computational efficiency
Computer architecture
Computer vision
Feature extraction
image and video synthesis
Network architecture
Real-time systems
Transforms
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Network architecture plays a pivotal role in style transfer. Most existing algorithms use VGG19 as the feature extractor, which incurs a high computational cost. In this work, we conduct an empirical study on the popular network architectures and find that some more efficient networks can replace VGG19 while having comparable style transfer performance. Beyond that, we show that an efficient network can be further accelerated by removing its empty channels via a simple channel pruning method tweaked for style transfer. To prevent the potential performance drop due to using a more lightweight network and obtain better style transfer results, we introduce a more accurate deep feature alignment strategy to improve existing style transfer modules. Taking GoogLeNet as an exemplary efficient network, the pruned GoogLeNet with the improved style transfer module is 2.3 ~ 107.4× faster than the state-of-the-art approaches and can achieve 68.03 FPS on 512×512 images. Extensive experiments demonstrate that VGG19 can be replaced by a more lightweight network with significantly improved efficiency and comparable style transfer quality.
ISSN:2642-9381
DOI:10.1109/WACV56688.2023.00407