Real-Time Masked Face Revealing for Video Conference

Video conferencing is an essential way for contactless conversation, which conveys abundant multimedia signals. Especially under COVID-19, the video conference has been becoming a common way for daily communications. However, for the sake of plague prevention, it usually happens that the people atte...

Full description

Saved in:
Bibliographic Details
Main Authors: Lin, Jinpeng, Liu, Pengfei, Zheng, Yinglin, Deng, Wenjin, Zeng, Ming
Format: Conference Proceeding
Language:English
Subjects:
Conferences
COVID-19
Face Inpainting with Multimodal Signals
Mouth
Real-time systems
Real-time Video Telephone
Realistic Face Synthesis
Shape
Streaming media
Temporal Consistent Inpainting
User experience
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Video conferencing is an essential way for contactless conversation, which conveys abundant multimedia signals. Especially under COVID-19, the video conference has been becoming a common way for daily communications. However, for the sake of plague prevention, it usually happens that the people attending the video conference are wearing a mouth mask, leading to inconvenient communication due to incomplete facial information. To tackle this problem, we develop a novel system that reveals the masked faces in real-time, making each participant feel like the others are mask-free. Moreover, we map the audio to 3DMM expression to guide the generation of various mouth shapes utilizing multi-modal information. Extensive experiments validate the revealing effectiveness and better user experience of the system. Furthermore, by applying lightweight networks design, the proposed system can run in real-time.
ISSN:1945-788X
DOI:10.1109/ICME51207.2021.9428117