Uncertainty Inspired RGB-D Saliency Detection

We propose the first stochastic framework to employ uncertainty for RGB-D saliency detection by learning from the data labeling process. Existing RGB-D saliency detection models treat this task as a point estimation problem by predicting a single saliency map following a deterministic learning pipel...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2022-09, Vol.44 (9), p.5761-5779
Main Authors: Zhang, Jing, Fan, Deng-Ping, Dai, Yuchao, Anwar, Saeed, Saleh, Fatemeh, Aliakbarian, Sadegh, Barnes, Nick
Format: Article
Language:English
Subjects:
alternating back-propagation
Back propagation
Coders
conditional variational autoencoders
Data models
Encoders-Decoders
Labeling
Labelling
Learning
Pipelines
Predictive models
RGB-D saliency detection
Salience
Saliency detection
Source code
Training
Uncertainty
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by cdi_FETCH-LOGICAL-c351t-5265778d3e9fae9c607d9efdcafdfe3a3c713e8e0a44451ea888e3b6e20b642c3
cites cdi_FETCH-LOGICAL-c351t-5265778d3e9fae9c607d9efdcafdfe3a3c713e8e0a44451ea888e3b6e20b642c3
container_end_page 5779
container_issue 9
container_start_page 5761
container_title IEEE transactions on pattern analysis and machine intelligence
container_volume 44
creator Zhang, Jing
Fan, Deng-Ping
Dai, Yuchao
Anwar, Saeed
Saleh, Fatemeh
Aliakbarian, Sadegh
Barnes, Nick
description We propose the first stochastic framework to employ uncertainty for RGB-D saliency detection by learning from the data labeling process. Existing RGB-D saliency detection models treat this task as a point estimation problem by predicting a single saliency map following a deterministic learning pipeline. We argue that, however, the deterministic solution is relatively ill-posed. Inspired by the saliency data labeling process, we propose a generative architecture to achieve probabilistic RGB-D saliency detection which utilizes a latent variable to model the labeling variations. Our framework includes two main models: 1) a generator model, which maps the input image and latent variable to stochastic saliency prediction, and 2) an inference model, which gradually updates the latent variable by sampling it from the true or approximate posterior distribution. The generator model is an encoder-decoder saliency network. To infer the latent variable, we introduce two different solutions: i) a Conditional Variational Auto-encoder with an extra encoder to approximate the posterior distribution of the latent variable; and ii) an Alternating Back-Propagation technique, which directly samples the latent variable from the true posterior distribution. Qualitative and quantitative results on six challenging RGB-D benchmark datasets show our approach's superior performance in learning the distribution of saliency maps. The source code is publicly available via our project page: https://github.com/JingZhang617/UCNet .
doi_str_mv 10.1109/TPAMI.2021.3073564
format article
fullrecord <record><control><sourceid>proquest_ieee_</sourceid><recordid>TN_cdi_ieee_primary_9405467</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9405467</ieee_id><sourcerecordid>2698828134</sourcerecordid><originalsourceid>FETCH-LOGICAL-c351t-5265778d3e9fae9c607d9efdcafdfe3a3c713e8e0a44451ea888e3b6e20b642c3</originalsourceid><addsrcrecordid>eNpdkLFOwzAQhi0EoqXwAiChSCwsKbbPTuwRWiiVikDQzpbrXKRUaVLsZOjbk9LSgemG-_7T_R8h14wOGaP6Yf7x-DYdcsrZEGgKMhEnpM806Bgk6FPSpyzhsVJc9chFCCtKmZAUzkkPQMlEK94n8aJy6BtbVM02mlZhU3jMos_JUzyOvmxZYOW20RgbdE1RV5fkLLdlwKvDHJDFy_N89BrP3ifT0eMsdiBZE0ueyDRVGaDOLWqX0DTTmGfO5lmOYMGlDFAhtUIIydAqpRCWCXK6TAR3MCD3-7sbX3-3GBqzLoLDsrQV1m0wXHZNNNUs7dC7f-iqbn3VfWd4V7Erz0B0FN9TztcheMzNxhdr67eGUbOTaX5lmp1Mc5DZhW4Pp9vlGrNj5M9eB9zsgQIRj2stqBRJCj9aGHan</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2698828134</pqid></control><display><type>article</type><title>Uncertainty Inspired RGB-D Saliency Detection</title><source>IEEE Electronic Library (IEL) Journals</source><creator>Zhang, Jing ; Fan, Deng-Ping ; Dai, Yuchao ; Anwar, Saeed ; Saleh, Fatemeh ; Aliakbarian, Sadegh ; Barnes, Nick</creator><creatorcontrib>Zhang, Jing ; Fan, Deng-Ping ; Dai, Yuchao ; Anwar, Saeed ; Saleh, Fatemeh ; Aliakbarian, Sadegh ; Barnes, Nick</creatorcontrib><description>We propose the first stochastic framework to employ uncertainty for RGB-D saliency detection by learning from the data labeling process. Existing RGB-D saliency detection models treat this task as a point estimation problem by predicting a single saliency map following a deterministic learning pipeline. We argue that, however, the deterministic solution is relatively ill-posed. Inspired by the saliency data labeling process, we propose a generative architecture to achieve probabilistic RGB-D saliency detection which utilizes a latent variable to model the labeling variations. Our framework includes two main models: 1) a generator model, which maps the input image and latent variable to stochastic saliency prediction, and 2) an inference model, which gradually updates the latent variable by sampling it from the true or approximate posterior distribution. The generator model is an encoder-decoder saliency network. To infer the latent variable, we introduce two different solutions: i) a Conditional Variational Auto-encoder with an extra encoder to approximate the posterior distribution of the latent variable; and ii) an Alternating Back-Propagation technique, which directly samples the latent variable from the true posterior distribution. Qualitative and quantitative results on six challenging RGB-D benchmark datasets show our approach's superior performance in learning the distribution of saliency maps. The source code is publicly available via our project page: https://github.com/JingZhang617/UCNet .</description><identifier>ISSN: 0162-8828</identifier><identifier>EISSN: 1939-3539</identifier><identifier>EISSN: 2160-9292</identifier><identifier>DOI: 10.1109/TPAMI.2021.3073564</identifier><identifier>PMID: 33856982</identifier><identifier>CODEN: ITPIDJ</identifier><language>eng</language><publisher>United States: IEEE</publisher><subject>alternating back-propagation ; Back propagation ; Coders ; conditional variational autoencoders ; Data models ; Encoders-Decoders ; Labeling ; Labelling ; Learning ; Pipelines ; Predictive models ; RGB-D saliency detection ; Salience ; Saliency detection ; Source code ; Training ; Uncertainty</subject><ispartof>IEEE transactions on pattern analysis and machine intelligence, 2022-09, Vol.44 (9), p.5761-5779</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c351t-5265778d3e9fae9c607d9efdcafdfe3a3c713e8e0a44451ea888e3b6e20b642c3</citedby><cites>FETCH-LOGICAL-c351t-5265778d3e9fae9c607d9efdcafdfe3a3c713e8e0a44451ea888e3b6e20b642c3</cites><orcidid>0000-0002-9343-9535 ; 0000-0002-5245-7518 ; 0000-0002-4432-7406 ; 0000-0002-0692-8411</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9405467$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>315,786,790,27957,27958,55147</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/33856982$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Zhang, Jing</creatorcontrib><creatorcontrib>Fan, Deng-Ping</creatorcontrib><creatorcontrib>Dai, Yuchao</creatorcontrib><creatorcontrib>Anwar, Saeed</creatorcontrib><creatorcontrib>Saleh, Fatemeh</creatorcontrib><creatorcontrib>Aliakbarian, Sadegh</creatorcontrib><creatorcontrib>Barnes, Nick</creatorcontrib><title>Uncertainty Inspired RGB-D Saliency Detection</title><title>IEEE transactions on pattern analysis and machine intelligence</title><addtitle>TPAMI</addtitle><addtitle>IEEE Trans Pattern Anal Mach Intell</addtitle><description>We propose the first stochastic framework to employ uncertainty for RGB-D saliency detection by learning from the data labeling process. Existing RGB-D saliency detection models treat this task as a point estimation problem by predicting a single saliency map following a deterministic learning pipeline. We argue that, however, the deterministic solution is relatively ill-posed. Inspired by the saliency data labeling process, we propose a generative architecture to achieve probabilistic RGB-D saliency detection which utilizes a latent variable to model the labeling variations. Our framework includes two main models: 1) a generator model, which maps the input image and latent variable to stochastic saliency prediction, and 2) an inference model, which gradually updates the latent variable by sampling it from the true or approximate posterior distribution. The generator model is an encoder-decoder saliency network. To infer the latent variable, we introduce two different solutions: i) a Conditional Variational Auto-encoder with an extra encoder to approximate the posterior distribution of the latent variable; and ii) an Alternating Back-Propagation technique, which directly samples the latent variable from the true posterior distribution. Qualitative and quantitative results on six challenging RGB-D benchmark datasets show our approach's superior performance in learning the distribution of saliency maps. The source code is publicly available via our project page: https://github.com/JingZhang617/UCNet .</description><subject>alternating back-propagation</subject><subject>Back propagation</subject><subject>Coders</subject><subject>conditional variational autoencoders</subject><subject>Data models</subject><subject>Encoders-Decoders</subject><subject>Labeling</subject><subject>Labelling</subject><subject>Learning</subject><subject>Pipelines</subject><subject>Predictive models</subject><subject>RGB-D saliency detection</subject><subject>Salience</subject><subject>Saliency detection</subject><subject>Source code</subject><subject>Training</subject><subject>Uncertainty</subject><issn>0162-8828</issn><issn>1939-3539</issn><issn>2160-9292</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><recordid>eNpdkLFOwzAQhi0EoqXwAiChSCwsKbbPTuwRWiiVikDQzpbrXKRUaVLsZOjbk9LSgemG-_7T_R8h14wOGaP6Yf7x-DYdcsrZEGgKMhEnpM806Bgk6FPSpyzhsVJc9chFCCtKmZAUzkkPQMlEK94n8aJy6BtbVM02mlZhU3jMos_JUzyOvmxZYOW20RgbdE1RV5fkLLdlwKvDHJDFy_N89BrP3ifT0eMsdiBZE0ueyDRVGaDOLWqX0DTTmGfO5lmOYMGlDFAhtUIIydAqpRCWCXK6TAR3MCD3-7sbX3-3GBqzLoLDsrQV1m0wXHZNNNUs7dC7f-iqbn3VfWd4V7Erz0B0FN9TztcheMzNxhdr67eGUbOTaX5lmp1Mc5DZhW4Pp9vlGrNj5M9eB9zsgQIRj2stqBRJCj9aGHan</recordid><startdate>20220901</startdate><enddate>20220901</enddate><creator>Zhang, Jing</creator><creator>Fan, Deng-Ping</creator><creator>Dai, Yuchao</creator><creator>Anwar, Saeed</creator><creator>Saleh, Fatemeh</creator><creator>Aliakbarian, Sadegh</creator><creator>Barnes, Nick</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0002-9343-9535</orcidid><orcidid>https://orcid.org/0000-0002-5245-7518</orcidid><orcidid>https://orcid.org/0000-0002-4432-7406</orcidid><orcidid>https://orcid.org/0000-0002-0692-8411</orcidid></search><sort><creationdate>20220901</creationdate><title>Uncertainty Inspired RGB-D Saliency Detection</title><author>Zhang, Jing ; Fan, Deng-Ping ; Dai, Yuchao ; Anwar, Saeed ; Saleh, Fatemeh ; Aliakbarian, Sadegh ; Barnes, Nick</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c351t-5265778d3e9fae9c607d9efdcafdfe3a3c713e8e0a44451ea888e3b6e20b642c3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>alternating back-propagation</topic><topic>Back propagation</topic><topic>Coders</topic><topic>conditional variational autoencoders</topic><topic>Data models</topic><topic>Encoders-Decoders</topic><topic>Labeling</topic><topic>Labelling</topic><topic>Learning</topic><topic>Pipelines</topic><topic>Predictive models</topic><topic>RGB-D saliency detection</topic><topic>Salience</topic><topic>Saliency detection</topic><topic>Source code</topic><topic>Training</topic><topic>Uncertainty</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Zhang, Jing</creatorcontrib><creatorcontrib>Fan, Deng-Ping</creatorcontrib><creatorcontrib>Dai, Yuchao</creatorcontrib><creatorcontrib>Anwar, Saeed</creatorcontrib><creatorcontrib>Saleh, Fatemeh</creatorcontrib><creatorcontrib>Aliakbarian, Sadegh</creatorcontrib><creatorcontrib>Barnes, Nick</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998–Present</collection><collection>IEEE Xplore</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>MEDLINE - Academic</collection><jtitle>IEEE transactions on pattern analysis and machine intelligence</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Zhang, Jing</au><au>Fan, Deng-Ping</au><au>Dai, Yuchao</au><au>Anwar, Saeed</au><au>Saleh, Fatemeh</au><au>Aliakbarian, Sadegh</au><au>Barnes, Nick</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Uncertainty Inspired RGB-D Saliency Detection</atitle><jtitle>IEEE transactions on pattern analysis and machine intelligence</jtitle><stitle>TPAMI</stitle><addtitle>IEEE Trans Pattern Anal Mach Intell</addtitle><date>2022-09-01</date><risdate>2022</risdate><volume>44</volume><issue>9</issue><spage>5761</spage><epage>5779</epage><pages>5761-5779</pages><issn>0162-8828</issn><eissn>1939-3539</eissn><eissn>2160-9292</eissn><coden>ITPIDJ</coden><notes>ObjectType-Article-1</notes><notes>SourceType-Scholarly Journals-1</notes><notes>ObjectType-Feature-2</notes><notes>content type line 23</notes><abstract>We propose the first stochastic framework to employ uncertainty for RGB-D saliency detection by learning from the data labeling process. Existing RGB-D saliency detection models treat this task as a point estimation problem by predicting a single saliency map following a deterministic learning pipeline. We argue that, however, the deterministic solution is relatively ill-posed. Inspired by the saliency data labeling process, we propose a generative architecture to achieve probabilistic RGB-D saliency detection which utilizes a latent variable to model the labeling variations. Our framework includes two main models: 1) a generator model, which maps the input image and latent variable to stochastic saliency prediction, and 2) an inference model, which gradually updates the latent variable by sampling it from the true or approximate posterior distribution. The generator model is an encoder-decoder saliency network. To infer the latent variable, we introduce two different solutions: i) a Conditional Variational Auto-encoder with an extra encoder to approximate the posterior distribution of the latent variable; and ii) an Alternating Back-Propagation technique, which directly samples the latent variable from the true posterior distribution. Qualitative and quantitative results on six challenging RGB-D benchmark datasets show our approach's superior performance in learning the distribution of saliency maps. The source code is publicly available via our project page: https://github.com/JingZhang617/UCNet .</abstract><cop>United States</cop><pub>IEEE</pub><pmid>33856982</pmid><doi>10.1109/TPAMI.2021.3073564</doi><tpages>19</tpages><orcidid>https://orcid.org/0000-0002-9343-9535</orcidid><orcidid>https://orcid.org/0000-0002-5245-7518</orcidid><orcidid>https://orcid.org/0000-0002-4432-7406</orcidid><orcidid>https://orcid.org/0000-0002-0692-8411</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 0162-8828
ispartof IEEE transactions on pattern analysis and machine intelligence, 2022-09, Vol.44 (9), p.5761-5779
issn 0162-8828
1939-3539
2160-9292
language eng
recordid cdi_ieee_primary_9405467
source IEEE Electronic Library (IEL) Journals
subjects alternating back-propagation
Back propagation
Coders
conditional variational autoencoders
Data models
Encoders-Decoders
Labeling
Labelling
Learning
Pipelines
Predictive models
RGB-D saliency detection
Salience
Saliency detection
Source code
Training
Uncertainty
title Uncertainty Inspired RGB-D Saliency Detection
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-09-22T04%3A57%3A48IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_ieee_&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Uncertainty%20Inspired%20RGB-D%20Saliency%20Detection&rft.jtitle=IEEE%20transactions%20on%20pattern%20analysis%20and%20machine%20intelligence&rft.au=Zhang,%20Jing&rft.date=2022-09-01&rft.volume=44&rft.issue=9&rft.spage=5761&rft.epage=5779&rft.pages=5761-5779&rft.issn=0162-8828&rft.eissn=1939-3539&rft.coden=ITPIDJ&rft_id=info:doi/10.1109/TPAMI.2021.3073564&rft_dat=%3Cproquest_ieee_%3E2698828134%3C/proquest_ieee_%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c351t-5265778d3e9fae9c607d9efdcafdfe3a3c713e8e0a44451ea888e3b6e20b642c3%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2698828134&rft_id=info:pmid/33856982&rft_ieee_id=9405467&rfr_iscdi=true