A Reconfigurable Architecture for Real-time Event-based Multi-Object Tracking

Although advances in event-based machine vision algorithms have demonstrated unparalleled capabilities in performing some of the most demanding tasks, their implementations under stringent real-time and power constraints in edge systems remain a major challenge. In this work, a reconfigurable hardwa...

Full description

Saved in:
Bibliographic Details
Published in:ACM transactions on reconfigurable technology and systems 2023-09, Vol.16 (4), p.1-26, Article 58
Main Authors: Gao, Yizhao, Wang, Song, So, Hayden Kwok-Hay
Format: Article
Language:English
Subjects:
Computer systems organization
Computing methodologies
Real-time system architecture
Reconfigurable computing
Tracking
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites cdi_FETCH-LOGICAL-a239t-189dc87a17b63a4243267ad47a9d41e19dfc61fb7d8e6605ff794e8ff4a003a63
container_end_page 26
container_issue 4
container_start_page 1
container_title ACM transactions on reconfigurable technology and systems
container_volume 16
creator Gao, Yizhao
Wang, Song
So, Hayden Kwok-Hay
description Although advances in event-based machine vision algorithms have demonstrated unparalleled capabilities in performing some of the most demanding tasks, their implementations under stringent real-time and power constraints in edge systems remain a major challenge. In this work, a reconfigurable hardware-software architecture called REMOT, which performs real-time event-based multi-object tracking on FPGAs, is presented. REMOT performs vision tasks by defining a set of actions over attention units (AUs). These actions allow AUs to track an object candidate autonomously by adjusting its region of attention and allow information gathered by each AU to be used for making algorithmic-level decisions. Taking advantage of this modular structure, algorithm-architecture codesign can be performed by implementing different parts of the algorithm in either hardware or software for different tradeoffs. Results show that REMOT can process 0.43–2.91 million events per second at 1.75–5.45 W. Compared with the software baseline, our implementation achieves up to 44 times higher throughput and 35.4 times higher power efficiency. Migrating the Merge operation to hardware further reduces the worst-case latency to be 95 times shorter than the software baseline. By varying the AU configuration and operation, a reduction of 0.59–0.77 mW per AU on the programmable logic has also been demonstrated.
doi_str_mv 10.1145/3593587
format article
fullrecord <record><control><sourceid>acm_cross</sourceid><recordid>TN_cdi_crossref_primary_10_1145_3593587</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>3593587</sourcerecordid><originalsourceid>FETCH-LOGICAL-a239t-189dc87a17b63a4243267ad47a9d41e19dfc61fb7d8e6605ff794e8ff4a003a63</originalsourceid><addsrcrecordid>eNo9kM1LwzAYxoMoOKd499Sbp2jSpElzLGN-wMZA5rm8Td_Mzn5Ikgr-91Y2d3oeeH48hx8ht5w9cC6zR5EZkeX6jMy4EYpqyeX5qTN1Sa5C2DOmhMrljKyL5A3t0LtmN3qoWkwKbz-aiDaOHhM3-GmHlsamw2T5jX2kFQSsk_XYxoZuqv1EJlsP9rPpd9fkwkEb8OaYc_L-tNwuXuhq8_y6KFYUUmEi5bmpba6B60oJkKkUqdJQSw2mlhy5qZ1V3FW6zlEpljmnjcTcOQmMCVBiTu4Pv9YPIXh05ZdvOvA_JWfln4XyaGEi7w4k2O4E_Y-_sDdW1A</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>A Reconfigurable Architecture for Real-time Event-based Multi-Object Tracking</title><source>Association for Computing Machinery:Jisc Collections:ACM OPEN Journals 2023-2025 (reading list)</source><creator>Gao, Yizhao ; Wang, Song ; So, Hayden Kwok-Hay</creator><creatorcontrib>Gao, Yizhao ; Wang, Song ; So, Hayden Kwok-Hay</creatorcontrib><description>Although advances in event-based machine vision algorithms have demonstrated unparalleled capabilities in performing some of the most demanding tasks, their implementations under stringent real-time and power constraints in edge systems remain a major challenge. In this work, a reconfigurable hardware-software architecture called REMOT, which performs real-time event-based multi-object tracking on FPGAs, is presented. REMOT performs vision tasks by defining a set of actions over attention units (AUs). These actions allow AUs to track an object candidate autonomously by adjusting its region of attention and allow information gathered by each AU to be used for making algorithmic-level decisions. Taking advantage of this modular structure, algorithm-architecture codesign can be performed by implementing different parts of the algorithm in either hardware or software for different tradeoffs. Results show that REMOT can process 0.43–2.91 million events per second at 1.75–5.45 W. Compared with the software baseline, our implementation achieves up to 44 times higher throughput and 35.4 times higher power efficiency. Migrating the Merge operation to hardware further reduces the worst-case latency to be 95 times shorter than the software baseline. By varying the AU configuration and operation, a reduction of 0.59–0.77 mW per AU on the programmable logic has also been demonstrated.</description><identifier>ISSN: 1936-7406</identifier><identifier>EISSN: 1936-7414</identifier><identifier>DOI: 10.1145/3593587</identifier><language>eng</language><publisher>New York, NY: ACM</publisher><subject>Computer systems organization ; Computing methodologies ; Real-time system architecture ; Reconfigurable computing ; Tracking</subject><ispartof>ACM transactions on reconfigurable technology and systems, 2023-09, Vol.16 (4), p.1-26, Article 58</ispartof><rights>Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-a239t-189dc87a17b63a4243267ad47a9d41e19dfc61fb7d8e6605ff794e8ff4a003a63</cites><orcidid>0000-0001-5673-3746 ; 0000-0002-1813-5865 ; 0000-0002-6514-0237</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>315,786,790,27957,27958</link.rule.ids></links><search><creatorcontrib>Gao, Yizhao</creatorcontrib><creatorcontrib>Wang, Song</creatorcontrib><creatorcontrib>So, Hayden Kwok-Hay</creatorcontrib><title>A Reconfigurable Architecture for Real-time Event-based Multi-Object Tracking</title><title>ACM transactions on reconfigurable technology and systems</title><addtitle>ACM TRETS</addtitle><description>Although advances in event-based machine vision algorithms have demonstrated unparalleled capabilities in performing some of the most demanding tasks, their implementations under stringent real-time and power constraints in edge systems remain a major challenge. In this work, a reconfigurable hardware-software architecture called REMOT, which performs real-time event-based multi-object tracking on FPGAs, is presented. REMOT performs vision tasks by defining a set of actions over attention units (AUs). These actions allow AUs to track an object candidate autonomously by adjusting its region of attention and allow information gathered by each AU to be used for making algorithmic-level decisions. Taking advantage of this modular structure, algorithm-architecture codesign can be performed by implementing different parts of the algorithm in either hardware or software for different tradeoffs. Results show that REMOT can process 0.43–2.91 million events per second at 1.75–5.45 W. Compared with the software baseline, our implementation achieves up to 44 times higher throughput and 35.4 times higher power efficiency. Migrating the Merge operation to hardware further reduces the worst-case latency to be 95 times shorter than the software baseline. By varying the AU configuration and operation, a reduction of 0.59–0.77 mW per AU on the programmable logic has also been demonstrated.</description><subject>Computer systems organization</subject><subject>Computing methodologies</subject><subject>Real-time system architecture</subject><subject>Reconfigurable computing</subject><subject>Tracking</subject><issn>1936-7406</issn><issn>1936-7414</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><recordid>eNo9kM1LwzAYxoMoOKd499Sbp2jSpElzLGN-wMZA5rm8Td_Mzn5Ikgr-91Y2d3oeeH48hx8ht5w9cC6zR5EZkeX6jMy4EYpqyeX5qTN1Sa5C2DOmhMrljKyL5A3t0LtmN3qoWkwKbz-aiDaOHhM3-GmHlsamw2T5jX2kFQSsk_XYxoZuqv1EJlsP9rPpd9fkwkEb8OaYc_L-tNwuXuhq8_y6KFYUUmEi5bmpba6B60oJkKkUqdJQSw2mlhy5qZ1V3FW6zlEpljmnjcTcOQmMCVBiTu4Pv9YPIXh05ZdvOvA_JWfln4XyaGEi7w4k2O4E_Y-_sDdW1A</recordid><startdate>20230901</startdate><enddate>20230901</enddate><creator>Gao, Yizhao</creator><creator>Wang, Song</creator><creator>So, Hayden Kwok-Hay</creator><general>ACM</general><scope>AAYXX</scope><scope>CITATION</scope><orcidid>https://orcid.org/0000-0001-5673-3746</orcidid><orcidid>https://orcid.org/0000-0002-1813-5865</orcidid><orcidid>https://orcid.org/0000-0002-6514-0237</orcidid></search><sort><creationdate>20230901</creationdate><title>A Reconfigurable Architecture for Real-time Event-based Multi-Object Tracking</title><author>Gao, Yizhao ; Wang, Song ; So, Hayden Kwok-Hay</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a239t-189dc87a17b63a4243267ad47a9d41e19dfc61fb7d8e6605ff794e8ff4a003a63</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Computer systems organization</topic><topic>Computing methodologies</topic><topic>Real-time system architecture</topic><topic>Reconfigurable computing</topic><topic>Tracking</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Gao, Yizhao</creatorcontrib><creatorcontrib>Wang, Song</creatorcontrib><creatorcontrib>So, Hayden Kwok-Hay</creatorcontrib><collection>CrossRef</collection><jtitle>ACM transactions on reconfigurable technology and systems</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Gao, Yizhao</au><au>Wang, Song</au><au>So, Hayden Kwok-Hay</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A Reconfigurable Architecture for Real-time Event-based Multi-Object Tracking</atitle><jtitle>ACM transactions on reconfigurable technology and systems</jtitle><stitle>ACM TRETS</stitle><date>2023-09-01</date><risdate>2023</risdate><volume>16</volume><issue>4</issue><spage>1</spage><epage>26</epage><pages>1-26</pages><artnum>58</artnum><issn>1936-7406</issn><eissn>1936-7414</eissn><abstract>Although advances in event-based machine vision algorithms have demonstrated unparalleled capabilities in performing some of the most demanding tasks, their implementations under stringent real-time and power constraints in edge systems remain a major challenge. In this work, a reconfigurable hardware-software architecture called REMOT, which performs real-time event-based multi-object tracking on FPGAs, is presented. REMOT performs vision tasks by defining a set of actions over attention units (AUs). These actions allow AUs to track an object candidate autonomously by adjusting its region of attention and allow information gathered by each AU to be used for making algorithmic-level decisions. Taking advantage of this modular structure, algorithm-architecture codesign can be performed by implementing different parts of the algorithm in either hardware or software for different tradeoffs. Results show that REMOT can process 0.43–2.91 million events per second at 1.75–5.45 W. Compared with the software baseline, our implementation achieves up to 44 times higher throughput and 35.4 times higher power efficiency. Migrating the Merge operation to hardware further reduces the worst-case latency to be 95 times shorter than the software baseline. By varying the AU configuration and operation, a reduction of 0.59–0.77 mW per AU on the programmable logic has also been demonstrated.</abstract><cop>New York, NY</cop><pub>ACM</pub><doi>10.1145/3593587</doi><tpages>26</tpages><orcidid>https://orcid.org/0000-0001-5673-3746</orcidid><orcidid>https://orcid.org/0000-0002-1813-5865</orcidid><orcidid>https://orcid.org/0000-0002-6514-0237</orcidid><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 1936-7406
ispartof ACM transactions on reconfigurable technology and systems, 2023-09, Vol.16 (4), p.1-26, Article 58
issn 1936-7406
1936-7414
language eng
recordid cdi_crossref_primary_10_1145_3593587
source Association for Computing Machinery:Jisc Collections:ACM OPEN Journals 2023-2025 (reading list)
subjects Computer systems organization
Computing methodologies
Real-time system architecture
Reconfigurable computing
Tracking
title A Reconfigurable Architecture for Real-time Event-based Multi-Object Tracking
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-09-22T11%3A26%3A24IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-acm_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20Reconfigurable%20Architecture%20for%20Real-time%20Event-based%20Multi-Object%20Tracking&rft.jtitle=ACM%20transactions%20on%20reconfigurable%20technology%20and%20systems&rft.au=Gao,%20Yizhao&rft.date=2023-09-01&rft.volume=16&rft.issue=4&rft.spage=1&rft.epage=26&rft.pages=1-26&rft.artnum=58&rft.issn=1936-7406&rft.eissn=1936-7414&rft_id=info:doi/10.1145/3593587&rft_dat=%3Cacm_cross%3E3593587%3C/acm_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-a239t-189dc87a17b63a4243267ad47a9d41e19dfc61fb7d8e6605ff794e8ff4a003a63%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true