Loading…
Event-Driven Intrinsic Plasticity for Spiking Convolutional Neural Networks
The biologically discovered intrinsic plasticity (IP) learning rule, which changes the intrinsic excitability of an individual neuron by adaptively turning the firing threshold, has been shown to be crucial for efficient information processing. However, this learning rule needs extra time for updati...
Saved in:
| Published in: | IEEE transaction on neural networks and learning systems 2022-05, Vol.33 (5), p.1986-1995 |
|---|---|
| Main Authors: | , , , |
| Format: | Article |
| Language: | English |
| Subjects: | |
| Citations: | Items that this one cites Items that cite this one |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| cited_by | cdi_FETCH-LOGICAL-c351t-b203c09430b8a538bc862bfea6d77845d5b9eea3f19ee80a05a74017832279753 |
|---|---|
| cites | cdi_FETCH-LOGICAL-c351t-b203c09430b8a538bc862bfea6d77845d5b9eea3f19ee80a05a74017832279753 |
| container_end_page | 1995 |
| container_issue | 5 |
| container_start_page | 1986 |
| container_title | IEEE transaction on neural networks and learning systems |
| container_volume | 33 |
| creator | Zhang, Anguo Li, Xiumin Gao, Yueming Niu, Yuzhen |
| description | The biologically discovered intrinsic plasticity (IP) learning rule, which changes the intrinsic excitability of an individual neuron by adaptively turning the firing threshold, has been shown to be crucial for efficient information processing. However, this learning rule needs extra time for updating operations at each step, causing extra energy consumption and reducing the computational efficiency. The event-driven or spike-based coding strategy of spiking neural networks (SNNs), i.e., neurons will only be active if driven by continuous spiking trains, employs all-or-none pulses (spikes) to transmit information, contributing to sparseness in neuron activations. In this article, we propose two event-driven IP learning rules, namely, input-driven and self-driven IP, based on basic IP learning. Input-driven means that IP updating occurs only when the neuron receives spiking inputs from its presynaptic neurons, whereas self-driven means that IP updating only occurs when the neuron generates a spike. A spiking convolutional neural network (SCNN) is developed based on the ANN2SNN conversion method, i.e., converting a well-trained rate-based artificial neural network to an SNN via directly mapping the connection weights. By comparing the computational performance of SCNNs with different IP rules on the recognition of MNIST, FashionMNIST, Cifar10, and SVHN datasets, we demonstrate that the two event-based IP rules can remarkably reduce IP updating operations, contributing to sparse computations and accelerating the recognition process. This work may give insights into the modeling of brain-inspired SNNs for low-power applications. |
| doi_str_mv | 10.1109/TNNLS.2021.3084955 |
| format | article |
| fullrecord | <record><control><sourceid>proquest_pubme</sourceid><recordid>TN_cdi_proquest_miscellaneous_2539888443</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>9449951</ieee_id><sourcerecordid>2539888443</sourcerecordid><originalsourceid>FETCH-LOGICAL-c351t-b203c09430b8a538bc862bfea6d77845d5b9eea3f19ee80a05a74017832279753</originalsourceid><addsrcrecordid>eNpdkE1LAzEQhoMoVmr_gIIsePGyNd-bHKVWLZYqtIK3kN1mJe12U5NdxX9v-mEPzuUdmGeG4QHgAsE-QlDeziaT8bSPIUZ9AgWVjB2BM4w4TjER4vjQZ-8d0AthAWNxyDiVp6BDKIJccHEGnodfpm7Se29jJqO68bYOtkheKx0aW9jmJymdT6Zru7T1RzJw9Zer2sa6WlfJxLR-G82388twDk5KXQXT22cXvD0MZ4OndPzyOBrcjdOCMNSkOYakgJISmAvNiMgLwXFeGs3nWSYom7NcGqNJiWIIqCHTGYUoEwTjTGaMdMHN7u7au8_WhEatbChMVenauDYozIgUQlBKInr9D1241sffI8WZJPEJlkUK76jCuxC8KdXa25X2PwpBtbGttrbVxrba245LV_vTbb4y88PKn9sIXO4Aa4w5jCWlUjJEfgFjiYJy</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2659343057</pqid></control><display><type>article</type><title>Event-Driven Intrinsic Plasticity for Spiking Convolutional Neural Networks</title><source>IEEE Electronic Library (IEL) Journals</source><creator>Zhang, Anguo ; Li, Xiumin ; Gao, Yueming ; Niu, Yuzhen</creator><creatorcontrib>Zhang, Anguo ; Li, Xiumin ; Gao, Yueming ; Niu, Yuzhen</creatorcontrib><description>The biologically discovered intrinsic plasticity (IP) learning rule, which changes the intrinsic excitability of an individual neuron by adaptively turning the firing threshold, has been shown to be crucial for efficient information processing. However, this learning rule needs extra time for updating operations at each step, causing extra energy consumption and reducing the computational efficiency. The event-driven or spike-based coding strategy of spiking neural networks (SNNs), i.e., neurons will only be active if driven by continuous spiking trains, employs all-or-none pulses (spikes) to transmit information, contributing to sparseness in neuron activations. In this article, we propose two event-driven IP learning rules, namely, input-driven and self-driven IP, based on basic IP learning. Input-driven means that IP updating occurs only when the neuron receives spiking inputs from its presynaptic neurons, whereas self-driven means that IP updating only occurs when the neuron generates a spike. A spiking convolutional neural network (SCNN) is developed based on the ANN2SNN conversion method, i.e., converting a well-trained rate-based artificial neural network to an SNN via directly mapping the connection weights. By comparing the computational performance of SCNNs with different IP rules on the recognition of MNIST, FashionMNIST, Cifar10, and SVHN datasets, we demonstrate that the two event-based IP rules can remarkably reduce IP updating operations, contributing to sparse computations and accelerating the recognition process. This work may give insights into the modeling of brain-inspired SNNs for low-power applications.</description><identifier>ISSN: 2162-237X</identifier><identifier>EISSN: 2162-2388</identifier><identifier>DOI: 10.1109/TNNLS.2021.3084955</identifier><identifier>PMID: 34106868</identifier><identifier>CODEN: ITNNAL</identifier><language>eng</language><publisher>United States: IEEE</publisher><subject>Artificial neural networks ; Biological neural networks ; Biological system modeling ; Biomembranes ; Brain - physiology ; Computational modeling ; Computational neuroscience ; Data processing ; Encoding ; Energy consumption ; Event-driven intrinsic plasticity (IP) ; Excitability ; Firing pattern ; Information processing ; input-driven IP ; IP networks ; Learning ; Neural coding ; Neural networks ; Neural Networks, Computer ; Neural plasticity ; Neurons ; Neurons - physiology ; Plastic properties ; Plasticity ; Recognition ; Recognition, Psychology ; self-driven IP ; Spiking ; spiking neural network (SNN)</subject><ispartof>IEEE transaction on neural networks and learning systems, 2022-05, Vol.33 (5), p.1986-1995</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2022</rights><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c351t-b203c09430b8a538bc862bfea6d77845d5b9eea3f19ee80a05a74017832279753</citedby><cites>FETCH-LOGICAL-c351t-b203c09430b8a538bc862bfea6d77845d5b9eea3f19ee80a05a74017832279753</cites><orcidid>0000-0002-4825-7054 ; 0000-0002-2398-4757 ; 0000-0002-0197-6043 ; 0000-0002-9874-9719</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/9449951$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>315,786,790,27957,27958,55147</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/34106868$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Zhang, Anguo</creatorcontrib><creatorcontrib>Li, Xiumin</creatorcontrib><creatorcontrib>Gao, Yueming</creatorcontrib><creatorcontrib>Niu, Yuzhen</creatorcontrib><title>Event-Driven Intrinsic Plasticity for Spiking Convolutional Neural Networks</title><title>IEEE transaction on neural networks and learning systems</title><addtitle>TNNLS</addtitle><addtitle>IEEE Trans Neural Netw Learn Syst</addtitle><description>The biologically discovered intrinsic plasticity (IP) learning rule, which changes the intrinsic excitability of an individual neuron by adaptively turning the firing threshold, has been shown to be crucial for efficient information processing. However, this learning rule needs extra time for updating operations at each step, causing extra energy consumption and reducing the computational efficiency. The event-driven or spike-based coding strategy of spiking neural networks (SNNs), i.e., neurons will only be active if driven by continuous spiking trains, employs all-or-none pulses (spikes) to transmit information, contributing to sparseness in neuron activations. In this article, we propose two event-driven IP learning rules, namely, input-driven and self-driven IP, based on basic IP learning. Input-driven means that IP updating occurs only when the neuron receives spiking inputs from its presynaptic neurons, whereas self-driven means that IP updating only occurs when the neuron generates a spike. A spiking convolutional neural network (SCNN) is developed based on the ANN2SNN conversion method, i.e., converting a well-trained rate-based artificial neural network to an SNN via directly mapping the connection weights. By comparing the computational performance of SCNNs with different IP rules on the recognition of MNIST, FashionMNIST, Cifar10, and SVHN datasets, we demonstrate that the two event-based IP rules can remarkably reduce IP updating operations, contributing to sparse computations and accelerating the recognition process. This work may give insights into the modeling of brain-inspired SNNs for low-power applications.</description><subject>Artificial neural networks</subject><subject>Biological neural networks</subject><subject>Biological system modeling</subject><subject>Biomembranes</subject><subject>Brain - physiology</subject><subject>Computational modeling</subject><subject>Computational neuroscience</subject><subject>Data processing</subject><subject>Encoding</subject><subject>Energy consumption</subject><subject>Event-driven intrinsic plasticity (IP)</subject><subject>Excitability</subject><subject>Firing pattern</subject><subject>Information processing</subject><subject>input-driven IP</subject><subject>IP networks</subject><subject>Learning</subject><subject>Neural coding</subject><subject>Neural networks</subject><subject>Neural Networks, Computer</subject><subject>Neural plasticity</subject><subject>Neurons</subject><subject>Neurons - physiology</subject><subject>Plastic properties</subject><subject>Plasticity</subject><subject>Recognition</subject><subject>Recognition, Psychology</subject><subject>self-driven IP</subject><subject>Spiking</subject><subject>spiking neural network (SNN)</subject><issn>2162-237X</issn><issn>2162-2388</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><recordid>eNpdkE1LAzEQhoMoVmr_gIIsePGyNd-bHKVWLZYqtIK3kN1mJe12U5NdxX9v-mEPzuUdmGeG4QHgAsE-QlDeziaT8bSPIUZ9AgWVjB2BM4w4TjER4vjQZ-8d0AthAWNxyDiVp6BDKIJccHEGnodfpm7Se29jJqO68bYOtkheKx0aW9jmJymdT6Zru7T1RzJw9Zer2sa6WlfJxLR-G82388twDk5KXQXT22cXvD0MZ4OndPzyOBrcjdOCMNSkOYakgJISmAvNiMgLwXFeGs3nWSYom7NcGqNJiWIIqCHTGYUoEwTjTGaMdMHN7u7au8_WhEatbChMVenauDYozIgUQlBKInr9D1241sffI8WZJPEJlkUK76jCuxC8KdXa25X2PwpBtbGttrbVxrba245LV_vTbb4y88PKn9sIXO4Aa4w5jCWlUjJEfgFjiYJy</recordid><startdate>20220501</startdate><enddate>20220501</enddate><creator>Zhang, Anguo</creator><creator>Li, Xiumin</creator><creator>Gao, Yueming</creator><creator>Niu, Yuzhen</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7QF</scope><scope>7QO</scope><scope>7QP</scope><scope>7QQ</scope><scope>7QR</scope><scope>7SC</scope><scope>7SE</scope><scope>7SP</scope><scope>7SR</scope><scope>7TA</scope><scope>7TB</scope><scope>7TK</scope><scope>7U5</scope><scope>8BQ</scope><scope>8FD</scope><scope>F28</scope><scope>FR3</scope><scope>H8D</scope><scope>JG9</scope><scope>JQ2</scope><scope>KR7</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>P64</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0002-4825-7054</orcidid><orcidid>https://orcid.org/0000-0002-2398-4757</orcidid><orcidid>https://orcid.org/0000-0002-0197-6043</orcidid><orcidid>https://orcid.org/0000-0002-9874-9719</orcidid></search><sort><creationdate>20220501</creationdate><title>Event-Driven Intrinsic Plasticity for Spiking Convolutional Neural Networks</title><author>Zhang, Anguo ; Li, Xiumin ; Gao, Yueming ; Niu, Yuzhen</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c351t-b203c09430b8a538bc862bfea6d77845d5b9eea3f19ee80a05a74017832279753</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>Artificial neural networks</topic><topic>Biological neural networks</topic><topic>Biological system modeling</topic><topic>Biomembranes</topic><topic>Brain - physiology</topic><topic>Computational modeling</topic><topic>Computational neuroscience</topic><topic>Data processing</topic><topic>Encoding</topic><topic>Energy consumption</topic><topic>Event-driven intrinsic plasticity (IP)</topic><topic>Excitability</topic><topic>Firing pattern</topic><topic>Information processing</topic><topic>input-driven IP</topic><topic>IP networks</topic><topic>Learning</topic><topic>Neural coding</topic><topic>Neural networks</topic><topic>Neural Networks, Computer</topic><topic>Neural plasticity</topic><topic>Neurons</topic><topic>Neurons - physiology</topic><topic>Plastic properties</topic><topic>Plasticity</topic><topic>Recognition</topic><topic>Recognition, Psychology</topic><topic>self-driven IP</topic><topic>Spiking</topic><topic>spiking neural network (SNN)</topic><toplevel>online_resources</toplevel><creatorcontrib>Zhang, Anguo</creatorcontrib><creatorcontrib>Li, Xiumin</creatorcontrib><creatorcontrib>Gao, Yueming</creatorcontrib><creatorcontrib>Niu, Yuzhen</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998–Present</collection><collection>IEEE Xplore</collection><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Aluminium Industry Abstracts</collection><collection>Biotechnology Research Abstracts</collection><collection>Calcium & Calcified Tissue Abstracts</collection><collection>Ceramic Abstracts</collection><collection>Chemoreception Abstracts</collection><collection>Computer and Information Systems Abstracts</collection><collection>Corrosion Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Engineered Materials Abstracts</collection><collection>Materials Business File</collection><collection>Mechanical & Transportation Engineering Abstracts</collection><collection>Neurosciences Abstracts</collection><collection>Solid State and Superconductivity Abstracts</collection><collection>METADEX</collection><collection>Technology Research Database</collection><collection>ANTE: Abstracts in New Technology & Engineering</collection><collection>Engineering Research Database</collection><collection>Aerospace Database</collection><collection>Materials Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Civil Engineering Abstracts</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>Biotechnology and BioEngineering Abstracts</collection><collection>MEDLINE - Academic</collection><jtitle>IEEE transaction on neural networks and learning systems</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Zhang, Anguo</au><au>Li, Xiumin</au><au>Gao, Yueming</au><au>Niu, Yuzhen</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Event-Driven Intrinsic Plasticity for Spiking Convolutional Neural Networks</atitle><jtitle>IEEE transaction on neural networks and learning systems</jtitle><stitle>TNNLS</stitle><addtitle>IEEE Trans Neural Netw Learn Syst</addtitle><date>2022-05-01</date><risdate>2022</risdate><volume>33</volume><issue>5</issue><spage>1986</spage><epage>1995</epage><pages>1986-1995</pages><issn>2162-237X</issn><eissn>2162-2388</eissn><coden>ITNNAL</coden><notes>ObjectType-Article-1</notes><notes>SourceType-Scholarly Journals-1</notes><notes>ObjectType-Feature-2</notes><notes>content type line 23</notes><abstract>The biologically discovered intrinsic plasticity (IP) learning rule, which changes the intrinsic excitability of an individual neuron by adaptively turning the firing threshold, has been shown to be crucial for efficient information processing. However, this learning rule needs extra time for updating operations at each step, causing extra energy consumption and reducing the computational efficiency. The event-driven or spike-based coding strategy of spiking neural networks (SNNs), i.e., neurons will only be active if driven by continuous spiking trains, employs all-or-none pulses (spikes) to transmit information, contributing to sparseness in neuron activations. In this article, we propose two event-driven IP learning rules, namely, input-driven and self-driven IP, based on basic IP learning. Input-driven means that IP updating occurs only when the neuron receives spiking inputs from its presynaptic neurons, whereas self-driven means that IP updating only occurs when the neuron generates a spike. A spiking convolutional neural network (SCNN) is developed based on the ANN2SNN conversion method, i.e., converting a well-trained rate-based artificial neural network to an SNN via directly mapping the connection weights. By comparing the computational performance of SCNNs with different IP rules on the recognition of MNIST, FashionMNIST, Cifar10, and SVHN datasets, we demonstrate that the two event-based IP rules can remarkably reduce IP updating operations, contributing to sparse computations and accelerating the recognition process. This work may give insights into the modeling of brain-inspired SNNs for low-power applications.</abstract><cop>United States</cop><pub>IEEE</pub><pmid>34106868</pmid><doi>10.1109/TNNLS.2021.3084955</doi><tpages>10</tpages><orcidid>https://orcid.org/0000-0002-4825-7054</orcidid><orcidid>https://orcid.org/0000-0002-2398-4757</orcidid><orcidid>https://orcid.org/0000-0002-0197-6043</orcidid><orcidid>https://orcid.org/0000-0002-9874-9719</orcidid></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 2162-237X |
| ispartof | IEEE transaction on neural networks and learning systems, 2022-05, Vol.33 (5), p.1986-1995 |
| issn | 2162-237X 2162-2388 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_2539888443 |
| source | IEEE Electronic Library (IEL) Journals |
| subjects | Artificial neural networks Biological neural networks Biological system modeling Biomembranes Brain - physiology Computational modeling Computational neuroscience Data processing Encoding Energy consumption Event-driven intrinsic plasticity (IP) Excitability Firing pattern Information processing input-driven IP IP networks Learning Neural coding Neural networks Neural Networks, Computer Neural plasticity Neurons Neurons - physiology Plastic properties Plasticity Recognition Recognition, Psychology self-driven IP Spiking spiking neural network (SNN) |
| title | Event-Driven Intrinsic Plasticity for Spiking Convolutional Neural Networks |
| url | http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-09-22T15%3A39%3A50IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Event-Driven%20Intrinsic%20Plasticity%20for%20Spiking%20Convolutional%20Neural%20Networks&rft.jtitle=IEEE%20transaction%20on%20neural%20networks%20and%20learning%20systems&rft.au=Zhang,%20Anguo&rft.date=2022-05-01&rft.volume=33&rft.issue=5&rft.spage=1986&rft.epage=1995&rft.pages=1986-1995&rft.issn=2162-237X&rft.eissn=2162-2388&rft.coden=ITNNAL&rft_id=info:doi/10.1109/TNNLS.2021.3084955&rft_dat=%3Cproquest_pubme%3E2539888443%3C/proquest_pubme%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c351t-b203c09430b8a538bc862bfea6d77845d5b9eea3f19ee80a05a74017832279753%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2659343057&rft_id=info:pmid/34106868&rft_ieee_id=9449951&rfr_iscdi=true |