Minimal Gated Unit for Recurrent Neural Networks

Recurrent neural networks （RNN） have been very successful in handling sequence data. However, understanding RNN and finding the best practices for RNN learning is a difficult task, partly because there are many competing and complex hidden units, such as the long short-term memory （LSTM） and the gat...

Full description

Saved in:
Bibliographic Details
Published in:International journal of automation and computing 2016-06, Vol.13 (3), p.226-234
Main Authors: Zhou, Guo-Bing, Wu, Jianxin, Zhang, Chen-Lin, Zhou, Zhi-Hua
Format: Article
Language:English
Subjects:
CAE) and Design
Computer Applications
Computer-Aided Engineering (CAD
Control
Engineering
MCU
Mechatronics
Neural networks
Recurrent neural networks
Research Article
RNN
Robotics
单元
单片机
序列数据
选通
递归神经网络
门控
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recurrent neural networks （RNN） have been very successful in handling sequence data. However, understanding RNN and finding the best practices for RNN learning is a difficult task, partly because there are many competing and complex hidden units, such as the long short-term memory （LSTM） and the gated recurrent unit （GRU）. We propose a gated unit for RNN, named as minimal gated unit （MCU）, since it only contains one gate, which is a minimal design among all gated hidden units. The design of MCU benefits from evaluation results on LSTM and GRU in the literature. Experiments on various sequence data show that MCU has comparable accuracy with GRU, but has a simpler structure, fewer parameters, and faster training. Hence, MGU is suitable in RNN＇s applications. Its simple architecture also means that it is easier to evaluate and tune, and in principle it is easier to study MGU＇s properties theoretically and empirically.
ISSN:1476-8186
1751-8520
DOI:10.1007/s11633-016-1006-2