###
工程科学与技术:2019,51(4):125-132
←前一篇   |   后一篇→
本文二维码信息
码上扫一扫!
基于串并行卷积门阀循环神经网络的短文本特征提取与分类
(1.重庆邮电大学 自动化学院, 重庆 400065;2.重庆邮电大学 计算机学院, 重庆 400065)
Short Text Feature Extraction and Classification Based on Serial-Parallel Convolutional Gated Recurrent Neural Network
(1.School of Automation, Chongqing Univ. of Posts and Telecommunications, Chongqing 400065, China;2.School of Computer Sci. and Technol., Chongqing Univ. of Posts and Telecommunications, Chongqing 400065, China)
摘要
图/表
参考文献
相似文献
本文已被:浏览 1681次   下载 755
投稿时间:2018-10-20    修订日期:2019-05-08
中文摘要: 针对短文本数据特征少、提供信息有限,以及传统卷积神经网络(convolutional neural network,CNN)和循环神经网络(recurrent neural network,RNN)对短文本特征表示不充分的问题,提出基于串并行卷积门阀循环神经网络的文本分类模型,处理句子特征表示与短文本分类。该网络在卷积层中去除池化操作,保留文本数据的时序结构和位置信息,以串并行的卷积结构提取词语的多元特征组合,并提取局部上下文信息作为RNN的输入;以门阀循环单元(gated recurrent unit,GRU)作为RNN的组成结构,利用文本的时序信息生成句子的向量表示,输入带有附加边缘距离的分类器中,引导网络学习出具有区分性的特征,实现短文本的分类。实验中采用TREC、MR、Subj短文本分类数据集进行测试,对网络超参数选择和卷积层结构对分类准确率的影响进行仿真分析,并与常见的文本分类模型进行了对比实验。实验结果表明:去掉池化操作、采用较小的卷积核进行串并行卷积,能够提升文本数据在多元特征表示下的分类准确率。相较于相同参数规模的GRU模型,所提出模型的分类准确率在3个数据集中分别提升了2.00%、1.23%、1.08%;相较于相同参数规模的CNN模型,所提出模型的分类准确率在3个数据集中分别提升了1.60%、1.57%、0.80%。与Text-CNN、G-Dropout、F-Dropout等常见模型相比,所提出模型的分类准确率也保持最优。因此,实验表明所提出模型可改善分类准确率,可实际应用于短文本分类场景。
Abstract:In order to address the problems that the features and information is limited in short text, the short text features are not fully expressed by traditional convolutional neural network (CNN) and recurrent neural network (RNN), a text classification model named convolutional gated recurrent neural network was proposed to represent sentence feature vector and classify short texts. The pooling operation was removed in convolution layerof the model to retain sequential structure and location information in text data. Series-parallel convolution structure was used to extract multi-feature combination of words and local context information as the input of RNN. Then, the gated recurrent unit (GRU) was used as the structure of RNN to represent the sentence features based on the sequential information of text. The features were input to the classifier with additive margin to guide network to learn distinguishing features and realize short text classification. The short text classification data set TREC, MR, and Subj were applied for testing. The influence of network hyper-parameters selection and convolution layer structures on classification accuracy were simulated and analyzed, and common text classification models were compared in experiments. Experimental results demonstrated that the classification accuracy of text data was improved by removing the pooling operation and using smaller convolution kernels for series-parallel convolution in the multi-feature representation. Compared with the GRU with the same number of parameters, the classification accuracy of the proposed model was increased by 2.00%, 1.23% and 1.08% in three datasets respectively. Compared with the CNN with the same number of parameters, the classification accuracy of the proposed model was increased by 1.60%, 1.57% and 0.80% in three datasets respectively. Compared with Text-CNN, G-Dropout, F-Dropout and other common models, the classification results also kept best. Therefore, experiments showed that the classification accuracy was effectively improved by the proposed model, which could be applied to short text classification scenarios.
文章编号:201801160     中图分类号:TP391    文献标志码:
基金项目:国家自然科学基金项目(61673079);重庆市基础科学与前沿技术研究项目(cstc2016jcyjA1919)
作者简介:唐贤伦(1977-),男,教授.研究方向:模式识别;计算智能.E-mail:tangxl@cqupt.edu.cn
引用文本:
唐贤伦,林文星,杜一铭,王婷.基于串并行卷积门阀循环神经网络的短文本特征提取与分类[J].工程科学与技术,2019,51(4):125-132.
TANG Xianlun,LIN Wenxing,DU Yiming,WANG Ting.Short Text Feature Extraction and Classification Based on Serial-Parallel Convolutional Gated Recurrent Neural Network[J].Advanced Engineering Sciences,2019,51(4):125-132.