Journal of the Korean Society for Library and Information Science
- Apply for Authority
- P-ISSN1225-598X
- E-ISSN2982-6292
Research on the Utilization of Recurrent Neural Networks for Automatic Generation of Korean Definitional Sentences of Technical Terms
- Downloaded
- Viewed
Abstract
In order to develop a semiautomatic support system that allows researchers concerned to efficiently analyze the technical trends for the ever-growing industry and market. This paper introduces a couple of Korean sentence generation models that can automatically generate definitional statements as well as descriptions of technical terms and concepts. The proposed models are based on a deep learning model called LSTM (Long Sort-Term Memory) capable of effectively labeling textual sequences by taking into account the contextual relations of each item in the sequences. Our models take technical terms as inputs and can generate a broad range of heterogeneous textual descriptions that explain the concept of the terms. In the experiments using large-scale training collections, we confirmed that more accurate and reasonable sentences can be generated by CHAR-CNN-LSTM model that is a word-based LSTM exploiting character embeddings based on convolutional neural networks (CNN). The results of this study can be a force for developing an extension model that can generate a set of sentences covering the same subjects, and furthermore, we can implement an artificial intelligence model that automatically creates technical literature.
- keywords
- Sentence Generation, Text Generation, Natural Language Generation(NLG), Automatic Report Generation, Deep Learning, 문장 생성, 텍스트 생성, 자연어 생성, 보고서 자동 생성, 딥 러닝
Reference
Bahdanau, D., Cho, K., and Bengio, Y. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. In conference ICLR 2015.
Bauer, A., Hoedoro, N., and Schneider, A. 2015. Rule-based Approach to Text Generation in Natural Language-Automated Text Markup Language (ATML3). In Challenge+ DC@RuleML 2015.
Bian, J., Gao, B., and Liu, T. Y. 2014. Knowledge-powered Deep Learning for Word Embedding.In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, September 15-19, 2014, Nancy: 132-148.
Bontcheva, K., and Wilks, Y. 2004. Automatic Report Generation from Ontologies: the MIAKT Approach. In International Conference on Application of Natural Language to Information Systems, 324-335.
Boulanger-Lewandowski, N., Bengio, Y., and Vincent, P. 2012. Modeling Temporal Dependencies in High-dimensional Sequences: Application to Polyphonic Music Generation and Transcription.In Proceedings of the Twenty-nine International Conference on Machine Learning ICML.
Bowman, S. et al. 2016. Generating Sentences from a Continuous Space. In SIGNLL Conference on Computational Natural Language Learning (CONLL), 2016.
Deng, L., and Yu, D. 2014. Deep Learning: Methods and Applications. Foundations and Trends® in Signal Processing, 7(3-4), 197-387.
Graves, A., Jaitly, N., and Mohamed, A. R. 2013. Hybrid Speech Recognition with Deep Bidirectional LSTM. In Automatic Speech Recognition and Understanding (ASRU), 2013IEEE Workshop on, 273-278.
Hochreiter, S. 1991. Untersuchungen zu Dynamischen Neuronalen Netzen. Ph.D. diss., Institut fur Informatik, Technische Universitat, Munchen.
Hochreiter, S., and Schmidhuber, J. 1997. Long Short-Term Memory. Neural Computation.Neural Computation, 9(8), 1735-1780.
Kalchbrenner, N., Grefenstette, E., and Blunsom, P. 2014. A Convolutional Neural Network for Modelling Sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 655-665.
Krizhevsky, A., Sutskever, I., and Hinton, G. E. 2012. Imagenet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems, 60(6), 1097-1105.
Langkilde-Geary, I. 2002. An Empirical Verification of Coverage and Correctness for a General-purpose Sentence Generator. In Proceedings of the 12th International Natural Language Generation Workshop, 17-24.
Nallapati, R. et al. 2016. Abstractive Text Summarization using Sequence-to-sequence Rnns and Beyond. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), August 7-12, 2016, Berlin: 280-290.
Mairesse, F. 2005. Natural Language Generation: APT on Dialogue Models and Dialogue Systems. [online] [cited 2017. 6. 30.]<http://farm2.user.srcf.net/research/papers/ART-NLG.pdf>
Srivastava, R. K., Greff, K., and Schmidhuber, J. 2015. Tranining Very Deep Networks.In Advances in Neural Information Processing Systems, (2015a), 2377-2385.
Sundermeyer, M., Schlüter, R., and Ney, H. 2012. LSTM Neural Networks for Language Modeling. In Thirteenth Annual Conference of the International Speech Communication Association.
Sutskever, I., Martens, J., and Hinton, G. E. 2011. Generating Text with Recurrent Neural Networks. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), 1017-1024.
Wikipedia. 2017. kowiki-latest-abstract.xml. [online] [cited 2017. 6. 27.]<https://dumps.wikimedia.org/kowiki/latest/>
Woodward, A., Sood, B., and Hare, J. 2016. Market Share Analysis: Business Intelligence and Analytics Software, 2015. [online] [cited 2017. 6. 2.]<https://www.gartner.com/doc/3365832/market-share-analysis-business-intelligence>
Zheng, X., Chen, H., and Xu, T. 2013. Deep Learning for Chinese Word Segmentation and POS Tagging. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 647-657.
Zilly, J. G. et al. 2016. Recurrent Highway Networks. In Proceedings of the 34 th International Conference on Machine Learning.
- Downloaded
- Viewed
- 0KCI Citations
- 0WOS Citations