-검정.jpg){kind=logo}
- P-ISSN1225-598X
- E-ISSN2982-6292
기술 용어에 대한 한국어 정의 문장 자동 생성을 위한 순환 신경망 모델 활용 연구
Research on the Utilization of Recurrent Neural Networks for Automatic Generation of Korean Definitional Sentences of Technical Terms
김한국 (한국과학기술정보연구원)
김광훈 (한국과학기술정보연구원)
김유일 (한국과학기술정보연구원)
최성필 (경기대학교)
초록
본 논문에서는 지속적으로 커져가는 산업․시장에 대해 관련 연구자들이 이를 효율적으로 분석할 수 있는 반자동 지원 체제 개발을 위한 기술 용어와 기술 개념에 대한 정의문 및 설명문을 자동으로 생성하는 한국어 문장 생성 모델을 제시한다. 한국어 정의 문장 생성을 위하여 딥러닝 기술 중 데이터의 전/후 관계를 포함한 시퀀스 레이블링이 가능한 LSTM을 활용한다. LSTM을 근간으로 한 두 가지 모델은 기술명을 입력할 시 그에 대한 정의문 및 설명문을 생성한다. 다양하게 수집된 대규모 학습 말뭉치를 이용해 실험한 결과, 본 논문에서 구현한 2가지 모델 중 CNN 음절 임베딩을 활용한 어절 단위 LSTM 모델이 용어에 대한 정의문 및 설명문을 생성하는데 더 나은 결과를 도출시킨다는 사실을 확인하였다. 본 논문의 연구 결과를 바탕으로 동일한 주제를 다루는 문장 집합을 생성할 수 있는 확장 모델을 개발할 수 있으며 더 나아가서는 기술에 대한 문헌을 자동으로 작성하는 인공지능 모델을 구현할 수 있으리라 사료된다.
- keywords
- Sentence Generation, Text Generation, Natural Language Generation(NLG), Automatic Report Generation, Deep Learning, 문장 생성, 텍스트 생성, 자연어 생성, 보고서 자동 생성, 딥 러닝
Abstract
In order to develop a semiautomatic support system that allows researchers concerned to efficiently analyze the technical trends for the ever-growing industry and market. This paper introduces a couple of Korean sentence generation models that can automatically generate definitional statements as well as descriptions of technical terms and concepts. The proposed models are based on a deep learning model called LSTM (Long Sort-Term Memory) capable of effectively labeling textual sequences by taking into account the contextual relations of each item in the sequences. Our models take technical terms as inputs and can generate a broad range of heterogeneous textual descriptions that explain the concept of the terms. In the experiments using large-scale training collections, we confirmed that more accurate and reasonable sentences can be generated by CHAR-CNN-LSTM model that is a word-based LSTM exploiting character embeddings based on convolutional neural networks (CNN). The results of this study can be a force for developing an extension model that can generate a set of sentences covering the same subjects, and furthermore, we can implement an artificial intelligence model that automatically creates technical literature.
- keywords
- Sentence Generation, Text Generation, Natural Language Generation(NLG), Automatic Report Generation, Deep Learning, 문장 생성, 텍스트 생성, 자연어 생성, 보고서 자동 생성, 딥 러닝
참고문헌
Bahdanau, D., Cho, K., and Bengio, Y. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. In conference ICLR 2015.
Bauer, A., Hoedoro, N., and Schneider, A. 2015. Rule-based Approach to Text Generation in Natural Language-Automated Text Markup Language (ATML3). In Challenge+ DC@RuleML 2015.
Bian, J., Gao, B., and Liu, T. Y. 2014. Knowledge-powered Deep Learning for Word Embedding.In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, September 15-19, 2014, Nancy: 132-148.
Bontcheva, K., and Wilks, Y. 2004. Automatic Report Generation from Ontologies: the MIAKT Approach. In International Conference on Application of Natural Language to Information Systems, 324-335.
Boulanger-Lewandowski, N., Bengio, Y., and Vincent, P. 2012. Modeling Temporal Dependencies in High-dimensional Sequences: Application to Polyphonic Music Generation and Transcription.In Proceedings of the Twenty-nine International Conference on Machine Learning ICML.
Bowman, S. et al. 2016. Generating Sentences from a Continuous Space. In SIGNLL Conference on Computational Natural Language Learning (CONLL), 2016.
Deng, L., and Yu, D. 2014. Deep Learning: Methods and Applications. Foundations and Trends® in Signal Processing, 7(3-4), 197-387.
Graves, A., Jaitly, N., and Mohamed, A. R. 2013. Hybrid Speech Recognition with Deep Bidirectional LSTM. In Automatic Speech Recognition and Understanding (ASRU), 2013IEEE Workshop on, 273-278.
Hochreiter, S. 1991. Untersuchungen zu Dynamischen Neuronalen Netzen. Ph.D. diss., Institut fur Informatik, Technische Universitat, Munchen.
Hochreiter, S., and Schmidhuber, J. 1997. Long Short-Term Memory. Neural Computation.Neural Computation, 9(8), 1735-1780.
Kalchbrenner, N., Grefenstette, E., and Blunsom, P. 2014. A Convolutional Neural Network for Modelling Sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 655-665.
Krizhevsky, A., Sutskever, I., and Hinton, G. E. 2012. Imagenet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems, 60(6), 1097-1105.
Langkilde-Geary, I. 2002. An Empirical Verification of Coverage and Correctness for a General-purpose Sentence Generator. In Proceedings of the 12th International Natural Language Generation Workshop, 17-24.
Nallapati, R. et al. 2016. Abstractive Text Summarization using Sequence-to-sequence Rnns and Beyond. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), August 7-12, 2016, Berlin: 280-290.
Mairesse, F. 2005. Natural Language Generation: APT on Dialogue Models and Dialogue Systems. [online] [cited 2017. 6. 30.]<http://farm2.user.srcf.net/research/papers/ART-NLG.pdf>
Srivastava, R. K., Greff, K., and Schmidhuber, J. 2015. Tranining Very Deep Networks.In Advances in Neural Information Processing Systems, (2015a), 2377-2385.
Sundermeyer, M., Schlüter, R., and Ney, H. 2012. LSTM Neural Networks for Language Modeling. In Thirteenth Annual Conference of the International Speech Communication Association.
Sutskever, I., Martens, J., and Hinton, G. E. 2011. Generating Text with Recurrent Neural Networks. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), 1017-1024.
Wikipedia. 2017. kowiki-latest-abstract.xml. [online] [cited 2017. 6. 27.]<https://dumps.wikimedia.org/kowiki/latest/>
Woodward, A., Sood, B., and Hare, J. 2016. Market Share Analysis: Business Intelligence and Analytics Software, 2015. [online] [cited 2017. 6. 2.]<https://www.gartner.com/doc/3365832/market-share-analysis-business-intelligence>
Zheng, X., Chen, H., and Xu, T. 2013. Deep Learning for Chinese Word Segmentation and POS Tagging. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 647-657.
Zilly, J. G. et al. 2016. Recurrent Highway Networks. In Proceedings of the 34 th International Conference on Machine Learning.
- 다운로드 수
- 조회수
- 0KCI 피인용수
- 0WOS 피인용수