-검정.jpg){kind=logo}
- P-ISSN1225-598X
- E-ISSN2982-6292
뇌파정보를 활용한 영상물 요약 알고리즘 설계와 평가
Design and Evaluation of Video Summarization Algorithm based on EEG Information
김용호 (부경대학교)
초록
본 연구는 비디오 스킴의 자동 생성을 위한 비디오 요약 알고리즘을 제안하고 이를 평가하였다. 제안된 알고리즘은 ERP(Event Related Potentials) 기반의 주제 적합성 모형, MMR(Maximal Marginal Relevance) 기법 및 판별분석기법을 사용하여 구현하였다. 제안한 ERP/MMR 기반 알고리즘을 이용하여 구성한 비디오 스킴의 품질과 유용성을 내재적 및 외재적 평가를 통해서 검증하였다. 내재적 및 외재적 평가에서 ERP/MMR 방법들의 평가 점수들은 각각 경쟁 기준으로 사용한 SBD(Shot Boundary Detection) 방법의 평가 점수 보다 유의미한 차이를 보이며 높게 나왔다. 그러나 이 두 평가에서 ERP/MMR(λ=0.6) 방법의 평가 점수와 ERP/MMR(λ=1.0) 방법의 평가 점수 간에 통계적으로 유의미한 차이는 없는 것으로 나타났다.
- keywords
- 사건관련유발전위, ERP/MMR 모형, 주제 적합성 모형, 비디오 스킴, 내재적 평가, 외재적 평가, ERP, ERP/MMR Model, Topic Relevance Model, Video Skim, Implicit Evaluation, Explicit Evaluation
Abstract
We proposed a video summarization algorithm based on an ERP (Event Related Potentials)-based topic relevance model, a MMR (Maximal Marginal Relevance), and discriminant analysis to generate a semantically meaningful video skim. We then conducted implicit and explicit evaluations to evaluate our proposed ERP/MMR-based method. The results showed that in the implicit and explicit evaluations, the average scores of the ERP / MMR methods were statistically higher than the average score of the SBD (Shot Boundary Detection) method used as a competitive baseline, respectively. However, there was no statistically significant difference between the average score of ERP/MMR (λ = 0.6) method and that of ERP/MMR (λ = 1.0) method in both assessments.
- keywords
- 사건관련유발전위, ERP/MMR 모형, 주제 적합성 모형, 비디오 스킴, 내재적 평가, 외재적 평가, ERP, ERP/MMR Model, Topic Relevance Model, Video Skim, Implicit Evaluation, Explicit Evaluation
참고문헌
권준수. 2000. 인지 기능 연구에서의 사건관련전위의 이용. 인지과학작업, 1(1), 79-98.
김용호, 김현희. 2016. 실시간 뇌파반응을 이용한 주제관련 영상물 쇼트 자동추출기법 개발연구:ERP의 N400 및 P600 가설검증을 중심으로. 멀티미디어학회논문지, 19(8), 1260-1274.
정영미. 2012. 정보검색연구. 서울: 연세대학교 대학출판문화원.
Allegretti, M. et al. 2015. When Relevance Judgement is Happening?: An EEG-based study." In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 719-722). New York: ACM.
Burmester, J., Spalek, K. and Wartenburger, I. 2014. Context Updating during Sentence Comprehension: The Effect of Aboutness Topic. Brain and Language, 137, 62-76.
Carbonell, J. and Goldstein, J. 1998. The Use of MMR, Diversity-based Reranking for Reordering Documents and Producing Summaries. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 335-336). New York: ACM.
Chen, F., Delannay, D. and De Vleeschouwer, C. 2011. An Autonomous Framework to Produce and Distribute Personalized Team-sport Video Summaries: A Basketball Case Study.IEEE Transactions on Multimedia, 13(6), 1381-1394.
Eugster, M. J. et al. 2016. Natural Brain-information Interfaces: Recommending Information by Relevance Inferred from Human Brain Signals. Scientific Reports, 6(38580), 1-10.
Evans, W. J., Cui, L. and Starr, A. 1995. Olfactory Event-related Potentials in Normal Human Subjects: Effects of Age and Gender. Electroencephalography and Clinical Neurophysiology, 95(4), 293-301.
Hu, W. et al. 2011. A Survey on Visual Content-based Video Indexing and Retrieval.IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 41(6), 797-819.
IBM Research Corp. 2016. Morgan | IBM Creates First Movie Trailer by AI. [online] [cited 2018. 8. 3.] <https://www.youtube.com/watch?v=gJEzuYynaiw>
Kim, H. H. and Kim, Y. H. 2010. Toward a Conceptual Framework of Key-frame Extraction and Storyboard Display for Video Summarization. Journal of the Association for Information Science and Technology, 61(5), 927-939.
Lin, C. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004), Barcelona, Spain.
Luck, S. J. 2014. An Introduction to the Event-related Potential Technique. Cambridge:MIT press.
Mayer, R. E. 2009. Multimedia Learning. New York: Cambridge University Press.
Mehmood, I. et al. 2016. Divide-and-conquer based Summarization Framework for Extracting Affective Video Content. Neurocomputing, 174, 393-403.
Mishra, R. et al. 2015. Real time and Non Real time Video Shot Boundary Detection using Dual Tree Complex Wavelet Transform. 2015 International Conference on Industrial Instrumentation and Control (ICIC, pp. 1495-1500). New York: IEEE.
Mostafa, J. and Gwizdka, J. 2016. Deepening the Role of the User: Neuro-Physiological Evidence as a Basis for Studying and Improving Search. In Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval (pp. 63-70). New York:ACM.
Mudrik, L. et al. 2014. Synchronous Contextual Irregularities Affect Early Scene Processing:Replication and Extension. Neuropsychologia, 56, 447-458.
Nakano, H. et al. 2014. Electrophysiological Response to Omitted Stimulus in Sentence Processing. NeuroReport, 25(14), 1169-1174.
Over, P., Smeaton, A. F. and Awad, G. 2008. The TRECVid 2008 BBC Rushes Summarization Evaluation. In Proceedings of the 2nd ACM TRECVid Video Summarization Workshop (pp. 1-20). New York: ACM.
Schumacher, P. B. and Hung, Y. C. 2012. Positional Influences on Information Packaging:Insights from Topological Fields in German. Journal of Memory and Language, 67(2), 295-310.
Seoane, L. F., Gabler, S. and Blankertz, B. 2015. Images from the Mind: BCI Image Evolution based on Rapid Serial Visual Presentation of Polygon Primitives. Brain-Computer Interfaces, 2(1), 40-56.
Sitnikova, T. et al. 2008. Two Neurocognitive Mechanisms of Semantic Integration during the Comprehension of Visual Real-world Events. Journal of Cognitive Neuroscience, 20(11), 2037-2057.
Tavassolipour, M., Karimian, M. and Kasaei, S. 2014. Event Detection and Summarization in Soccer Videos using Bayesian Network and Copula. IEEE Transactions on Circuits and Systems for Video technology, 24(2), 291-304.
Wang, L. and Schumacher, P. B. 2013. New is not always Costly: Evidence from Online Processing of Topic and Contrast in Japanese. Frontiers in Psychology, 4, 363.
Xu, R. et al. 2014. Enhanced low-latency Detection of Motor Intention from EEG for Closedloop Brain-computer Interface Applications. IEEE Transactions on Biomedical Engineering, 61(2), 288-296.
Yang, J. et al. 2012. Channel Selection and Classification of Electroencephalogram Signals:An Artificial Neural Network and Genetic Algorithm-based Approach. Artificial Intelligence in Medicine, 55(2), 117-126.
Zhu, X. et al. 2007. A Text-to-picture Synthesis System for Augmenting Communication.In Proceedings of the 22nd National Conference on Artificial Intelligence-Volume 2 (pp. 1590-1595). Menlo Park, CA: AAAI Press.
- 다운로드 수
- 조회수
- 0KCI 피인용수
- 0WOS 피인용수