- P-ISSN1013-0799
- E-ISSN2586-2073
- KCI
럿거스 정보검색 평가 프로젝트에 관한 연구
Rutgers Information Retrieval Evaluation Project on IR Performance on Different Precision Levels
초록
이 논문의 주요목적은 정보이용자들이 어떤 수준의 정확률 차이에서 유의미하게 차이를 인지하는지를 알아보고자 하는 것이다. 그에 관련한 몇 가지 흥미 있는 결과가 도출되었다. 그 외에 적합성 판정은 이용자의 판정시간과 관계가 없는 것으로 나타났다. 그리고 주제에 대한 이용자의 배경지식과 적합성 판정의 관계가 두드러졌다. 또한, 적합문서의 숫자가 적었을 때 이용자들은 적합성 판정에 더욱 어려움을 겪었다. 마지막으로, 검색결과리스트중 상위 N 문서의 적합성 판정에 대한 중요성을 확인할 수 있었다.
- keywords
- Precision, Relevance Judgement, Information Retrieval System Evaluation, User Behavior., Precision, Relevance Judgement, Information Retrieval System Evaluation, User Behavior., 정확률, 적합성판정, 정보검색시스템 평가, 이용자행태
Abstract
The purpose of this study is to investigate what level of difference in precision would be significantly perceived by a human user of an information retrieval system. Not many researches have been conducted with regards to this issue in information retrieval field. Despite the non-significant results, there were several interesting findings in recognizing different levels of precision rates. The correctness of relevance task had little to do with the taken time for the task. In addition, the strong relationship between the subjects' topic familiarity and rate of correct judgments is one of the most interesting results in this study. It turned out that the subjects have more difficulty in a situation they have to judge between the two lists having more non-relevant documents than in a situation they do between the lists having more relevant documents. Finally, the serious influence from the first top N documents in a list for relevance judgment task has been confirmed.
- keywords
- Precision, Relevance Judgement, Information Retrieval System Evaluation, User Behavior., Precision, Relevance Judgement, Information Retrieval System Evaluation, User Behavior., 정확률, 적합성판정, 정보검색시스템 평가, 이용자행태
참고문헌
Saracevic, T. (1969). Comparative effect of titles, abstracts and full texts on relevance judgments.. Proceedings of the American Society for Information Science, 6, 293-299.
Sormunen, E. (2002). iberal Relevance Criteria of TREC - Counting on Negligible Documents? (324-330). Proceedings of the Twenty-Fifth Annual ACM SIGIR Conference on Research and Development in Information Retrieval (Tampere, Finland August 11-15, 2002), Special Issue of SIGIR Forum,.
Veerasamy, A. Effectiveness of a graphical display of retrieval results. (236-245). Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 97),.
(1995). NIST Special Publication 500-236 (-). The Fourth Text REtrieval Conference (TREC-4): the proceedings of the fourth Text REtrieval Conference (TREC-4).
(1997). NIST Special Publication 500-240. (-). The Sixth Text REtrieval Conference (TREC-6): the proceedings of the sixth Text REtrieval Conference (TREC-6.
- 다운로드 수
- 조회수
- 0KCI 피인용수
- 0WOS 피인용수