바로가기메뉴

본문 바로가기 주메뉴 바로가기

logo

  • P-ISSN1013-0799
  • E-ISSN2586-2073
  • KCI

Article Contents

Prev Next
e-Submission

Vol.23 No.2

Citation Share

Rutgers Information Retrieval Evaluation Project on IR Performance on Different Precision Levels

Journal of the Korean Society for Information Management / Journal of the Korean Society for Information Management, (P)1013-0799; (E)2586-2073
2006, v.23 no.2, pp.97-111
https://doi.org/10.3743/KOSIM.2006.23.2.097

(2006). Rutgers Information Retrieval Evaluation Project on IR Performance on Different Precision Levels. Journal of the Korean Society for Information Management, 23(2), 97-111, https://doi.org/10.3743/KOSIM.2006.23.2.097

Abstract

The purpose of this study is to investigate what level of difference in precision would be significantly perceived by a human user of an information retrieval system. Not many researches have been conducted with regards to this issue in information retrieval field. Despite the non-significant results, there were several interesting findings in recognizing different levels of precision rates. The correctness of relevance task had little to do with the taken time for the task. In addition, the strong relationship between the subjects' topic familiarity and rate of correct judgments is one of the most interesting results in this study. It turned out that the subjects have more difficulty in a situation they have to judge between the two lists having more non-relevant documents than in a situation they do between the lists having more relevant documents. Finally, the serious influence from the first top N documents in a list for relevance judgment task has been confirmed.

keywords
Precision, Relevance Judgement, Information Retrieval System Evaluation, User Behavior., Precision, Relevance Judgement, Information Retrieval System Evaluation, User Behavior., 정확률, 적합성판정, 정보검색시스템 평가, 이용자행태

Reference

1.

Saracevic, T. (1969). Comparative effect of titles, abstracts and full texts on relevance judgments.. Proceedings of the American Society for Information Science, 6, 293-299.

2.

Sormunen, E. (2002). iberal Relevance Criteria of TREC - Counting on Negligible Documents? (324-330). Proceedings of the Twenty-Fifth Annual ACM SIGIR Conference on Research and Development in Information Retrieval (Tampere, Finland August 11-15, 2002), Special Issue of SIGIR Forum,.

3.

Veerasamy, A. Effectiveness of a graphical display of retrieval results. (236-245). Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 97),.

4.

(1995). NIST Special Publication 500-236 (-). The Fourth Text REtrieval Conference (TREC-4): the proceedings of the fourth Text REtrieval Conference (TREC-4).

5.

(1997). NIST Special Publication 500-240. (-). The Sixth Text REtrieval Conference (TREC-6): the proceedings of the sixth Text REtrieval Conference (TREC-6.

Journal of the Korean Society for Information Management

e-Submission OA Policy