We show first that the pooling of multiple human judgments of relevance provides a predictor of relevance that is superior to that obtained from a single human's relevance judgments. A learning algorithm applied to a set of relevance judgments obtained from a single human would be expected to perform on new material at a level somewhat below that human. However, we examine two learning methods which when trained on the superior source of pooled human relevance judgments are able to perform at the level of a single human on new material. All performance comparisons are based on an independent human judge. Both algorithms function by producing term weights; one by a log odds calculation and the other by producing a least squares fit to human relevance ratings. Some characteristics of the algorithms are examined. An article has been published and further work remains to be done in examining learning methods applied to this system.

Agency
National Institute of Health (NIH)
Institute
National Library of Medicine (NLM)
Type
Intramural Research (Z01)
Project #
1Z01LM000083-04
Application #
6432760
Study Section
(CBB)
Project Start
Project End
Budget Start
Budget End
Support Year
4
Fiscal Year
2000
Total Cost
Indirect Cost
Name
National Library of Medicine
Department
Type
DUNS #
City
State
Country
United States
Zip Code