Domain-independent automatic keyphrase indexing with small training sets

Keyphrases are widely used in both physical and digital libraries as a brief, but precise, summary of documents. They help organize material based on content, provide thematic access, represent search results, and assist with navigation. Manual assignment is expensive because trained human indexers...

Full description

Saved in:

Bibliographic Details
Published in:	Journal of the American Society for Information Science and Technology 2008-05, Vol.59 (7), p.1026-1040
Main Authors:	Medelyan, Olena, Witten, Ian H.
Format:	Article
Language:	eng
Subjects:	Automatic indexing Cataloging rules Exact sciences and technology French language Indexing Information and communication sciences Information processing and retrieval Information retrieval Information retrieval. Man machine relationship Information science. Documentation Jargon Libraries Library cataloging Research process. Evaluation Sciences and techniques of general use Search strategies Semantics Spanish language Studies Thesauri Training Training sets
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Description
Summary:	Keyphrases are widely used in both physical and digital libraries as a brief, but precise, summary of documents. They help organize material based on content, provide thematic access, represent search results, and assist with navigation. Manual assignment is expensive because trained human indexers must reach an understanding of the document and select appropriate descriptors according to defined cataloging rules. We propose a new method that enhances automatic keyphrase extraction by using semantic information about terms and phrases gleaned from a domain‐specific thesaurus. The key advantage of the new approach is that it performs well with very little training data. We evaluate it on a large set of manually indexed documents in the domain of agriculture, compare its consistency with a group of six professional indexers, and explore its performance on smaller collections of documents in other domains and of French and Spanish documents.
ISSN:	1532-2882 1532-2890