Unsupervised low-dimensional vector representations for words, phrases and text that are transparent, scalable, and produce similarity metrics that are not redundant with neural embeddings

[Display omitted] •Low-dimensional vector representations for words, phrases and text are popular.•Current vectors lack transparency and/or require elaborate training and tuning.•We present novel vectors that overcome these issues.•Our vectors perform well on term similarity benchmarks.•The datasets...

Full description

Saved in:
Bibliographic Details
Published in:Journal of biomedical informatics 2019-02, Vol.90, p.103096-103096, Article 103096
Main Authors: Smalheiser, Neil R., Cohen, Aaron M., Bonifield, Gary
Format: Article
Language:English
Subjects:
Data Mining
Dimensional reduction
Implicit features
Natural language processing
PubMed
Pvtopic
Semantic similarity
Semantics
Text mining
Vector representation
Word2vec
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:[Display omitted] •Low-dimensional vector representations for words, phrases and text are popular.•Current vectors lack transparency and/or require elaborate training and tuning.•We present novel vectors that overcome these issues.•Our vectors perform well on term similarity benchmarks.•The datasets are public for query and download. Neural embeddings are a popular set of methods for representing words, phrases or text as a low dimensional vector (typically 50–500 dimensions). However, it is difficult to interpret these dimensions in a meaningful manner, and creating neural embeddings requires extensive training and tuning of multiple parameters and hyperparameters. We present here a simple unsupervised method for representing words, phrases or text as a low dimensional vector, in which the meaning and relative importance of dimensions is transparent to inspection. We have created a near-comprehensive vector representation of words, and selected bigrams, trigrams and abbreviations, using the set of titles and abstracts in PubMed as a corpus. This vector is used to create several novel implicit word-word and text-text similarity metrics. The implicit word-word similarity metrics correlate well with human judgement of word pair similarity and relatedness, and outperform or equal all other reported methods on a variety of biomedical benchmarks, including several implementations of neural embeddings trained on PubMed corpora. Our implicit word-word metrics capture different aspects of word-word relatedness than word2vec-based metrics and are only partially correlated (rho = 0.5–0.8 depending on task and corpus). The vector representations of words, bigrams, trigrams, abbreviations, and PubMed title + abstracts are all publicly available from http://arrowsmith.psych.uic.edu/arrowsmith_uic/word_similarity_metrics.html for release under CC-BY-NC license. Several public web query interfaces are also available at the same site, including one which allows the user to specify a given word and view its most closely related terms according to direct co-occurrence as well as different implicit similarity metrics.
ISSN:1532-0464
1532-0480
DOI:10.1016/j.jbi.2019.103096