Ensemble‐ and distance‐based feature ranking for unsupervised learning

In this study, we propose two novel (groups of) methods for unsupervised feature ranking and selection. The first group includes feature ranking scores (Genie3 score, RandomForest score) that are computed from ensembles of predictive clustering trees. The second method is URelief, the unsupervised e...

Full description

Saved in:
Bibliographic Details
Published in:International journal of intelligent systems 2021-07, Vol.36 (7), p.3068-3086
Main Authors: Petković, Matej, Kocev, Dragi, Škrlj, Blaž, Džeroski, Sašo
Format: Article
Language:English
Subjects:
Algorithms
Clustering
Computation
extra trees
feature ranking
Intelligent systems
Parameters
Ranking
relief
tree ensembles
Trees
Unsupervised learning
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this study, we propose two novel (groups of) methods for unsupervised feature ranking and selection. The first group includes feature ranking scores (Genie3 score, RandomForest score) that are computed from ensembles of predictive clustering trees. The second method is URelief, the unsupervised extension of the Relief family of feature ranking algorithms. Using 26 benchmark data sets and 5 baselines, we show that both the Genie3 score (computed from the ensemble of extra trees) and the URelief method outperform the existing methods and that Genie3 performs best overall, in terms of predictive power of the top‐ranked features. Additionally, we analyze the influence of the hyper‐parameters of the proposed methods on their performance and show that for the Genie3 score the highest quality is achieved by the most efficient parameter configuration. Finally, we propose a way of discovering the location of the features in the ranking, which are the most relevant in reality.
ISSN:0884-8173
1098-111X
DOI:10.1002/int.22390