Feature selection is an important problem in statistical machine learning, and is a common method for dimensionality reduction that encourages model interpretability. Classical feature selection asks for a subset of features that are most informative for the entire data set.
Recently, model interpretation has gained increasing attention in the research community. Modern machine learning models can be difficult to probe and understand after they have been trained. This is a major problem for the field, with consequences for trustworthiness, diagnostics, debugging, robustness, and a range of other engineering and human interaction issues surrounding the deployment of a model. Given a predictive model, an interpretation method yields, for each instance to which the model is applied, a vector of importance scores associated with the underlying features. Similar to classical feature selection, it evaluates importance features. However, there are two main differences. First, such methods are applied in an instancewise fashion. Second, feature importance is evaluated with respect to a given model, instead of the data set.
Our research covers both classical feature selection, and model interpretation. We have developed both parametric and nonparametric methods for feature selection. Our ultimate goal is to provide a better interpretation of complex models such as deep neural networks, kernel methods, and tree ensembles.