user2016

This talk reviews some of the most relevant statistical approaches for outlier and anomaly detection in R. It covers statistical approaches, clustering based approaches, nearest neighbor approaches, random forest, and autoencoders. This talk will pull from a variety of existing R packages including DMwR, fclust, dbscan, isolation forest, and autoencoder. nThe talk is relevant because outlier detection is a necessary step to clean the data and in other instances, the outliers may be of interest themselves. For example, identification of credit card fraud involves identifying outliers. While the need for outlier detection has increased, there has also been a rise in the number of techniques for identifying outliers. nThis talk provides a theoretical background for each approach. This provides an understanding of the assumptions and limitations of each approach. This will then be demonstrated with examples of different datasets to show how performance varies for differing approaches. The specific methods include: Extreme Value, Expectation Maximization, Kmeans, Fuzzy Clustering, DBSCAN, Isolation Forests, an Autoencoders. nThis talk includes sharing/demonstrating two interactive shiny apps that illustrate these algorithms. The first is for low dimensional data and is available at: http://projects.rajivshah.com/shiny/outlier/ The second is a shiny app that must be run locally on a computer (due to computational requirements), for higher dimension methods. By including a shiny application, this allows people to try these different methods out on datasets. In my experience, these sort of talks resonate much better, because the audience can be directly involved.