Rule based Learning Ensembles

RuleFit3 with  R

Beta version (7/07/12)

RuleFittm implements the learning method and interpretational tools described in Predictive Learning via Rule Ensembles (pdf).

RuleFit3tm is a newer version with improved analytics and some additional options. A principal improvement is the use of glmnet (Friedman, Hastie, and Tibshirani 2008)  to perform rule fitting . This provides the full sprectum of elastic net procedures from the lasso to ridge regression, thereby allowing the user to decrease the sparsity  (increase the number of terms)  in the final model from that provided by the lasso.  Sparser models than the lasso can be obtained by choosing forward stepwise or forward statewise rule fitting. Also a different method for cross-validated model selection  is implemented that usually results in more accurate models and more honest (less optimistic) estimates of future prediction error. This is especially the case for smaller training data sets. There is also a wider variety of model selection criteria from which to choose for selecting the number of terms in the final model. The previous version of RuleFittm can be obtainded from here.

The files provided here implement a rudimentary interface for using RuleFit3tm with the R statistical package.  R is freely available from The R Project for Statistical Computing. This R/RuleFit3 interface runs on PC compatible computers with Windows XP/Vista/7, Linux, and Mac OS X (version 10.6 or later). This present version of the R/RuleFit3 interface will not function after December 29, 2014. Future versions will be available after that date. Note that this is a beta version of the program. Please report any problems to and periodically check this site for updated versions.

Installation Instructions:

It is assumed that R has already been installed on the computer. The first step is to create a (target) directory in which to store the downloaded files. This directory should be used only for the R/RuleFit installation and contain no other files or sub-directories. Its full path name must contain NO imbedded blanks. For Windows, this directory must reside on the same logical disk where R is installed.

                                          Windows XT/Vista/7


                                 Mac OS X (version 10.6 or later)


Complete documentation for all R/RuleFit procedures is provided in the html help file RuleFit_help.html . Some familiarity with the above paper is recommended, especially when experimenting with some of the more advanced features and options.