Adaptive Factorization Network: Learning Adaptive-Order Feature Interactions

V arious factorization-based methods have been proposed to leverage second-order, or higher-order cross features for boosting the performance of predictive models. They generally enumerate all the cross features under a predefined maximum order, and then identify useful feature interactions through model training, which suffer from two drawbacks. First, they have to make a tradeoff between the expressiveness of higher-order cross features and the computational cost, resulting in suboptimal predictions. Second, enumerating all the cross features, including irrelevant ones, may introduce noisy feature combinations that degrade model performance. In this work, we propose the Adaptive Factorization Network (AFN), a new model that learns arbitrary-order cross features adaptively from data. The core of AFN is a logarithmic transformation layer to convert the power of each feature in a feature combination into the coefficient to be learned. The experimental results on four real datasets demonstrate the superior predictive performance of AFN against the start-of-the-arts. 1 Introduction Feature engineering is typically recognized as central to successful machine learning tasks, such as recommender systems (Lian et al. 2017), computational advertising (He et al. 2014) and search ranking (Lian and Xie 2016). Except for exploiting raw features, it is usually crucial to find effective transformations of raw features to boost the performance of predictive models. Cross features are a major type of feature transformations, where multiplication is performed over sparse raw features to form new features (Cheng et al. 2016). However, handcrafting useful cross features is inevitably expensive and time-consuming, and the results may not generalize to unseen feature interactions.