How to Compare Machine Learning Algorithms

Under the RAM model [1], the "time" an algorithm takes is measured by the elementary operations of the algorithm. While users and developers may concern more about the wall clock time an algorithm takes to train the models, it would be fairer to use the standard worst case computational time complexity to compare the time the models take to train. Using computational complexity has the benefits of ignoring the differences like the computer power and architecture used at runtime and the underlying programming language, allowing users to focus on the fundamental differences of the elementary operations of the algorithms. Note that the time complexity can be very different during training and testing. For example, parametric models like linear regression could have long training time but they are efficient during test time.