Reviews: Cyclades: Conflict-free Asynchronous Machine Learning

The Cyclades method actually only conditionally achieves the serial equivalence according to Theorem 1. The conflict degree is highly related to the sparsity of the data, and the condition of Theorem 1 cannot be satisfied in many practical scenarios. Even for sparse applications, when the data block is large, or the delay among different local machines is large, the serial equivalence cannot be achieved either. To some extent, Cyclades adds a data partitioning step to Hogwild, and this data partitioning is done by leveraging existing bipartite allocation algorithms. However, speed up is only a system measure, and from the machine learning perspective, we care more about the balance between accuracy and efficiency.