Is there anyone working on either a decentralized deep learning algorithm, or a consumer facing app that uses AI to help people diagnose themselves? She's had this her entire life. She was misdiagnosed 3 or 4 times, most recently she was eating gluten free for the last 8 years because she was diagnosed as celiac disease. She's lost most of her hair over the last 6 months and has been in the hospital 3-4 times this year. It turns out, she never had celiac, she has always had CVID.
The Media is going crazy about Bitcoin, Ethereum and the rise of crypto markets. Entrepreneurs from the sector have kind of a Rockstar status raising millions of USD in seconds through ICOs. However, the crypto sector is much more than Bitcoin, fintech, trading and crypto currencies -- it's about building a better, decentralized, (digital) world. By using the term "decentralization" I refer to a process of redistributing functions, people, powers or things away from a central authority. The problem with centralized systems is that they lack transparency, allow for single points of failure, censorship, abuse of power and inefficiencies.
Unique cryptocurrencies are multiplying so fast these days it's hard to keep up. Aram Barnett, CEO of the crypto hedge fund Alluminate, told International Business Times his team has evaluated more than 2,000 tokens. There is a ton of money flowing through these platforms, although online exchange markets get hacked all the time. Unfortunately, exchanges don't have the same protections as banks. So when hackers steal from an exchange, individual customers can lose millions of dollars worth of bitcoin and blockchain-based assets.
In this paper, we propose and analyze SPARQ-SGD, which is an event-triggered and compressed algorithm for decentralized training of large-scale machine learning models. Each node can locally compute a condition (event) which triggers a communication where quantized and sparsified local model parameters are sent. In SPARQ-SGD each node takes at least a fixed number (H) of local gradient steps and then checks if the model parameters have significantly changed compared to its last update; it communicates further compressed model parameters only when there is a significant change, as specified by a (design) criterion. We prove that the SPARQ-SGD converges as O(1/nT) and O(1/ (nT)) in the strongly-convex and non-convex settings, respectively, demonstrating that such aggressive compression, including event-triggered communication, model sparsification and quantization does not affect the overall convergence rate as compared to uncompressed decentralized training; thereby theoretically yielding communication efficiency for "free". We evaluate SPARQ-SGD over real datasets to demonstrate significant amount of savings in communication over the state-of-the-art.