Learning in recurrent neural networks has been a topic fraught with difficulties and problems. We here report substantial progress in the unsupervised learning of recurrent networks that can keep track of an input signal. Specifically, we show how these networks can learn to efficiently represent their present and past inputs, based on local learning rules only. Our results are based on several key insights. First, we develop a local learning rule for the recurrent weights whose main aim is to drive the network into a regime where, on average, feedforward signal inputs are canceled by recurrent inputs.

This is the accepted version of an article with the same name, published in the Special Issue "Federated Learning and Blockchain Supported Smart Networking in Beyond 5G (B5G) Wireless Communication" in Computer Networks. Abstract Federated machine learning (FL) allows to collectively train models on sensitive data as only the clients' models and not their training data need to be shared. However, despite the attention that research on FL has drawn, the concept still lacks broad adoption in practice. One of the key reasons is the great challenge to implement FL systems that simultaneously achieve fairness, integrity, and privacy preservation for all participating clients. To contribute to solving this issue, our paper suggests a FL system that incorporates blockchain technology, local differential privacy, and zero-knowledge proofs. Our implementation of a proof-of-concept with multiple linear regression illustrates that these state-of-the-art technologies can be combined to a FL system that aligns economic incentives, trust, and confidentiality requirements in a scalable and transparent system. A Blockchain blockchain eliminates the need for a centralized authority, provides transparency, enforces the federated learning protocol, and provides a decentralized infrastructure for the collection of fees and the distribution of rewards. The reward payment is calculated based on the client's clients' Federated learning enables multiple clients FIM Research Center 1. Introduction The application of machine learning (ML) promises far-reaching potentials across industries [1]. ML has already proven successful in many areas, such as web search or recommender systems in e-commerce, in which a lot of high-quality data exists [2]. While researchers address ML's growing demand for compute power and use of data with, e.g., distributed ML approaches where multiple computing nodes share their resources [3, 4, 5] and quality issues with data processing, access to data is not only a technical issue. Both traditional ML and distributed ML approaches assume that their training data is centralized by nature, preventing the applicability of ML approaches to domains in which data is sensitive and distributed at the same time. To avoid that ML approaches must rely on data to which only a centralized organization or individual has full access, federated machine learning (FL) can aggregate the less sensitive ML models that were independently and locally trained by individual clients [6, 7].

Learning in recurrent neural networks has been a topic fraught with difficulties and problems. We here report substantial progress in the unsupervised learning of recurrent networks that can keep track of an input signal. Specifically, we show how these networks can learn to efficiently represent their present and past inputs, based on local learning rules only. Our results are based on several key insights. First, we develop a local learning rule for the recurrent weights whose main aim is to drive the network into a regime where, on average, feedforward signal inputs are canceled by recurrent inputs.