Neural networks with redundant representation: detecting the undetectable

Neural networks with redundant representation: detecting the undetectable Elena Agliari, 1, Francesco Alemanno, 2, 3 Adriano Barra, 2, 4 Martino Centonze, 2 and Alberto Fachechi 2, 4 1 Dipartimento di Matematica "Guido Castelnuovo", Sapienza Università di Roma, Roma, Italy 2 Dipartimento di Matematica e Fisica "Ennio De Giorgi", Università del Salento, Lecce, Italy 3 C.N.R. Nanotec, Lecce, Italy 4 Istituto Nazionale di Fisica Nucleare, Sezione di Lecce, Italy (Dated: December 2, 2019) We consider a three-layer Sejnowski machine and show that features learnt via contrastive divergence have a dual representation as patterns in a dense associative memory of order P 4 . The latter is known to be able to Hebbian-store an amount of patterns scaling as N P 1, where N denotes the number of constituting binary neurons interacting P -wisely. We also prove that, by keeping the dense associative network far from the saturation regime (namely, allowing for a number of patterns scaling only linearly with N, while P 2) such a system is able to perform pattern recognition far below the standard signal-to-noise threshold. In particular, a network with P 4 is able to retrieve information whose intensity is O (1) even in the presence of a noise O ( N) in the large N limit. This striking skill stems from a redundancy representation of patterns - which is afforded given the (relatively) low-load information storage - and it contributes to explain the impressive abilities in pattern recognition exhibited by new-generation neural networks. The whole theory is developed rigorously, at the replica symmetric level of approximation, and corroborated by signal-to-noise analysis and Monte Carlo simulations. Artificial intelligence is nearly everywhere in today's society and has rapidly changed the face of economy, communication and science.