Summary of approach and contributions: The authors resurrect the pioneering work of Hirose on complex valued neural networks in order to provide a new RNN based on a complex valued activation/transition function and a complex argument gating mechanism. In order to obtain a differentiable function that is not constant and yet bounded, the authors step away from holomorphic functions and employ CR calculus. The authors show experimental improvements on two synthetic tasks and one actual data set. Strengths of the paper: o) Moving away from strict holomorphy and using CR calculus to apply complex valued networks to RNNs is interesting as a novel technique. I think that the authors should spend more time explaining how phases can be easily encoded in the complex domain and therefore why such complex representations can be advantageous for sequential learning.

Computer engineers at Duke University have demonstrated that using complex numbers--numbers with both real and imaginary components--can play an integral part in securing artificial intelligence algorithms against malicious attacks that try to fool object-identifying software by subtly altering the images. By including just two complex-valued layers among hundreds if not thousands of training iterations, the technique can improve performance against such attacks without sacrificing any efficiency. The research was presented at the Proceedings of the 38th International Conference on Machine Learning. "We're already seeing machine learning algorithms being put to use in the real world that are making real decisions in areas like vehicle autonomy and facial recognition," said Eric Yeats, a doctoral student working in the laboratory of Helen Li, the Clare Boothe Luce Professor of Electrical and Computer Engineering at Duke. "We need to think of ways to ensure that these algorithms are reliable to make sure they can't cause any problems or hurt anyone." One way that machine learning algorithms built to identify objects and images can be fooled is through adversarial attacks.

Is it not odd to anyone that Deep Learning uses only real numbers? Or perhaps, it would be even odder if Deep Learning uses complex numbers (note: the kind with imaginary numbers). One viable argument is that it is highly unlikely that the brain uses complex numbers in its computation. However, you can make the argument also that the brain doesn't perform matrix multiplication or perform chain rule differentiation. Besides, Artificial Neural Networks (ANN) have a cartoonish model of actual neurons.