Towards a Practical Neural-Symbolic Framework

Deep learning is still bearing fruits. However, the standard types of networks are exhausting their possibilities, and researchers seek out such extensions to the basic neural network models, which will weaken their inherent limitations. Some extensions such as self-attention layers have enjoyed great practical success. Remarkably, many shortcomings of neural networks mirror the advantages of symbolic systems (and vice versa). Indeed, one can note that both self-attention layers and capsule networks are attempts to work around the notorious variable binding problem described in the Fodor and Pylyshyn's paper, which is easily solved in symbolic systems but is very inconvenient for neural networks.