Can we do better than Convolutional Neural Networks?

The British Machine Vision Conference (BMVC), finished about two weeks ago in Cardiff, UK, is one of the top conferences in computer vision & pattern recognition with a competitive acceptance rate of 28%. Compared to others, it's a small event, so you have plenty of time to walk around posters and talk to presenters one-on-one, which I found really nice. I presented a poster on Image Classification with Hierarchical Multigraph Networks on which I mainly worked during my internship at SRI International under the supervision of Xiao Lin, Mohamed Amer (homepage) and my PhD advisor Graham Taylor. In the paper, we basically try to answer the question "Can we do better than Convolutional Neural Networks?". Here I discuss this question and support my arguments by results.