Incorporating Side Information by Adaptive Convolution