The Ramifications of Making Deep Neural Networks Compact