Building a 320* Teraflops Deep Learning Box for Under $10,000

Look what I got for Christmas!! If you don't recognize it, it's two Titan V cards from Nvidia. A single Titan V has a systolic array unit dubbed as a TensorCore that is capable of 110 teraflops peak performance. In addition, it includes a conventional GPU that's capable of 25 teraflops half-precision. So we are roughly speaking here about 135 teraflops (half-precision) per card.