CUDA is of special interest to our users, since it dramatically boosts performance in parallel computations. This in turn significantly lowers the time required for tuning and training models. While we seamlessly support NVIDIA's cuDNN library of deep learning primitives, we also make the power and performance of the GPUs accessible to end users without cuDNN installed. This kind of stack - all in Java with a "native" backend, however, comes with it's own set of unique challenges. These native operations are essentially a set of independent individual operations applied to the same data.

To this end, we use JavaCPP (created and maintained by Skymind engineer Samuel Audet). The auto-generated JNI code from JavaCPP has zero overhead compared to manually-coded JNI functions. All matrices are stored off-heap and memory is managed through JavaCPP. In Java, these matrices are accessed by simply passing around pointers. We are therefore not restricted by addressing arrays with "ints" which are always 32 bits.