Parallelizing GPU-intensive Workloads via Multi-Queue Operations

GPUs have proven extremely useful for highly parallelizable data processing use-cases. The computational paradigms found in machine learning & deep learning for example fit extremely well to the processing architecture graphics cards provide. One would assume that GPUs would be able to process any submitted tasks concurrently -- the internal steps within a workload are indeed run in parallel, however separate workloads are actually processed sequentially. Recent improvements in graphics card architectures are now enabling for hardware parallelization across multiple workloads, which can be achieved by submitting the workloads to different underlying physical GPU queues. Practical tecniques in machine learning that would benefit from this include model parallelism and data parallelism.