Goto

Collaborating Authors

 Country





85b6841eaf79327b1777f9e64af3835d-Paper-Datasets_and_Benchmarks.pdf

Neural Information Processing Systems

In recent years there has been significant progress in the field of 3D learning on classification, detection and segmentation problems. The vast majority of the existing studies focus oncanonical closed-set conditions, neglecting theintrinsic open nature ofthereal-world.



OnlineMultitaskLearningwithLong-TermMemory

Neural Information Processing Systems

Associatedwitheach segment is a hypothesis from some hypothesis class. We give algorithms that are designed to exploit the scenario where there are many such segments but significantly fewer associated hypotheses.



Increasing GPU Utilization during Generative Inference for Higher Throughput

Neural Information Processing Systems

Apart from the already-large model parameters, the key/value (KV) cache that holds information about previous tokens in a sequence can grow to be even larger than the model itself. This problem is exacerbated in one of the current LLM serving frameworks which reserves the maximum sequence length of memory for the KV cache to guarantee generating a complete sequence as they do not know the output sequence length. This restricts us to use a smaller batch size leading to lower GPU utilization and above all, lower throughput. We argue that designing a system with a priori knowledge of the output sequence can mitigate this problem.