pallas
MultiKernelBench: A Multi-Platform Benchmark for Kernel Generation
Wen, Zhongzhen, Zhang, Yinghui, Li, Zhong, Liu, Zhongxin, Xie, Linna, Zhang, Tian
The automatic generation of deep learning (DL) kernels using large language models (LLMs) has emerged as a promising approach to reduce the manual effort and hardware-specific expertise required for writing high-performance operator implementations. However, existing benchmarks for evaluating LLMs in this domain suffer from limited hardware support, coarse-grained kernel categorization, and imbalanced task coverage. To address these limitations, we introduce MultiKernelBench, the first comprehensive, multi-platform benchmark for LLM-based DL kernel generation. MultiKernelBench spans 285 tasks across 14 well-defined kernel categories and supports three major hardware platforms: Nvidia GPUs, Huawei NPUs, and Google TPUs. To enable future extensibility, we design a modular backend abstraction layer that decouples platform-specific logic from the core benchmarking infrastructure, allowing easy integration of new hardware platforms. We further propose a simple yet effective category-aware one-shot prompting method that improves generation quality by providing in-category exemplars. Through systematic evaluations of seven state-of-the-art LLMs, we reveal significant variation in task difficulty, poor generalization to platforms with less training exposure, and the effectiveness of targeted prompting strategies. MultiKernelBench is publicly available at https://github.com/wzzll123/MultiKernelBench.
- North America > United States > District of Columbia > Washington (0.05)
- Asia > China > Jiangsu Province > Nanjing (0.05)
- North America > United States > New York > New York County > New York City (0.04)
- (9 more...)
Rare images show endangered Pallas's wildcat making adorable facial expressions
Rare images of a wildcat on the brink of extinction have been caught by animal protection volunteers in Siberia. The endangered Pallas's wildcats have been hunted by poachers for their fur which can be sold on the black market to be made into mittens. Native to remote regions of southern Siberia, as well as Central Asia and China, they are seldom seen, and known for their reclusive and solitary lives. 'They are secretive and do not like to be seen, making these images rather special,' reported The Siberian Times. This cat was caught on camera a whisker away from an animal camera trap as it surveyed the scene in the Altai Nature Reserve in mountainous southern Siberia.
- Asia > China (0.28)
- Asia > Central Asia (0.28)
- Europe > Russia (0.08)
- Asia > Russia > Far Eastern Federal District > Primorsky Krai > Vladivostok (0.08)