Goto

Collaborating Authors

 pytorch code



Learning Bug Context for PyTorch-to-JAX Translation with LLMs

Phan, Hung, Vu, Son Le, Jannesari, Ali

arXiv.org Artificial Intelligence

Despite recent progress of large language models (LLMs) on code translation among mainstream languages, translating PyTorch to JAX remains nontrivial. The two libraries, though both embedded in Python, differ in core design, execution semantics, and ecosystem maturity; JAX is newer and comparatively underrepresented in public code, and parallel PyTorch--JAX corpora are limited. Weaknesses in existing evaluation further complicate cross-framework benchmarking. We present T2J, a prompt-augmentation framework that strengthens LLM-based PyTorch to JAX translation. Our pipeline (i) assembles two PyTorch sources -- the problem-solving set from TorchLeet (Aroori & Chien, 2025) and a GitHub-derived set from CodeParrot (Wolf et al., 2022) -- and uses GPT-4o-mini to produce initial JAX drafts; (ii) engages two professional developers to iteratively repair those drafts until functional equivalence, yielding a curated fixed-bug dataset of common errors and patches; and (iii) constructs augmented prompts that inject structured guidance from these fixes to steer lightweight LLMs (e.g., GPT-4o-mini). We also introduce three metrics tailored to PyTorch to JAX: T2J CodeTrans Score, T2J FixCost Score (an LLM-based estimate of bug-fix effort), and T2J Comparison Score (LLM-as-judge). Empirically, T2J raises GPT-4o-mini performance by up to 10% on CodeBLEU, 50% on T2J FixCost Score, 1.33 points on T2J CodeTrans Score (0--4 scale), and 100% on T2J Comparison Score; moreover, the generated code runs up to 2.5x faster than the baseline.



Reliable Estimation of KL Divergence using a Discriminator in Reproducing Kernel Hilbert Space Supplementary Material

Neural Information Processing Systems

This supplementary material is presented in a format parallel to the main paper. The section numbers and titles are consistent with the main paper. Similarly, the Theorem numbers are consistent with the main paper, but we also have several additional theorems and lemmas which were not included in the main paper. We start with the set of assumptions based on which our theory is developed. The input domains X and W are compact.


Introduction to Lightning Fabric

#artificialintelligence

Lightning Fabric is a new, open-source library that allows you to quickly and easily scale models while maintaining full control over your training loop. In the past, getting PyTorch code to run efficiently on GPUs and scaling it up to many machines and large datasets was possible with PyTorch Lightning. As time went on, however, we became aware of the need to provide a scaling option that landed somewhere between a raw deep learning framework like PyTorch on the one hand, and a high-level, feature-rich framework like PyTorch Lightning. Lightning Fabric is just that. While PyTorch Lightning provides many features to save time and improve readability and collaboration, there are complex use cases where full control over the training loop is needed.


Does Zero-Shot Reinforcement Learning Exist?

Touati, Ahmed, Rapin, Jérémy, Ollivier, Yann

arXiv.org Artificial Intelligence

A zero-shot RL agent is an agent that can solve any RL task in a given environment, instantly with no additional planning or learning, after an initial reward-free learning phase. This marks a shift from the reward-centric RL paradigm towards "controllable" agents that can follow arbitrary instructions in an environment. Current RL agents can solve families of related tasks at best, or require planning anew for each task. Strategies for approximate zero-shot RL ave been suggested using successor features (SFs) [BBQ+ 18] or forward-backward (FB) representations [TO21], but testing has been limited. After clarifying the relationships between these schemes, we introduce improved losses and new SF models, and test the viability of zero-shot RL schemes systematically on tasks from the Unsupervised RL benchmark [LYL+21]. To disentangle universal representation learning from exploration, we work in an offline setting and repeat the tests on several existing replay buffers. SFs appear to suffer from the choice of the elementary state features. SFs with Laplacian eigenfunctions do well, while SFs based on auto-encoders, inverse curiosity, transition models, low-rank transition matrix, contrastive learning, or diversity (APS), perform unconsistently. In contrast, FB representations jointly learn the elementary and successor features from a single, principled criterion. They perform best and consistently across the board, reaching 85% of supervised RL performance with a good replay buffer, in a zero-shot manner.


GitHub - rentruewang/koila: Prevent PyTorch's `CUDA error: out of memory` in just 1 line of code.

#artificialintelligence

Koila solves CUDA error: out of memory error painlessly. Fix it with just one line of code, and forget it. To use it, download release v0.1.1 here.* Automatically accumulates gradients when batch sizes are too large. Lazily evaluates PyTorch code to save computing power.


Learn PyTorch for Deep Learning – Free 26-Hour Course

#artificialintelligence

My comprehensive PyTorch course is now live on the freeCodeCamp.org The best way to learn is by doing. And that's just what we'll do in the Learn PyTorch for Deep Learning: Zero to Mastery course. If you're new to data science and machine learning, consider the course a momentum builder. By the end, you'll be comfortable navigating the PyTorch documentation, reading PyTorch code, writing PyTorch code, searching for things you don't understand and building your own machine learning projects.


Learn PyTorch in a day. Literally.

#artificialintelligence

My latest YouTube video is 25 hours, 36 minutes and 57 seconds long (actually a little over a day). And its sole purpose is to be a momentum builder, to help you learn PyTorch for deep learning. In each section, we'll get hands-on and learn important machine learning concepts by writing PyTorch code together, apprenticeship style. It still amazes me to think how much the field of machine learning is progressing. There are things in the video that weren't possible 10 years ago.


Scale your PyTorch code with LightningLite

#artificialintelligence

LightningLite comes with distributed utilities and basic built-in properties that you can rely on to ensure your distributed training works as expected. LightningLite provides you only with the tool to scale your training. You need to know several significant challenges ahead of you, but they are already resolved in PyTorch Lightning. If you are facing one of those challenges, then you are already meeting the limit of LightningLite. We recommend you convert to Lightning, so you never have to worry about those.