Generative large language models (LLMs) pre-trained on code have shown impressive effectiveness in code generation, program repair, and document analysis. However, existing generative LLMs focus on source code and are not specialized for binaries. There are three main challenges for LLMs to model and learn binary code: hex-decimal values, complex global dependencies, and compiler optimization levels. To bring the benefit of LLMs to the binary domain, we develop Nova and Nova$^+$, which are LLMs pre-trained on binary corpora. Nova is pre-trained with the standard language modeling task, showing significantly better capability on five benchmarks for three downstream tasks: binary code similarity detection (BCSD), binary code translation (BCT), and binary code recovery (BCR), over GPT-3.5 and other existing techniques. We build Nova$^+$ to further boost Nova using two new pre-training tasks, i.e., optimization generation and optimization level prediction, which are designed to learn binary optimization and align equivalent binaries. Nova$^+$ shows overall the best performance for all three downstream tasks on five benchmarks, demonstrating the contributions of the new pre-training tasks.

In this post I'm going to explore how a mathematical concept can be redefined in progressively more computer-oriented terms, all the way from high level languages down to machine code, ready for direct execution by a computer. To that end, I'm going to define the same logic in several different but related formats: If you're interested in how language styles can differ or curious about what your code might look like after being compiled, keep reading! A factorial is the product of an integer and all smaller integers greater than 0. There are lots of ways to describe a definition like this. This definition states that n! is the product of all integers from 1 to n. One important use of factorials is calculating the total number of permutations of a set. For example, the string "cat" can be rearranged in 6 possible ways: "cat", "act", "atc", "tac", "tca", and "cta". This string has 3 letters and 3! 6. The string "a", which has one character, can only be arranged in that one way.