The safe and effective deployment of Large Language Models (LLMs) involves a critical step called alignment, which ensures that the model's responses are in accordance with human preferences. Prevalent alignment techniques, such as DPO, PPO and their variants, align LLMs by changing the pre-trained model weights during a phase called post-training. While predominant, these post-training methods add substantial complexity before LLMs can be deployed. Inference-time alignment methods avoid the complex post-training step and instead bias the generation towards responses that are aligned with human preferences. The best-known inference-time alignment method, called Best-of-N, is as effective as the state-of-the-art post-training procedures. Unfortunately, Best-of-N requires vastly more resources at inference time than standard decoding strategies, which makes it computationally not viable.

Positional encoding plays a crucial role in transformers, significantly impacting model performance and length generalization. Prior research has introduced absolute positional encoding (APE) and relative positional encoding (RPE) to distinguish token positions in given sequences.

Weight decay is helpful in both: the over-training and the under-training, one-pass regime.

Lastly, we evaluate RegionCLIP's performance with

However, current research for decision-making, like reinforcement learning (RL), has primarily required numerous real interactions with the target environment to learn a skill, while failing to utilize the existing knowledge already summarized in the text.

In this work, we introduce ChatQA, a suite of models that outperform GPT -4 on retrieval-augmented generation (RAG) and conversational question answering (QA).

Self-attention is the key mechanism of transformers, which are the essential building blocks of modern foundation models.

As a proof-of-concept, we show that--at scale--even a standard transformer network is a powerful semantic routing system and achieves non-trivial performance on our benchmark.