Scaling model capacity has been vital in the success of deep learning.

Thought, the LLM can verbalize that the unknown city is Paris and use this fact to answer downstream questions.

The acquisition of training data is crucial for machine learning applications.

Preference learning algorithms (e.g., RLHF and DPO) are frequently used to steer LLMs to produce generations that are more preferred by humans, but our understanding of their inner workings is still limited.

Propensity Score (MIPS) estimator, proving that MR achieves lower variance among a generalized family of MIPS estimators.

The sufficiency result can be extendedtodeepernetworks;weshowthatan L-layernetworkwith W parameters in the hidden layers can memorizeN data points ifW = Ω(N).

We investigate algorithmic principles that enable efficient data deletion in ML.