Scaling Data-Constrained Language Models

Oct-9-2025, 02:46:03 GMT–Neural Information Processing Systems

We propose and empirically validate a scaling law for compute optimality that accounts for the decreasing value of repeated tokens and excess parameters.

arxiv preprint arxiv, large language model, machine learning, (17 more...)

Neural Information Processing Systems

Oct-9-2025, 02:46:03 GMT

Conferences PDF

Country:
- Asia > Middle East
  - Jordan (0.04)
- Europe
  - Austria (0.04)
  - Belgium > Brussels-Capital Region
    - Brussels (0.04)
  - Finland > Southwest Finland
    - Turku (0.04)
  - Ireland > Leinster
    - County Dublin > Dublin (0.04)
  - Italy > Calabria
    - Catanzaro Province > Catanzaro (0.04)
  - United Kingdom
    - England > Hampshire
      - Southampton (0.04)
    - Scotland > City of Edinburgh
      - Edinburgh (0.04)

Genre:
- Research Report > New Finding (0.68)

Technology:
- Information Technology > Artificial Intelligence
  - Machine Learning > Neural Networks
    - Deep Learning (1.00)
  - Natural Language
    - Chatbot (0.68)
    - Large Language Model (1.00)
  - Representation & Reasoning (1.00)