All publications · ayazdan.com
Efficient LLMs · 2025

The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws

ICLR 2025 · International Conference on Learning Representations (ICLR)
@inproceedings{yazdanbakhsh2025journey,
  title     = {{The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws}},
  author    = {Yazdanbakhsh, Amir and others},
  booktitle = {International Conference on Learning Representations (ICLR)},
  year      = {2025}
}