μP Hyperparameter Transfer Has an Embedding Layer Hole, New arXiv Paper Says
An arXiv paper shows the embedding learning rate accounts for most of μP's advantage over standard parameterization, and a single scaling fix recovers the bulk of the benefit.








