LLaMA: Open and Efficient Foundation Language Models

The paper introduces LLaMA, a series of open-source foundation language models ranging in size from 7B to 65B parameters, trained on trillions of tokens from publicly available datasets. LLaMA-13B surpasses GPT-3 on most benchmarks, and LLaMA-65B is competitive with the best models, Chinchilla-70B and PaLM-540B. The authors demonstrate that training state-of-the-art models with publicly available data is possible, and argue that the release of these models will accelerate the development of LLMs. They further highlight the importance of responsible AI practices by examining the biases and toxicity encoded in their models.