Orca: The Model Few Saw Coming
from AI Explained Hosted by AI Explained
The first model set to be opensourced that actually comes close to ChatGPT, and is just 13B (that's small enough for a laptop). The 51 page report from Microsoft was released just 48 hours ago but I have gone through it all, and bring relevant insights from 5 other papers.By imitating the logic and explanations of GPT 4 (and using GPT 3.5 as an assistant), as well as by training on diverse tasks and an order of magnitude more examples, we have Orca. I will showcase it on a dozen benchmarks and go through in detail how it works and why. I will also end on comments from Sam Altman and Ilya Sutskever on whether Opensource will catch-up...Orca Paper: https://arxiv.org/pdf/2306.02707.pdfFalse Promise Paper: https://arxiv.org/pdf/2305.15717.pdfhttps://sharegpt.com/FLAN: https://arxiv.org/pdf/2301.13688.pdfVicuna: https://lmsys.org/blog/2023-03-30-vicuna/No Moat memo: https://www.semianalysis.com/p/google-we-have-no-moat-and-neitherLLM Leaderboard: https://chat.lmsys.org/?leaderboardAGIEval: https://arxiv.org/pdf/2304.06364.pdfBIG-Bench Hard: https://arxiv.org/pdf/2210.09261.pdfLanguage Models as Tool Makers: https://arxiv.org/pdf/2305.17126.pdfAltman Interview: https://www.youtube.com/watch?v=VWUhASix9wsDERA Paper: https://arxiv.org/abs/2303.17071Let's Verify Step by Step: https://arxiv.org/abs/2305.20050https://www.patreon.com/AIExplained Non-Hype, Free Newsletter: https://signaltonoise.beehiiv.com/
Embed this episode
No transcript yet.