Nous Hermes 3 and exploiting underspecified evaluations

The latest model from one of the most popular fine-tuning labs makes us question how a model should be identified as a "frontier model." This is AI generated audio with Python and 11Labs. Source code: https://github.com/natolambert/interconnects-tools Original post: https://www.interconnects.ai/p/nous-hermes-3 0:00 Nous Hermes 3 and exploiting underspecified evaluations 5:29 Parsing training lessons from Hermes 3 Fig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/nous-hermes-3/img_005.png Fig 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/nous-hermes-3/img_010.png Fig 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/nous-hermes-3/img_012.png Fig 4: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/nous-hermes-3/img_020.png Fig 5: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/nous-hermes-3/img_027.png Fig 6: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/nous-hermes-3/img_030.png Fig 7: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/nous-hermes-3/img_032.png Fig 8: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/nous-hermes-3/img_036.png This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.interconnects.ai/subscribe