EPISODE · Feb 1, 2025 · 14 MIN
Qwen-1
from Large Language Model (LLM) Talk · host AI-Talk
Qwen-1, also known as QWEN, is a series of large language models that includes base pretrained models, chat models, and specialized models for coding and math. These models are trained on a massive dataset of 3 trillion tokens using byte pair encoding for tokenization, and they feature a modified Transformer architecture with untied embeddings and rotary positional embeddings. The chat models (QWEN-CHAT) are aligned to human preferences using Supervised Finetuning (SFT) and Reinforcement Learning from Human Feedback (RLHF). QWEN models have strong performance, outperforming many open-source models, but they generally lag behind models like GPT-4.
What this episode covers
Qwen-1, also known as QWEN, is a series of large language models that includes base pretrained models, chat models, and specialized models for coding and math. These models are trained on a massive dataset of 3 trillion tokens using byte pair encoding for tokenization, and they feature a modified Transformer architecture with untied embeddings and rotary positional embeddings. The chat models (QWEN-CHAT) are aligned to human preferences using Supervised Finetuning (SFT) and Reinforcement Learning from Human Feedback (RLHF). QWEN models have strong performance, outperforming many open-source models, but they generally lag behind models like GPT-4.
NOW PLAYING
Qwen-1
No transcript for this episode yet
Similar Episodes
Feb 8, 2026 ·4m
Jan 30, 2026 ·6m
Dec 15, 2025 ·2m
Nov 30, 2025 ·5m
Oct 26, 2025 ·14m
Oct 26, 2025 ·61m