Episode 3 Preschooler -(Jan Allen Destajo)
Listen to this episode
Summary
The development of preschoolers, a detailed podcast explaining facts and teachings.
First published
03/13/2021
Genres
Duration
9 minutes
Parent Podcast
Developmental Stages of the Learner Across the Lifespan - Preschooler
View PodcastSimilar Episodes
-
Developmental Stages of GPTs by orthonormal
12/10/2021
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Developmental Stages of GPTs, published by orthonormal on the AI Alignment Forum. Epistemic Status: I only know as much as anyone else in my reference class (I build ML models, I can grok the GPT papers, and I don't work for OpenAI or a similar lab). But I think my thesis is original. Related: Gwern on GPT-3 For the last several years, I've gone around saying that I'm worried about transformative AI, an AI capable of making an Industrial Revolution sized impact (the concept is agnostic on whether it has to be AGI or self-improving), because I think we might be one or two cognitive breakthroughs away from building one. GPT-3 has made me move up my timelines, because it makes me think we might need zero more cognitive breakthroughs, just more refinement / efficiency / computing power: basically, GPT-6 or GPT-7 might do it. My reason for thinking this is comparing GPT-3 to GPT-2, and reflecting on what the differences say about the "missing pieces" for transformative AI. My Thesis: The difference between GPT-2 and GPT-3 has made me suspect that there's a legitimate comparison to be made between the scale of a network architecture like the GPTs, and some analogue of "developmental stages" of the resulting network. Furthermore, it's plausible to me that the functions needed to be a transformative AI are covered by a moderate number of such developmental stages, without requiring additional structure. Thus GPT-N would be a transformative AI, for some not-too-large N, and we need to redouble our efforts on ways to align such AIs. The thesis doesn't strongly imply that we'll reach transformative AI via GPT-N especially soon; I have wide uncertainty, even given the thesis, about how large we should expect N to be, and whether the scaling of training and of computation slows down progress before then. But it's also plausible to me now that the timeline is only a few years, and that no fundamentally different approach will succeed before then. And that scares me. Architecture and Scaling GPT, GPT-2, and GPT-3 use nearly the same architecture; each paper says as much, with a sentence or two about minor improvements to the individual transformers. Model size (and the amount of training computation) is really the only difference. GPT took 1 petaflop/s-day to train 117M parameters, GPT-2 took 10 petaflop/s-days to train 1.5B parameters, and the largest version of GPT-3 took 3,000 petaflop/s-days to train 175B parameters. By contrast, AlphaStar seems to have taken about 30,000 petaflop/s-days of training in mid-2019, so the pace of AI research computing power projects that there should be about 10x that today. The upshot is that OpenAI may not be able to afford it, but if Google really wanted to make GPT-4 this year, they could afford to do so. Analogues to Developmental Stages There are all sorts of (more or less well-defined) developmental stages for human beings: image tracking, object permanence, vocabulary and grammar, theory of mind, size and volume, emotional awareness, executive functioning, et cetera. I was first reminded of developmental stages a few years ago, when I saw the layers of abstraction generated in this feature visualization tool for GoogLeNet. We don't have feature visualization for language models, but we do have generative outputs. And as you scale up an architecture like GPT, you see higher levels of abstraction. Grammar gets mastered, then content (removing absurd but grammatical responses), then tone (first rough genre, then spookily accurate authorial voice). Topic coherence is mastered first on the phrase level, then the sentence level, then the paragraph level. So too with narrative flow. Gwern's poetry experiments (GPT-2, GPT-3) are good examples. GPT-2 could more or less continue the meter of a poem and use words that fit the existing theme, but even...
Clean -
How People Learn Series: Learning Across the Lifespan
04/21/2023
The Owl Pellets crew chats about learning across the lifespan in the final episode of the How People Learn series.
Clean -
GUEST EXPERT - Dr Carrie Ruxton: The benefits of tea across the lifespan
08/24/2022
Dr Carrie Ruxton talks about the many health benefits of tea across the lifespan. #tea #perfectcuppa #wellbeingforchildren #childrenshealth
Clean -
Easy A and Developmental Stages
08/26/2022
This exploration of the movie Easy A ventures to tackle the concepts of processing developmental stages, identifying core values, and understanding internal worth through the world of Olive Penderghast and her misadventures in this modern day rendition of the Scarlet Letter. Email us at filmsandfeelingspodcast@gmail.com Music credit to "April Showers" by ProleteR (https://proleter.bandcamp.com)
Clean
Similar Podcasts
-
Shepherd of the Valley Bible Church
08/12/2020
Tommy Moon
Weekly messages from the pastor at Shepherd of the Valley Bible Church in Hood River, Oregon.
Clean -
Retro Blissed
08/12/2020
The Network
Two gamers with a love for the history of video games play and discuss the best and worst titles in existence. Blow out your NES cartridge, check the batteries in your remote, and memorize your Game Genie codes… It’s time to get RETRO!
Clean -
Ear to Asia
08/12/2020
Asia Institute, The University of Melbourne
On Ear to Asia, we talk with Asia experts to unpack the issues behind news headlines in a region that is rapidly changing the world. Ear to Asia is produced by Asia Institute, the Asia research specialists at the University of Melbourne, Australia. Subscribe to us on Apple Podcasts, Spotify, or wherever you get your podcasts.
Clean -
Skaana with Mark Leiren-Young | Oceans, Eco-Ethics & The Environment
08/12/2020
Mark Leiren-Young | Oceans, orcas, eco-ethics and the environment.
Mark Leiren-Young, author of Orcas Everywhere and director of The Hundred-Year-Old Whale, meets the humans who are fighting to save our oceans, orcas and environment. Find out how you can make waves.Join the Pod…… https://www.patreon.com/mobydoll Skaana home….. https://www.skaana.orgFacebook……….. https://www.facebook.com/skaanapod/Twitter…………... @leirenyoung
Clean -
Clayton Church of Christ
08/12/2020
Clayton Church of Christ
Listen to our latest series of messages preached during our Sunday morning services
Clean -
Rock The Walls
08/12/2020
idobi Network
Always on the frontlines, Rock The Walls is hosted by music fan and devoted radio host Patrick Walford. Over one thousand interviews are already in the can—from being the first ever radio interview for bands like I Prevail & The Story So Far, to speaking with heavy & alternative music legends such as The Used, Anthrax, Parkway Drive, Godsmack, Korn, Sum 41, Bring Me The Horizon, A Day To Remember, and hundreds more.After doing the show for over a decade, hosting Warped Radio, bringing you your idobi Music News, and Music Directing idobi Howl, along with hitting the road for coverage on the Warped at Sea Cruise in 2017 & the final Vans Warped Tour in 2018, Walford is a long trusted voice in the music scene. Tune in to hear in-depth interviews you won't hear anywhere else with all your favorite heavy & alternative artists, along with spinning the best in new music.
Clean -
Community of Hope Lutheran Church
08/12/2020
Community of Hope Lutheran Church
Sermons from Community of Hope Church Lutheran Church
Clean -
Clean and Sober Radio
08/12/2020
Gary Hendler
Clean and Sober Radio is dedicated to promoting a lifetime free of alcohol and drugs. Our mission is to inform, share, and discuss, across multiple media platforms, the story of addiction and the road to recovery.
Clean -
Just the Right Book with Roxanne Coady
08/12/2020
Roxanne Coady
Just the Right Book is a podcast hosted by Roxanne Coady, owner of famous independent bookstore R.J. Julia Booksellers in Madison, CT, that will help you discover new and note-worthy books in all genres, give you unique insights into your favorite authors, and bring you up to date with what’s happening in the literary world.
Clean -
Experts On The Wire (An SEO Podcast!)
08/12/2020
Dan Shure (SEO)
Downloaded over 500,000 times! Experts On The Wire is a monthly SEO podcast hosted by Dan Shure. Discover new trends, tactics, tools, people, and businesses doing remarkable work in the world of Search Engine Optimization. Past guests include Rand Fishkin, Brian Dean, Annie Cushing, Noah Kagan & Vanessa Fox. I'm an SEO consultant myself, so we dig deep into SEO, technical challenges, growth stories, mobile SEO, eCommerce, crawling, content marketing & more.
Clean -
PODCASTS - WELCOME TO HILLSIDE
08/12/2020
First Evangelical Free Church of Tahlequah
Come hear the Word of God preached at EFreeTahlequah anywhere in the world.
Clean -
Dawg Pound Daily Podcast on the Cleveland Browns
08/12/2020
FanSided
FanSided's Dawg Pound Daily Podcast discusses the latest Cleveland Browns news, analysis and more from the staff at DawgPoundDaily.com.
Clean
Comments
Sign in to leave a comment.
No comments yet. Be the first to comment!