Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related technologies are changing the world, this is the show for you. Thank you to our partners at Fastly for shipping all of our pods super fast to wherever you listen. Check them out at fastly.com. And to our friends at Fly, deploy your app servers and database close to your users. No ops required. Learn more at fly.io.
Hello, this is Daniel Whitenack. I am here on site at ODSC East in Boston at the Open Data Science Conference. And I am super excited because I get to sit down with Aaron McHale Staples, who is a developer community advocate at Label Studio. And yeah, what do you think of the conference so far, Aaron? It's been, first off, super fan of kind of what you've been doing and a lot of anybody who's creating stuff out there in the space, especially with the current, like, zeitgeist and explosion of interest in AI and machine learning. It's a little crazy. It's a little wild. It's a little wild. I, you know, I would be lying if I'm not, wouldn't be saying I'm newer to the field myself, but it's been something I've been very fascinated about.
But all that being said, this conference is really cool to see just the breadth of, first off, people here. There are people who are very new to the industry, people who came to just learn more for their first time. But there are people who have been practicing for years and years, and this is their third or fourth time at ODSC. And I'm also really interested about the number of people concerned about data integrity here. Yeah, lots of, like, interpretability, integrity, reliability type talks. Yeah, lots of reliability. The other one is, like, also on missing data and, like, how do we approach these problems, especially with the rise of foundational models and generative AI. Like, how does that impact it for the long, which is crucial conversations, I think, to have.
Yeah, definitely. And what sorts of different players in the space are you seeing at this conference, both in terms of, like, open source or, like, different kind of targets, like ML Ops platforms, that sort of thing? How do you see that developing? First off, I'm personally a huge fan of open source. It's not only how I learned to code in the first place, but just a big believer in the ecosystem. I'm a huge believer in open data. I'm a participant in Open Data Week, so I think all these things are, like... And you're wearing a Pi Lady shirt, which is awesome. Yeah, I'm a member of Pi Ladies. So, again, super important, I think, to have all these things in the ecosystem, but...
One of the things I think that stands out is there's so many new innovations that if you're starting a tech stack from ground zero, it's really fun to see all the different players in the game. So selfishly, working at Label Studio, one of the best things about being in the space right now is we're a cool platform because we can integrate with so many different data types. It means that I always get to play with almost every other tool or workshop or players in the ecosystem, which is selfishly fun. It means I get to have more things to integrate with or build. As always, we're huge fans of what the Packarderm team is always doing. We work very closely with them. We work very closely.
I've got a lot of friends and fans, the DVC crew. They're not here at this conference around, but did get to work with them at PyCon, which was really amazing to see kind of the work that they're coming out with. The Condor crew is always fun to see around. So that's always exciting. Cool. Yeah. Yeah. There's so many awesome things going on. And I've seen like maybe like three or four open source packages that I like, I don't know if I've been ignoring or I haven't heard about. So that's like one of the fun things about coming to these things. I know also you gave a recent talk at PyData Berlin about reinforcement learning from human feedback, I believe was the topic. Could you tell us a little bit about like...
the general like pitch or angle on that talk, which is definitely like a key topic these days with all the instruction tune models that are coming out and all of that. So what, what was your kind of angle in terms of what you're thinking about there? Yeah. So I think one of the cool things is it was a talk that Nikolai and I gave and Nikolai is the CTO and one of the co-founders of label studio. And what we did at Berlin is we really made sure to expand on.
This idea that, like, yes, these generative models, these larger models are kind of becoming the norm. I think we met yesterday. I was talking to someone who was like, I got interested in AI because I made a thousand things with MidJourney. I'm like, cool. And I'm very fascinated by, and I'm a believer, like, I don't care how you got into it, but just the curiosity to show up to a conference and learn more is very fascinating. But explaining to someone how it works and then also explaining the best practices behind it is really important. Personally, I have a journalism background. I have a liberal arts background. And I think it's really important that we incorporate the humanities in technology for the long run. And so when it comes to reinforcement learning.
all of these large generative models, they can all be made just a little bit better with the human signal that we can provide. And we can say a lot of things like, you know, get into prompt engineering, which is another whole other topic, but it will never be as good as like if you can retrain your own data set with subject matter experts or to a specific use case or condition that you're trying to output that data turns. Yeah. And I think one of the things that's been on my mind recently is like this topic, reinforcement learning from human feedback, especially with what's gone on with chat GPT and all, sometimes it feels like out of reach for like day-to-day data scientists. Like how do I, like I could leverage this model, but what is the tooling around reinforcement learning from human feedback? Like how could I use that framework or use tooling around that to like impact my own models or my own life? Like how could I connect my domain experts input and their preferences into a system that I'm designing? Do you have any thoughts there? Yeah. So one of the examples I love to point to is actually Bloomberg did this and they probably did this early April now.
the financial data that they had and Bloomberg from all the way back from like you know many of us know about it Bloomberg from Bloomberg News but that was actually the financial terminal that was used for stock trading but they have these I mean mass amounts of financial data and how do they stack on top of it like how do they get and access that data even faster and kind of train it to the best use case that we have like currently our larger models can't do that they're not experts in financial data they're not combing just financial data but what Bloomberg did is they took and they retrained and they built things. I probably fangirled over, sorry, if you're on the Bloomberg team and I fangirled over you at PyCon, because I was like, this is the coolest thing ever. I use this as an example. Also, I learn machine learning off of your repo. Okay, thanks, bye. But we do have a model. If you want to learn and see reinforcement learning in action, there is an open source repo. It is built by myself, Nikolai, and Jimmy Wicker, who we have as a data scientist in residence at Label Studio and Hartex, but also is at Pachydermy as well. But all of that is built. It's open. You can play around with it. It's based off of GPT-2 right now, so you can go have some fun and get your hands dirty. And it's all runnable within a Google CoLab notebook.
That's awesome. Yeah. What is, so you mentioned it being run in a Google Colab notebook, which I think is awesome. And using also a bit of a smaller model to start with. And we've seen a lot of kind of directions towards smaller open models that are accessible to data scientists with like Llama and other things like that. How do you see that trajectory going and how will that impact sort of like day-to-day practitioners in terms of what they're able to do with this sort of technology? I think the biggest thing, and I'm actually going to zoom out to answer this, is the biggest thing we need to think about is context. Like what are you using?
model to solve or AI to solve or ML to solve. And the more that I've been diving into these conferences and the ecosystem, especially at a conference where it's a blended conference, where you have folks that are not necessarily deep in the field or ML practitioner or they're like new to ML, it is so easy. And there's a meme I always point to that. It's like, oh, it's, you know, we're an AI back so-and-so. And it's like, JK, we're just an AI. We're just, you know, calling the API and put a nice pretty shiny front end on it, which is no shade to anybody who is putting a front end on a GPT API. Like there's no shade at all to that. But it's like, think about what you need a model for in the first place or what you want to use machine learning. Like that context is so important.
I'm currently playing around with a Naked and Afraid data set just to play around. There's an open source data set out there. That's awesome. Like videos? No, it's context from the TV show of how many days they survived. Oh, so literally, yeah, statistics and features of the different survival situations. Yeah, it's like country, their name, gender, and then how many days they made it. And climate. Yeah, based on that, yeah, that's so intriguing. I watch a lot. So confession, I watch also the Alone show, which is another survival show. I'm a reality TV.
terrible junkie. I, that is how I do stress is reality TV. But so I always wonder, I have this conversation with my wife around like, could I do it? And maybe with a model trained off of, off of your survival data set, I could like say I'm from here and this is my background. Could I survive? Yeah. And I can't take credit for the original data set. Um, it is someone who I have made friends with in my reality TV subreddit. So if you need to know where I spend my time, but, um, it is actually.
Very good. He's very awesome updating it. It's available on Reddit. I can share it with you and you can post it in the links. But I'm just kind of playing around with the data set for fun. But in this context, like I'm playing around building demos and just having some fun, teaching myself some new skills. I don't need a large foundational model for that. And I think going back to like your original question of like, well, all these models are getting smaller, more accessible. We can run it in a notebook. We don't need the high powered computer models, everything single times. And if we stop and think about the context of the problem that we're trying to solve, it can give us a lot of answers and it can save us time, energy and computing power. And I think that's why I get really excited about being on the data labeling side. Again, I have a background in humanities. I'm a self-taught programmer, but I think.
I don't want to be like, we need more people like me in data science, but we need more of the humanities in data science because we're missing the context. Yeah, we recently had a guest on the show that was talking about the intersection of art history and computer science and how computer scientists who are analyzing and doing computer vision could actually learn a lot from what we know about art and how scenes are composed or how art has changed over time and how the features that they're actually engineering are connected to some of those things. So yeah, I think that there's a lot of different areas where this could apply and domain experts are so important. And I assume that with all of this reinforcement learning from human feedback, I always mess it up. It's okay, I've been doing the same thing.
LHF, I get it. Yeah, especially since you're from the label studio side, do you have any, could you give like a general picture or workflow for people of like, hey, I maybe want to take one of these models, GPT-2, LAMA, MPT now, whatever the one is, but I also want to gather some domain expert feedback and eventually get to some type of like instruction or fine-tuned model off of that. Like, could you just give a general picture for like what that looks in today's world? Yeah, and we'll try, I feel like this is better when you have a whiteboard and a diagram and some arrows. Oh, for sure, yes, it's hard. I'll do a quick walkthrough. So first off,
you'll create a sort of prompt. So typically these models work with a prompt and then you're given a large language model. And then you start to train it. Usually what happens when you're training these models is you get a set of two outputs. And so in this case, we can use what is opossums because we're opossum fans at Label Studio. I feel like that's natural. And you can be like an opossum is a marsupial creature or an opossum is a great character for memes. Technically, both of those are correct. But depending on context, and this is where that human signal side comes in, one answer is more correct than the other. So if we were training, let's say, a opossum meme bot thing or a meme bot generator.
Let's go that direction. We'll have some fun with it. We would take the latter answer of this, a possum is a great animal to make memes. And that would be the better answer. If we were going for what type of animal are we doing like maybe a biology assignment homework, probably would pick the marsupial one. But this gives insight of like the details that you give your annotation team can really directly influence the model. That's the labeling side. When we move this on, all of this is put through kind of this result from human feedback. Your answers are ranked. I did a binary situation, so just two options, but you can't have a multitude of options that you put in. It is all weighted. It is then looped back around, this is when you wish you had the whiteboard, to a reward or a preference model. And this reward or the preference model kind of tells you like, hey,
I probably want to go for answers that look like this. Now, computers don't speak memes or marsupial or biology textbooks, but they do know patterns and trends, which is like what they pick up on. So based on that context clues that we give them, this preference model will start to preference those type of answers. Now, it's really important that these reward preference models also hold in place kind of the original things that we had that it knows, like how is language structured or other things from our original model that we enjoyed, like we like, like language is always structured like this. Here's a proper noun. We like to capitalize the first letter of the sentences. Things that are important, but like we kind of overthink sometimes when talking about generative language models, at least.
After that, we want to make sure that we're not just gaming a system. Models are, again, I don't think models are sentient. They're kind of just like math numbers. They're just trying to game a system. They're playing like, I always compare it to like, you're trying to like, it's Moneyball, essentially. I'm a baseball fan here. So it's Moneyball. You're statsing out the system. And they, in order so that they're just not giving you what you want to hear every time, you'll have to calculate an error rule in there. So put an error metric or an update rule. And it basically says, all right, we're going to almost like dunk you down a little bit. So you're not too perfect. And that'll prevent unwanted model drift.
Then once you've done that a few times, combine that with a copy of your original model that you had. Again, you're kind of doing that checks and balances, making sure it doesn't run away. After that, you will have a tuned language model. And then rinse, wash, repeat until you've got that model right where you want it. set it off to production and then talk to your friends at the other parts of your MLOps ecosystem. Yes. And it'll come in handy. Awesome. Yeah. And I hope that we can link some of your slides from that talk in our show notes. Of course. They're awesome, including emojis and the full deal, which helps. So make sure and check out the show notes. The link to the slides will be in there so you can take a look at these figures while you're listening to the show. One follow-up question on this, we're talking about like gathering this feedback data. People can think about like, okay, in the context of my company or where I'm working, I'm going to gather some of this data, you know, tune a model.
But what is your perspective on sort of like the open data ecosystem? And like, what would you encourage people to think about in terms of like data that they could make openly available to help others who are also trying to do this or the other way around, like people that are searching for maybe a place to start? What is the sort of open data ecosystem look like right now? And how important is that as this sort of field advances? Yeah. First off, this is you've got me on my other favorite soapbox at the moment. And this goes back to my days when I was a journalism student working in journalism.
But open data is one of my favorite topics to geek out on. Basically, it was something that really came actually as part of the Obama administration. He actually established federal funding for a lot of our public and civic data as a part of government accountability and transparency. So there was actual federal grants that went out to make a lot of our civic data public. So there's a really cool example. I believe it's the city of Philadelphia that actually built a SimCity-like game off of their public data. It's so cool. It was like a grant given. Super fascinating. I'll link it to you and it'll be in the plethora of show notes on all of that. But open data is just open, freely accessible, freely used data that is made available to the public.
I love open data. I'm a participant in Open Data Week. But when it's been federally funded, it's not always the best thing to be federally funded. And, like, we all know how government grants go. And if you aren't aware how government grants go, they're very niche-specific and they run out. And they're not always maintained and it's not always the coolest sexy job that we have. So they're always not the best maintained or context applicants. What a lot of these early machine learning models did and what a lot of machine learning models did is these open data sets have given opportunities for people like myself to even learn how to do data science. I learned Python in Open Data Week. I remember going back and, like, let's get the traffic data in New York City. And it's, like, basic, you know, using curl and, like...
Getting things started for the first time. Can you query an API? Like, they're not the most organized data sets out there. They're not the most clean. Sometimes you get some really messy garbage data. You know, the 2020 census is actually a great example. I was speaking to someone yesterday at the conference about this. The 2020 census was the first time that we were able to do it digitally. Well, she gave the example of like, you know, hey, I started the census on my phone. Oh, no, the pot boiled over. Oops, I accidentally counted myself twice in the census. Or I didn't fill out my address. Or now I've got two people or a person who lives at this address or a typo. Crap. Now that's a very messy data set. So open data can be a problem. Let's go to like the practical application of this. If you are working in open data.
or you are interested in getting more involved in open data, one of my favorite sources is if you're publishing a story, making tutorials, making content, put your data out there and put how you processed it. And it's not just one thing to put your data out there, but also how you processed it. In journalism, you have this phrase, how you frame the story is how you tell the story. Leaving out details, context, or even how you came across the source can influence how the story comes across. Yeah, for sure. And that's especially evident in data-driven journalism and solutions journalism, which is interesting. And it also can be really damaging to trust and reputation. But I think ML runs the same risk right now if we're not transparent of, here's how I prepared the data set.
here's how I trained an annotator, or here's the tools that I used, or here's how I obtained the data in the first place. Yeah, and like you were saying, certain things like how you give instructions to a data annotator or how you set up your prompt, that has such an influence on the downstream performance of these things, but it's very frequently, I've definitely found the instructions you give data annotators is something very often left out of the story of how people tell what they did, right? It's like, oh, we gathered this data with these labels. Okay, well, I can imagine my own set of instructions for getting those labels, but it... could result in a totally different thing that's happening, like all sorts of biases and other things that go into that. I mean, well, I have a perfect case example of this. In January, we met many of the team members at Hartex and Label Studio met up. Basically, we got our entire customer success and sales team and, you know, the community side of things and a bunch of our support engineers to all sit together and, like, we had a data labeling competition for fun at the end. And I had just finished, like, how did you get started with data labeling and, like, best practices? And I was like, easy. Like, I'm going to kick all of your butts. Like, I was totally going in, like, hot shit and everything and, like, thinking.
Well, I sped through. I was like, whatever. Next. Great. Done. And I sped through because I was like, speed was a metric, but also accuracy. Well, I sped through this thing because I was like, whatever. I'm going to ace this. I know the keyboard shortcuts. My systems are set up. I have the lowest accuracy score, everybody. My data was all. I was like, you failed, Aaron. It was like...
I was like, man, I'm going to go embarrass myself right now after all that crap I just talked. Yeah, yeah. I think that's the other thing. Like, I don't know if you have any encouragement here, but like data scientists out there who have not like actively participated in like the data labeling process. I think, yeah, that's like such a learning experience because like it gives you a perspective, even if in the future, like you're not part of one of those processes, it gives you good questions to ask.
If like, oh, someone gives you this data set that was labeled, you should probably ask a few follow-up questions about like, how did that go? What did you do there? Well, in academic research, you actually have to disclose things like, did you pay your annotators? Or how did you prepare the annotators when you're doing research? Because that can put so much of a bias on a model that is built off of that data. And that academic, like you can't get peer-reviewed studies done without disclosing that information. It's part of data ethics now. And one of the biggest things, and we don't talk about it enough, is how do you pay your annotators? Or do you outsource your annotators? Which isn't saying that's a bad thing to do. But again, we have to remember that so many of these models...
And I see it, like, I think a lot of it times it actually is probably, I'm going to guess here, I don't know, but I'd be even wondering if these smaller models that are generated because they're generated at home or people working around on their computer, they might even have more bias because we're not trading an annotator. Like, I know when I'm goofing around with my naked and afraid data set, I'm not annotating. Like, I'm playing some 30-second... goofing around stuff and watching YouTube videos, just seeing what's out there. I'm not doing the work, which is a problem. Yeah. I guess kind of bringing things full circle a little bit. Like we started talking about like some of these like players and ML ops and sort of the ops around this process. We talked about human feedback, reinforcement learning. We talked about open data. What excites you about like the trends that we're seeing and what impact they could have on, on our industry moving forward. Maybe that's related to like people that weren't able to like participate in this process before the tooling's better. So they can, or maybe it's something totally different, like around like tasks or other things that you see.
In the future, what are you personally excited about looking forward as you bring this stuff together? First off, I've been really impressed with what the Hugging Face team is doing. I noticed the Hugging Face shirt. The Hugging Face spaces have been amazing. We do have a label studio of Hugging Face space, but the ability to get up and going in the browser has been super awesome. There is a talk I went to at PyData Berlin that's running Streamlight. They're running entirely Python-based models right in the browser. And tools, I think it's Binders, I believe, is another tool that's doing another, again, very similar to notebook processing, all in the browser. It makes it more accessible than ever before. And it's just really exciting, especially as, like, I love that we have more people interested in this industry, but it's also...
not only the interest, but the tools to do it correctly and ethically. And again, jumping on my soapbox here, this is why the open data is so important. So putting when we can, putting our sources, our references, building in the public, building an open source and making kind of a, almost, I don't want to say a paper trail, but like show your work. sort of process is really important for the future. Yeah. Awesome. That's great. And, um, as we kind of, uh, close out here, where can people find you online? And also tell us a little bit about your own podcast, which sounds awesome and like includes pickles. Yeah. Um, so I am available online at Aaron McHale on all the platforms, um, or Aaron.bio has a link to everything that I'm at. You can also chase me down at label studio. So it's label studio, but the last.io is like a things, um, join the community coming out with me there. We have an open coming upcoming town hall and getting into more workshops.
So very excited about that. I also run the DevRelish podcast. So it's everything about DevRel and ish. Also, you know, naturally, some people made sourdough bread. I got into fermentation. We got a fun pickle fact and cool pickle logos because you got to relish the developer moments in open source. Yeah. This was definitely not a sour experience. I've relished it very much. Thank you so much for joining, Aaron. It's been a great pleasure to talk to you and looking forward to following up with all the cool community stuff you got going on. Again, people, check out the show notes. And thank you so much.
Thank you so much. This was quite a big deal that we had going on here. Thank you for listening to Practical AI. Your next step is to subscribe now, if you haven't already. And if you're a longtime listener of the show, help us reach more people by sharing Practical AI with your friends and colleagues. Thanks once again to Fastly and Fly for partnering with us to bring you all Change Talk podcasts. Check out what they're up to at Fastly.com and Fly.io. And to our Beat Freaking residents, Breakmaster Cylinder, for continuously cranking out the best beats in the biz. That's all for now. We'll talk to you again next time.