I can relate to Jess. I thought that I knew the language, but then it feels like, okay, I actually don't know anything. And I don't know, there's been that interesting curve. It's called a curve of like David something, where it technically shows that, okay, when you think that you know something, you have high confidence, and that's actually when you actually don't know anything. But then when you start learning, and then you're adopting more of yourself, it's technically like that process of acknowledging and learning more things, and then the curve goes up at some point again.
Welcome to Practical AI, a weekly podcast making artificial intelligence practical, productive, and accessible to everyone. Subscribe now if you haven't already. Head to practicalai.fm for all the ways. Special thanks to our partners at Fastly for delivering our shows super fast to wherever you listen. Check them out at fastly.com. And to our friends at fly.io. We deploy our app servers close to our users, and you can too. Learn more at fly.io.
Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I'm a data scientist with SIL International, and I'm at the EMNLP conference in Abu Dhabi, and I've got some old friends, some new friends here with me, a very exciting community show that we have for you today. Why don't I just have everyone just give a brief introduction of who they are and where they're coming from? Hi, everyone. My name is Andi Swababula from South Africa. I work for the South African Center for Digital Language Resources as an Isikosa researcher, Isikosa being one of the 11 official languages spoken in South Africa.
Hi, everyone. I am Rueida Mabuya, better known as Rue. I am also from South Africa. I work with Andiswa at the South African Center for Digital Language Resources. I work as an Isisulu researcher. It's also part of the 11 official languages of the country. Thank you. Hi, all. My name is Yusvenecker. I've been working as a data engineer for over 20 years. Recently, I wanted to switch towards data science, so I did a master's in that direction. For my thesis, I created a translation system for San Antongo, which is an English-based Creole language from Suriname.
where my father's from, and I'm of mixed origin. So my mother's from the Netherlands, and my father's from Suriname, and I've been born and raised in the Netherlands. Okay, hi. My name is Bonaventure Dusu. I'm originally from Benin, but I'm also known as the citizen of the world, traveling around conferences. I am soon to be a PhD student at the Quebec Air Institute and also at Migio. And I work on low-resource languages, African low-resource languages. I mean, focusing on formative languages, but also extending to other languages as well. And I'm also interested in machine learning for healthcare, like drug discovery, gene therapy, medical imaging, and all those type of things.
Great. Yeah. Thank you all. Thank you all for taking time to do this. This is so great. I mean, one of the really encouraging things about being in this room with you all is that, you know, so many talks here at EMNLP are great talks, but they'll say things like massively multilingual or like this works for all languages or something like that. And that's definitely one perspective that's not totally accurate. So maybe if we start with Rue and Andy, what, from your perspective, or you're passionate about, what is the area that you're working? And maybe just highlight some of those things that maybe are another perspective of either linguistics or NLP that are important for people to understand in terms of the work.
world's languages? Okay, so my passion is working on making sure that Isisulu is also a language of teaching and learning because currently in South Africa, English is a predominant language that is used in media and governance and also in higher education. So what we're doing now is ensuring that our languages are also at a level where they are at par with English in terms of developing it in tools. in human language technologies, in machine learning, etc. So we also want to have that privilege, if I can put it like that, in ensuring that our languages are also more accessible even online. I've been working on looking at specific literature materials, but you find that I can't find those online. I literally have to scan the book if I want to do an analysis of a particular morphological structure that I'm working on currently. But then you find that all the Shakespearean books, for example, are readily available online. So that's my passion in ensuring that even African scholars can do research in their own languages with writers or authors that are like them, that writes in the languages that we're interested in looking at.
We also have those readily available and they can be used for research throughout for posterity. And I think for me, also, for the institution that we are working with, or working for, is that we're trying to also build a bridge, a gap between NLP practitioners and linguists, because it's always a matter of linguists are doing their projects on their own side, and NLP practitioners are doing their own projects on the side. But I feel like if we can work together, because for you to understand the language that you're working on, you should collaborate with the linguists. And for us to understand how these language technologies work, we need to work with the NLP practitioners, because now in South Africa, NLP is not a big field. That's why our institutions are able to send us to such conferences, so that we know what other people are doing, so that when we go back...
to our institutions and going back to actually assisting the universities in the country we can impart that knowledge to them because currently really how we're doing research is still a bit traditional and time is moving technology is advancing we don't want to be left behind so i think our greatest passion is to make sure that our languages because we keep saying that under-resourced under-resourced until what like we've been saying that our languages have been under-resourced for many years now that now we have the opportunity to actually be sitting in a room like this with people who are actually doing the things that we've always heard of but never knew were possible for our languages so we basically have to say collaborate with us assist us to get our languages to a level where there are languages of teaching and learning and our data is also easily accessible because the other issue we have is accessibility of data we don't have a lot of data in our languages because we don't digitize the material that we have but at least now we know that we know what the possibilities are so yeah That's great. And just to follow up on one of the things that you mentioned, you mentioned this sort of idea of linguists and NLP people collaborating together. And you kind of mentioned the problem of sort of the availability of data, the scarcity of data. Are there any either misconceptions or things that you would like to highlight in terms of, you know, a lot of times what we see are these models, NLP models getting bigger and bigger and bigger and requiring more and more and more data to train. But the reality is for a lot of languages of the world, like you said, we either kind of have just a small amount of data and it's not growing quickly. And, you know, those models are getting further and further away from being applicable in those scenarios. So anything?
you'd like to highlight or point out there? Yeah, I think, yeah, that's the issue that I don't really know who builds the models, because I wanted to refer specifically to the people who build the models, is that when they're building them, it's like they don't have the idea of the structures of the languages that they're building the models for. Because we also have in our repository certain models that we're using, but as soon as we train them with data from our languages, the accuracy levels are always very poor, because it's like the systems or the technologies do not understand that, for instance, my language is alternative. So when they're being built, hence the collaboration, it's like... Also take into consideration the languages that you're building the systems for. Because I feel like the systems are always built for specific structured languages. So that's always an issue because now we do have access to the languages, but there's so little that we can do with them because they don't understand the structures of our languages. Hence, again, I will emphasize.
Emphasize the collaboration between NLP practitioners and linguists. Just to add on, a case and example is when looking at Google Translate, for example. It has improved quite a lot now, but previously you'd find that you'd want to translate something from my language into English or vice versa. The results were poor. But I think now because there's data that is available and they're doing something to ensure that there's an improvement in there, then you actually can see the positive results. But still, you find that in some other tools, even now, you still find that the accuracy levels are still very low. And as you say, it's the fact that when the tools are being...
built or created it's like they don't have the language structure in mind i feel like it's a matter of okay we've built this so it should work for every other language which is not the case each language has a unique structure we have a unique morphology even though they're spoken maybe for like the case in south africa we have nine indigenous languages but the structure is not the same so the people that actually build tools or create tools need to have that in mind and need to also work collaboratively with linguists people that are trained in these languages to ensure that the structure is also represented when the tools are created Yeah, that's such an amazing point. I'm glad you brought that up. I know even in our work, you know, encountering tools that are very popular, you know, things like word segmenters or subword packages where maybe it doesn't work quite right for like an Arabic script language or right to left language or whatever it is, just because that was never envisioned, you know, from the beginning. On that same theme of kind of the types of languages that people are building language technology for, I wanted to ask you just about Creole languages. Maybe people listening to this podcast aren't that familiar with Creole languages or understand kind of how they're used around the world or what they are. Could you describe that a little bit and then also a little bit about your language and the language?
you've been working with? Yeah, sure. So a Creole language is basically a language that emerges in places where people of different cultural backgrounds come together. So in the case of Suriname, that is during slavery time, basically the 17th century, people were brought from Africa to Suriname. Also people from Europe came there to basically, you know what they did, all kinds of horrible stuff. But they needed a means to communicate amongst each other. And basically it's a language that has characteristics from the different languages that participate from the people that participated in those communities, like English, Portuguese, Dutch, and some African languages.
So it's basically a melting pot of languages. I don't know if I can put it that way, but usually it's grammatically a bit simpler and it's easier to learn a language. Let's put it that way. Yeah, and one of the things you were mentioning to me in our discussions even previous to this is that for some of those reasons that you mentioned, you know, maybe Creole languages weren't and aren't always treated sort of at the same status of other languages. Could you speak to that a little bit? Yeah, definitely that's the case. So when I started this project, I found that it was really low resource. I was even amazed to find out that I couldn't find a single book or novel written in San Antonio.
And that's because there's a lot of stigmatization going on with that language. So for instance, in school, the Suriname is a former colony of the Netherlands. The first language still to this day, they're independent since 1975. The first language still is, and the official language is Dutch spoken in Suriname. San Antongo is the second language. So there's basically no sources available in San Antongo. And it was for a long time forbidden to speak the language in schools, for instance. Also, parents often discourage their children to speak the language. So that stigmatization caused the low availability that we are dealing with now.
Yeah, and describe a little bit about sort of your vision, I guess, for this project that you're working on and how it fits into maybe some of the needs of the language community that you're aware about just from being part of that community and how that shaped your view of the project and what you're actually working on in your project. Yeah, so I have two sons. My youngest son asked me a couple of, last year he asked me why I hadn't taught him how to speak Sranantongo. And that gave me actually the idea to build a translating system so that there's a lot of people living in the Netherlands with Suriname roots. Like I just explained, like in 1975, they became independent. So around that time, lots of people migrated from Suriname to the Netherlands. So this first generation is mostly fluent in Sranangtongo. I myself am a second generation. Before starting this project, I thought I was pretty okay in Sranangtongo, maybe 80, 90%. So I now would say maybe 50%, 60% after studying the language better. My son, who's a third generation, really wants to connect with his culture. So when we visited Suriname, for instance, and we're meeting with family members, he wants to know what they're saying. So I see this need.
within the Netherlands that people from the second and third generation, they want to connect with the culture, but they don't speak the language. So I hope that this translation system will support them in reconnecting with their culture. Awesome. Yeah. So I want to maybe ask Bona a couple of questions. I feel like I've been trying to get you on the podcast for a while now, so I'm glad you're actually here. I was actually already on. So yeah, I wonder, one of the things that we were chatting about just before this is that people view Masakane and some of the things that are going on there with a lot of respect and because of the momentum it has and how so much has been done. And I know you're part of some of the things that are being presented here with Masakane. Could you describe maybe, we've actually mentioned Masakane on the podcast before, but for those that aren't familiar, could you describe what it is?
Okay, I hope I will do justice to everyone. So I would describe Masakali as a grassroots organization, an NLP movement that wants to build NLP on language technologies for African languages by Africans. And as everyone talked about, as everyone said earlier here, we need more people speaking those languages to be more involved in the building. of those language technologies. Because someone, for instance, who works in a big company and be like, oh, okay, I found maybe this data on, let me say, Oscar or like on like Flores or any data set that is saved to be high quality and like multilingual, you know, he's just like training a model on it and then assume that everything is supposed to be working, you know, but while at the beginning, those like language models like SLM, those type of things have been created for like high resource languages like what, like English, Chinese, you know, even like in those initial papers, you can see that the downstream test, they evaluated those models on.
language is like French, English, Chinese, those type of things. So there's the assumption that, okay, we train those massive pre-trained language models, and well, we can just do some transfer learning to low resource settings, which is not always true, which is one of the ideas and papers that I presented here, talking about active learning for language modeling. So as everyone has been saying, this is important, and that's the gap. Also, Andy said, and Ruth said, and just said, we need to reduce that gap between languages. and people who are practitioners or NLP practitioners. We need domain expertise, domain knowledge. And of course, as an NLP researcher, I can have those, but languages also have to come in to be able to say, okay, whatever this model is predicting is rubbish or it makes sense. And I think as the most recent ongoing stuff on Twitter and all, models actually don't understand language. They understand data distribution. They understand words. But then...
We need, again, as I'm emphasizing, we need that expert knowledge to be able to make sense of whatever those models produce, to be able to say, okay, this is actually something useful. And, yeah, that's what Master Candy is trying to, like, build a community of, like, NLP, not, I want to say NLP research, a community of, like, let me say, people, Africans, who are working on African technologies. So it includes linguists, it includes people who have theoretical, like, NLP mathematical background, like me and like you. You're also part of the community. We have also, like, Sebastian. We have people who actually are not Africans, but who have, like, interest.
in building a company, that effort. And that's what the community is all about, putting forward and representing, bringing more people like us at this type of big NLP conference, let me say, to the world or to the map, or however, increasing representation and making sure our language are preserved through those technologies, which I navigate about today, because somehow everything we're doing now is based on it somehow. So yeah. Yeah, that's great. And I can speak, as you mentioned, I've had the great privilege to interact with a lot of people from Masakane and, you know, on a few small areas. But even in those small things, I would just encourage researchers out there that have an interest in what's being discussed here to engage with those communities, engage with local language communities and speakers of those languages as you're building these technologies. Because you do, you know, I'm benefited so much by getting to interact with Masakane and the things that I learn and, you know, getting to have these sorts of discussions to have better awareness and understand how I can join and partner with people in the building of language technology. So as we kind of have gone around and talked about various things.
wondering if you all could maybe share there's probably people list a lot of people listening to this podcast that are actually listening to it in English but or maybe have a mother tongue their English is their second language and they're thinking hey I wonder what I could do in in my mother tongue in my in my first language I know it's not well supported in language technology any encouragements from any of you of like if you're a language speaker and you you're wanting to sort of get into this somehow and kind of partner together collaborate with others to help build you know more language a higher level of digital language support for your language what would you say? Yeah, if I can add to that. So what you usually see, you need a lot of data to train a good translation system or any NLP application. And although there is not much data available in San Antonio in my case, the thing that you can usually find are Bible translations. So in my case, I use the Jehovah Witness 300 corpus, which has a translation of 300,000 parallel sentences from Dutch into San Antonio. And even in some smaller, even smaller languages spoken in Suriname like Saramakan and Aukkan, which is closer to some African languages.
So that's the starting point. Of course, the idea would probably be that you create a general purpose translation system. That was also my plan and still is my plan. So on top of that data, you would need some more data from a general domain and from different domains next to the religious domain. So it was funny actually to see that Daniel from SIL, I found online, I found a dictionary in San Antonio and it contains a lot of words and with those words, some example sentences. So before even knowing Daniel, I found his website of his company and was able to scrape 3,000 sentences from there. Yeah, hopefully next time you don't have to scrape it. Shoot me an email. Good to know you now. And on top of that, what I did is basically scan some smaller sources and OCR them basically and then manually align sentences. So it's a lot of manual work.
I think also Bona has the same experience of doing a lot of manual work for his translation system. So now that I finally was able to get my first model up and running, I also built a translation system around it, a web app. So I'm now in the pilot phase where people are starting to use some speakers using the system and evaluating the results. And so basically by just using it, they enter a sentence in Dutch, they get the translation back and they rate it. And if they don't think it's good, I ask them to enter a better translation and they submit it. And I collect that data in my database. So I hope.
in this way to collect more data from a more modern use of a San Tong instead of the religious one and collect enough data to eventually build a system that is more potent. Yeah, that's great. I think you highlight something that's definitely good for people to realize. This is also our colleague Colin Leong from the University of Dayton. He told me about this, that his parents, who are speakers of a local language in East Asia, he asked them, hey, give me all of the data you have for your language and I'll try to build something. And he showed me the folder and it included MP3 files and Word documents and images and PDFs and all these things. So I think, yeah, that's something important for people to realize that not everything has a nicely curated data set on Hugging Face and even being involved in some of that work to get that data put together is a hugely...
beneficial thing so yeah anyone else uh things you want to highlight for people out there i'm wanting to to start some of this before leaving the floor to the ladies yeah i would like to just like second uh what just said i also had the same let me say struggle with phone because i started and nobody was working on phone nobody knew about the language and that was also something interesting and exciting you know like going into a direction where nobody's looking at and like unveiling it so not too sure but a lot of people nowadays just like quote me as the phone guy you know like when somebody's talking about phone or whatever and there's dub they just tag me on like Twitter or whatever. And yeah, I envision just to be the same, like for Sanatongo. And you know, the moral of the story is that you need to get started because there's always going to be a point where there's no data and someone has to do some little effort. You know, for instance, we have GW300, but what if those people didn't do anything? We wouldn't have even, we would not even have a starting point.
So I started with GW200, then I tried to manually scrape with my friends and all through Google Forms. I created something like 25,000 sentences. And then out of that, then I've been able to bring some proof of concept. And it grew up, and people are now more knowing about the language. Still is not, I mean, I've used FFR Translate with Chris. People are using it. People are very sending feedback. They are happy. It helps them. Artists, people, they are more awareness, people willing to be.
more contributing, creating more content. It's not yet on something like a Google Translate or a centralized translation for those African low-resource language or low-resource language in general, but I hope that something's going to be coming. So I'll just say just that. Yeah, honestly, like my name says, I like adventures. And I like good adventures. So I just like to go where nobody's focusing on, and none is exciting. You bring something that people haven't been focusing on tonight. I don't think I would have had the same impact if I, for instance, started with Igbo, because that project, the first FFR project that then went on DPC, all those type of things, we were dubting whether we should use for Igbo.
So finally, Chris and I decided to go for phone because Ibo has at least some effort done already, but nobody heard about phone. Nothing was on phone. Today, there are a lot of papers, people citing the work. It's been cited in the paper that led to the extension of Google Translate to 24 or 20 more African languages or 24. It's been cited in no language left behind of Meta. And also being part of the Master Candy, you collaborate with people like Sebastian, with Julia, with Angela Fan who work on NNLB. So just get started and people will know about it and then I will just keep supporting. Yeah, if you don't have support, just be a self-supporter. And at some point...
Like, you know, when people are seeing the effort, they will definitely then join, and they need to, like, pick it up from there. Yeah, Rue or Andy, anything to add? I also share the same sentiments with the case for AC Zulu. You find that people should just start, even though it's difficult, because data is available, but people are not coming forth. They're not wanting to share their data.
you find that, okay, you collect or you do whatever with it and then you just keep it to yourself. And then that now hinders the progress of the language or the development of the language in itself. I think it'd be a great idea if people get the understanding that when you are allowing your data to be accessible or make it open resource, you're not, it's not a matter of, I want to steal your idea. I can do something different than what you did with the data. And also just ensuring that it's more researchers have accessibility to it so that they can use it for whatever that they want to use it for. The only issue that I have with getting sure that the data is more, is collected in a general sense, because you find that newspaper articles are very much easily accessible. But in the case where you find that novels, you can't, you need to actually do OCR.
and do the scans by hand, which takes a lot of time. So if we can find something that would work, that would be much quicker, we'd be grateful for it. So that at least we can get it to be at a level where we have enough data to train models and train tools with it. Yeah, what I would add is that language preservation is very important. Let's find ways in which we can preserve our languages in the sense that they do not go extinct. For instance, what we're doing now in South Africa is that because most of our languages have dialects.
and dialects, because they're not standard languages, they're not documented. So one of the projects that people can do if they're in similar situations is where you collect speech data of those dialects so that they can be accessible somewhere. So if in the next 10 years a dialect is not spoken, there is data available for people to hear that, okay, there was actually once a language that was spoken in a certain area. And yeah, I think more than anything, it's just preserving the language, creating more data, because now for this Mazakana project, we're able to access our data via online newspapers. So if people can also digitize the work that they have, because it is true that it's so difficult to find material because it's not being...
Not digitized, published, yes. Some books are out of publication now. So for that to not happen, we must digitize our literature, our text, and everything else so that it can be easily accessible so that we can actually run such projects. Yeah. Great. Well, I really appreciate all of you getting to have a chance to join us here. I know it's a busy conference and there's a lot of great things to look at, but I'm so happy that we get to bring this conversation to a wider audience. And maybe, you know, if anyone wants to leave with a greeting in their language, please go ahead. So before finishing, I will do a little bit of like promotion. Yeah, please do. Regarding the language discoverability and other stuff of things, Chris and I have been working on Lanfrica, which is an innovation, an idea of like...
putting out those language resources for those African language resources that are not discoverable on the internet. So you can access research papers, you can access data sets, you can access tools like keyboard or those type of things and anything like dictionaries, anything. Even sometimes like YouTube videos. So people are doing great things like educating people, for instance, in Afra or those type of languages, languages I've never heard about before. And they are doing great effort, but then it's like on YouTube and nobody knows. So on Africa, they can easily kind of access those resources. And if you also have your work, like on mainly on...
African languages, but if it's on low-resource languages, we have the Lanfrica talk, where people from all around the world, we got people from CMU, we got people from Google Research, we got people from UCL, people from around the world, students, researchers, anybody who has been working and passionate about low-resource languages, NLP technologies, tend to come and give a talk for people to know more about what they've been doing. So it's pretty simple, you just go on www.lanfrica.com, L-A-N-F-R. and then like you can find all the information about languages and also how to assess the or like how to book for a meeting for the Lanfrica talk. So that being said, like something basic in my language would be, that means, thank you so much, Daniel, for, well, I don't know about the forum for this invitation, you know, and that's something like I can relate to just, I thought that I knew the language, but then it feels like, okay, I actually don't know anything.
And I don't know, they've been that interesting curve. It's called a curve of, like, David something, where it technically shows that, okay, when you think that you know something, you have high confidence, and that's actually when you actually don't know anything. But then when you start learning, and then you're adopting more of yourself, it's technically, like, that process of acknowledging and, like, learning more things, and then, like, the curve, like, goes up at some point again. So, yeah, I'll just stick to Enna Chen, Rika, Daniel, and I will let just Andy and Ro finish in the languages. Thank you so much. I'd also just like to maybe just promote also the organization we're coming from, the South African Center for Digital Language Resources. Any researchers that are working in any of the South African languages, they can check us out at sarila.org.za. A parting word that I can say in my language is, which means thank you so much for listening to me. Now, I don't know what I'm going to say, but I'm going to keep it very short, which means thank you very much. Yeah, before finishing, I must express that I'm actually a bit jealous of the Mexicana community.
Free of charge. Okay, that's good to know. Thank you very much for the invitation. Also, I wanted to add that maybe if there's other people listening who are into Creole languages, maybe give me a shout-out. My name is just Svenneker, just.svenneker at gmail.com. I'm happy to join you guys, but I'm not sure if it fits in as being an African language.
As we say, there's Daniel, for instance, there's Sebastian, there is Graham, you know, people actually who are not necessarily from the continent or speaking those languages, but who just share the same vision. So, like, well, many people, I'm sure that many people will be interested in, like, maybe, like, writing research papers or reading language models or those type of things for Sganatongo. So, I mean, as they say, alone, you can do better or something like that, but, like, together with a bigger community, you can go definitely faster and, like, you know, and, like, that's how technical ideas come, you know. So, um... I'm definitely going to share the link with you. I have your link in to join the Slack. And we'll put it in our show notes as well. All the links to everything we've talked about. Yeah. Okay. Okay. So I'm definitely going to share with you and you're afraid to just call me. I just want to warn you in a big place. People find it messy. We usually work in that chaotic environment. You mean community is messy? Well, I won't say it. I mean, it's just like... That means it's a real community. Yeah, yeah, yeah. So things are not like, don't wait for people to be like, okay, this, this, this, you know, take ownership, take initiative, like on whatever you want to work on. And then people will just like easily follow you.
And yeah, I'm pretty sure it's going to be beneficial. I would like with pleasure, for instance, to jump on the project with you. I speak, what, French, a little bit of German, Russian, English, phone a little bit. So it wouldn't hurt me to learn a sixth or fifth language. At least, you know, I learned a lot, like, working with people about, I mean, working with Igbo people or those type of things. So let us know the African past KU. Just join. It's a bad law resource in general. And I know there's even, for instance, in Nigeria, they've spoken Nigerian pigeon, which I had some ideas of.
doing some pre-training transfer learning from that line because I think there's some similarities going on there. So that already brings the connection to Masakana closer, I guess. So I would like to finish with just one word, which means thank you. Well, thank you all. This has been so much fun. Appreciate it. And we'll share links in our show notes for everything that we've talked about and all the great things that you've shared. So thank you all. Thank you. Thank you.
All right, that is our show for this week. If you dig it, don't forget to subscribe. Head to practicalai.fm for all the ways. And if Practical AI has benefited your life, pay it forward by sharing the show with a friend or a colleague. Word of mouth is the number one way people find shows like ours. Thanks again to Fastly for fronting our static assets, to Fly.io for backing our dynamic requests, to Breakmaster Cylinder for the beats, and to you for listening. We appreciate you. That's all for now. We'll talk to you again on the next one.