Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI-related technologies are changing the world, this is the show for you. Thank you to our partners at Fastly for shipping all of our pods super fast to wherever you listen. Check them out at fastly.com. And to our friends at Fly, deploy your app servers and database close to your users. No ops required. Learn more at fly.io.
What's up, friends? I'm here with Vijay Raji, CEO and founder of Statsig, where they help thousands of companies from startups to Fortune 500s to ship faster and smarter with a unified platform for feature flags, experimentation and analytics. So Vijay, what's the inception story of Statsig? Why did you build this? Yeah, so Statsig started about two and a half years ago. And before that, I was at Facebook for 10 years where I saw firsthand the set of tools that people or engineers inside Facebook had access to. And this breadth and depth of the tools that actually led to the formation of the canonical engineering culture that Facebook is famous for. And that also got me thinking about, you know, how do you distill all of that and bring it out?
to everyone if every company wants to like build that kind of an engineering culture of building and shipping things really fast using data to make data informed decisions and then also informed like what do you need to go invest in next and all of that was like fascinating was really really powerful so so much so that I decided to quit Facebook and start this company Yeah, so in the last two and a half years, we've been building those tools that are helping engineers today to build and ship new features and then roll them out. And as they're rolling it out, also understand the impact of those features. Does it have bugs? Does it impact your customers in the way that you expected it? Or are there some side effects, unintended side effects? And knowing those things help you make your product better. It's somewhat common now to hear this train of thought where an engineer, developer was at one of the big companies, Facebook, Google, Airbnb, you name it. And they get used to certain tooling on the inside. They get used to certain workflows, certain developer culture, certain ways of doing things, tooling, of course.
And then they leave and they miss everything they had while at that company. And they go and they start their own company like you did. What are your thoughts on that? What are your thoughts on that kind of tech being on the inside of the big companies and those of us out here not in those companies without that tooling? In order to get the same level of sophistication of tools that companies like Facebook, Google, Airbnb, and Uber have, you need to invest quite a bit. You need to take some of your best engineers and then go have them go build.
tools like this. And not every company has the luxury to go do that, right? Because it's a pretty large investment. And so the fact that the sophistication of those tools inside these companies have advanced so much and that's like left behind most of the other companies and the tooling that they get access to, that's exactly the opportunity that I was like, okay, well, we need to bring those sophistication outside so everybody can be benefiting from these. Okay, the next step is to go to statsig.com slash changelaw. They're offering our fans free white glove onboarding, including migration support, in addition to 5 million free events per month. That's massive. Test drive statsig today at statsig.com slash changelaw. That's S-T-A-T-S-I-G.com slash changelaw. The link is in the show notes. Welcome to another episode of Practical AI. This is Daniel Whitenack. I'm the founder at Prediction Guard, and I'm joined as always by my co-host, Chris Benson, who is a tech strategist at Lockheed Martin. How are you doing, Chris? Doing well today, Daniel. How are you?
I can't complain. It's like these two weeks in the Midwest in the United States that I enjoy each year where it's like between hot and really, really cold. And so, yeah, I'm excited about that. And yeah, just cranking away, I'll be at the Intel Innovation Conference next week, which is going to be a fun experience to see some of what they're doing in the AI space and talk to them about some of the stuff we've been trying on Intel hardware. So yeah, it's a lot of good stuff coming up. I'm really excited. Actually, it was through that Intel community that I met Dominic from Ask UI, which is our guest today. So welcome, Dominic. It's great to have you here.
Thanks, Darlan, for having me. Yeah, well, AskUI, could you tell us a little bit about first, maybe just what is it about UI that AskUI is concerned with and how did you get to start thinking about UI and automation? Maybe a little bit back on what are we doing? We try to... free humans from being robots. What does it mean in particular? Not only do you have repetitive tasks on user interfaces, but what we are trying to do to bridge the gap between you have to be the programmer versus you can describe in a natural language what you want to do, so your intention, what you want to automate. For example, if you want to log in in your Facebook, then you can say with us, please click the log in button, please fill out this and this credential with natural language, which allows also non-technical guys to automate user interface and lower the
water, which you can't automate or start to automate things. And the second question was a little bit about... Why did you start caring about this problem? Why, why, why, why I started caring? A little bit background about this. Previously, my background is a software developer. I was working previously at Siemens before I founded with Jonas, my co-founder of Askii. So there, what I was doing there, I was in the plant automation system where we had to test everything because our systems have to run 365 days a year.
So everything should be tested well enough. So we were standing a lot of inside the testing level and test everything through. Then I switched also a little bit to Palmon, the same thing. You have to test, you have to stand directly on to test the application. And then I was moving to a new organization, which tries to modernize or bring the agility within Siemens. I learned a lot about Scrum agility, bring all the new stuff, tools. But I had also the problem because Selenium and other tools couldn't solve the pain to write the user interface tests in the way. And then I was thinking, hey,
can we not solve this with AI? Because AI can understand visual information and can understand initial information. So I thought, hey, can we not combine this? And all of this, the journey started. This might be a completely ignorant question, but I just want it for my own context as well. Now, in my background as a data scientist, I've participated many times in the fun activity of web scraping. And through that, I know like, oh, there's a lot of ways in which web data or UI data might be exposed. So could you talk a little bit about like how?
What does the data look like? What does the problem look like as you're trying to get an AI model to interact or use an AI model or natural language input to an AI model to then interact with either a UI or a web page, especially when a lot of those things can be quite varied in terms of how they're built. Is it more of a visual thing or is it something else? Yeah, we are on the visual part. For example, when you're talking about Selenium, Selenium is only working with web interfaces and you cannot use this, for example, on Android and so on. What we are doing, we're doing really a screenshot of the system.
So it means we can take a screenshot of the application, we can take a screenshot directly off the operating system. And then our AI model, what we trained, can detect the user interfaces. This means we're detecting buttons, we're detecting text, we're detecting text fields, we're detecting checkboxes, we're detecting icons. And so we have already trained a model which can understand the visual representations of user interfaces. And then we connected our natural language part to it to match the intention, what you have. For example, click on the lock-in button to the real lock-in button on the user interface, and then we are moving them out there. So this is a little bit different from the concept versus...
I connect directly to the application and then try to scrape the source code of the application to get my information. Maybe this is the best way to differentiate this. So you're kind of starting with a screenshot instead of doing web scraping. Correct. And then just doing classification on the screenshot itself to do that? Correct. So we have object detection model in place, which really detects the screenshot and detect all the elements on it.
As a quick follow-up, when you do the classification and identify what you have in the screenshot, how do you tie that in with the test desired? So you have a collection of tests that you're trying to automate. How are you, like, tying, like, I got a screenshot, I have a button, I have a text field. How does that tie into a particular, like, a unit test or something? Currently, we have our textbook application, which you can download, which is also available as an RSI campaign package, which you can use. And with this, you can directly start, install it, and then generate a standard test. And then you have, for example, the JEST test log, the Java JUnit test log.
where you then can write askui.click.withtext.so on. And the background, what happens, we have a controller then installed on your local system, which connects to its operating system, takes a screenshot, and also have the ability to move the mouse. Then we're taking the screenshot, we are connecting the instruction, the click on button, for example, then this to our inference backend, then we get the result back, and then we're moving the mouse there. And with those single steps, as you would do it in Selenium, you can then write your workflows or tests to automate every operating system that we're currently running on. Windows on Mac and on iOS, especially for a legacy application in the Windows environment, we can test also these applications that we are not limited to web. Did this answer the question? It does, it does. Thank you. I appreciate it. It was a good explanation. So you obviously had a certain perspective of when you came to this problem because you had worked at Siemens, you had thought about this sort of automation. I'm wondering, as you've developed this technology and seen others that might have...
had this need or experienced this pain. What is the sort of range of things that you're seeing people either do or want to do with this kind of automation technology that might fit the use case, sometimes fit the use case that you had in mind initially, but sometimes might be a sort of new things that you didn't think about previously? Because we are using AI technology and with AI technology, you're a little bit more flexible what you can describe because you can learn it based on data and have not to explicitly describe which things you want to do. New use cases are what we started where yeah, we want to do regression tests for classical user interface tests. Then we moved a little bit in the direction of robot process automation. And I think where we would be planning in the future is to automatically transfer unstructured data to user interfaces. So if you have, imagine you have a PDF, which in a formula, and then you want to say, hey, please, normally normal workers will do this, which are copying, for example, the PDF or information from the PDF over to another formula. And what you can do then with our technology is to say, hey,
please automatically copy all the information over, which you see on the left side to the right side. This is where we think where the technology will go so that you can definitely find the matching of the files, everything, when you then connect also the language model, which have the ability to understand the language a little bit better or a little bit closer as humans can define it, then you can build such systems. Yeah, that's really interesting. I'm thinking of even like in I had a friend that works in the offices, same office building that we're in here. And he stopped by and we were talking today. He he takes screenshots of his work as he goes along in the day, like every so often. And he does this because he wants like he can't always remember, like all the things that he was doing throughout the day. And so he takes these screenshots as a sort of historical record of like the various interfaces and what he was doing and such. So I had that in my mind as you were talking through all these things. I'm like, hey, this would be really cool for him. He could, you know, potentially query certain things about his day.
interfaces and what he did and create potential automations off of those. This is also a use case where we're thinking about to be a recorder in the background between collecting repetitive tasks and then say, for example, hey, we have not detected you have done this over the last week five times. Should we automate this for you, for example? There are also cases where the technology can go because we have the understanding of YouTube interfaces. There was a lot of engineering stuff around this, then you can build such nice systems. That seems like an automation, like automation and AI seems to like scare people for some maybe justified reasons, some unjustified, or I don't know what's justified. But that's like an automation that like, why wouldn't I want that to save myself some time? I would love to automate some of my repetitive tasks. So one of the things I'm wondering as you're having looked at your website is you talk about like all the different platforms, you know, you get web apps, enterprise apps and everything. Are you able to take the same approach across the different platforms that you're targeting for that?
And is that what makes it so flexible by doing screenshots and stuff? Is that what does it? Or do you have, how do you break down the challenge of getting from, you know, addressing one platform in your initial and then starting to spread out across the other platforms? From the other, the main technology is to accessing the screenshot from the operating system and to controlling it. I think if you have tried to do it, there are a lot of open source software already available, which can do this. If you have such technology accessible, then you can take the screenshot. For example, we have Android support where we can...
And it takes a screenshot and it's the same model and the same technology behind them as we are using this under Windows operating system or under Linux operating system. And we have a general model which can solve all the tasks. So Dominic, I'm already thinking of a lot of use cases that maybe I might want to automate and interact with the system. One of the things we were chatting about prior to actually hitting the record button on this episode was maybe the unique approach that Ask UI has taken in terms of more of a software engineering approach to understanding how these machine learning and AI systems work and utilizing them in this sort of like more practical software engineering approach to this. I'm wondering if you could talk about that in a little bit more detail and how you've approached these problems that you think is maybe unique or at least represents your perspective on how to build systems like this. Yeah, first of all, what I see or what we saw already in the past, the research area has built a lot of models, has released them on the public, but then it stopped it. After you have published your paper, you have no interest anymore to bring this to production.
What we also see in the beginning of the 2020s is that a lot of new applications are coming, which try to solve this, to formalize this as software patterns. As I know at the beginning, I started with machine learning and I was wondering, are there now software patterns, like metric pattern, like trainer pattern, like other kinds of patterns which you can reuse and which you can communicate in a better way? Why are we good at software development nowadays? It's because we have standardized the patterns which we use and use them everywhere. So this is what I was always searching a little bit and also the tooling.
And how did we approach this from the machine learning? We have different teams a little bit. The first thing what we have done, we have built an application, which used directly our model. And we used to plug in the best of the first model, which we can produce. And we started with our model, which has only five images for object detection for detecting this, five images training data set, which is not a lot. But we've done this to prove that it's end-to-end possible. And we mocked a lot of things away. And then we go on out to the customers and say, hey, can you work with us? And then you complain.
yeah, it's nice, but your model object detection model don't work so good. So what we've done in the next one, we collected more data, trained a better model, then we go over to the customers, now enough. And then they were, okay, it's going the direction, but we could only support one application at the beginning. And then we had to relate it based on this and tries to connect all our things together. What this means on the other side, we started directly to taking, going out to the customer and then increasing everything. And now we are making, because this was hard at the beginning, because I mentioned already that in 2020, all the...
tools, Piposh Lightning, and Metaflow. And so everything came up, which you could reuse. And now we are migrating more and more to the data pipeline and trying to bring everything to the customer so that they can also train by themselves. But this is our software engineering approach where we say, hey, bring everything to the customer. Let the customer complain.
then you're doing the next situation steps. So this is the lesson. So as a follow-up, I want to ask you to kind of flip-flop what you just covered a little bit. And what's it like to engage from the customer's perspective? Because you're kind of taking it from your perspective and we take it over. So if you're a customer and you start to use Ask UI and deploy it, what does that picture look like? What does the customer go through as they start deploying or start utilizing the service? What he's doing too? There's also a little bit of history, but what is this now? You're going now to our website, then log in with your credentials, and then you can directly upload your first screenshot and then simulate on this.
So do a simulation directly on the screenshot so that you see the aha effect. And then the next step, what you want to do, if you have created your workflow a little bit, then you want to automate this. And you send and then schedule this, for example, in the Docker container in the background we have already, if you're in the web environment. And then after time, you get the result and you're happy that you have automated with really easy things, the workflow. And what we are now doing is to reduce also the hurdle that you have to learn less as now because this is the thing that we're trying to bringing all the problems from the user perspective away. The user has a really easy life to create automations, to maintain automations, and to schedule all the stuff, set up all the testing environments. Because it's not only the automation part, what you're interested in, you're also interested in where can you schedule it, and what the broader thing, how can you connect data and so on. This is what we are doing. That's awesome. I want to propose what I think is probably a bad idea, but I want to get your reaction to it. So I'm wondering, there's this way now that you've enabled people to automate their interactions with various UIs. And there's...
plenty of cases where I don't really care to interact with a UI, but I do need to accomplish a task. But I also, like, for example, one of these that I struggle with all the time, right, is like AWS and its interface, which is just like, you can do everything right, but it's super hard to understand how to do anything, right? Is there an opportunity? Let's say, and again, I think this is probably from the start a bad idea, but let's say I just gave some sort of agent tied in to ask UI my credit card information and whatever, and just said, hey, I want you to create a AWS account and spin up this infrastructure and do this. And then when you're done, give me the URL so I can access my, here's the GitHub repo, tell me when it's ready, and here's my URL. Now, I imagine that would also need to tie into other external knowledge, like the documentation from AWS or something. But is this type of scenario anything that...
like you've been talking about internally or see as things that might come about in the future, as soon as you start automating things around UIs, there's the one side of it, which I think we've talked about a lot, which is automating the things you've already done with UIs. But what if you want to do things with UIs that you haven't done yet, but don't really care to learn how to do? This is also our plan to do this to leverage that. We already played a little bit around with large language models, giving all the documentation to do this and also tries to translate, for example, Google documentation and create out of this AskUI step. This was working quite good.
and the direction will go there. So in the future, we can do this. So there was another part that was the attention base. Please create me an ET2 instance and share my credit card information. This was an intention. Please do this. You can do this. But the main problem with this is, humans, if you talk also to your colleague or to someone from another nation and so on, you will always have communication hurdles. So you have always to have the tiny steps in between where you then can correct a little bit up. This is one thing. The other thing is, are we trustful for credit card information? For this, I have to say, you're submitting also GitHub or GitLab.
your security tokens to your AWS successor. There are standards already in place, which are also validated, which can do this. And this is what we have to follow. We have also business, also enterprise customers, which wants to have the standards. So we have to be compliant with the standards. So from our side, it's no problem. You can trust us. But on the other way, we have also the possibility, because it's nice to create the tests online, download everything and execute this locally on your machine without any access on our side. You can also hide all the information, all the secrets, I would say.
on your device and I have the guarantee that they're not leaked on anything. Yeah, that was another thing I was going to ask is like, hey, if I'm going to a site, like, let's just say I'll give an example because I love the product and what they're doing. So, Chris, you remember we had Josh from Koki on talking about their voice studio and all that. I've been using that recently. You go in, you can input your text, like a sentence and synthesize a voice and then you can change like the language or something and then like go through export the file. So there's like.
there's not just a ui interaction here there's like input data and output data that might include things like passwords it might also include things like hey i generated a file with this ui like where does that go like a quick download or something i don't know what's the best practices around as you're automating these things how should a customer think about these like inputs and outputs and how do you handle those in terms of what you're building i would forward this question to how are you normally doing integration tests or end-to-end tests you have always the same data there so first of all to recommend customers uh try to use always um synthetic or generated data, don't leak production data to it because it's security. This is the processing. What you learn when you start automating and testing, don't use production data. And then the other thing, you have to apply security standards. So there are environment variables or secret files, which you can inject to it, go with it, do it, use our security function that you're not sending to us anything that's related to it. And this is what I could recommend to the customer.
I'm curious. We've been talking about testing for a while, and most of the use cases have been unit tests. Are there any other types of testing, like integration testing, that you're able to do? Is there a workflow for those kinds of things? Or is this really focused on kind of the screenshot itself and everyone stands alone? Is there any way to tie them together? Yeah, you can tie them together. What we are, we are only a library, which you can use in TypeScript. In the future, we will also support Python and other languages to make this technology also available. But you can also combine this with Selenium or some other techniques you can connect to a Mongo database, getting the data out, processing this in terms of other systems and so on.
Yeah, we are really flexible in this race because our main concept, this is maybe also another thing that's unique on our side, we are thinking that automation or user interface automation or the local user interface automation is to a certain level nice because it can give the ability to other people. But at a certain point, we are reaching the limit. And for this, you need developers to build some nice stuff to automate or to connect, for example, to MongoDB or some other sort of stuff. In this case, you can always have always the possibility. to go from our low code view to our code view and insert directly code. So there's no problem. You can do this and you can also install other libraries.
This is a changelog news break. PageFind looks pretty cool. It's a fully static search library that aims to perform well on large sites while using as little of your users' bandwidth as possible and without hosting any infrastructure. It runs after your static site generator, like Hugo, Eleventy, Astro, etc., and generates a static search bundle to add to your built files. It then exposes a JavaScript search API that can be used anywhere on your site. Quote, the goal of PageFind is that websites with tens of thousands of pages should be searchable by someone in their browser while consuming as little bandwidth as possible.
PageFind's search index is split into chunks so that searching in the browser only ever needs to load a small subset of the search index. PageFind can run a full-text search on a 10,000-page site with a total network payload under 300 kilobytes, including the PageFind library itself. For most sites, this will be closer to 100 kilobytes. End quote. I'd love to see a comparison, link me up if you know of one, but my guess is that this could easily replace Algolia on lots of open-source docs and websites. One less service to depend on. Why not, right?
You just heard one of our five top stories from Monday's Changelog News. Subscribe to the podcast to get all of the week's top stories and pop your email address in at changelog.com slash news to also receive our free companion email with even more developer news worth your attention. Once again, that's changelog.com slash news. I always like to ask guests who have really manifested some new idea that's really driven fundamentally by AI and machine learning this question. And that's, you know, as you are building out this product, what challenges did you find in using? You already alluded this a little bit with like, oh, researchers release like all of these models. And then like what happens after that? They're not really supported or maybe they die off or other things.
What specifically were the kind of machine learning or AI challenges that you faced as you were trying to make this work? You alluded a little bit to the data side of things and kind of adding data over time, but I imagine there's much, much more than that. So what are some of those things that stand out, just practical things that you faced in trying to apply this technology to a real world automation problem? Yeah, maybe to start at the beginning, previously I was a software engineer. I had no clue about machine learning. I get a little bit theoretical knowledge, but theoretical is nice, but in practice, it's completely different. So I had no clue that, for example,
If you build up the learning rates, you can bring your model to convergence. So such things I struggled at the beginning with also how you connect the layers at the beginning to make everything work. But then you've solved this problem. Then the next step where you start to struggle is to making the experiments visible so that you generate the right metrics that you see that you understand what you're learning or what you're not learning. Then when you have solved this challenge for yourself, so you have tried out a TensorBoard, you have the next challenge to go to how can I manage the data? How can I increase the data? How can I version the data?
And then you came in the challenge, how can you do repeatable experiments that you can say, hey, you have done now progress step by step. And for this, you have to then to search a lot about what tools are out there, which tools are good, so you need to know a feeling. Then the next step is you've figured out that you have messed everything up and your code is totally out there. So you have to think a little bit about how can I structure the code. This is what I mentioned previously with patterns. So you're looking in other repositories or other developers to structure the code that it's more maintainable and more reusable.
And then you're coming across, for example, PyTorch Lightning, which is what I say, which are doing this really great, to build up models in a modular way. And then you're not only one developer, then you're two developers, two machine learnings, a researcher, then you have to communicate in a way. So you have to exchange data. So then the thing is, you're starting to copy and paste data and send it over Slack or some stuff. Then you have to say, hey, it's totally stupid what we are doing. We need a data platform. And then with you going through step by step, you have reached the complete expertise that you say, hey, now we need a complete data storage. We need a metric system. There's how we exchange data between the teams. This is the way how we label data.
For example, also the other challenge is not only exchanging data, getting data. This is also the challenge labeling goods data. Then you're looking at the labeling tools. So then you figure out, yeah, labeling tools are nice for standard use cases, but sometimes, especially in our case, because we have five different models which are chained together in a nice way, that you have labeled different kinds of data for different models because these model types are fitting perfectly for this use case. So you're then thinking about how you can improve the labeling process. So now we're building a new labeling tool based on Streamlit so that we can easily connect, for example, our inference part.
that we can do a little bit preloading, then can automate stuff and to improve everything. And then at the beginning, you remember that you talked once with some guy, which always said, if you do machine learning, you will end up to building a labeling tool where we now reach. You've reached the pinnacle. But it's a journey. And I think when somebody would now ask me how to start, I would answer completely different in a completely different way. So that's literally what I was about to ask you, because that was a fantastic journey that you just took us on about all the practicalities. And, you know, you solve one problem and you hit the next and you hit the next and you hit the next. And you just described.
coming in completely new to this and taking it all the way to being very, very productive and all the practicalities. So that's actually what I want to ask you is you said I wouldn't do it the same way. I'd like to know there is at least one person out there right now, if not many, that are thinking about AI. They may have dabbled in it. Maybe, maybe not. They have an idea for a startup. They're listening to you and they're going, that's the guy, you know, who started doing this. But I have my own idea.
What would you tell them? Like, how do you get started, you know, to get going? Because this is a daunting field to break into. You mean only the machine learning part or to start a startup based on machine learning? Mostly the machine learning. Like, how did you learn? It's a skill set. It takes the time to digest and it's constantly evolving. How did you digest that skill set so that you could be productive? Okay. First of all, what I would recommend, directly introduce tools which will support you. Use directly HuggingFace. Try to build models based on HuggingFace so that you have libraries or beta on PyTorch Lightning because they're giving things for free.
I would always say. Then the other thing, I would directly introduce version control system for data. Beginning, I would recommend using Cleama. I would now recommend DVC at the beginning. And then try to find, especially for the machine learning part, try to find one researcher and one software engineer and bring them together and let them communicate. Because then you get the efficiency from software engineers, especially cloud software engineers, and you get the research knowledge and bring them together so they can learn from each other to exchange ideas, how to do this in the software manner and how to do research. And this, when I would start again, I would now say, hey,
Here you have one team, two people, one software guy, which DevOps background, also the software development and DevOps background a little bit, and then bring a machine running researcher to it. And then I think they would benefit the best. Yeah. And you talked a little bit about your... journey in terms of the technical side and learning about these tools and also the kind of bringing on more people side as you look forward to the next steps in the roadmap i love it how you publish your roadmap on your site which is really cool as you look forward to the future of that roadmap what do you feel like are the challenges that you're facing right now as someone who is does it have to do with oh now what do i do with all these generative ai stuff and how does that factor into our product or does it have to do with how do we make these models better and support a wider set of use cases or is it is it a combination or something completely different? I would say it's not more about the technical challenge, what you could solve, because technical challenges you can solve with knowledge and a little bit of research. So normally, if it's not physically impossible, you can solve those things with a certain time. This is normally a normal problem. The main problem is what I see is to speed up the development process itself so that the right things are researched, the right things are designed, and the right things are started to develop so that you're bringing more the focus on one topic. And if you have a lot of people in your company, or a lot of, I would say, interfaces,
then you have to bring them one common understanding what you want to achieve, how you want to work, how you want to define requirement. This is, I would say, currently the main challenge when I would say so. And then from the technical challenge, we have to talk to customers, get the feedback, build the stuff as quick as possible and iterate in terms of what the business wants to have. So as we kind of wind up here, and this is pretty typical, we love to get kind of the benefit of your insight, not only for these short-term practicalities, but a little bit of the dreaming. And as you're, as you are kind of, you've come this far in this journey that you've described, and you now have this capability that didn't exist as an entrepreneur, as you're looking at the future, what are the ideas that may be speculative? It doesn't have to be based in the realities of what we have today. Where do you want to go with this? Like, what do you envision building, you know, over the next couple of years? You know, you can pick the horizon, two years, five years, whatever you think. So when you lay in bed at night, you're like, that's the place I'm going.
eventually. What does that look like? Maybe to look a little bit back in the history, how this project started, I think I haven't heard this before. When I was also at Siemens, then I also done my master's thesis about visual question answering, which should solve the task that we are doing now in separate tasks as an end-to-end task. So what my dream is, is to take the now available technology, large language models, including the visual part and the natural part, combine everything so that we can teach or bring every kind of data inside the model. What does this mean to every kind of data?
So this means we get, for example, manuals for software, which says, hey, please click on this button. So then the person coming to you and say, hey, please create my account and this one. You give this a manual and then it's completely automatically without any kinds of learning because it's learned how to interact with your operating system. This is one thing where we want to go to bring nowadays technology in to make it accessible for the users. And also that everyone, really everyone can use it. Also your grandma.
That's great. I am excited to see some of those things come down the line. And I think one of the things I've enjoyed about this conversation is that you brought a lot of the sort of positive side of automation that is really, really so helpful to technical people, but also other people that are doing these tasks that they really actually don't want to do or can't scale to a certain point, right? So I think it's really awesome. And yeah, I'm looking forward to seeing your future work with Ask UI. And thank you so much for joining the podcast. Really appreciate it, Dominic. Thank you for having me.
Thank you for listening to Practical AI. Your next step is to subscribe now, if you haven't already. And if you're a longtime listener of the show, help us reach more people by sharing Practical AI with your friends and colleagues. Thanks once again to Fastly and Fly for partnering with us to bring you all Change Talk podcasts. Check out what they're up to at Fastly.com and Fly.io. And to our Beat Freakin' Residence Breakmaster Cylinder for continuously cranking out the best beats in the biz. That's all for now. We'll talk to you again next time.