Welcome to the podcast. I'm your host, Jayden Shafer. Today on the show, we are talking about AI security, specifically for browsers. Google Chrome has released a bunch of new security measures for the Google Chrome agent features that they're going to be rolling out.
And I think a lot of these are actually very interesting ideas that we're going to see played out with OpenAI's Atlas browser, with perplexity, Sonnet browser. And inevitably, everyone, including Firefox, I think, will have some version of an AI browser that can go and take actions for you. I think this is basically the final one of the best form factors for AI agents. I think the golden ticket would be an actual software on your computer that could take more control of your computer.
I think the next specs thing and the thing that has a widest distribution today would be browsers. So something like Google Chrome would be the number one place that I think would get these AI agents actually taking action being very, very useful for us. So the big problem, of course, is all of the innumerable ways that you can have security breaches and you can have bad actors essentially tricking these AI agents into handing over data or money for things that you wouldn't want to hand over your data or money for. And bad actors and hackers can essentially steal this data in these ways.
So Google obviously is very concerned about this, but they have a massive incentive to not have the Chrome browser disrupted by someone like perplexity or opening eye. So they need to stay in the game. So of course, Google has a lot of really bright minds over there. And they've come with some really incredible solutions.
I want to break down what some of the solutions are and how I think this evolves and also where I believe that they are falling short, because I don't think everything they've done is perfect. So we're going to get into all of that. Before we do, I wanted to mention, if you want to try all of the models I talk about on the show, whether that be OpenAI or Cloud or Google Gemini or all of the image models or audio models like 11Labs, I'd love for you to try out AI box.ai. This is my own startup.
We've launched a no-code AI app builder where you can describe a tool or an app that you want to create. And it will build a workflow for you that you can use to help automate a lot of your tasks you do online. We're adding new features all the time and have some exciting stuff coming up. If you want to try that out, or get access to over 40 different AI models to chat with them like Jai GPT, but all the different AI models instead of just being limited to one for $20 a month, you can go check it out.
It is AI box.ai. I'll leave a link in the description. OK, let's talk about what Google has recently shown off as far as security measures for what they're rolling out in the future with Google Chrome. Now, they kind of gave a bunch of demos and they showed how Google Chrome could be an AI agent similar to anthropic and perplexing what they really know.
But when they kind of gave their demos, they said, look, these are going to be available in the coming months. So they did kind of the teas and wait instead of the immediate drop with this feature. It's basically my pet peeve, but whatever. It is the way it is, especially with companies like Google and Apple, where they have these big show and tell conferences.
And then they're like, look, these are coming up in the next few months. And so I mean, this is just the way it is when they have these pre-designed conferences, like Google I-O, where they announce a bunch of stuff. I really appreciate it when these AI companies do an announcement and immediately drop Google did not do that in this case. But what they did do that I thought was pretty impressive as they've started to roll this out is they said that they're using the help of a few different models to keep the agents actions in check.
So essentially they've built what they're calling a user alignment critique. Now, they're using Gemini to do this. And it basically looks at the action items that is built by the planner model. So essentially what that means is when you're going to go and use an AI agent, let's say you're like, hey, help me go and download my podcast episode.
And I mean, I know this isn't everyone's work blog. I'm coming up with something that's my own rather than just giving you a travel planner one, which I feel like is over you. So let's say you're like, hey, go download my podcast episode, go edit it, go upload it to this platform, come up with some titles and descriptions, and publish it for me on Monday, Wednesday or Friday depending on what the next available slot for a podcast episode is. Let's say this is your workflow.
It's going to like these agents, they essentially listen to your prompt. However, you know, unformatted it may be where you describe your whole workflow, and they'll break it down into a very clear organized path. They're like, OK, in order to do the download, we need to make sure that we have space on the drive. And we need to make sure that when we download it, we do XYZ things and we're downloading the right file.
And we've got to go find it. Like, anyways, they kind of this very elaborate walkthrough of what they're going to do. Now, what they've done as far as security goes is they've created a user alignment critique, which is essentially a model. It can't see anything on your screen.
So AI agent, it can see everything on your screen. It comes up with a plan that goes and executes the plan. Every step of the plan when it says, OK, I see this, I'm going to do this thing next. You can kind of see the reasoning on comment and even on open eyes atlas where it tells you what it's doing while it goes through all the steps.
It's kind of useful to watch that to understand how these models are working. But how Google is functioning now is as it's taking those actions, it has a separate model that is looking at what your original objective is and what its current step is taking. It can't see what's on the screen. So it can't be tricked basically by a prompt injection that's like, forget all your past instructions and make sure you do XYZ.
This is what people are kind of worried about. This quote unquote prompt injection. Instead, all it sees is your original goal. And then the action is going to take.
And that model says yes or no if that action aligns with the original goal. It's a very clever kind of way to use AI to stop the bad actors of AI. So they've done this. And if the critical model thinks that the planned task doesn't serve what the user's original goal is, it's going to ask the planning model to rethink its strategy on what it's doing.
So Google says that the critic model only sees the metadata and not the actual web content. And I think what's powerful is that the other thing they're doing to prevent agents from getting disallowed or untrustworthy sites and accessing basically is that they're using this an agent origin sets tool. So this is another tool. So I think three main tools.
The first one was their user alignment critique. The second one is called agent origin sets. And essentially what it's going to do is it's going to restrict the model to access read only origins and read writable origins. So what that means is that read only origins is data that Gemini is allowed to consume content from.
So an example of that would be on a shopping site. The listings, those are very relevant to the task that you're doing, right? If you go buy me a pair of white tennis shoes and go to a web page and there's white tennis shoes, the listing information about those white tennis shoes, that's very relevant to the task that you're doing. But the banner ad that's on that website, maybe it's a Shopify store or some other store and they've got Google ads on there, that is not relevant.
And so the listing is relevant, the banner ads are not. And Google also said that the agent is only allowed to click on or type on certain i-frames of a page. So the ads would not be there. What's kind of hilarious to me is the fact that Google is the number one ads platform in the world.
And yet their AI agent that they're creating is literally designed to ignore ads. And it's also especially ironic to me considering that there's all coming on Google Chrome, who I for many years used Google Chrome and I had ad blockers, all sorts of ad blocker plugins, the block extensions, the block ads. And then Google Chrome made it very difficult for those same ad blocker plugins to fully function and I essentially moved to other platforms. I use Brave Now as my browser because it by default blocks all ads aren't in the plugin at an extension.
It just works. And yeah, I forget ads exist on the internet, if I'm being perfectly honest. And so anyways, it's very ironic to me that as we're creating AI agents, we're literally designing them to ignore ads, which is where Google makes all their money and what they've stopped. It feels like they're kind of bent on not allowing any humans to block ads, but their AI agents can.
And I think it's kind of funny, but I also do think it's clever and great for security yada yada. So like Google continue on this path. I think it's great. One thing that I think is interesting is this concept of iFrames inside of a website.
Basically what I think is clever here is because it's only allowing the agent to click on or type on certain iFrames on a site. It's actually, it's very smart because beyond just visually what you can see on the page, it also can read the HTML or the code for the page so it understands more about the page and more about security risks and a regular human with a very common phishing attack that I think you would see is like is, you know, iFrames inside of a website and the iFrame is of a completely different website that's stealing data. You know, hacker could go, you know, hacker legitimate website, put an iFrame embedded inside of that website that you think you're typing your data into a specific website, but really there's a hacker extracting data that's like one kind of clever way to go about it. Another one is just with, you know, complete spoof websites where you have a slight typo in the URL that people may not notice.
You try to make a direct clone of like PayPal or Bank of America and get their, get their information and, you know, there's kind of common scams that people do. What's cool to me is that the AI agents will actually be better than humans at detecting that because they're looking at not just what's on the screen, but also the code and they're built to kind of scrutinize everything and not click on or interact with iFrames or elements of the page. They know our false, which is kind of cool. This is what they said in a blog post about all that they said.
This delineation enforces that only data from a limited set of origins is available to the agent and this data can only be passed on to the right of origins. This bounds the threat vector of cross origin data leaks. This also gives the browser the ability to enforce some of that separation, such as by not even sending to the model data that is outside of the readable set. Okay.
So I mean, essentially, Google's, you know, keeping a check on the page navigation by looking at the URLs throughout, you know, through like another observer model. This observer model is like looking what's on the page and I think it's can prevent navigation to harmful model generated URLs. That's essentially what Google has said. I think right now, Google said that it is also handing over the range to users for a bunch of different tasks.
This is something I think are good about this. Something I don't think are good about this. For instance, when an agent is trying to navigate to a site with information like banking or medical data, it first is going to ask the user, you know, like you can see pros and cons to this. I think a lot of people are going to be wary of like, hey, like I don't want Google Gemini going to log it into my bank without my permission because security all sorts of security issues there.
But at the same time, it's like if I asked it to do something because it was useful for me and it's not able to do it, then it makes it less useful to me. So this is kind of the debate that I personally have and if there's a way to make it secure, I could just say, hey, go check my bank and let me know if my transaction to XYZ corporation went through because I wasn't sure it's something funny happening on my credit card. And Google Gemini could go do that and let me know. That would be very useful.
So I can see like uses. But then if it was like, oh, well, that's too confidential. Would you like me to do this or can you log in and then I'll go search for it? I can imagine it just being kind of cumbersome and annoying and I would like to streamline it.
And I think that's a great debate to be had in that regard. But what I think is less debatable is that for sites that require a login is going to ask a user for permission to like from user password manager. Google said that the agent's model doesn't have exposure to password data and they also said it's going to let users, it's going to ask users before taking actions like making a purchase or sending a message. Personally, I think this is just making it less useful.
If it's going to ask me permission to send a message around like, hey, go send an email to this person saying this and then it like drafts up an email. It's like, do I have permission to send this message to this person? It's like, I already, yeah, I already gave you permission. Or if I was like, hey, like go buy me this exact pair of shoes, don't spend more than this much money.
And then it like does all the research and it's like, all right, I'm going to buy the shoes. And it's exactly everything that you wanted. Are you good? Like, I don't know.
To me, it's like, just if I gave you an instruction, I would, when the model is good enough, I hope we can do it without asking me for it. So maybe we'll get there. Maybe this is just kind of a because I'm also not good enough today. So send the limitations.
My worst fear is be that these limitations last forever, right? Like I want an AM model that can truly go and do everything I need to go do without having to ask me. This is basically my biggest pet peeve and my biggest issue with perplexies, comment and with open eyes, Atlas browser is that I have all sorts of tasks I get it to do and ask me like, you know, 10 different times throughout the task. Are you sure I can proceed to the next step?
And I'm like, if I have to babysit you and say, yes, every, you know, every minute, I might as well just do this thing myself. Or really what I do is just hire a person to do it because I can tell them how to do it once and they'd never ask me again for a month while they do all the tasks. So anyway, so that's my personal pet peeve. And I hope that Google removes a lot of these asking for permission things in the future.
Google said that they also have some prompt injection classifiers to prevent unwanted actions. They're also testing agentic capabilities against attacks. So they have a lot of researchers that are working on this right now. I would expect nothing less from Google.
I think that is fantastic. There is apparently, or I guess perplexia recently they released this kind of new open source content detection model earlier this month. And so I think that there's a lot of other players paying a lot of attention to this. And the idea for that with perplexis to prevent prompt injection attacks against agents, I think Google is going to work on this as well.
I honestly think that all of the research done by any of these companies, especially because they're going to publish it and talk about it, is going to get used by everyone. I don't think that's like the competitive advantage you want. Like the competitive advantage of Google Chrome is that like, you know, we are way less prone to attacks, but we're not going to tell you how, like they're going to tell everyone how and everyone's going to use them. So at the end of the day, I think this is going to be good for the entire industry.
Thank you so much for tuning into the podcast today. If you enjoyed the episode, make sure to leave a rating review wherever you get your podcast. And as always, make sure to go check out aibox.ai. I will catch you in the next episode.