The AI Security Podcast

If you’re trying to break into AI security, it can feel confusing — do you need to be a machine learning expert, a cybersecurity professional, or both? In this episode, we break down practical tips for getting hired in AI security, from the skills that actually matter to the types of projects and experience that can help you stand out. We discuss how to build relevant expertise in areas like adversarial machine learning, AI risk, and model security, as well as how to position yourself for roles in startups, research labs, and large tech companies. Whether you’re coming from a cybersecurity, data science, or general tech background, this episode will give you actionable advice on how to start building a career in one of the fastest-growing areas of technology. 🚀

Want to give a great conference talk (and not bore everyone to death)? In this episode, I share practical tips for giving a strong conference talk — from structuring your idea to actually delivering it on stage. #PublicSpeaking #Conferences #CFP #TechTalks #Cybersecurity #AI

In this episode, we dig into model weight security — what it means, why it’s emerging as a critical issue in AI security, and whether the framing in the recent RAND report on securing AI model weights actually helps defenders and policymakers.We discuss the RAND report Securing AI Model Weights: Preventing Theft and Misuse of Frontier Models — exploring its core findings, including how model weights (the learnable parameters that encode what a model “knows”) are becoming high-value targets and the kinds of attack vectors that threat actors might use to steal or misuse them.#ai #aisecurity #cybersecurity 👉 Read the full RAND report here:https://www.rand.org/pubs/research_reports/RRA2849-1.html

In this episode, we dig into Model Context Protocol (MCP) and agent-to-agent (A2A) communication — what they are, why they matter, and where the real risks start to emerge.We cover:- What MCP actually enables beyond “tool calling”- How A2A changes the threat model for AI systems- Where trust boundaries break down when agents talk to each other- Why existing security assumptions don’t hold in agentic systems- What practitioners should be thinking about now (before this ships everywhere)This one’s for anyone working on AI systems, security, or governance who wants to understand what’s coming before it becomes a headline incident.As always: curious to hear your takes — especially where you think the biggest risks (or overblown fears) really are.

As promised, I’m back with Tania for a deep dive into the wild world of agentic AI security — how modern AI agents break, misbehave, or get exploited, and what real case studies are teaching us. We’re unpacking insights from the Taxonomy of Failure Modes in Agentic AI Systems, the core paper behind today’s discussion, and exploring what these failures look like in practice.We also break down three great resources shaping the conversation right now:Microsoft’s Taxonomy of Failure Modes in Agentic AI Systems — a super clear breakdown of how agent failures emerge across planning, decision-making, and action loops: https://cdn-dynmedia-1.microsoft.com/is/content/microsoftcorp/microsoft/final/en-us/microsoft-brand/documents/Taxonomy-of-Failure-Mode-in-Agentic-AI-Systems-Whitepaper.pdfOWASP’s Agentic AI Threats & Mitigations — a practical, security-team-friendly guide to common attack paths and how to defend against them: https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/Unit 42’s Agentic AI Threats report — real-world examples of adversarial prompting, privilege escalation, and chain-of-trust issues showing up in deployed systems: https://unit42.paloaltonetworks.com/agentic-ai-threats/Join us as we translate the research, sift through what’s real vs. hype, and talk about what teams should be preparing for next 🚨🛡️.

A quick end-of-year message to say thanks. Thanks for being part of the channel this year — whether you’ve been watching quietly, sharing, or arguing with me in the comments. I really appreciate it.I hope you have a good Christmas and holiday period, whatever that looks like for you. Take a break if you can. See you in 2026.

In this episode, I’m joined by Bandana Kaur — a cybersecurity researcher, speaker, and all-round superstar who somehow managed to do in her teens what most people are still figuring out in their thirties. 🤔Bandana is just 18 years old, entirely self-taught in cybersecurity, already working in the field, and recently gave three talks at Black Hat. Yes, three! 😱We talk about how she taught herself cybersecurity as a teenager, how she broke into the industry without a traditional pathway, and what it’s actually like being young (and very competent) in a field that still struggles with gatekeeping. Bandana shares what she focused on while learning, how she approached opportunities like conference speaking, and what she thinks matters most for people trying to get into security today.This conversation is part career advice, and part reminder that you don’t need permission — or a perfectly linear path — to do meaningful work in cybersecurity.Follow Bandana: @hackwithher

Remember that time I invited myself over to Greg's place with my camera? This is part 2 from that great conversation. I'm curious to hear whether you've heard a lot about EA? It's something really big in the AI world but I'm conscious a lot of people outside the bubble haven't heard of it. Let me know in the comments! Check out Greg's work here: https://www.goodancestors.org.au/MIT AI Risk Repository: https://airisk.mit.edu/The Life You Can Save (book): https://www.thelifeyoucansave.org/book/80,000 hours: https://80000hours.org/Learn more about AI capability and impacts: https://bluedot.org/

This week I invited myself over to Greg Sadler's place, the CEO of Good Ancestors, about AI safety. I brought sushi but I didn't have lunch so I ate most of it, and then I almost made him late for his next meeting. We specifically chat through AI capabilities, his work in policy, and building a not-profit. Greg is the kind of person who is so smart and cool that I feel like an absolute dummy interviewing him - so I know you're all going to like this episode. Stay tuned for part 2 where we dive into effective altruism and its intersection with AI!Check out Greg's work here: https://www.goodancestors.org.au/MIT AI Risk Repository: https://airisk.mit.edu/The Life You Can Save (book): https://www.thelifeyoucansave.org/book/80,000 hours: https://80000hours.org/Learn more about AI capability and impacts: https://bluedot.org/

Hi! 👋 In this episode, we’re diving into the US AI Action Plan — the White House’s new roadmap for how America plans to lead in AI.. and beat China.We’ll look at what’s inside the plan, what it really means for AI security and regulation, and whether it’s more of a policy promise… or a political one.📄 You can read the full plan here:https://www.whitehouse.gov/wp-content/uploads/2025/07/Americas-AI-Action-Plan.pdfLet me know what you think — is this the kind of leadership AI needs, or a dangerous framing of AI capability?

Welcome back! 👋After taking a little break to reset and redesign everything behind the scenes, I’m back — and consolidating all my content. This episode is part of both The AI Security Podcast (on Spotify and Apple Podcasts) and my YouTube channel, HarrietHacks — so whether you prefer to listen or watch, you’ll get the same great conversations (and bad jokes) across both platforms.From now on, I’ll be posting at least fortnightly (with the occasional bonus episode when something big happens… like when I announced the book!).I’ve been in a few conversations lately where people have tried to convince me that AI Security is just Application Security in disguise. Naturally, I disagree. 🤷‍♀️ So in this episode, we dive into AI Security vs Application Security — how they overlap, where they diverge, and why securing AI systems demands new thinking beyond traditional AppSec.💌 Sign up for the newsletter: http://eepurl.com/i7RgRM📘 Pre-order The AI Security Handbook: [link coming soon]🎥 Watch this episode and more on YouTube: https://www.youtube.com/@HarrietHacks🔗 Useful LinksSQL Injection Examples (W3Schools): https://www.w3schools.com/sql/sql_injection.aspApplication Security Blog (Medium): https://medium.com/@pixelprecisionengineering1/application-security-appsec-in-cybersecurity-855ad9ce5e5eEcholeak Zero-Click Copilot Exploit (Dark Reading): https://www.darkreading.com/application-security/researchers-detail-zero-click-copilot-exploit-echoleakTraditional AppSec vs AI Security (Pillar Security): https://www.pillar.security/blog/traditional-appsec-vs-ai-security-addressing-modern-risks

For a while we've been wanting to talk about Agentic AI Security.. the thing is that we could spend multiple episodes talking about it! So we decided to do just that. This is part 1 - a primer - where we talk about exactly what AI agents are and why we may need to consider their security a bit differently. Stay tuned for the rest of the series!

Six months ago Tania and I made an episode about the interim report for our AI Security Likelihood Project.. and it is finally time to discuss the final report! You'll see it live at this link shortly: https://www.aisecurityfundamentals.com/The premise was simple: are AI security incidents happening in the wild? What can we learn about future incidents from these historic ones? We answer some of these questions.

In this episode, Tania and I discuss the debate around closed or open model weights. What do you think?The RAND report we mention: https://www.rand.org/pubs/research_reports/RRA2849-1.html

In this episode, Tania and I talk through some creative examples of prompt injection/engineering we've seen in the wild.. think prompts hidden in papers, red-teaming and web-scraping.Your Brain on ChatGPT: https://arxiv.org/pdf/2506.08872Paper with hidden text (p. 12):  https://arxiv.org/abs/2502.19918v2Interesting overview: https://www.theregister.com/2025/07/07/scholars_try_to_fool_llm_reviewers/Echoleak blog post: https://www.aim.security/lp/aim-labs-echoleak-m365

This week we discussed multiple AI vulnerabilities, including Echolink in M365 Copilot, Agent Smith in Langchain, and a SQL injection flaw in Llama Index, all of which have been patched. We also covered a data exposure bug in Asana's MCP server and OWASP's project to create an AI vulnerability scoring system, while also outlining Google's defense layers for Gemini, Thomas Roccia's Proximity tool for MCP server security, news regarding AI and legal/security concerns, and research on AI hacking AI, prompt compression, multi-agent security protocols, and the security of reasoning models versus LLMs.

I'm back from holiday, and this week Tania and I talk about a project she completed as part of the ARENA AI safety curriculum to replicate the findings of evaluations on frontier AI capabilities.Link to reasoning paper: https://arxiv.org/abs/2502.09696Link to the Inspect dashboard: https://inspect-evals-dashboard.streamlit.app/ARENA AI Safety course: https://www.arena.education/

This week we try a new condensed format for the AI security digest! we covered critical CVEs, including vulnerabilities in AWS MCP, Llama Index, GitHub MCP integration, and tool poisoning attacks. We also reported on malware campaigns using spoofed AI installers, a supply chain attack via fake PyTorch models, and the AI-guided discovery of a Linux kernel vulnerability by Sean Healin using OpenAI's 03 model. We addressed OpenAI's actions against malicious use of their models, Reddit's lawsuit against Anthropic for data scraping, the creation of an AI model for reconstructing 3D faces from DNA by Chinese researchers, a zero-trust framework for AI agent identity management proposed by the Cloud Security Alliance, research on an agent-based red teaming framework, the impact of context length on LLM vulnerability, and CSIRO's technique for improving deep fake detection. We also highlighted the vulnerablemcp.info project and the ongoing evolution of AI security best practices.Sign up to get the digest in your inbox: http://eepurl.com/i7RgRM

Sign up to receive in your inbox: http://eepurl.com/i7RgRMTania Sadhani and Miranda R discussed various AI security topics, including critical CVEs affecting platforms like ChatGPT and Hugging Face, the potential for SharePoint Copilot in internal reconnaissance, and malicious npm packages targeting Cursor developers. They also covered the OASP Gen AI security initiative's Agent Name Service (ANS), the proposed AI.txt for controlling AI agent interactions, and Unit 42's framework for agentic AI attacks. Furthermore, Miranda highlighted security guidance from international agencies, Anthropic triggering ASL 3 for Claude Opus 4, Microsoft's AI red teaming playground, a significant data leak from an AI vendor, and the Israeli police's use of AI-hallucinated laws.

Ever wondered how security vulnerabilities are found in AI? Join us as we chat with Aditya, a Vulnerability Researcher at Mileva Security Labs!

Sign up to receive in your inbox: http://eepurl.com/i7RgRMThis week we note regular CVEs in AI libraries such as Nvidia TensorFlow and PyTorch. We discuss a novel prompt injection technique called "policy puppetry", along with malware dispersal through fake AI video generators and Meta's release of an open-source AI security tool set including Llama Firewall. We also covered Israel's experimental use of AI in warfare, Russia's AI-enabled drones in Ukraine, China's crackdown on AI misuse, Dreadnode's research on AI in red teaming, geolocation doxing via multimodal LLMs, safety research on autonomous vehicle attacks targeting inference time, Config Scan for analyzing malicious configurations on Hugging Face, Spotlight as a physical solution against deepfakes, and Reply Bench for benchmarking autonomous replication of LLM agents.

This week I'm joined by my friend Alberto, he has an incredible storied career - from data science, insurance, AI risk, advising Tesla.. check out his book here! https://www.amazon.com.au/Ethics-I-Facts-Fictions-Forecasts/dp/1636763650

We talk about Stanford Human-Centred AI's latest AI Index report, check it out here: https://hai.stanford.edu/ai-index/2025-ai-index-report

Did you know we have a fortnightly threat intel newsletter? We decided there was so much good research in there we have to talk about it here! We're joined by threat intel lead Miranda for this fortnight's biggest AI security news, coming out in this week's digest! http://eepurl.com/i7RgRM

My friend Shain joins me on the podcast to talk about his work with the OWASP MLSec Top 10 list and organisational guidance, as well as how he got here!For info about the list and how to contribute, check out the link: https://owasp.org/www-project-machine-learning-security-top-10/ 

This week we talk about AI red teaming.. I can't quite believe we haven't talked about it already! We cover the origins of red teaming in the military, how red teaming is done in cyber security, and the fundamentals of AI red teaming. Resources:https://academy.hackthebox.com/course/preview/introduction-to-red-teaming-aihttps://tryhackme.com/path/outline/redteaming

I didn't manage to get my act into gear to record a fresh episode between getting back from the UK and heading to New Zealand so I've pulled one from the vault.. it's a good one though! A year ago I was interviewed by Microsoft for their AI security series. Massive thanks to Microsoft for having me and Sarah Young for excellent interviewing. Check out the rest of the series, they have some really cool people on. This is also a good episode for anyone new to AI security, it covers most of the bases.Link here: https://www.youtube.com/watch?v=sPQaJVnBSRQ&pp=ygUYaGFycmlldCBmYXJsb3cgbWljcm9zb2Z0

I didn't want to do a solo episode so instead I enlist free content support by speaking to the vendors at AI UK, run by the Alan Turing Institute, in London! Thanks very much to those people, links below here:Datambit: https://datambit.com/RAISE: https://raise-project.uk/Contact us at [email protected]

We're in Manchester but we can't talk about it much.. so instead we reflect on some of the latest digest incidents (and how we wish the hot tub in our airbnb had water in it).Sign up to the digest here: http://eepurl.com/i7RgRM

In this episode we discuss the AI lifecycle.. we talk through important machine learning stages (training, inference etc), how they apply at the systems levels (agentic systems, APIs etc) and and example in context (facial recognition at airports but please forgive us we are not airport experts).We refer to useful content from MITRE's resources: https://atlas.mitre.org/resources/ai-security-101Feel free to compliment us at [email protected].

In this episode I interview James Webb, host of the Cyber Expanse. We discuss careers at the intersection of AI and cyber security, how to get into the security workforce, and what it's like being 'influencers' (content creators) in the security space.This episode also has a video equivalent at Harriet Hacks on YouTube, and will have a companion episode on James' YouTube channel as well.

Tania and I went to Melbourne to attend the Society of Risk Analysis Annual General Meeting. What a great group of people.. we learnt so much about risk analysis from these awesome people. We all came from totally different disciplines - climate, biosecurity, technology - but everyone working in risk has the same challenges!They were also a great group of people to go out for cocktails with (we had a few)[email protected]

Today I'm joined by Miranda, our threat intel lead at Mileva, to discuss how threat intel and vulnerability research works in the world of AI security. We also recently released a threat digest, sign up here to understand what AI security incidents and research are happening in the wild: http://eepurl.com/[email protected]

This is part 2 of our discussion on the TikTok ban and its AI security implications. We have not yet recorded a podcast about DeepSeek (look, we're very busy) but you'll find that a lot of the discussion about Chinese national security laws is relevant and echoes those same conversations.Links discussed:The Infographic: https://newsroom.tiktok.com/en-au/comparative-study-of-social-media-data-collection-practices-au?utm_source=chatgpt.comBing vs Google: https://searchengineland.com/google-bing-is-cheating-copying-our-search-results-62914Reach out at [email protected].

We chat about the TikTok ban and its AI security implications. We got so into this topic we had to split it in two parts!The AI security creative comms comp: https://milev.ai/reports/ai-security-communication-competition-2000-in-prizes/Sign up to our new threat digest:  http://eepurl.com/i7RgRM Get in touch with us at [email protected].

Harriet turned 30 recently and decided to try this trend - but AI security edition! We talk through 30 lessons - AI, security and life.Contact us at [email protected]

Welcome to the AI security podcast! We decided to rebrand and claim this space since no-one else has taken it! We're looking forward to being your go-to podcast for AI security news, research and casual yaps.In this episode Tania and I each discuss our top three trends in AI security and AI for security as we enter 2025.Links discussed:Cyber risk management podcast: https://open.spotify.com/show/43k7780x6wSvKCq75StIsMAbout the number 2025: https://mindyourdecisions.com/blog/2025/01/02/5-fun-facts-about-2025-you-probably-didnt-know/Our AI security research: https://aisecurityfundamentals.com/Reach out to Mileva at [email protected]

We continue our discussion on key trends in GRC and its intersection with AI!Annie-Mei is also about to launch her own podcast and I highly recommend you check it out if you want more no-nonsense content and advice about the cyber space.

This week we released the interim report and subset of the AI incident database for the AI Security Likelihood research Tania has been working on! We chat through the motivations for the project, our initial insights and takes, and where we hope to take it next. Now we want your input! Check it out at aisecurityfundamentals.com and let us know if you have insights on the risk modelling, AI incidents, or access to data/collaborations that could make this project even better.(If you don't want to hear us talk about our Christmas party skip to around 8 minutes in).Big thanks to Foresight for funding this research, and our collaborators at the Australian National University and University of New South Wales.We mention this book, How to Measure Anything in Cyber Security and recommend giving it a read: https://www.oreilly.com/library/view/how-to-measure/9781119892304/cover.xhtml

Today we chat to my good friend Annie-Mei about GRC (governance, risk and compliance). She is a senior GRC consultant with a background in journalism! We discuss her career journey and an overview of what on earth GRC is.

This time we sit down with Howjer Gu, Founder of Planoverse, a start-up in our stream within the UNSW 10x accelerator! Fun fact - Howjer and I used to work together at Deloitte back in 2016 and couldn't believe it when we ran into each other during the application process. Howjer has a background in data analytics and recently founded Planoverse, which is like a digital twin for grocery stores. Learn about his start-up journey and the importance of data.Planoverse: https://www.planoverse.com.au/

2024 has seen a lot of change for Mileva.. and we often get people asking about what we're doing and we got here, so this time Tania interviews Harriet about starting Mileva! We know lots of our audience are curious about starting a business and the different trade-offs that entails (quitting your full time job, looking for funding, services vs products, when is the right time) so we address some of these questions. We also talk about our experience in the UNSW 10x accelerator (hint: I'll be interviewing some of these founders in upcoming episodes!)Oh also Harriet somehow ended up in Vogue!

In this episode Tania and I chat through how we learnt about AI security and adversarial machine learning, and how you can too! Some resources we mention:- Crash Course AI https://thecrashcourse.com/topic/ai/- 3 Blue 1 Brown https://www.youtube.com/c/3blue1brown- Kaggle https://www.kaggle.com/learn/intro-to-machine-learning- Hands-on ML textbook (more for 'classic' machine learning - still really important to understand 'modern' models) https://www.booktopia.com.au/hands-on-machine-learning-with-scikit-learn-keras-and-tensorflow-3e-aurelien-geron/book/9781098125974.html?gad_source=1&gclid=CjwKCAiA9dGqBhAqEiwAmRpTC3W7SbJJOrI8fEi-19OxvyNLElA8Nk7iKH6nE4KDm1iLlySKgb114hoCcG8QAvD_BwE- The paper that started it all (or at least a lot of it) https://arxiv.org/abs/1312.6199- Adversarial Robustness Toolbox https://adversarial-robustness-toolbox.org/- Cleverhans https://cleverhans-lab.github.io/- Darknet diaries (please someone introduce me to Jack) https://darknetdiaries.com/- Rob Miles https://www.youtube.com/@RobertMilesAI- Import AI https://jack-clark.net/- AI Daily Brief https://www.youtube.com/channel/UCKelCK4ZaO6HeEI1KQjqzWA- Less Wrong https://www.lesswrong.com/- Unsupervised learning https://danielmiessler.com/- OWASP ML Top 10: https://owasp.org/www-project-machine-learning-security-top-10/- ML Safety https://course.mlsafety.org/about- Anatomy of an AI system: https://anatomyof.ai/

People kept asking us about the recent AI laws in California and why some were passed but one was vetoed, so we decided we'd finally better do a podcast episode on it! Disclaimers: we're not lawyers (this will be immediately obvious) but I hope this is enlightening to other AI/tech folk. We certainly loved researching this one, and we encourage everyone who's interested to join the discussion!In this episode we shout out Civ.ai, a non-profit we met at Berkeley when we were in the US. They create demos to show policymakers just how impactful and risky AI can be, so please do get in touch with them if you're interested! https://civai.org/

In this bonus episode we're still with Miranda and we're talking about some of the trends we're seeing around LLMs (language models) being used by adversaries to augment attacks. This could be another episode, but we have a bite sized discussion for you here!

This time we are joined by another vital member of the Mileva team, Miranda R! Unlike Tania and myself, who are data scientists who got into security, Miranda actually started as a cyber security analyst and made her way into AI. She chats about her career journey and insights on the cyber/AI intersection.

When we were in the US we rode a Waymo.. and now we chat about it here! We definitely want to chat about job displacement in another episode, but for now - here is our excuse to tax deduct the Waymo ride!

We're finally back after a bit of a break (travel is tiring). Over the last couple of weeks we attended and spoke at BSides Singapore and BSides Canberra! In this episode we cover the research discussed in our talks and the highlights of BSides in general (you know we're massive BSides fans).  

We're exhausted but we're still here! And to our knowledge we have not been hacked... in this episode we reflect on our AI and cyber security conversations from BSides Las Vegas, the AI Security Forum, DEF CON 32 and our week in the Bay Area. We also want to shout out these organisations:Far Labs for hosting us: https://far.ai/labs/CivAI, this awesome new company we met who are creating AI demos: https://far.ai/labs/Dreadnode for their awesome LLM red teaming comp at the AI Village: https://www.dreadnode.io/Rob Miles for his AI safety youtube channel: https://www.youtube.com/c/RobertMilesAI

Are you excited to attend hacker summer camp (BSides LV, Black Hat and/or DEF CON) in 2024? So are we!! Harriet shares some tips and tricks based on her experience last year. If you'll be in Vegas this year and see myself and Tania please say hi!Hackertracker: https://hackertracker.app/conferences/HSCPARTIES2024/schedule/