Securing the GenAI Ecosystem: Risks and Defenses

The sources collectively examine the significant security and privacy risks introduced by the rapid adoption of generative AI systems, ranging from LLM vulnerabilities to flaws in AI-generated code. One paper addresses general GenAI threats, highlighting that the open-ended nature of these models makes them highly susceptible to prompt injection attacks and potential abuse for automating cyber-attacks like sophisticated phishing. A separate framework specifically reviews the challenges inherent in Retrieval Augmented Generation (RAG) pipelines, which expand the attack surface and raise concerns regarding sensitive data leakage through processes like data retrieval and embedding. Additionally, a report focusing on AI code generation reveals that 45% of tested LLM-written code failed security checks, emphasizing that model sophistication does not guarantee safe code output. The final source proposes a non-adversarial, optimal transport-based method, DP-Sinkhorn, designed to generate data while ensuring differential privacy without the training instabilities common to Generative Adversarial Networks (GANs). Across all sources, various mitigation strategies are proposed, including technical methods like rejection sampling and safety fine-tuning for models, alongside administrative controls such as stringent access limitation and continuous system monitoring.