Securing the GenAI Ecosystem: Risks and Defenses

The collected texts collectively explore the burgeoning security and risk landscape of Generative Artificial Intelligence (GAI) systems, outlining both governmental guidance and advanced technical research for mitigating harms. The National Institute of Standards and Technology (NIST) offers a structured risk management framework (AI RMF), identifying numerous societal and security risks unique to GAI, including harmful bias, the generation of abusive content, and confabulation. Concurrent academic research confirms these issues, detailing how GAI introduces new threat vectors like prompt injection and jailbreaking, necessitating the deployment of defenses such as AI firewalls and robust content watermarking. Specific application architectures, like Retrieval Augmented Generation (RAG), introduce unique vulnerabilities related to data leakage and require specialized security policies and controls like access limitation. Furthermore, innovative algorithms like DP-Sinkhorn are being developed to facilitate the stable and differentially private training of generative models, thus addressing fundamental concerns over data security during development.