Latest

In recent months, I've observed "agents" and "agentic" terms supplanting generic concepts like "language models" and "generative AI". Overall, this is a net-positive for the industry as it acknowledges that single-turn, stateless language models alone are not enough to solve

Highlights from Last Week * 💉SQL Injection Jailbreak: a structural disaster of large language models * ❇ Fixing Security Vulnerabilities with AI in OSS-Fuzz * 🏭 LLMs for Domain Generation Algorithm Detection * 🧫 AutoPT: How Far Are We from the End2End Automated Web Penetration Testing?  * 🎭 Targeted Manipulation and Deception Emerge when Optimizing LLMs for User Feedback

Highlights from Last Week * 🤖 Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattacks * 🔊 Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models * 🌱 Sorting Out the Bad Seeds: Automatic Classification of Cryptocurrency Abuse Reports * 🦟 Fine-Tuning LLMs for Code Mutation: A New Era of Cyber Threats

Every week for the past 7 months, I've found myself executing a python script I call "deepthought" to summarize research papers that talk about applying generative AI to security problems. This process is what powers the "Last Week in GAI" digests I send out

Highlights from Last Week * 🛡 Countering Autonomous Cyber Threats * 🚪 Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In  * 📱 MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control  * 🧾 ProveRAG: Provenance-Driven Vulnerability Analysis with Automated Retrieval-Augmented LLMs * 🦾 Towards Automated Penetration Testing: Introducing LLM Benchmark, Analysis, and Improvements Partner Content Pillar Security is

Highlights from Last Week * 🤔 Cognitive Overload Attack:Prompt Injection for Long Context * 💂‍♂ On Calibration of LLM-based Guard Models for Reliable Content Moderation  * 🕷 When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs * 🧌 Can LLMs be Scammed? A Baseline Measurement Study * 🌱 Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue

Highlights from Last Week * 🪱 Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems * ⚠️ AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents * 🪝 APOLLO: A GPT-based tool to detect phishing emails and generate explanations that warn users * 🎶 Coevolving with the Other You: Fine-Tuning LLM with Sequential Cooperative Multi-Agent Reinforcement Learning * 🐞 RealVul: Can

Highlights from Last Week * 💂‍♂ MoJE: Mixture of Jailbreak Experts, Naive Tabular Classifiers as Guard for Prompt Attacks  * 📚 Unit Test Generation for Vulnerability Exploitation in Java Third-Party Libraries * 🔗 LLMs are One-Shot URL Classifiers and Explainers * 🛻 Enhancing LLM-based Autonomous Driving Agents to Mitigate Perception Attacks * ⛔ Holistic Automated Red Teaming for Large Language