Latest

Highlights from Last Week * 🚸 Trust & Safety of LLMs and LLMs in Trust & Safety * 🤓 Hacking CTFs with Plain Agents  * 👻 Unleashing GHOST: An LLM-Powered Framework for Automated Hardware Trojan Design * 📇 OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation * 📊 AI Benchmarks and Datasets for LLM Evaluation Partner

Since April, I've been tracking the security startup space, specifically as it relates to agents and agentic workflows. Back then, there were a handful of security companies leaning into the concept of agents and demonstrating real progress. In just over 6 months, I've seen sizable increase

Highlights from Last Week * 📚 EnStack: An Ensemble Stacking Framework of Large Language Models for Enhanced Vulnerability Detection in Source Code  * 👤 Sensitive Content Classification in Social Media: A Holistic Resource and Evaluation * ☣️ Knowledge Database or Poison Base? Detecting RAG Poisoning Attack through LLM Activations * 🏛 Fine-Tuning LLMs with Noisy Data for Political

Highlights from Last Week * 🦹‍♂ RAG-Thief: Scalable Extraction of Private Data from Retrieval-Augmented Generation Applications with Agent-based Attacks * 🐠 Adapting to Cyber Threats: A Phishing Evolution Network (PEN) Framework for Phishing Generation and Analyzing Evolution Patterns using Large Language Models * 🧵 A Flexible Large Language Models Guardrail Development Methodology Applied to Off-Topic Prompt

Highlights from Last Week * 👹 Unmasking the Shadows: Pinpoint the Implementations of Anti-Dynamic Analysis Techniques in Malware Using LLM * 🐑 LLM App Squatting and Cloning * ✅ Trustful LLMs: Customizing and Grounding Text Generation with Knowledge Bases and Dual Decoders * 💔 RedCode: Risky Code Execution and Generation Benchmark for Code Agents  * 🎓 MultiKG: Multi-Source Threat Intelligence

In recent months, I've observed "agents" and "agentic" terms supplanting generic concepts like "language models" and "generative AI". Overall, this is a net-positive for the industry as it acknowledges that single-turn, stateless language models alone are not enough to solve

Highlights from Last Week * 💉SQL Injection Jailbreak: a structural disaster of large language models * ❇ Fixing Security Vulnerabilities with AI in OSS-Fuzz * 🏭 LLMs for Domain Generation Algorithm Detection * 🧫 AutoPT: How Far Are We from the End2End Automated Web Penetration Testing?  * 🎭 Targeted Manipulation and Deception Emerge when Optimizing LLMs for User Feedback

Highlights from Last Week * 🤖 Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattacks * 🔊 Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models * 🌱 Sorting Out the Bad Seeds: Automatic Classification of Cryptocurrency Abuse Reports * 🦟 Fine-Tuning LLMs for Code Mutation: A New Era of Cyber Threats

Highlights from Last Week * 🛡 Countering Autonomous Cyber Threats * 🚪 Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You In  * 📱 MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control  * 🧾 ProveRAG: Provenance-Driven Vulnerability Analysis with Automated Retrieval-Augmented LLMs * 🦾 Towards Automated Penetration Testing: Introducing LLM Benchmark, Analysis, and Improvements Partner Content Pillar Security is

Highlights from Last Week * 🤔 Cognitive Overload Attack:Prompt Injection for Long Context * 💂‍♂ On Calibration of LLM-based Guard Models for Reliable Content Moderation  * 🕷 When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs * 🧌 Can LLMs be Scammed? A Baseline Measurement Study * 🌱 Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue

Highlights from Last Week * 🪱 Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems * ⚠️ AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents * 🪝 APOLLO: A GPT-based tool to detect phishing emails and generate explanations that warn users * 🎶 Coevolving with the Other You: Fine-Tuning LLM with Sequential Cooperative Multi-Agent Reinforcement Learning * 🐞 RealVul: Can