Applied GAI in Security

Sign in Subscribe

Latest

Last Week in GAI Security Research - 03/03/25

Last Week in GAI Security Research - 03/03/25

Highlights from Last Week * 🖊 RapidPen: Fully Automated IP-to-Shell Penetration Testing with LLM-based Agents * ⭕ Worse than Zero-shot? A Fact-Checking Dataset for Evaluating the Robustness of RAG Against Misleading Retrievals * 🛡 Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents * 🚧 LongSafety: Evaluating Long-Context Safety of Large Language Models * 🕸 Why Are

Last Week in GAI Security Research - 02/24/25

Last Week in GAI Security Research - 02/24/25

Highlights from Last Week * 🤔 What are Models Thinking about? Understanding Large Language Model Hallucinations "Psychology" through Model Inner State Analysis * 🤖 Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks * 🔐 Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions * 🤑 Fraud-R1 : A Multi-Round Benchmark

Last Week in GAI Security Research - 02/17/25

Last Week in GAI Security Research - 02/17/25

Highlights from Last Week * 💦 RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage * 💰 Auditing Prompt Caching in Language Model APIs * 🤖 Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks * 🔊 Improving Acoustic Side-Channel Attacks on Keyboards Using Transformers and Large Language Models * 🧠 On the Emergence of Thinking in

Last Week in GAI Security Research - 02/10/25

Last Week in GAI Security Research - 02/10/25

Highlights from Last Week * ☣️ Exploring the Security Threats of Knowledge Base Poisoning in Retrieval-Augmented Code Generation * 🥡 LLMSecConfig: An LLM-Based Approach for Fixing Software Container Misconfigurations * 🤓 Can LLMs Hack Enterprise Networks? Autonomous Assumed Breach Penetration-Testing Active Directory Networks * 📐 Rule-ATT&CK Mapper (RAM): Mapping SIEM Rules to TTPs Using LLMs * 🦥 OverThink:

Last Week in GAI Security Research - 02/03/25

Last Week in GAI Security Research - 02/03/25

Highlights from Last Week * 🎩 Illusions of Relevance: Using Content Injection Attacks to Deceive Retrievers, Rerankers, and LLM Judges * 🌐 On the Feasibility of Using LLMs to Execute Multistage Network Attacks * 🦄 Comparing Human and LLM Generated Code: The Jury is Still Out! * 🏛 Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours

Beans to Bots: Hacking My Coffee Machine with AI

Beans to Bots: Hacking My Coffee Machine with AI

This post is not sponsored in any form. Speciality coffee has been in my life for over a decade. What used to be near-impossible to find is now commonplace, even in chains like Starbucks. Over the years, I've roasted coffee, formed a business around it and quested for

Last Week in GAI Security Research - 01/27/25

Last Week in GAI Security Research - 01/27/25

Highlights from Last Week * 🧑‍🔧️ An Empirically-grounded tool for Automatic Prompt Linting and Repair: A Case Study on Bias, Vulnerability, and Optimization in Developer Prompts * 🎶 Tune In, Act Up: Exploring the Impact of Audio Modality-Specific Edits on Large Audio Language Models in Jailbreak * 🦺 Can Safety Fine-Tuning Be More Principled? Lessons Learned

Last Week in GAI Security Research - 01/20/25

Last Week in GAI Security Research - 01/20/25

Highlights from Last Week * 😡 AfriHate: A Multilingual Collection of Hate Speech and Abusive Language Datasets for African Languages * 🧙 Gandalf the Red: Adaptive Security for LLMs * 🧭 I Can Find You in Seconds! Leveraging Large Language Models for Code Authorship Attribution * 🗳 Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards Partner Content Pillar

Running in Circles: Security Insights from a Treadmill

Running in Circles: Security Insights from a Treadmill

I run ultramarathons, so I run often and sometimes end up indoors when conditions aren’t ideal or I need to juggle family needs. When running inside, I use a Nordictrack X22i, which is a wonderful piece of equipment but is limited to the built-in iFit application and nothing more;

Last Week in GAI Security Research - 01/13/25

Last Week in GAI Security Research - 01/13/25

Highlights from Last Week * 📝 A survey of textual cyber abuse detection using cutting-edge language models and large language models * 🪲 Exploring Large Language Models for Semantic Analysis and Categorization of Android Malware * 📦 FlipedRAG: Black-Box Opinion Manipulation Attacks to Retrieval-Augmented Generation of Large Language Models * 🌊 RAG-WM: An Efficient Black-Box Watermarking Approach for

Last Week in GAI Security Research - 01/06/25

Last Week in GAI Security Research - 01/06/25

Highlights from Last Week * 🌨 Toward Intelligent and Secure Cloud: Large Language Model Empowered Proactive Defense * ♠️ SPADE: Enhancing Adaptive Cyber Deception Strategies with Generative AI and Structured Prompt Engineering * 🎼 On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks against LLMs * 🔒 TrustRAG: Enhancing Robustness and Trustworthiness in RAG * 😎 SafeSynthDP: Leveraging

Last Week in GAI Security Research - 12/23/24

Last Week in GAI Security Research - 12/23/24

Highlights from Last Week * 📰 Evaluation of LLM Vulnerabilities to Being Misused for Personalized Disinformation Generation * 📏 Trust Calibration in IDEs: Paving the Way for Widespread Adoption of AI Refactoring * 🛠️ Can LLMs Obfuscate Code? A Systematic Analysis of Large Language Models into Assembly Code Obfuscation * 🤖 SpearBot: Leveraging Large Language Models in a

Last Week in GAI Security Research - 12/16/24

Last Week in GAI Security Research - 12/16/24

Highlights from Last Week * 🙅 Trust No AI: Prompt Injection Along The CIA Security Triad * 🔁 Enhancing Adversarial Resistance in LLMs with Recursion * 🕰 Adversarial Vulnerabilities in Large Language Models for Time Series Forecasting * 🐇 MAGIC: Mastering Physical Adversarial Generation in Context through Collaborative LLM Agents * 🎭 From Allies to Adversaries: Manipulating LLM Tool-Calling through

Last Week in GAI Security Research - 12/09/24

Last Week in GAI Security Research - 12/09/24

Highlights from Last Week * 🚸 Trust & Safety of LLMs and LLMs in Trust & Safety * 🤓 Hacking CTFs with Plain Agents * 👻 Unleashing GHOST: An LLM-Powered Framework for Automated Hardware Trojan Design * 📇 OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generation * 📊 AI Benchmarks and Datasets for LLM Evaluation Partner