Last Week in GAI Security Research - 04/22/24

Highlights from Last Week

• 🔇 Advancing the Robustness of Large Language Models through Self-Denoised Smoothing
• 🥷🏼 The Power of Words: Generating PowerShell Attacks from Natural Language
• 🛡️ TransLinkGuard: Safeguarding Transformer Models Against Model Stealing in Edge Deployment 
• 🔍 Uncovering Safety Risks in Open-source LLMs through Concept Activation Vector 
• 📈 Sampling-based Pseudo-Likelihood for Membership Inference Attacks

🔇 Advancing the Robustness of Large Language Models through Self-Denoised Smoothing (http://arxiv.org/pdf/2404.12274v1.pdf)

• Self-denoised smoothing, or SELFDENOISE, significantly improves the robustness of Large Language Models (LLMs), enhancing empirical and certified robustness in adversarial defense, and defending against both downstream task attacks and jailbreak attacks.
• SELFDENOISE achieves up to 19.7% improvement in empirical robustness against adversarial attacks in downstream tasks without sacrificing clean accuracy, illustrating an optimal accuracy-robustness trade-off.
• In defending against jailbreak attacks, SELFDENOISE demonstrates superior defense success rates, outperforming existing methods across different attack scenarios, including both transfer and adaptive attacks.

🥷🏼 The Power of Words: Generating PowerShell Attacks from Natural Language (http://arxiv.org/pdf/2404.12893v1.pdf)

• Fine-tuned models, specifically CodeT5+ and CodeGPT, demonstrate notable improvements and outperform ChatGPT across all metrics in generating offensive PowerShell code.
• Extensive evaluation shows that models without fine-tuning show limited ability in generating PowerShell code, highlighting the significant impact of fine-tuning on model performance.
• Static and execution analysis confirms that fine-tuned models can generate code with high syntax accuracy and closely aligned behavior with intended malicious activities.

🛡️ TransLinkGuard: Safeguarding Transformer Models Against Model Stealing in Edge Deployment (http://arxiv.org/pdf/2404.11121v1.pdf)

• TransLinkGuard fulfills four critical protection properties for edge-deployed transformer models: post-physical-copy protection, request-level authorization, runtime reverse engineering safeguarding, and delivering high security with minimal runtime overhead.
• Through a lightweight authorization module in a secure environment, TransLinkGuard achieves black-box security level protection with negligible overhead, showcasing superior performance compared to existing PTSE approaches.
• Extensive testing demonstrates that TransLinkGuard maintains the original model's accuracy without compromise, ensuring efficient and secure deployment of transformer models on edge devices.

🔍 Uncovering Safety Risks in Open-source LLMs through Concept Activation Vector (http://arxiv.org/pdf/2404.12038v1.pdf)

• Utilizing safety concept activation vectors (SCAVs) achieves an impressive attack success rate (ASR) over 95% on well-aligned LLMs, indicating potential safety risks despite careful alignment efforts.
• The SCAVs exhibit transferability across different open-source LLMs, suggesting a fundamental linkage to LLMs' inherent safety mechanisms.
• A comprehensive evaluation method that includes ASR, GPT-4 rating, and human evaluation confirms the effectiveness of the proposed attack method in generating truly harmful content.

📈 Sampling-based Pseudo-Likelihood for Membership Inference Attacks (http://arxiv.org/pdf/2404.11262v1.pdf)

• SaMIA, a Sampling-based Pseudo-Likelihood method for Membership Inference Attacks, achieves performance on par with likelihood-based methods, even surpassing them for longer texts without requiring access to model likelihoods.
• Incorporating zlib compression with SaMIA (SaMIA*zlib) improves leakage detection across all text lengths, suggesting benefits from compressing the influence of repeated substrings.
• The effectiveness of SaMIA increases with the length of the target text, as demonstrated by progressively distinct distributions of ROUGE-1 scores between leaked and unleaked texts across different text lengths.

Strengthen Your Professional Network

In the ever-evolving landscape of cybersecurity, knowledge is not just power—it's protection. If you've found value in the insights and analyses shared within this newsletter, consider this an opportunity to strengthen your network by sharing it with peers. Encourage them to subscribe for cutting-edge insights into generative AI.