Last Week in GAI Security Research - 05/06/24

Highlights from Last Week

• 🚀 Boosting Jailbreak Attack with Momentum
• ✍🏻 AppPoet: Large Language Model based Android malware detection via multi-view prompt engineering
• 🚪 Transferring Troubles: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuning
• 👻 Assessing Cybersecurity Vulnerabilities in Code Large Language Models
• 👾 Assessing LLMs in Malicious Code Deobfuscation of Real-world Malware Campaigns

Partner Content

Codemod is the end-to-end platform for code automation at scale. Save days of work by running recipes to automate framework upgrades

• Leverage the AI-powered Codemod Studio for quick and efficient codemod creation, coupled with the opportunity to engage in a vibrant community for sharing and discovering code automations.
• Streamline project migrations with seamless one-click dry-runs and easy application of changes, all without the need for deep automation engine knowledge.
• Boost large team productivity with advanced enterprise features, including task automation and CI/CD integration, facilitating smooth, large-scale code deployments.

🚀 Boosting Jailbreak Attack with Momentum (http://arxiv.org/pdf/2405.01229v1.pdf)

• Introducing a momentum term into gradient-based adversarial attacks on Large Language Models (LLMs) significantly increases attack success rates and efficiency.
• The Momentum Accelerated Greedy Coordinate Gradient (MAC-GCG) attack achieved a higher success rate of 48.6% compared to 38.1% with the standard GCG method, within just 20 steps of iteration.
• Experimental evaluations across multiple prompts demonstrate that the MAC method not only enhances attack success rates but also stabilizes suffix optimization, indicating a crucial advancement in adversarial attack methodologies against aligned language models.

✍🏻 AppPoet: Large Language Model based Android malware detection via multi-view prompt engineering (http://arxiv.org/pdf/2404.18816v1.pdf)

• AppPoet, leveraging a large language model (LLM)-assisted system for Android malware detection, achieved a high detection accuracy of 97.15% and F1 score of 97.21%, demonstrating superior performance over traditional baseline methods.
• Through the employment of multi-view engineering, including feature extraction from application permissions, API uses, and URL features, AppPoet generates comprehensive diagnostic reports on potential malware, enhancing interpretability and providing actionable insights.
• The research illustrates the vulnerability of Android devices to malware, citing that by the third quarter of 2023, 438,000 instances of mobile malware were detected, underscoring the need for advanced detection systems like AppPoet.

🚪 Transferring Troubles: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuning (http://arxiv.org/pdf/2404.19597v1.pdf)

• Cross-lingual backdoor attacks on multilingual large language models (LLMs) like mT5, BLOOM, and GPT-3.5-turbo demonstrate an average success rate of 50% across 25 languages, revealing significant vulnerabilities and the need for enhanced security measures.
• Larger LLMs pre-trained primarily on English data show increased susceptibility to backdoor attacks, with successful trigger paraphrasing, indicating a worrying trend towards higher security risks in multilingual contexts.
• Effective defense against such backdoor attacks remains challenging as poisoned training instances constitute less than 1% of the dataset; thus, the insidious nature of these triggers poses a continuous security threat to the development and deployment of LLMs.

👻 Assessing Cybersecurity Vulnerabilities in Code Large Language Models (http://arxiv.org/pdf/2404.18567v1.pdf)

• EvilInstructCoder framework demonstrates that instruction-tuned large language models (LLMs) for code generation are susceptible to adversarial cyberattacks, with vulnerabilities allowing for the injection of malicious code through adversarial code injections.
• Adversarial attacks, including backdoor attacks and data poisoning, were successful with a relatively low poisoning rate of 0.5%, altering model outputs to include malicious code in 76%-86% of cases.
• The research underscores the need for robust security measures for instruction-tuned code LLMs, revealing the potential for significant cybersecurity risks within software development processes that rely on these AI tools.

👾 Assessing LLMs in Malicious Code Deobfuscation of Real-world Malware Campaigns (http://arxiv.org/pdf/2404.19715v1.pdf) -

• OpenAI's GPT-4 outperformed other large language models in deobfuscating malicious PowerShell scripts with a success rate of 69.56% for extracting URLs.
• The study demonstrated the potential of utilizing large language models for automating the deobfuscation of malicious code, presenting an important tool in enhancing malware analysis.
• Manual effort in malware analysis can be significantly reduced through the integration of LLMs, which assist in identifying and extracting obfuscated information from malicious payloads.

Other Interesting Research

• Human-Imperceptible Retrieval Poisoning Attacks in LLM-Powered Applications (http://arxiv.org/pdf/2404.17196v1.pdf) - Retrieval poisoning poses a significant threat to LLM security, demonstrating high success rates and challenging existing defense strategies.
• Evaluating and Mitigating Linguistic Discrimination in Large Language Models (http://arxiv.org/pdf/2404.18534v1.pdf) - LDFighter improves safety and quality in LLM responses, addressing linguistic discrimination across languages, yet disparities persist particularly for low-resource languages.
• Silencing the Risk, Not the Whistle: A Semi-automated Text Sanitization Tool for Mitigating the Risk of Whistleblower Re-Identification (http://arxiv.org/pdf/2405.01097v1.pdf) - A novel text sanitization tool significantly lowers the risk of whistleblower re-identification while maintaining a majority of the original text's semantics.
• Learnable Linguistic Watermarks for Tracing Model Extraction Attacks on Large Language Models (http://arxiv.org/pdf/2405.01509v1.pdf) - Innovative linguistic watermarking techniques ensure intellectual property protection in language models by blending undetectability, learnability, and efficiency without compromising text quality or model performance.
• Adversarial Attacks and Defense for Conversation Entailment Task (http://arxiv.org/pdf/2405.00289v2.pdf) - Innovative fine-tuning and defense strategies significantly improve NLP model resilience to adversarial attacks, emphasizing the need for continuous advancements in model robustness.
• Safe Training with Sensitive In-domain Data: Leveraging Data Fragmentation To Mitigate Linkage Attacks (http://arxiv.org/pdf/2404.19486v1.pdf) - Fragmenting sensitive domain data into syntactic chunks emerges as a practical, privacy-preserving strategy that marginally impacts the efficacy of fine-tuned language models.
• Generative AI in Cybersecurity (http://arxiv.org/pdf/2405.01674v1.pdf) - Generative AI is a double-edged sword in cybersecurity, simultaneously advancing defenses and empowering adversaries.

Strengthen Your Professional Network

In the ever-evolving landscape of cybersecurity, knowledge is not just power—it's protection. If you've found value in the insights and analyses shared within this newsletter, consider this an opportunity to strengthen your network by sharing it with peers. Encourage them to subscribe for cutting-edge insights into generative AI.