We use a few cookies

Essential cookies keep the library working. With your permission, we'd also like to use analytics to see what's useful and improve the experience. Read our Privacy Policy or Cookie Policy.

Skip to main content
Home/e-Resources/Artificial Intelligence & Machine Learning/SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models
Conference ProceedingsArtificial Intelligence & Machine Learning

SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models

2024

Checking your access to this record.

Abstract & Overview

In the rapidly evolving landscape of Large Language Models (LLMs), ensuring robust safety measures is paramount.To meet this crucial need, we propose SALAD-Bench, a safety benchmark specifically designed for evaluating LLMs, attack, and defense methods.Distinguished by its breadth, SALAD-Bench transcends conventional benchmarks through its large scale, rich diversity, intricate taxonomy spanning three levels, and versatile functionalities.SALAD-Bench is crafted with a meticulous array of questions, from standard queries to complex ones enriched with attack, defense modifications and multiple-choice.To effectively manage the inherent complexity, we introduce an innovative evaluators: the LLMbased MD-Judge for QA pairs with a particular focus on attack-enhanced queries, ensuring a seamless, and reliable evaluation.Above components extend SALAD-Bench from standard LLM safety evaluation to both LLM attack and defense methods evaluation, ensuring the joint-purpose utility.Our extensive experiments shed light on the resilience of LLMs against emerging threats and the efficacy of contemporary defense tactics.Data and evaluator are released under https://github.com/ OpenSafetyLab/SALAD-BENCH. Warning: this paper includes examples that may be offensive or harmful.

Cite this record

Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, et al. (2024). SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models. https://doi.org/10.18653/v1/2024.findings-acl.235

Copy in another style

Li, L., Dong, B., Wang, R., Hu, X., Zuo, W., Lin, D., Qiao, Y., & Shao, J. (2024). SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models. https://doi.org/10.18653/v1/2024.findings-acl.235
Author names are used as deposited, so check their order before you submit.

Metadata is harvested from open-access indexes and can be incomplete. Check a citation against the published version before relying on it — see our open access & attribution policy.