Multimodal Situational Safety

Kaiwen Zhou*¹, Chengzhi Liu*¹, Xuandong Zhao², Anderson Compalas¹, Dawn Song², Xin Eric Wang†¹

¹University of California, Santa Cruz, ²University of California, Berkley

^*Equal contribution

Dataset Structure

The Dataset can be downloaded from Hugging Face.

Each entry in the Chat Task dataset contains the following fields:

• safe_image_path: the file path to the safe image.
• intent: The user's intent in the context of images.
• unsafe_image: The description of unsafe image.
• unsafe_image_path: the file path to the unsafe image.
• Type: The multimodal situational safety category of the entry.
• queries: The user's question in Chat Task.

Each entry in the Embodied Task dataset contains the following fields:

• task: the specific embodied task.
• category: The multimodal situational safety category of the entry.
• safe_instruction/safe_instructions: The user's safe instructions and related variations.
• unsafe_instruction/unsafe_instructions: The user's unsafe instructions and related variations.
• safe: the file path to the safe image.
• unsafe: the file path to the unsafe image.

Evaluation

You can evaluate different MLLMs by running our evaluation code inference.py and changing the "--mllm" parameter:

python inference.py --mllm gemini --data_root xxx --output_dir xxx

The deployment of the model can refer to models. For proprietary models, please set up your API key first.

Citation

@misc{zhou2024multimodalsituationalsafety,
      title={Multimodal Situational Safety}, 
      author={Kaiwen Zhou and Chengzhi Liu and Xuandong Zhao and Anderson Compalas and Dawn Song and Xin Eric Wang},
      year={2024},
      eprint={2410.06172},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2410.06172}, 
}