EgoNormia

Can VLMs make normative decisions in physical social interactions?

Input Modality Types:

  • Blind: Models receive only the questions with no visual input
  • Pipeline: Models receive text-only descriptions of the scene (generated by Gemini 1.5 Flash)
  • Video: Models receive both video input (1 fps, concatenated into a single image) and questions
Model
Modality
Both
Act
Jus
Sen
Date
H
Human
Human
Video92.492.492.485.12025-02-15
Google logo - light
🥇 Gemini 2.5 Flash
Google
Video53.758.956.856.72025-04-27
OpenAI logo - light
🥈o4-mini
OpenAI
Video50.060.252.352.82025-04-27
OpenAI logo - light
🥉 GPT-4.1
OpenAI
Video49.855.552.655.22025-04-27
Google logo - light
Gemini 2.5 Pro
Google
Video47.252.048.447.12025-04-27
Google logo - light
Gemini 1.5 Pro
Google
Video45.351.947.861.12025-02-15
Google logo - light
Gemini 2.0 Thinking
Google
Video42.751.745.357.32025-02-15
Google logo - light
Gemini 1.5 Flash
Google
Video41.746.544.354.42025-02-15
OpenAI logo - light
o3-mini
OpenAI
Pipeline41.545.745.265.02025-02-15
Alibaba logo - light
Qwen2.5 VL 72B
Alibaba
Video41.548.343.862.82025-02-15
OpenAI logo - light
GPT-4o
OpenAI
Video39.845.144.859.62025-02-15
Google logo - light
Gemini 2.0 Flash
Google
Video38.949.641.360.02025-02-15
Alibaba logo - light
QwQ-32B
Alibaba
Video37.846.742.244.62025-04-27
Google logo - light
Gemini 2.0 Thinking
Google
Pipeline37.546.342.158.82025-02-15
Deepseek logo - light
Deepseek R1
Deepseek
Pipeline36.542.940.061.02025-02-15
Anthropic logo - light
Claude 3.5 Sonnet
Anthropic
Video36.043.541.059.32025-02-15
S
InternVL 2.5
Shanghai AI Lab
Pipeline32.740.938.062.52025-02-15
Google logo - light
Gemini 1.5 Pro
Google
Pipeline30.737.334.864.02025-02-15
R
Constant Choice
Random
Blind25.325.325.340.52025-02-15
Anthropic logo - light
Claude 3.5 Sonnet
Anthropic
Pipeline23.936.733.561.22025-02-15
Google logo - light
Gemini 1.5 Pro
Google
Blind21.224.623.654.02025-02-15
OpenAI logo - light
GPT-4o
OpenAI
Pipeline21.023.723.566.02025-02-15
OpenAI logo - light
GPT-4o
OpenAI
Blind17.719.919.955.92025-02-15
Deepseek logo - light
Deepseek R1
Deepseek
Blind16.119.417.127.32025-02-15
S
InternVL 2.5
Shanghai AI Lab
Blind15.318.317.455.42025-02-15
S
InternVL 2.5
Shanghai AI Lab
Video15.118.717.650.72025-02-15
OpenAI logo - light
o3-mini
OpenAI
Blind15.016.817.151.92025-02-15
Google logo - light
Gemini 1.5 Flash
Google
Pipeline14.717.716.754.22025-02-15
Google logo - light
Gemini 1.5 Flash
Google
Blind12.215.014.146.62025-02-15
Meta logo - light
Llama 3.2
Meta
Video2.219.910.154.72025-02-15