Atnafu Lambebo Tonja

Afri-MCQA: Multimodal Cultural Question Answering for African Languages

Atnafu Lambebo Tonja, Srija Anand, Emilio Villa-Cueva, Israel Abebe Azime, Jesujoba Oluwadara Alabi, Muhidin A. Mohamed, Debela Desalegn Yadeta, Negasi Haile Abadi, Abigail Oppong, Nnaemeka Casmir Obiefuna, Idris Abdulmumin, Naome A. Etori, Eric Peter Wairagala, Kanda Patrick Tshinu, Imanigirimbabazi Emmanuel, Gabofetswe Malema, Alham Fikri Aji, David Ifeoluwa Adelani, Thamar Solorio

ACL 2026 · ★ Social Impact Paper Award

Two Afri-MCQA examples: a Hausa and English question about a clay pan from Nigeria, and an Amharic and English question about a castle in Gondar, Ethiopia, beside a map of the 12 countries covered.
Example questions in Hausa and Amharic with their English counterparts. Each question is available as text and speech.

Abstract

Africa is home to over one-third of the world's languages, yet remains underrepresented in AI research. We introduce Afri-MCQA, the first Multilingual Cultural Question-Answering benchmark covering 7.5k Q&A pairs across 15 African languages from 12 countries. The benchmark offers parallel English-African language Q&A pairs across text and speech modalities and was entirely created by native speakers. Benchmarking large language models (LLMs) on Afri-MCQA shows that open-weight models perform poorly across evaluated cultures, with near-zero accuracy on open-ended VQA when queried in native language or speech. To evaluate linguistic competence, we include control experiments meant to assess this specific aspect separate from cultural knowledge, and we observe significant performance gaps between native languages and English for both text and speech. These findings underscore the need for speech-first approaches, culturally grounded pretraining, and cross-lingual cultural transfer.

Data

The dataset is on Hugging Face, released under an academic license or CC BY-NC 4.0.

BibTeX

@inproceedings{tonja-etal-2026-afri,
    title = "Afri-{MCQA}: Multimodal Cultural Question Answering for {A}frican Languages",
    author = "Tonja, Atnafu Lambebo  and
      Anand, Srija  and
      Villa-Cueva, Emilio  and
      Azime, Israel Abebe  and
      Alabi, Jesujoba Oluwadara  and
      Mohamed, Muhidin A.  and
      Yadeta, Debela Desalegn  and
      Abadi, Negasi Haile  and
      Oppong, Abigail  and
      Obiefuna, Nnaemeka Casmir  and
      Abdulmumin, Idris  and
      Etori, Naome A  and
      Wairagala, Eric Peter  and
      Tshinu, Kanda Patrick  and
      Emmanuel, Imanigirimbabazi  and
      Malema, Gabofetswe  and
      Aji, Alham Fikri  and
      Adelani, David Ifeoluwa  and
      Solorio, Thamar",
    booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
    month = jul,
    year = "2026",
    address = "San Diego, California, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.acl-long.1869/",
    doi = "10.18653/v1/2026.acl-long.1869",
    pages = "40249--40282"
}