MovieQA logo

MovieQA: Story Understanding Benchmark
A Question Answering data set for Automatic Story Comprehension

NEW: Joint Video and Language Understanding Workshop at ICCV, 23rd October 2017
Click here to know more


About MovieQA

We introduce the MovieQA dataset which aims to evaluate automatic story comprehension from both video and text. The data set consists of almost 15,000 multiple choice question answers obtained from over 400 movies and features high semantic diversity.

Each question comes with a set of five highly plausible answers; only one of which is correct. The questions can be answered using multiple sources of information: movie clips, plots, subtitles, and for a subset scripts and DVS. Click here to see examples of the data set.

NEW: As part of the ICCV 2017 workshop, we are adding about 80 more movies for video-based answering. We are also introducing an intermediate task, that could be used to learn improved descriptors: alignment between video clips and plot synopses sentences, evaluated through retrieval.

Cite this paper if you use the data:
@inproceedings{MovieQA,
author = {Makarand Tapaswi and Yukun Zhu and Rainer Stiefelhagen and Antonio Torralba and Raquel Urtasun and Sanja Fidler},
title = {MovieQA: Understanding Stories in Movies through Question-Answering},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2016}
}
University of Toronto
Karlsruhe Institute of Technology
Massachusetts Institute of Technology

Sponsors


Website Changelog

  • 2017.08.15: Stormpath API user management discontinued
  • 2017.06.13: ICCV 2017 Workshop announcement
  • 2016.12.09: A searchable dataset browser
  • 2016.09.01: Evaluation benchmark and leaderboard are open
  • 2016.03.30: v1.0 data release! Open for public registrations
  • 2016.02.01: v1.0 beta today. Register and help us fix some bugs
  • 2015.12.04: Hello world! The website goes online

Data Release log

  • 2017.01.26: Plot sentence to video clip alignments available
  • 2016.11.08: Patch for 65 missing video clips
  • 2016.09.10: Video meta-data released: shot boundaries, timestamps
  • 2016.04.06: Removed missing video-clips from QAs
  • 2016.03.30: v1.0 data release!