Leader Board

The leader board presents results on the test set of the MovieQA benchmark. The tables here are sorted based on the used answering source. Please follow the buttons for quick access to different sources.

Movie: Video+Subtitles

Date Team Name Affiliation Accuracy Notes Paper Code
10 Feb 2017, 03:36 BI_kmkim Seoul National University 29.97 - -
16 Feb 2017, 23:20 BI_kmkim Seoul National University 29.41 - -
06 Feb 2017, 20:31 BI_kmkim Seoul National University 28.54 - -
21 Oct 2016, 10:58 Anonymous Anonymous 23.61 Only_Video_Question_Answers_Pairs - -

Subtitles only

Date Team Name Affiliation Accuracy Notes Paper Code
24 Oct 2016, 13:22 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 36.87
28 Aug 2016, 22:52 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 26.45
25 Aug 2016, 00:59 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 23.65

DVS only

Date Team Name Affiliation Accuracy Notes Paper Code
24 Oct 2016, 13:22 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 35.09
25 Aug 2016, 00:59 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 24.91
28 Aug 2016, 22:52 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 23.33

Scripts only

Date Team Name Affiliation Accuracy Notes Paper Code
24 Oct 2016, 13:22 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 37.05
25 Aug 2016, 00:59 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 24.41
28 Aug 2016, 22:52 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 23.90

Plot Synopses only

Date Team Name Affiliation Accuracy Notes Paper Code
26 Jan 2017, 09:07 UCLMR University College London, Machine Reading Group 77.63 - -
23 Jan 2017, 04:32 UCLMR University College London, Machine Reading Group 76.64 - -
17 Nov 2016, 10:59 UCLMR University College London, Machine Reading Group 75.78 - -
18 Sep 2016, 01:45 Sure Singapore Management University 72.91
02 Jan 2017, 10:58 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 57.97
12 Nov 2016, 00:41 Anonymous Anonymous 47.45 TFIDF replicate of Makarand's work Anonymous Anonymous
28 Aug 2016, 22:52 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 47.36
25 Aug 2016, 00:59 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 45.63
24 Oct 2016, 13:22 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 38.43
30 Oct 2016, 01:28 Anonymous Anonymous 20.65 Using plain old simple tf - idf - -

Open ended

Date Team Name Affiliation Accuracy Notes Paper Code
20 Aug 2016, 00:15 KIT-UofT-MIT Karlsruhe Inst. of Tech. + Univ. of Toronto + Massachusetts Inst. of Tech. 25.56