Terms and Conditions
The training data provided for each task may be used to
train model parameters. The test data should be used strictly
for reporting of results, i.e., it should not be used in any
way to train or tune systems, for example, by evaluating multiple
parameters or feature choices and reporting the best results
obtained. It is the responsibility of the participant to divide
the training set into appropriate training and validation splits,
e.g., using 5-fold cross-validation. The tuned algorithms should
then be run only once on the test data.
We strongly discourage multiple submissions to the server
as the evaluation server should not be used for parameter tuning.
The number of submissions for the same task is strictly controlled.
If participants would like to report results in their papers for
multiple versions of their algorithm (e.g., parameters or features)
they can do so on a validation split of the training data and only
submit the best performing setting for evaluation to our server.
We enforce a minimum of 72 hours between submissions. If
someone would like to submit earlier, e.g., due to a corrupt upload
we kindly ask you to open a ticket for the same.
Users should not register multiple times using different email
addresses. This also means that for collaborative effort across groups
or within a lab, only one account should be used by all participants.
Multiple accounts will be deleted and participants will be warned
or black listed from future submissions. When registering, we ask all
participants to provide their full name, institution affiliation and
institutional email address (i.e., .edu). If you have any question
regarding these policies, or wish to use a different email address
please do not hesitate to get in touch with us!