The goal of NICO Challenge is to facilitate the OOD (Out-of-Distribution) generalization in visual recognition through promoting the research on the intrinsic learning mechanisms with native invariance and generalization ability. The training data is a mixture of several observed contexts while the test data is composed of unseen contexts. Participants are tasked with developing reliable algorithms across different contexts (domains) to improve the generalization ability of models.
The NICO++ dataset is reorganized to training, open validation, open test and private test sets for each track. There are 60 categories for both two tracks and 40 of them are shared in both tracks (totally 80 categories in NICO++). For the common context generalization, 88866 samples are for training, 13907 for public test (images are public while labels are unavailable) and 35920 (both images and labels are unavailable) for private test. For the hybrid context generalization, 57425 samples are for training, 8715 for public test and 20079 for private test.
The training and public data are available in data-link. The general procedure of NICO Challenge includes two phases. First in phase 1, all the competitors are requested to submit their results on public validation data. The evaluation will be automatically done via the online platform based on our proposed metric (i.e., overall accuracy on all test images). A public leaderboard will keep rolling during this phase and the top 10 teams will be invited into phase 2 where they are required to upload their source code and their models are tested on private test data.