Open-Set Semi-Supervised Object Detection



ECCV 2022


Figure: (a) Open-Set Semi-Supervised Object Detection is a more practical semi-supervised setting, where the goal is to improve an object detector with unconstrained/open-set unlabeled images. (b) We find most of the self-training semi-supervised methods (e.g., Unbiased Teacher) cannot address this problem setting, since some OOD objects in unlabeled images are mispredicted as inlier objects with high confidence and misused as pseudo-labels.




Motivation

For a more practical semi-supervised object detection scenrario, we consider open-set semi-supervised settings for object detection tasks. Prior SSOD works often assume the label spaces of labeled and unlabeled data are identical, while this closed-set assumption is unlikely to happen in real-world situations, where unlabeled images collected in the wild might contain out-of-distribution (OOD) objects, which are unseen, undefined, and unknown in the available labeled set.



Analysis: Open-set setting is harder than closed-set setting

Figure: Performance comparison of the existing SSOD method between closed-set and open-set settings.

We find the existing SSOD method show limited performance gain in the open-set scenario compared with closed-set scenario as shown in the above figure. We attribute this lower performance gain to semantic expansion. Specifically, when OOD/novel objects exist in the unlabeled set, the closed-set classifier might mispredict these OOD instances as ID objects with high confidence. These over-confident and incorrect OOD instances are prone to be selected as the pseudo-labels of ID objects even after confidence thresholding.



Our Method: Online OOD detector vs. Offline OOD detector

Figure: Illustration of online and offline OOD detection.

To improve object detector with uncontrained unlabeled images, we combine the existing closed-set SSOD methods with OOD detection, which aims to remove the OOD objects in pseudo-labels and prevent semantic expansion in OSSOD. For the online OOD detector, we add an additional branch on the object detector, but it suffers from the instability of pseudo-labels used for SSOD and leads to degraded performance of the object detector. In constrast, the offline OOD detector, which is learned individually and seperated from the object detector, does not have these limitations and produce better results on OOD detection due to its nature of independent model architecture. To understand more details about our method, we encourage the readers to take a closer look at our paper.



Paper



Open-set Semi-Supervised Object Detection

European Conference on Computer Vision 2022 (ECCV 2022)
Y.-C. Liu, C.-Y. Ma, X. Dai, J. Tian, P. Vajda, Z. He, Z. Kira




Code (Coming soon)





Acknowledgements

We thank Ross Girshick for many helpful suggestions and valuable discussions.
This project is done partially while Yen-Cheng Liu was interning at Meta.
Yen-Cheng Liu and Zsolt Kira were partly supported by DARPA’s Learning with Less
Labels(LwLL) program under agreement HR0011-18-S-0044, as part of their affiliation with Georgia Tech.