Skip to content Skip to navigation
Qualifying Exam
1/31/2019 12:00 pm
CBIM 22

Zero-shot Learning from Noisy Text Descriptions

Yizhe Zhu, Dept. of Computer Science

Examination Committee: Prof. Ahmed Elgammal (chair), Prof. Gerard de Melo, Prof. Yongfeng Zhang, Prof. Badri Nath

Abstract

The main shortcoming of deep learning methods is the inevitable requirement of large-scale labeled training data that need to be collected and annotated by costly human labor.  Zero-shot learning aims to recognize objects with no training samples available by introducing semantic representations of classes. We propose two methods to tackle zero-shot learning problem from noisy text descriptions as semantic representations of classes.  The first one is a visual-semantic embedding-based method that embeds visual data and semantic data to a shared embedding space, while connecting text terms to its relevant visual parts. The second one is a GAN(Generative Adversarial Network)-based method that is able to generate visual feature based on semantic information, and the synthetic visual features are then used as training samples for unseen class classification.