CS Events Monthly View

PhD Defense

Machine Learning and Understanding Art

 

Download as iCal file

Wednesday, May 18, 2022, 10:00am
 

Speaker: Diana Kim

Location : Virtual

Committee: 

Professor Ahmed Elgammal (Advisor)

Professor Konstantinos Michmizos

Professor Karl Stratos

Professor Gennaro Vessio (University of Bari, Italy)

Event Type: PhD Defense

Abstract: Recent advances in machine learning on various computer vision tasks have shown the great potential for developing an AI system for art through the successful applications in prediction of style, genre, medium, attribution, school of art, etc. Beyond the categorical information, in this dissertation, a more fundamental level of artistic knowledge is pursued by developing machine learning systems for art principles. The art principles are to know how art is visually formed and identify what content in it and what symbolic meaning it has. The task necessitates fine-grained semantics to describe art, but art data does not generally accommodate the fine details. The scarcity of data annotation is a primary challenge in this machine learning study. Three research problems are explored; (1) first is to find principal semantics for style recognition. (2) second is to lay the groundwork for computational iconography, i.e. recognize content and discover the co-occurrence and visual similarities among the content in fine art paintings. (3) third is to quantify paintings with finite visual semantics from style through language models. In the system design, well-established knowledge and facts in art theory are leveraged, or general knowledge of art is integrated into the hierarchical architecture of deep-CNN as a numerical form after learning it from a corpus of art-texts through contemporary language models in Natural Language Processing. Language modeling is a practical and scalable solution requiring no direct annotation, but it is inevitably imperfect. This dissertation shows how deep learning's hierarchical structure and adaptive nature can create a stronger resilience to the incompleteness of the practical solution than other related methods.

Organization: 

Rutgers University School of Arts and Sciences