CS Events

Qualifying Exam

Picture-to-Amount (PITA): Predicting Relative Ingredient Amounts from Food Images


Download as iCal file

Tuesday, October 19, 2021, 02:00pm - 03:30pm


Speaker: Jiatong Li

Location : Remote via Zoom


Prof. Vladimir Pavlovic (Chair)

Prof. Dimitris Metaxas

Prof. Sungjin Ahn

Prof. David Pennock

Event Type: Qualifying Exam

Abstract: Increased awareness of the impact of food consumption on health and lifestyle today has given rise to novel data-driven food analysis systems. Although these systems may recognize the ingredients, a detailed analysis of their amounts in the meal, which is paramount for estimating the correct nutrition, is usually ignored. Here, we study the novel and challenging problem of predicting the relative amount of each ingredient from a food image. We propose PITA, the Picture-to-Amount deep learning architecture to solve the problem. More specifically, we predict the ingredient amounts using a domain-driven Wasserstein loss from image-to-recipe cross-modal embeddings learned to align the two views of food data. Experiments on a dataset of recipes collected from the Internet show the model generates promising results and improves the baselines on this challenging task.


Join Zoom Meeting

Join by SIP

Meeting ID: 967 2298 4705
Password: 359929
One tap mobile
+13126266799,,96722984705# US (Chicago)
+16465588656,,96722984705# US (New York)

Join By Phone
+1 312 626 6799 US (Chicago)
+1 646 558 8656 US (New York)
+1 301 715 8592 US (Washington DC)
+1 346 248 7799 US (Houston)
+1 669 900 9128 US (San Jose)
+1 253 215 8782 US (Tacoma)
Meeting ID: 967 2298 4705
Find your local number: https://rutgers.zoom.us/u/aexFZQ1TSK

Join by Skype for Business