Past Events

PhD Defense

Semantic Modeling, Integration and Episodic Organization of Personal Digital Traces


Download as iCal file

Thursday, June 23, 2022, 12:00pm


Speaker: Varvara Kalokyri

Location : Virtual


Prof. Amelie Marian (Chair)
Prof. Alexander Borgida
Prof. Gerard De Melo
Prof. Chirag Shah, University of Washington

Event Type: PhD Defense

Abstract: Memory plays a fundamental role in life and is critical to our everyday functioning. We use memories to maintain our personal identity, to support our relationships, to learn, and to solve problems. Today, a vast amount of tools supports digital capture of different aspects of people's lives. These tools produce a multitude of data objects, which we call Personal Digital Traces - PDTs, which can be used to help reconstruct people‚Äôs episodic memories and connect to their past personal events. This reconstruction may have several applications, from helping the recall of patients with neurodegenerative diseases to helping people remember past events and better manage their data and time if this information is used in activity-centric applications, like personal assistants.This dissertation takes steps towards supporting autobiographical memory by associating heterogeneous PDTs according to their higher-level purposes and usages and summarizing them into episodic narratives. We start by presenting a unified and intuitive conceptual modeling language whose novel features include the properties "who, what, when, where, why, how" applied uniformly to both personal digital traces and their corresponding atomic events/tasks that produce them. We then proceed by describing an ontology for prototypical higher level plans ("scripts") for common everyday events and show how families of related scripts can be defined by incrementally modifying more general scripts through inheritance. We then present a multiplayer web-based game called OneOfUs, which creates possible associated digital trace descriptions that can be produced by each of these activities/scripts through crowdsourcing techniques that act as evidence for the execution of such scripts. The game is able to automatically validate and assess knowledge at the time of the game, as well as dynamically learn new pieces of information, as it has a way of not neglecting uncommon answers through players' votes. For instantiating those scripts based on the lower level actions of which scripts are composed of, we present a bottom up merging algorithm that groups and relates several digital traces from many different sources into script instances (episodes) as well as a software architecture that supports systematic and declarative specification of evidence. This also utilizes a scoring scheme to account for the varied strength of evidence provided by PDTs or script steps.Finally, to evaluate the efficacy of our methodology, we designed and implemented YourDigitalSelf, an Android mobile device application that gathers and integrates personal digital traces into narratives. A thorough evaluation performed over real user's data collections shows that our approach is able to integrate and combine successfully different traces from different popular sources into coherent episodes/activities. In addition, we show evidence that our approach does augment user's memory of their past actions, thereby forms a powerful retrospective memory aid.


Department of Computer Science

School of Arts & Sciences

Rutgers University

Contact  Professor Amelie Marian