Subscribe to RSS FeedCS Events
Qualifying ExamReachability Analysis Enables Provably Correct Controller Synthesis and Safe Exploration in Reinforcement Learning |
|
||
Thursday, May 22, 2025, 11:00am - 12:00pm |
|||
Speaker: Yuning Wang
Location : CoRE 305
Committee:
Professor He Zhu
Professor Srinivas Narayana
Professor Santosh Nagarakatte
Professor Yongfeng Zhang
Event Type: Qualifying Exam
Abstract: In this talk, we introduce how to integrate reachability analysis into Reinforcement Learning (RL) training process to enable provably correct controller synthesis and safe exploration. First, we present a verification-based learning framework VEL that synthesizes safe programmatic controllers for environments with continuous state and action spaces. VEL performs abstraction-based program verification to reason about a programmatic controller and its environment as a closed-loop system. VEL minimizes the amount of safety violation in the proof space of the system, which approximates the worst-case safety loss, using gradient-descent style optimization. Experimental results demonstrate the substantial benefits of leveraging verification feedback for synthesizing provably correct controllers. Second, we introduce VELM, a RL framework that conducts formal reachability analysis similar to VEL but for each iteration in the RL training loop with a learned symbolic environment model. An online shielding layer is then constructed to confine the RL agent's exploration solely within a state space verified as safe in the learned model, thereby bolstering the overall safety profile of the RL system. Our experimental results demonstrate that VLEM significantly reduces safety violations in comparison to existing safe learning techniques, all without compromising the RL agent's reward performance.
:
Contact Professor He Zhu
Zoom Link: https://rutgers.zoom.us/my/yw895?pwd=eFBoQmV5YyttdTJrUXQvcTE2a2RpUT09&omn=95981147564
