Funded Projects

Start of funding 01.01.2025

Unpacking Failure Modes of Generative Robot Policies

Prof. Dr. Matthias Althoff
Technische Universität München
Informatik 6 - Lehrstuhl für Robotik, Künstliche Intelligenz und Echtzeitsysteme

Christopher Agia
Stanford University
Interactive Perception and Robot Learning Lb, Department of Computer Science

Robot behavior policies trained via imitation learning are prone to failure under conditions that deviate from their training data. Thus, algorithms that monitor learned policies at test time and provide early warnings of failure are necessary to facilitate scalable deployment. We propose a runtime monitoring framework that splits the detection of failures into two complementary categories: 1) Erratic failures, which we propose to detect using a statistical measure of temporal action consistency, and 2) task progression failures, where we use VLMs to detect when the policy confidently and consistently takes actions that do not solve the task. Our approach has two key strengths. First, because learned policies exhibit diverse failure modes, combining complementary detectors leads to significantly higher accuracy at failure detection. Second, using a statistical temporal action consistency measure ensures that we quickly detect when multi-modal, generative policies exhibit erratic behavior at negligible computation cost. In contrast, we only use VLMs to detect failure modes that are less time sensitive. We demonstrate our approach in the context of diffusion policies trained on a multi-modal domain and robotic mobile manipulation domains both in simulation and in the real world.