You are here: Home Research



Interactive Learning for Robotic Manipulation

We propose CEILing, a framework which combines both corrective and evaluative feedback from a human teacher to train a stochastic policy in an asynchronous manner. We present extensive simulation and real-world experiments that demonstrate that CEILing can effectively solve complex robot manipulation tasks directly from raw images in less than one hour of real-world training.

Read more

Panoptic Segmentation of LiDAR Point Clouds

We present EfficientLPS that addresses multiple challenges in segmenting LiDAR point clouds including distance-dependent sparsity, large scale-variations, and re-projection errors. We also formulate a regularized pseudo labeling framework for training on unlabelled data. EfficientLPS sets the new state-of-the-art on SemanticKITTI and nuScenes benchmarks.

Read more

Learning Kinematic Feasibility through Reinforcement Learning

We frame mobile manipulation as a reinforcement learning problem in which the base ensures kinematic feasibility of arbitrary end-effector motions. This results in a simple objective with which we can deploy the same approach across a variety of mobile robots, alleviating the need for expert-based adaptations. It also generalizes to unseen tasks at test time.

Read more

Dynamic Object Removal & Inpainting

We address the problem of synthesizing plausible color, texture and geometry in regions occluded by dynamic objects. We propose the novel geometry-aware DynaFill architecture that follows a coarse-to-fine topology and incorporates our gated recurrent feedback mechanism. Our model achieves state-of-the-art performance compared to existing methods.

Read more

Multi-Object Panoptic Tracking

We introduce and investigate a new perception task that we call MOPT which unifies the conventionally disjoint tasks of semantic segmentation, instance segmentation, and multi-object tracking into a single coherent scene understanding problem. We present PanopticTrackNet and several new baselines to address this task using either LiDAR scans or images.

Read more

Visual Localization in LiDAR Maps

We present novel CNN-based methods for monocular camera localization in LiDAR-maps that are independent of both the map and camera intrinsics. Our networks achieve state-of-the-art performance on KITTI, Argoverse, and Lyft Level5 while being the first deep learning methods to effectively generalize to unseen environments as well as to different sensors.

Read more

Efficient Panoptic Segmentation

We present the novel EfficientPS architecture that consists of our shared backbone with 2-way FPN, followed by new instance and semantic segmentation heads, and our panoptic fusion module. Our network sets the new state-of-the-art on Cityscapes, KITTI, Mapillary Vistas and IDD while being the most efficient and fast panoptic segmentation model to date.

Read more