You are here: Home Teaching SS 2022 Seminar: Learning with Limited …

Seminar: Learning with Limited Supervision

Nikhil Gosala, Daniel Honerkamp, Juana Valeria Hurtado

Deep learning has become a key enabler of real world autonomous systems. While classical supervised learning methods typically rely on ground truth information, the area autonomous robotics requires less dependence on manual supervision. The research directions of semi- and self-supervised learning instead aim to learn representations without explicit and potentially even manual supervision. Especially, the domain of robot learning requires scaling to large amounts of unlabeled data in a lifelong manner. Self- and semi-supervised learning already have had a significant impact on fields such as perception, state estimation, control, or graph representation learning, thereby making important progress in object manipulation, scene understanding, visual recognition, object tracking, and learning-based control, amongst others. In this seminar, we will study a selection of state-of-the-art works that propose deep learning techniques for tackling various challenges in autonomous systems. In particular, we will analyze contributions in architecture design and learning paradigms in the areas of computer vision, reinforcement learning, imitation learning, and deep graph learning.


Course Information

Course Number: 11LE13S-7310-M
Places: 12
Zoom Session Details:
Meeting ID: 671 4380 4755
Passcode: Nx0kS9D76
Course Program:
Introduction: 29/04/2022 @ 14:00
How to make a presentation: 08/07/2022 @ 14:00
Evaluation Program:
  Abstract Due Date: 24/06/2022
  Seminar Presentation: 22/07/2022
  Summary Due Date: 05/08/2022
  Basic knowledge of Deep Learning or Reinforcement Learning
We will be taking a vote on doing the block course in person. (Covid restrictions permitting)
Topics will be assigned for the seminar via a preference voting. If there are more interested students than places, places will be assigned based on priority suggestions of the HisInOne system and motivation (tested by asking for a short summary of the preferred paper). The date of registration is irrelevant. In particular, we want to avoid that students grab a topic and then leave the seminar. Please have a coarse look at all available papers to make an informed decision before you commit. 

Course Material

Lecture 1: Introduction - Valid Till: 29/07/2022
Lecture 1: Introduction

Additional Information

Enrollment Procedure

  • Enroll through HISinOne, the course number is 11LE13S-7310-M.
  • The registration period for the seminars are from 25/04/2022 to 02/05/2022.
  • Attend the introductory session on 29/04/2022 via Zoom (Session details have been provided above).
  • Select three papers from the topic list (see below) and complete this form by 02/05/2022.
  • Places will be assigned based on priority suggestions of HISInOne and motivation of the student on 10/05/2022.

Evaluation Details

  • Students are expected to write an abstract, prepare a 20-minute long presentation and draft a summary.
  • The abstract should not exceed two pages and is due on 24/06/2022
  • The seminar will be held as a "Blockseminar" on 22/07/2022
  • The slides of your presentation should be discussed with the supervisor two weeks before the Blockseminar.
  • The summary should not exceed seven pages (excluding bibliography and images) and is due on 05/08/2022. Significantly longer summaries will not be accepted.
  • Ensure you cite all work you use including images and illustrations. Where possible, try to use your own illustrations.
  • The final grade is based on the oral presentation, the written abstract, the summary, and participation in the blockseminar.

What should the Summary contain?

The summary should address the following questions:

  • What is the paper's main contribution and why is it important?
  • How does it relate to other techniques in the literature?
  • What are the strengths and weaknesses of the paper?
  • What would be some interesting follow-up work? Can you suggest any possible improvements in the proposed methods? Are there any further interesting applications that the authors might have overlooked?

Graded Component Submission

  • Save your document as a PDF and directly submit it to your topic supervisor via email.
  • The filename should be in the format "FirstName_LastName_X.pdf" where X is the evaluation component (Abstract / Summary / Presentation).


  1. Scribble-Supervised LiDAR Semantic Segmentation
    Supervisor: Daniele Cattaneo
  2. FreeSOLO: Learning to Segment Objects without Annotations
    Supervisor: Daniele Cattaneo
  3. TossingBot: Learning to Throw Arbitrary Objects with Residual Physics
    Supervisor: Eugenio Chisari
  4. Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Clutter
    Supervisor: Eugenio Chisari
  5. Video Autoencoder: Self-Supervised Disentanglement of Static 3D Structure and Motion
    Supervisor: Nikhil Gosala
  6. Audio-Visual Instance Discrimination with Cross-Modal Agreement
    Supervisor: Nikhil Gosala
  7. Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
    Supervisor: Daniel Honerkamp
  8. Adversarially Guided Actor Critic
    Supervisor: Daniel Honerkamp
  9. How Far Can I Go ?: A Self-Supervised Approach for Deterministic Video Depth Forecasting
    Supervisor: Juana Valeria Hurtado
  10. Self-Supervised Multi-Object Tracking with Cross-Input Consistency
    Supervisor: Juana Valeria Hurtado
  11. Lifelong Graph Learning
    Supervisor: Martin Büchner
  12. 3D Neural Scene Representations for Visuomotor Control
    Supervisor: Martin Büchner


If you have any questions, please direct them to Martin Büchner before the topic allotment, and to your supervisor after you have received your topic.