TRIPODS Data Science Boot Camp:
Matching Robot Simulations to Reality

Winter Break 2021, January 5-14

Abdeslam Boularias and Mridul Aanjaneya

Overview: In the field of simulation, computer scientists write programs that model the behavior of virtual worlds unfolding over time. Simulations underpin everything from the engaging imagery of video games and films to the predictions of complex scientific theories. Simulations are increasingly used in robotics as well, where they can be used to anticipate the rich dynamics of the physical world and plan accordingly. This tutorial is a brief exploration of how roboticists combine first principles from physics, machine learning techniques, and numerical methods to simulate realistic robotic systems. The area brings many opportunities for research across physics, mathematics, statistics and computer science.

We will start by presenting model-free methods that are used for controlling robots. The model-free methods are elegantly simple and easy to implement, but they suffer from their needs for large quantities of training data. To improve the situation, we can take advantage of the governing equations of motion, which are well-understood for many practical applications in robotics. In particular, in the second session, we present global optimization techniques that can be utilized to efficiently identify and reason with models of robotic systems, combining appropriate governing equations and parameter values. Advances in modern computing and interactive computer graphics will be highlighted in the third session to showcase just how far we have progressed in the last few decades in developing more expressive and powerful simulations. A key insight, which we explore in the fourth session, is the use of differentiable physics simulators, which naturally support inference to bring simulations into correspondence with observed behavior in robotic systems.

Research Methods: Students will be introduced to the basics of numerical solvers, global optimization techniques, and numerical methods for integrating differential equations. The vibrant field of “differentiable programming” will be introduced for automatically estimating unknown physical parameters. Methods will be discussed for estimating the derivatives analytically, and also through the use of automatic differentiation techniques. Hands-on exposure to the Taichi programming language will be provided, which is a convenient platform for writing differentiable simulators for a variety of phenomena.

Intended Audience: These sessions are aimed for all students with a curiosity about physics and programming. We welcome undergraduates and graduate students, and invite interested students from computer science, mathematics, statistics and physics and related disciplines. To follow along with the interactive sessions, students should have basic familiarity with Python and Matlab programming. Prior experience with numerical methods and physics simulation will be helpful, but is not necessary, as the sessions will be self-contained.

Schedule and Participation:

Meetings will be held on zoom. Please register at for the zoom session details. Meetings are Tuesday and Thursday, 4pm-5:30pm EST, January 5, 7, 12 and 14. Each session will include an interactive, hands-on component that’s best enjoyed with synchronous participation, although recordings will be available for those that cannot attend.

1. January 5, 4pm: Model-free Reinforcement Learning (Abdeslam Boularias)
– Brief introduction to RL
– Examples
– Implementing simple RL agents in Python: grid-world and inverted pendulum.
Watch the Recording (pass: K.Gn4!BW)

2. January 7, 4pm: Model Identification with Bayesian Optimization (Abdeslam Boularias)
– Overview of model identification problems in robotics
– Gaussian Processes (GPs) for Bayesian Optimization
– Implementing a GP in Matlab for model identification

3. January 12, 4pm: Interactive Computer Graphics and Physics Simulation (Mridul Aanjaneya)
– Show some demos of large-scale simulations.
– Show some demos of interactive applications.
– Present some mathematical details of underlying numerics.
– Provide Python code for playing with some numerical solvers.
– Provide Python code for integrating simple systems of equations.

4. January 14, 4pm: Differentiable Physics (Mridul Aanjaneya and Abdeslam Boularias)
– Introduce the basics of rigid body dynamics (this is a continuation of 1e) above).
– Talk about methods for handling collisions and contact.
– Introduce the Taichi differentiable programming language.
– Show an example using Taichi for model identification.
– Present work on a mobile robot for model identification and control.