Math lasts forever, even in the AI era.
The best way to attack Math is to write it down and think twice yourself.
This course is designed to cover the in-depth math knowledge essential for research in Robotics & State Estimation, with direct applications to:
- Robot perception and navigation
- Simultaneous Localization and Mapping (SLAM)
- Structure from Motion (SfM)
- Visual-Inertial Odometry (VIO)
- Multi-sensor fusion & calibration
| # | Topic | Contents | Instructor | Recording | |
|---|---|---|---|---|---|
| 1 | Basics | probability, estimator, linear system | Yulin Yang | Video · 91 min | |
| 2 | 3D Geometry | rotation, Lie group (SE(3), SE2(3)) | Yulin Yang | Video · 121 min | |
| 3 | Visual SLAM I | camera model, point / line / plane | Yulin Yang | Video · 60 min | |
| 4 | Visual SLAM II | information, marginalization and pipeline | Yulin Yang | Video · 63 min | |
| 5 | IMU | IMU model and integration | Yulin Yang | Video · 52 min | |
| 6 | Kalman Filter For SLAM | KF and observability analysis, KF-based VINS | Yulin Yang | Video · 112 min | |
| 7 | Batch Optimization For SLAM | IMU pre-integration based VINS, multi-visual-inertial | Yulin Yang | Video · 78 min | |
| 8 | Information Theory for Estimation | TBD | Chuchu Chen | — | — |
| 9 | System Consistency | TBD | Chuchu Chen | — | — |
| 10 | Learning-based SLAM | dense mapping | Xingxing Zuo | Video · 83 min | Slides |
See resources/ for the full reading list and links.
Lecture 2 — rotation, Lie groups, SE(3), quaternions
- Barfoot, State Estimation for Robotics, Cambridge Univ. Press, 2017 — Ch. 7 (matrix Lie groups): http://asrl.utias.utoronto.ca/~tdb/bib/barfoot_ser17.pdf
- Dellaert et al., Lie Groups for 2D and 3D Transformations (GTSAM): https://github.com/borglab/gtsam/blob/develop/doc/LieGroups.pdf
- Atanasov, Rotations SO(3) and Rigid-Body Motions SE(3) (ECE276A, UCSD, 2020): https://natanaso.github.io/ece276a2020/ref/ECE276A_12_SO3_SE3.pdf
- Solà, Deray & Atchuthan, A Micro Lie Theory for State Estimation in Robotics, 2018: https://arxiv.org/abs/1812.01537
- Trawny & Roumeliotis, Indirect Kalman Filter for 3D Attitude Estimation, UMN TR 2005-002 — Sec. 1: https://mediawiki.isr.tecnico.ulisboa.pt/images/d/db/Indirect_Kalman_Filter_for_3D_Attitude_Estimation.pdf
- Solà, Quaternion Kinematics for the Error-State Kalman Filter, 2017 — Sec. 1–3: https://arxiv.org/abs/1711.02508
Lecture 3 — camera model, calibration, point / line / plane
- Yang & Huang, Observability Analysis of Aided INS with Heterogeneous Features of Points, Lines and Planes, IEEE T-RO 2019: https://arxiv.org/abs/1805.05876
- Yang & Huang, Aided Inertial Navigation: Unified Feature Representations and Observability Analysis, ICRA 2019: https://ieeexplore.ieee.org/document/8793507
- Yang, Geneva, Eckenhoff & Huang, Visual-Inertial Navigation with Point and Line Features, IROS 2019: https://pgeneva.com/downloads/papers/Yang2019IROS.pdf
- Yang, Geneva, Zuo, Eckenhoff, Liu & Huang, Tightly-Coupled Aided Inertial Navigation with Point and Plane Features, ICRA 2019: https://ieeexplore.ieee.org/document/8794078
- Yang, Geneva, Eckenhoff & Huang, Degenerate Motion Analysis for Aided INS with Online Spatial and Temporal Sensor Calibration, RA-L 2019: https://ieeexplore.ieee.org/document/8616792
- Yang, Geneva, Zuo & Huang, Online Self-Calibration for Visual-Inertial Navigation Systems: Models, Analysis and Degeneracy, IEEE T-RO 2023: https://arxiv.org/abs/2201.09170
Lecture 4 — information, marginalization, pipeline
- Yang, Maley & Huang, Null-Space-based Marginalization: Analysis and Algorithm, IROS 2017: https://ieeexplore.ieee.org/document/8206592
- Geneva, Eckenhoff, Lee, Yang & Huang, OpenVINS: A Research Platform for Visual-Inertial Estimation, ICRA 2020: https://ieeexplore.ieee.org/document/9196524 · https://docs.openvins.com
Lecture 5 — IMU model and integration
- Yang, Geneva, Zuo & Huang, Online IMU Intrinsic Calibration: Is It Necessary?, RSS 2020: https://www.roboticsproceedings.org/rss16/p026.pdf
- Yang, Geneva, Zuo & Huang, Online Self-Calibration for Visual-Inertial Navigation Systems, IEEE T-RO 2023 — IMU model: https://arxiv.org/abs/2201.09170
- Yang, Babu, Chen, Huang & Ren, Analytic Combined IMU Integration (ACI²) for Visual-Inertial Navigation, ICRA 2020: https://ieeexplore.ieee.org/document/9197280
- Yang, Chen, Lee & Huang, Decoupled Right Invariant Error States for Consistent Visual-Inertial Navigation, RA-L 2022 — SE₂(3) error states: https://www.researchgate.net/publication/357588032_Decoupled_Right_Invariant_Error_States_for_Consistent_Visual-Inertial_Navigation
Lecture 6 — Kalman filter, observability, KF-based VINS
- Yang & Huang, Aided Inertial Navigation with Geometric Features: Observability Analysis, ICRA 2018: https://ieeexplore.ieee.org/document/8460670 · TR: https://udel.edu/~ghuang/papers/tr_obs.pdf
- Yang & Huang, Observability Analysis of Aided INS with Heterogeneous Features of Points, Lines and Planes, IEEE T-RO 2019: https://arxiv.org/abs/1805.05876
- Geneva, Eckenhoff, Lee, Yang & Huang, OpenVINS, ICRA 2020 — EKF-based VINS reference implementation: https://docs.openvins.com
Lecture 7 — preintegration, graph VINS, multi-visual-inertial
- Yang, Babu, Chen, Huang & Ren, Analytic Combined IMU Integration (ACI²), ICRA 2020: https://ieeexplore.ieee.org/document/9197280
- Yang, Geneva & Huang, Multi-Visual-Inertial System: Analysis, Calibration and Estimation, IJRR 2024: https://arxiv.org/abs/2308.05303
- One lecture per week.
- Bring a pen and paper — follow along with the equation derivations.
- Video recordings and notes are shared after each lecture. Please do not redistribute (e.g., to YouTube) for now.
- Discussions and office hours are held on GitHub: https://github.com/yangyulin/rise-tutorial/discussions
Weekly on Sunday:
| Location | Local time |
|---|---|
| Seattle | 6:30 AM – 7:45 AM PDT |
| Washington DC | 9:30 AM – 10:45 AM EDT |
| Abu Dhabi | 5:30 PM – 6:45 PM |
| Beijing | 9:30 PM – 10:45 PM |