Course Description

Title: IFT 6757 Autonomous Vehicles (aka. Duckietown)

Instructor: Liam Paull - Office AA3341 (but probably best contact me some other way)

Class TAs: Charlie Gauthier, Miguel Saavedra


Lab Space: AA3256


Office Hours (in AA3256):

Liam: Monday’s 2-3pm

Charlie: Wednesday’s 3-4pm

Miguel: Wednesday’s 10-11am

Background and Description:

Self-driving vehicles are poised to become one of the most pervasive and impactful applications of autonomy, and have received a great deal of attention recently.

This course considers problems in perception, navigation, planning and control, and their systems-level integration in the context of self-driving vehicles through an open-source curriculum for autonomy education that emphasizes hands-on experience. Integral to the course, students will collaborate to implement concepts covered in lecture on a low-cost autonomous vehicle with the goal of navigating a model town complete with roads, signage, traffic lights, obstacles, and citizens.

Important Note 1: Every student will be given their own robot to personalize and love for the semester.

Important Note 2: This is a completely open source class. As a result, this class will never be the same twice since we always build on what already exists. Students who do a great job have the potential that their work will become the new repository standard for others to try and beat in subsequent iterations.


The teaching objectives that we most care about are:

  • Have the students learned the basic abstractions that comprise an autonomous vehicle? For example, if they pick up a new research paper they be able to put it in context.

  • Do the students understand how methods from heterogeneous disciplines such as control theory, machine learning, computer vision, and artificial intelligence are integrated together to create a complex autonomous system?

  • Have the students familiarized themselves with the standard tools that are used in the class, such as Git, ROS, Docker, and others?


The course will cover the theory and application of probabilistic techniques for autonomous mobile robotics with particular emphasis on their application in the context of self-driving vehicles. Topics include probabilistic state estimation and decision making for mobile robots; stochastic representations of the environment; dynamics models and sensor models for mobile robots; algorithms for mapping and localization; planning and control in the presence of uncertainty; cooperative operation of multiple mobile robots; mobile sensor networks; deep learning for perception; imitation from expert trajectories; reinforcement learning.

Following is a list of topics discussed in the class:

  • Autonomy architectures

  • Sensors, models, and representations (projective geometry, kinematics/dynamics)

  • Computer vision (intrinsic/extrinsic calibration, illumination invariance, feature extraction, line detection, place recognition)

  • Nonlinear filtering and state estimation (Bayes filter, Kalman filter, particle filter, SLAM)

  • Navigation and planning (mission planning, motion planning and control basics)

  • Complex perception pipelines (use of object detection, reading traffic signs, and tracking)

  • Tools for making robots work (Docker, ROS, Git, network basics)

  • Reinforcement learning and sim2real transfer

  • Deep learning for perception


Meeting Times: M 10:30-12:30, W 11:30am–1:30pm

Location: M Z-337, W Z-260

Pre-requisites: Permission of the instructor. Please come to the first class and fill out an application and/or email the instructor to discuss.

Intended Enrollment: TBD

Intended Degree Level: Senior Undergraduates and Graduates

Tentative Grading Scheme:

  • Exercises (best 5 of 6) - 25%

    • Setup, perception, estimation, planning, control, end-to-end

    • Simulator version submitted to challenge server (2/5)

    • Hardware implementation and demonstration (2/5)

    • Inspection of your code (1/5)

  • Project - 45% (Will be in groups):

    • Design Documents and Presentations - 10%

    • Implementation and final demo - 25%

    • Final Report - 10%

  • 30 min Seminar - 15%

  • “Being a Good Citizen” - 15%

    • Feedback on others’ design reports and seminars - 5%

    • Responding to questions, filing issues, fixing documentation etc - 5%

    • Allocated by by the TAs and Professor - 5%

Late Policy: Exercises are due on Sunday. -10% for two days and then 0% after. You should demonstrate the functionality of your agent to one of the course staff in the office hours.