Publications

2020

The AI Driving Olympics at NeurIPS 2018 Julian Zilly, Jacopo Tani, Breandan Considine, Bhairav Mehta, Andrea F Daniele, Manfred Diaz, Gianmarco Bernasconi, Claudio Ruch, Jan Hakenberg, Florian Golemo, and others 2020 BibTeX

MapLite: Autonomous Intersection Navigation without a Detailed Prior Map Teddy Ort, Krishna Murthy, Rohan Banerjee, Sai Krishna Gottipati, Dhaivat Bhatt, Igor Gilitschenski, Liam Paull, and Daniela Rus IEEE Robotics and Automation Letters 2020 BibTeX

2019

A Data-Efficient Framework for Training and Sim-to-Real Transfer of Navigation Policies Homanga Bharadhwaj, Zihan Wang, Yoshua Bengio, and Liam Paull 2019 Abstract arXiv BibTeX

Learning effective visuomotor policies for robots purely from data is challenging, but also appealing since a learning-based system should not require manual tuning or calibration. In the case of a robot operating in a real environment the training process can be costly, time-consuming, and even dangerous since failures are common at the start of training. For this reason, it is desirable to be able to leverage \textitsimulation and \textitoff-policy data to the extent possible to train the robot. In this work, we introduce a robust framework that plans in simulation and transfers well to the real environment. Our model incorporates a gradient-descent based planning module, which, given the initial image and goal image, encodes the images to a lower dimensional latent state and plans a trajectory to reach the goal. The model, consisting of the encoder and planner modules, is trained through a meta-learning strategy in simulation first. We subsequently perform adversarial domain transfer on the encoder by using a bank of unlabelled but random images from the simulation and real environments to enable the encoder to map images from the real and simulated environments to a similarly distributed latent representation. By fine tuning the entire model (encoder + planner) with far fewer real world expert demonstrations, we show successful planning performances in different navigation tasks.

Deep Active Localization Sai Krishna, Keehong Seo, Dhaivat Bhatt, Vincent Mai, Krishna Murthy, and Liam Paull 2019 Abstract arXiv Code BibTeX

Active localization is the problem of generating robot actions that allow it to maximally disambiguate its pose within a reference map. Traditional approaches to this use an information-theoretic criterion for action selection and hand-crafted perceptual models. In this work we propose an end-to-end differentiable method for learning to take informative actions that is trainable entirely in simulation and then transferable to real robot hardware with zero refinement. The system is composed of two modules: a convolutional neural network for perception, and a deep reinforcement learned planning module. We introduce a multi-scale approach to the learned perceptual model since the accuracy needed to perform action selection with reinforcement learning is much less than the accuracy needed for robot control. We demonstrate that the resulting system outperforms using the traditional approach for either perception or planning. We also demonstrate our approaches robustness to different map configurations and other nuisance parameters through the use of domain randomization in training. The code is also compatible with the OpenAI gym framework, as well as the Gazebo simulator.

Kotlin∇: A shape-safe DSL for differentiable programming Breandan Considine, Michalis Famelis, and Liam Paull 2019 BibTeX

gradSLAM: Dense SLAM meets Automatic Differentiation Krishna Murthy Jatavallabhula, Ganesh Iyer, and Liam Paull arXiv preprint arXiv:1910.10672 2019 BibTeX

2018

Probabilistic cooperative mobile robot area coverage and its application to autonomous seabed mapping Liam Paull, Mae Seto, John J Leonard, and Howard Li The International Journal of Robotics Research 2018 Abstract BibTeX

There are many applications that require mobile robots to autonomously cover an entire area with a sensor or end effector. The vast majority of the literature on this subject is focused on addressing path planning for area coverage under the assumption that the robot’s pose is known or that error is bounded. In this work, we remove this assumption and develop a completely probabilistic representation of coverage. We show that coverage is guaranteed as long as the robot pose estimates are consistent, a much milder assumption than zero or bounded error. After formally connecting robot sensor uncertainty with area coverage, we propose an adaptive sliding window filter pose estimator that provides a close approximation to the full maximum a posteriori estimate with a computation cost that is bounded over time. Subsequently, an adaptive planning strategy is presented that automatically exploits conditions of low vehicle uncertainty to more efficiently cover an area. We further extend this approach to the multi-robot case where robots can communicate through a (possibly faulty and low-bandwidth) channel and make relative measurements of one another. In this case, area coverage is achieved more quickly since the uncertainty over the robots’ trajectories is reduced. We apply the framework to the scenario of mapping an area of seabed with an autonomous underwater vehicle. Experimental results support the claim that our method achieves guaranteed complete coverage notwithstanding poor navigational sensors and that resulting path lengths required to cover the entire area are shortest using the proposed cooperative and adaptive approach.

Autonomous Vehicle Navigation in Rural Environments without Detailed Prior Maps Teddy Ort, Liam Paull, and Daniela Rus In IEEE International Conference on Robotics and Automation (ICRA) 2018 Abstract arXiv BibTeX

State-of-the-art autonomous driving systems rely heavily on detailed and highly accurate prior maps. However, outside of small urban areas, it is very challenging to build, store, and transmit detailed maps since the spatial scales are so large. Furthermore, maintaining detailed maps of large rural areas can be impracticable due to the rapid rate at which these environments can change. This is a significant limitation for the widespread applicability of autonomous driving technology, which has the potential for an incredibly positive societal impact. In this paper, we address the problem of autonomous navigation in rural environments through a novel mapless driving framework that combines sparse topological maps for global navigation with a sensor-based perception system for local navigation. First, a local navigation goal within the sensor view of the vehicle is chosen as a waypoint leading towards the global goal. Next, the local perception system generates a feasible trajectory in the vehicle frame to reach the waypoint while abiding by the rules of the road for the segment being traversed. These trajectories are updated to remain in the local frame using the vehicle’s odometry and the associated uncertainty based on the least-squares residual and a recursive filtering approach, which allows the vehicle to navigate road networks reliably, and at high speed, without detailed prior maps. We demonstrate the performance of the system on a full-scale autonomous vehicle navigating in a challenging rural environment and benchmark the system on a large amount of collected data.

Local Positioning System Using UWB Range Measurements for an Unmanned Blimp Vincent Mai, Mina Kamel, Matthias Krebs, Andreas Schaffner, Daniel Meier, Liam Paull, and Roland Siegwart IEEE Robotics and Automation Letters 2018 Abstract arXiv BibTeX

Unmanned blimps are a safe and reliable alternative to conventional drones when flying above people. On-board real-time tracking of their pose and velocities is a necessary step toward autonomous navigation. There is a need for an easily deployable technology that is able to accurately and robustly estimate the pose and velocities of a blimp in 6 DOF, as well as unexpected applied forces and torques, in an uncontrolled environment. We present two multiplicative extended Kalman filters using ultrawideband radio sensors and a gyroscope to address this challenge. One filter is updated using a dynamics model of the blimp, whereas the other uses a constant speed model. We describe a set of experiments in which these estimators have been implemented on an embedded flight controller. They were tested and compared in accuracy and robustness in a hardware-in-loop simulation as well as on a real blimp. This approach can be generalized to any lighter than air robot to track it with the necessary accuracy, precision, and robustness to allow autonomous navigation.

Geometric Consistency for Self-Supervised End-to-End Visual Odometry Ganesh Iyer, J Krishna Murthy, K Gunshi Gupta, and Liam Paull In CVPR Workshop on Deep Learning for Visual SLAM 2018 Abstract arXiv Project Page BibTeX

With the success of deep learning based approaches in tackling challenging problems in computer vision, a wide range of deep architectures have recently been proposed for the task of visual odometry (VO) estimation. Most of these proposed solutions rely on supervision, which requires the acquisition of precise ground-truth camera pose information, collected using expensive motion capture systems or high-precision IMU/GPS sensor rigs. In this work, we propose an unsupervised paradigm for deep visual odometry learning. We show that using a noisy teacher, which could be a standard VO pipeline, and by designing a loss term that enforces geometric consistency of the trajectory, we can train accurate deep models for VO that do not require ground-truth labels. We leverage geometry as a self-supervisory signal and propose "Composite Transformation Constraints (CTCs)", that automatically generate supervisory signals for training and enforce geometric consistency in the VO estimate. We also present a method of characterizing the uncertainty in VO estimates thus obtained. To evaluate our VO pipeline, we present exhaustive ablation studies that demonstrate the efficacy of end-to-end, self-supervised methodologies to train deep models for monocular VO. We show that leveraging concepts from geometry and incorporating them into the training of a recurrent neural network results in performance competitive to supervised deep VO methods.

Learning steering bounds for parallel autonomous systems Alexander Amini, Liam Paull, Thomas Balch, Sertac Karaman, and Daniela Rus In IEEE International Conference on Robotics and Automation (ICRA) 2018 Abstract arXiv BibTeX

Deep learning has been successfully applied to “end-to-end” learning of the autonomous driving task, where a deep neural network learns to predict steering control commands from camera data input. While these previous works support reactionary control, the representation learned is not usable for higher-level decision making required for autonomous navigation. This paper tackles the problem of learning a representation to predict a continuous control probability distribution, and thus steering control options and bounds for those options, which can be used for autonomous navigation. Each mode of the distribution encodes a possible macro-action that the system could execute at that instant, and the covariances of the modes place bounds on safe steering control values. Our approach has the added advantage of being trained on unlabeled data collected from inexpensive cameras. The deep neural network based algorithm generates a probability distribution over the space of steering angles, from which we leverage Variational Bayesian methods to extract a mixture model and compute the different possible actions in the environment. A bound, which the autonomous vehicle must respect in our parallel autonomy setting, is then computed for each of these actions. We evaluate our approach on a challenging dataset containing a wide variety of driving conditions, and show that our algorithm is capable of parameterizing Gaussian Mixture Models for possible actions, and extract steering bounds with a mean error of only 2 degrees. Additionally, we demonstrate our system working on a full scale autonomous vehicle and evaluate its ability to successful handle various different parallel autonomy situations.

2017

Duckietown: an open, inexpensive and flexible platform for autonomy education and research Liam Paull, Jacopo Tani, Heejin Ahn, Javier Alonso-Mora, Luca Carlone, Michal Cap, Yu Fan Chen, Changhyun Choi, Jeff Dusek, Yajun Fang, and others In IEEE International Conference on Robotics and Automation (ICRA) 2017 Abstract arXiv BibTeX

Duckietown is an open, inexpensive and flexible platform for autonomy education and research. The platform comprises small autonomous vehicles (“Duckiebots”) built from off-the-shelf components, and cities (“Duckietowns”) complete with roads, signage, traffic lights, obstacles, and citizens (duckies) in need of transportation. The Duckietown platform offers a wide range of functionalities at a low cost. Duckiebots sense the world with only one monocular camera and perform all processing onboard with a Raspberry Pi 2, yet are able to: follow lanes while avoiding obstacles, pedestrians (duckies) and other Duckiebots, localize within a global map, navigate a city, and coordinate with other Duckiebots to avoid collisions. Duckietown is a useful tool since educators and researchers can save money and time by not having to develop all of the necessary supporting infrastructure and capabilities. All materials are available as open source, and the hope is that others in the community will adopt the platform for education and research.

Hybrid control and learning with coresets for autonomous vehicles Guy Rosman, Liam Paull, and Daniela Rus In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2017 Abstract arXiv BibTeX

Modern autonomous systems such as driverless vehicles need to safely operate in a wide range of conditions. A potential solution is to employ a hybrid systems approach, where safety is guaranteed in each individual mode within the system. This offsets complexity and responsibility from the individual controllers onto the complexity of determining discrete mode transitions. In this work we propose an efficient framework based on recursive neural networks and coreset data summarization to learn the transitions between an arbitrary number of controller modes that can have arbitrary complexity. Our approach allows us to efficiently gather annotation data from the large-scale datasets that are required to train such hybrid nonlinear systems to be safe under all operating conditions, favoring underexplored parts of the data. We demonstrate the construction of the embedding, and efficient detection of switching points for autonomous and nonautonomous car data. We further show how our approach enables efficient sampling of training data, to further improve either our embedding or the controllers

Two-stage focused inference for resource-constrained minimal collision navigation Beipeng Mu, Liam Paull, Ali-Akbar Agha-Mohammadi, John J Leonard, and Jonathan P How IEEE Transactions on Robotics 2017 BibTeX

Parallel autonomy in automated vehicles: Safe motion generation with minimal intervention Wilko Schwarting, Javier Alonso-Mora, Liam Paull, Sertac Karaman, and Daniela Rus In Robotics and Automation (ICRA), 2017 IEEE International Conference on 2017 BibTeX

A parallel autonomy research platform Felix Naser, David Dorhout, Stephen Proulx, Scott Drew Pendleton, Hans Andersen, Wilko Schwarting, Liam Paull, Javier Alonso-Mora, Marcelo H Ang, Sertac Karaman, and others In 2017 IEEE Intelligent Vehicles Symposium (IV) 2017 BibTeX

Safe nonlinear trajectory generation for parallel autonomy with a dynamic vehicle model Wilko Schwarting, Javier Alonso-Mora, Liam Paull, Sertac Karaman, and Daniela Rus IEEE Transactions on Intelligent Transportation Systems 2017 BibTeX

2016

A Unified Resource-Constrained Framework for Graph SLAM Liam Paull, Guoquan Huang, and John J Leonard In IEEE International Conference on Robotics and Automation (ICRA) 2016 Abstract arXiv Poster Slides Code BibTeX

Graphical methods have proven an extremely useful tool employed by the mobile robotics community to frame estimation problems. Incremental solvers are able to process incoming sensor data and produce maximum a posteriori (MAP) estimates in realtime by exploiting the natural sparsity within the graph for reasonable-sized problems. However, to enable truly longterm operation in prior unknown environments requires algorithms whose computation, memory, and bandwidth (in the case of distributed systems) requirements scale constantly with time and environment size. Some recent approaches have addressed this problem through a two-step process - first the variables selected for removal are marginalized which induces density, and then the result is sparsified to maintain computational efficiency. Previous literature generally addresses only one of these two components. In this work, we attempt to explicitly connect all of the aforementioned resource constraint requirements by considering the node removal and sparsification pipeline in its entirety. We formulate the node selection problem as a minimization problem over the penalty to be paid in the resulting sparsification. As a result, we produce node subset selection strategies that are optimal in terms of minimizing the impact, in terms of Kullback-Liebler divergence (KLD), of approximating the dense distribution by a sparse one. We then show that one instantiation of this problem yields a computationally tractable formulation. Finally, we evaluate the method on standard datasets and show that the KLD is minimized as compared to other commonly-used heuristic node selection techniques.

Slam with objects using a nonparametric pose graph Beipeng Mu, Shih-Yuan Liu, Liam Paull, John Leonard, and Jonathan P How In IEEE/RSJ International Conference onnIntelligent Robots and Systems (IROS) 2016 Abstract arXiv Video BibTeX

Mapping and self-localization in unknown environments are fundamental capabilities in many robotic applications. These tasks typically involve the identification of objects as unique features or landmarks, which requires the objects both to be detected and then assigned a unique identifier that can be maintained when viewed from different perspectives and in different images. The data association and simultaneous localization and mapping (SLAM) problems are, individually, well-studied in the literature. But these two problems are inherently tightly coupled, and that has not been well-addressed. Without accurate SLAM, possible data associations are combinatorial and become intractable easily. Without accurate data association, the error of SLAM algorithms diverge easily. This paper proposes a novel nonparametric pose graph that models data association and SLAM in a single framework. An algorithm is further introduced to alternate between inferring data association and performing SLAM. Experimental results show that our approach has the new capability of associating object detections and localizing objects at the same time, leading to significantly better performance on both the data association and SLAM problems than achieved by considering only one and ignoring imperfections in the other.

Information-based active SLAM via topological feature graphs Beipeng Mu, Matthew Giamou, Liam Paull, Ali-akbar Agha-mohammadi, John Leonard, and Jonathan How In 2016 IEEE 55th Conference on Decision and Control (CDC) 2016 BibTeX

Requirements for building an ontology for autonomous robots Behzad Bayat, Julita Bermejo-Alonso, Joel Carbonera, Tullio Facchinetti, Sandro Fiorini, Paulo Goncalves, Vitor AM Jorge, Maki Habib, Alaa Khamis, Kamilo Melo, and others Industrial Robot: An International Journal 2016 BibTeX

Decoupled, consistent node removal and edge sparsification for graph-based SLAM Kevin Eckenhoff, Liam Paull, and Guoquan Huang In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2016 BibTeX

Duckietown: an innovative way to teach autonomy Jacopo Tani, Liam Paull, Maria T Zuber, Daniela Rus, Jonathan How, John Leonard, and Andrea Censi In International Conference EduRobotics 2016 2016 BibTeX

2015

Communication-constrained multi-AUV cooperative SLAM Liam Paull, Guoquan Huang, Mae Seto, and John J Leonard In 2015 IEEE international conference on robotics and automation (ICRA) 2015 BibTeX

Occupancy grid map merging for multiple robot simultaneous localization and mapping Sajad Saeedi, Liam Paull, Michael Trentini, and Howard Li International Journal of Robotics and Automation 2015 BibTeX

Toward Object-based Place Recognition in Dense RGB-D Maps Ross Finman, Liam Paull, and John J Leonard In ICRA Workshop on Visual Place Recognition 2015 BibTeX

Two-Stage Focused Inference for Resource-Constrained Collision-Free Navigation Beipeng Mu, Ali-akbar Agha-mohammadi, Liam Paull, Mathew Graham, Jonathan How, and John Leonard In Robotics: Science and Systems 2015 BibTeX

An Overview of MIT-Olin’s Approach in the AUVSI RobotX Competition Arthur Anderson, Erin Fischell, Thom Howe, Tom Miller, Arturo Parrales-Salinas, Nick Rypkema, David Barrett, Michael Benjamin, Alex Brennen, Michael DeFillipo, and others In Field and Service Robotics 2015 BibTeX

Bridging Text Spotting and SLAM with Junction Features Hsueh-Cheng Wang, Chelsea Finn, Liam Paull, Michael Kaess, Ruth Rosenholtz, Seth Teller, and John Leonard In IROS 2015 BibTeX

Cooperative Area Coverage Liam Paull, Mae Seto, and John J Leonard In RSS Workshop on Principles of Multi-Robot Systems 2015 BibTeX

2014

Sensor-Driven Area Coverage for an Autonomous Fixed-Wing Unmanned Aerial Vehicle Liam Paull, Carl Thibault, Amr Nagaty, Mae Seto, and Howard Li IEEE Transactions on Cybernetics 2014 BibTeX

AUV Navigation and Localization: A Review Liam Paull, Sajad Saeedi, Mae Seto, and Howard Li IEEE Journal of Oceanic Engineering 2014 BibTeX

Group mapping: A topological approach to map merging for multiple robots Sajad Saeedi, Liam Paull, Michael Trentini, Mae Seto, and Howard Li IEEE Robotics & Automation Magazine 2014 BibTeX

Area coverage planning that accounts for pose uncertainty with an AUV seabed surveying application Liam Paull, Mae Seto, and Howard Li In 2014 IEEE International Conference on Robotics and Automation (ICRA) 2014 BibTeX

Map merging for multiple robots using Hough peak matching Sajad Saeedi, Liam Paull, Michael Trentini, Mae Seto, and Howard Li Robotics and Autonomous Systems 2014 BibTeX

Decentralized cooperative trajectory estimation for autonomous underwater vehicles Liam Paull, Mae Seto, and John J Leonard In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems 2014 BibTeX

Physical Words for Place Recognition in Dense RGB-D Maps Ross Finman, Thomas Whelan, Liam Paull, and John J Leonard In ICRA Workshop on Visual Place Recognition 2014 BibTeX

2013

Path planning for autonomous underwater vehicles Liam Paull, Sajad Saeedi, and Howard Li 2013 BibTeX

Introduction to autonomy for marine robots ML Seto, Liam Paull, and Sajad Saeedi 2013 BibTeX

Sensor driven online coverage planning for autonomous underwater vehicles Liam Paull, Sajad Saeedi, Mae Seto, and Howard Li IEEE/ASME Transactions on Mechatronics 2013 BibTeX

Robust Online Adaptive Sensor-Driven Survey Planning for Single and Multiple Autonomous Underwater Vehicles and Liam Paull 2013 BibTeX

2012

Map merging using Hough peak matching Sajad Saeedi, Liam Paull, Michael Trentini, Mae Seto, and Howard Li In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems 2012 BibTeX

Towards an ontology for autonomous robots Liam Paull, Gaetan Severac, Guilherme V Raffo, Julian Mauricio Angel, Harold Boley, Phillip J Durst, Wendell Gray, Maki Habib, Bao Nguyen, S Veera Ragavan, and others In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems 2012 BibTeX

Efficient map merging using a probabilistic generalized voronoi diagram Sajad Saeedi, Liam Paull, Michael Trentini, Mae Seto, and Howard Li In 2012 IEEE/RSJ international conference on intelligent robots and systems 2012 BibTeX

Sensor driven online coverage planning for autonomous underwater vehicles Liam Paull, Sajad SaeediGharahbolagh, Mae Seto, and Howard Li In Intelligent Robots and Systems (IROS), 2012 IEEE/RSJ International Conference on 2012 BibTeX

2011

A multi-agent framework with MOOS-IvP for autonomous underwater vehicles with sidescan sonar sensors Liam Paull, Sajad Saeedi, Mae Seto, and Howard Li In International Conference on Autonomous and Intelligent Systems 2011 BibTeX

Neural network-based multiple robot Simultaneous Localization and Mapping S. Saeedi, L. Paull, M. Trentini, and H Li In Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on 2011 BibTeX

Neural network-based multiple robot simultaneous localization and mapping Sajad Saeedi, Liam Paull, Michael Trentini, and Howard Li IEEE transactions on neural networks 2011 BibTeX

Multiple robot simultaneous localization and mapping Sajad Saeedi, Liam Paull, Michael Trentini, and Howard Li In 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems 2011 BibTeX

2010

A novel domestic electric water heater model for a multi-objective demand side management program Liam Paull, Howard Li, and Liuchen Chang Electric Power Systems Research 2010 BibTeX

A novel demand side management program using water heaters and particle swarm optimization Arnaldo Sepulveda, Liam Paull, Walid G Morsi, Howard Li, CP Diduch, and Liuchen Chang In 2010 IEEE Electrical Power & Energy Conference 2010 BibTeX

An information gain based adaptive path planning method for an autonomous underwater vehicle using sidescan sonar Liam Paull, Sajad Saeedi, Howard Li, and Vincent Myers In 2010 IEEE International Conference on Automation Science and Engineering 2010 BibTeX

Document classification using information theory and a fast back-propagation neural network Howard Li, Liam Paull, Yevgen Biletskiy, and Simon X Yang Intelligent Automation & Soft Computing 2010 BibTeX

2009

A water heater model for increased power system efficiency Liam Paull, Derek MacKay, Howard Li, and Liuchen Chang In Electrical and Computer Engineering, 2009. CCECE’09. Canadian Conference on 2009 BibTeX

Path planning for multiple Unmanned Aerial Vehicles using genetic algorithms Howard Li, Yi Fu, Khalid Elgazzar, and Liam Paull In 2009 Canadian Conference on Electrical and Computer Engineering 2009 BibTeX

Model of domestic water heaters for aggregated load control L Paull, and L Chang 2009 BibTeX

2008

The development of a fuzzy neural system for load forecasting Liam Paull, Howard Li, and Liuchen Chang In 2008 Canadian Conference on Electrical and Computer Engineering 2008 BibTeX