Open design autonomous vehicle navigation
Open design autonomous vehicle navigation refers to the development of navigation and control systems for autonomous vehicles using open design, open source software, open standards, openly documented hardware, and reproducible research. The goal is to develop systems that allow vehicles to understand where they are, perceive their environment, determine where they should go, plan a safe route, and control the vehicle while making the technical design available for study and improvement.
Autonomous navigation can apply to:
|
|
The exact technology differs between a highway vehicle, airplane, warehouse robot, and drone, but many of the underlying problems are similar.
A useful simplified model is:
sense → locate → understand → plan → control → move → measure again
This process may repeat many times each second.
Major parts of autonomous navigation
A complete autonomous navigation system can contain several interacting systems.
|
|
No single component makes a vehicle autonomous. Reliable autonomy generally requires several systems working together.
Sensors
An autonomous vehicle needs information about itself and its surroundings.
Common sensors include:
- Cameras.
- LiDAR.
- Radar.
- Ultrasonic sensors.
- GNSS receivers.
- Inertial measurement units.
- Wheel encoders.
- Vehicle speed sensors.
- Barometers and air-data sensors for aircraft.
Different sensors have different strengths.
Cameras provide rich visual information but can be affected by lighting and weather. Radar can measure distance and relative velocity and can operate in conditions where cameras may perform less effectively. LiDAR can provide detailed three-dimensional measurements of the surrounding environment.
Using several sensor types together is called sensor fusion.
Localization
Before choosing where to go, a vehicle needs to estimate where it currently is.
Localization can use:
- GNSS.
- Inertial navigation.
- Maps.
- Wheel odometry.
- LiDAR matching.
- Visual odometry.
- Simultaneous localization and mapping.
- Combinations of several methods.
GNSS alone may not always provide sufficient reliability or accuracy, especially indoors, underground, between tall buildings, or where satellite signals are unavailable.
Autonomous systems can therefore combine several sources of information.
For aircraft, inertial navigation and satellite navigation are particularly important. Ground robots may rely heavily on LiDAR, cameras, wheel encoders, and maps.
Mapping
Autonomous navigation can use several kinds of maps.
A road vehicle might use information about:
- Roads.
- Lanes.
- Intersections.
- Traffic signals.
- Speed limits.
- Curbs.
- Crosswalks.
Robots may use occupancy maps showing which areas are free or obstructed.
Some autonomous vehicles depend on detailed previously created maps. Others attempt to understand the environment primarily from sensors in real time.
An open design system should ideally document the map formats and allow users to create, modify, and exchange maps.
Computer vision
Computer vision allows computers to extract useful information from cameras and other visual sensors.
An autonomous vehicle may use computer vision to identify:
- Vehicles.
- Pedestrians.
- Bicycles.
- Traffic lights.
- Road boundaries.
- Lane markings.
- Signs.
- Obstacles.
- Drivable areas.
Modern computer vision often relies heavily on machine learning and neural networks.
The system does not simply need to recognize that an object exists. It may also need to estimate where the object is, how quickly it is moving, and what it may do next.
Perception
Perception creates a useful model of the surrounding environment.
Data from cameras, radar, LiDAR, maps, and other sensors can be combined into information such as:
- There is a vehicle 30 meters ahead.
- A pedestrian is entering the road.
- A traffic light is red.
- The current lane bends to the left.
- An obstacle blocks part of the planned route.
Perception is one of the areas where artificial intelligence has become especially important.
Reliable perception must also deal with unusual conditions such as rain, darkness, construction zones, unusual vehicles, faded road markings, and objects not represented well in training data.
Prediction
Autonomous navigation involves predicting how other objects may move.
A vehicle might estimate whether another car will:
- Continue straight.
- Turn.
- Change lanes.
- Slow down.
- Stop.
Predicting human behavior is difficult because people do not always behave consistently.
A good navigation system therefore needs to account for uncertainty rather than assuming that every road user will behave exactly as expected.
Route and path planning
Route planning chooses a broad route toward a destination.
Path planning determines a more specific path through the environment.
Motion planning can also consider velocity, acceleration, turning radius, obstacles, and vehicle dynamics.
For example, a car may know that its route requires taking the next exit. The motion planner must still determine how and when to change lanes safely.
Robotic navigation systems often maintain a global plan toward the destination and a local plan for avoiding nearby obstacles.
Vehicle control
Once a trajectory is selected, the vehicle must follow it.
A car controller may command:
- Steering.
- Acceleration.
- Braking.
An aircraft controller may command:
- Roll.
- Pitch.
- Yaw.
- Throttle.
- Control surfaces.
Control systems continually compare the desired motion with actual vehicle motion and make corrections.
Techniques such as PID control and model predictive control can be used depending on the vehicle and application.
Artificial intelligence
AI can appear throughout an autonomous navigation system.
Possible applications include:
- Object recognition.
- Road understanding.
- Prediction.
- Route selection.
- Trajectory generation.
- Driver monitoring.
- Sensor interpretation.
- End-to-end driving models.
Traditional autonomous systems frequently divide navigation into separate modules such as perception, localization, planning, and control.
Newer approaches increasingly investigate end-to-end systems in which neural networks learn larger portions of the driving process directly from data.
Modular and end-to-end approaches each create different research questions concerning performance, interpretability, training, verification, and safety.
Open source autonomous driving
Several major open-source projects provide platforms for studying vehicle autonomy.
Autoware
Autoware is an open-source autonomous driving project supported by the Autoware Foundation.
It provides a modular architecture containing functions for:
- Sensing.
- Perception.
- Localization.
- Planning.
- Control.
Autoware is based on the Robot Operating System ecosystem and is intended for research as well as real-world autonomous vehicle applications.
Apollo
Apollo is an open autonomous driving platform originally developed by Baidu.
Its platform includes modules for:
- Localization.
- Perception.
- Prediction.
- Planning.
- Control.
- High-definition mapping.
Apollo uses its Cyber RT runtime framework for communication and execution of autonomous driving components.
openpilot
openpilot is an open-source advanced driver-assistance system developed by comma.ai.
It can provide functions such as adaptive cruise control and automated lane centering on supported production vehicles.
It should be distinguished from a fully autonomous vehicle system. Its current design requires a human driver to remain attentive and capable of taking control.
Open navigation for robots
Nav2 is a major open-source navigation framework for ROS 2 mobile robots.
Nav2 provides tools for:
- Localization.
- Mapping integration.
- Costmaps.
- Global planning.
- Local planning.
- Obstacle avoidance.
- Behavior trees.
- Navigation task execution.
A small autonomous rover can therefore provide a practical environment for learning many of the same concepts used in larger autonomous vehicles without beginning with a full-size automobile.
Autonomous aircraft
Open navigation is also extensively used in aircraft and drones.
ArduPilot is an open-source autopilot system that supports fixed-wing aircraft, multicopters, helicopters, rovers, boats, submarines, and other vehicles.
An autonomous aircraft may use:
- GNSS.
- Inertial sensors.
- Barometric altitude.
- Airspeed sensors.
- Magnetometers.
- Cameras.
- Mission waypoints.
- Flight-control algorithms.
Aircraft autonomy creates different constraints from road navigation because the system must manage three-dimensional motion, flight stability, altitude, weather, and limited opportunities to stop if something goes wrong.
Hardware
Open autonomous navigation can involve both general-purpose computers and dedicated control hardware.
Possible hardware includes:
|
|
A system may use a powerful computer for AI perception while a separate embedded controller handles safety-critical low-level vehicle control.
Simulation
Testing autonomy entirely on real vehicles would be expensive and potentially dangerous.
Simulation allows developers to test many situations before physical deployment.
CARLA is an open-source simulator developed for autonomous driving research. It can simulate roads, traffic, pedestrians, weather conditions, cameras, LiDAR, GPS, and other sensors.
AWSIM provides another simulation environment developed particularly for testing Autoware systems.
Simulation can be useful for:
- Software development.
- Sensor testing.
- AI training.
- Testing unusual traffic situations.
- Replaying failures.
- Evaluating planning algorithms.
Simulation cannot perfectly reproduce reality, so physical testing remains necessary before relying on a system in real-world conditions.
Open design and reproducibility
A genuinely open autonomous navigation project can publish more than source code.
Useful materials include:
- Software source code.
- Hardware schematics.
- Sensor specifications.
- Vehicle interface documentation.
- Maps.
- Simulation scenarios.
- Training methods.
- Test datasets.
- Safety requirements.
- Experimental results.
This can make it possible for researchers to reproduce experiments and compare alternative approaches.
Open development can also make autonomous navigation useful as a platform for teaching robotics, AI, computer vision, controls, and transportation engineering.
Discussion questions, essay ideas, and learning related AI prompt ideas
- What components are necessary for an autonomous vehicle navigation system?
- When is computer vision sufficient, and when are LiDAR or radar useful?
- How should a vehicle navigate when GNSS becomes unavailable?
- What is the difference between route planning, path planning, and motion planning?
- What advantages do modular autonomous driving architectures provide?
- What are the advantages and limitations of end-to-end AI driving systems?
- How can autonomous vehicles estimate what pedestrians or other drivers will do?
- Why is simulation important before real-world vehicle testing?
- Compare Autoware, Apollo, Nav2, ArduPilot, and openpilot according to their intended applications.
- Ask an AI system to describe the information flow from a vehicle camera to a steering command. Verify the major technical claims.
- Design a simulated autonomous rover that travels between several locations while avoiding obstacles.
- Research how aircraft navigation differs from autonomous automobile navigation.
- What parts of an autonomous vehicle should remain operational if the main AI computer fails?
- How can open-source autonomous vehicle software improve learning and research?
- Could common open standards allow autonomous navigation software to operate across many different types of vehicles?
Readings
Wikipedia
- Self-driving car
- Autonomous robot
- Robot navigation
- Computer vision
- Simultaneous localization and mapping
- Pathfinding
- Motion planning
- Sensor fusion
- LiDAR
- Radar
- Inertial measurement unit
- Robot Operating System
- Unmanned aerial vehicle
- Autopilot
- Advanced driver-assistance system