Machine Learning Based Prediction Of Human Interactions With Autonomous Vehicles

What is this patent about?

’889 is related to the field of data analytics and, more specifically, to systems and methods for predicting how humans will interact with vehicles, particularly in the context of autonomous driving. Existing autonomous vehicle systems often struggle to accurately predict the behavior of pedestrians, cyclists, and other drivers, especially in complex urban environments. This deficiency can lead to unsafe driving conditions and hinder the widespread adoption of autonomous vehicles.

The underlying idea behind ’889 is to leverage the collective intelligence of human observers to train a machine learning model that can predict the state of mind and likely actions of road users. This is achieved by presenting human observers with images and video segments of road scenes, collecting their responses regarding the intentions and awareness of other road users in those scenes, and then using this data to train a supervised learning model . The trained model can then be used by an autonomous vehicle to predict the behavior of road users in real-time.

The claims of ’889 focus on a computer-implemented method for controlling an autonomous vehicle. The method involves receiving images of road scenes and user responses describing the state of mind of road users in those images. A training dataset of summary statistics is generated from the user responses and used to train a supervised learning model. The autonomous vehicle then receives a new image, predicts the state of mind of road users in the image using the trained model, and controls the vehicle based on this prediction.

In practice, the system captures video from a vehicle's camera and extracts frames or segments. These are presented to human observers, potentially with manipulations to highlight relevant aspects. The observers' responses, indicating their assessment of the road users' intentions (e.g., whether a pedestrian will cross the street), are aggregated into summary statistics. These statistics, along with the corresponding image data, are used to train a model, such as a deep convolutional neural network , to predict the state of mind of road users in new, unseen images.

This approach differs significantly from prior methods that rely solely on motion vectors or other kinematic data to predict behavior. By incorporating human judgment, the system can capture subtle cues and contextual information that are often missed by purely sensor-based approaches. The system can also be continuously improved by incorporating new data and feedback from human observers, leading to more accurate and reliable predictions of road user behavior and, ultimately, safer autonomous driving.