Estimating Object Properties Using Visual Image Data

What is this patent about?

’755 is related to the field of autonomous driving systems, specifically addressing the challenge of reducing the number and cost of sensors required for accurate environmental perception. Traditional autonomous vehicles rely on a suite of sensors, including cameras, radar, and lidar, to gather data about their surroundings. The patent aims to minimize the need for expensive emitting distance sensors like lidar by leveraging machine learning techniques.

The underlying idea behind ’755 is to train a machine learning model to estimate the distance of objects from a vehicle using only image data captured by a camera. This is achieved by initially training the model with data from both a camera and an emitting distance sensor (e.g., radar). The correlated output of the distance sensor is used as the ground truth to teach the model how to infer distance directly from visual cues in the camera images.

The claims of ’755 focus on a system and method where image data from a vehicle's camera is fed into a trained machine learning model. The model, having been trained with paired camera and emitting distance sensor data, then outputs the distance to an object in the image. Crucially, the model is designed to output the distance using only the image data , effectively replacing the need for a dedicated distance sensor during operation.

In practice, the system involves a two-stage process. First, a training phase uses a vehicle equipped with both a camera and a distance sensor (like radar) to collect synchronized data. This data is then used to train a machine learning model, such as a convolutional neural network, to associate visual features in the camera images with corresponding distance measurements from the radar. Once trained, this model can be deployed to other vehicles that only have a camera, allowing them to estimate object distances without needing the radar sensor.

This approach differentiates itself from prior solutions by reducing reliance on costly and complex sensor setups. Instead of directly measuring distance with dedicated sensors, the system learns to infer distance from visual information. This can lead to lower vehicle production costs, reduced sensor maintenance, and potentially lower bandwidth requirements for data processing. The trained model can also serve as a redundant distance data source , improving accuracy and fault tolerance even when used in conjunction with a dedicated distance sensor.