Annotation Cross-Labeling For Autonomous Control Systems

What is this patent about?

’434 is related to the field of autonomous vehicle systems and, more specifically, to the training of computer models used in such systems. Autonomous vehicles rely on sensors like cameras and LIDAR to perceive their environment. Training the computer models that interpret this sensor data requires accurately labeled data, a process that can be time-consuming and computationally expensive, especially for 3D data like LIDAR point clouds.

The underlying idea behind ’434 is to leverage readily available and easily annotated 2D image data to assist in the annotation of more complex 3D sensor data. The system uses the 2D annotation of an object in an image, along with the camera's viewpoint, to project a spatial region into the 3D space of the LIDAR data. This region then serves as a focused search area for annotating the corresponding object in the 3D point cloud.

The claims of ’434 focus on a method, system, and storage medium for annotating sensor measurements. Specifically, the claims cover obtaining a 2D image with an object annotated, obtaining 3D sensor measurements of the same scene, determining a spatial region in the 3D space corresponding to the 2D annotation and the camera's viewpoint, and then annotating the 3D sensor measurements within that spatial region to locate the object in 3D.

In practice, the system would first identify an object in a 2D camera image and create a bounding box around it. Knowing the camera's position and orientation, the system then projects a viewing frustum from the camera's perspective into the 3D LIDAR point cloud. This frustum defines a limited volume in the 3D space where the corresponding object is likely to be found. An annotation model is then applied only to the data within this frustum, significantly reducing the computational burden and improving annotation accuracy.

This approach differs from prior methods that either rely on manual 3D annotation, which is labor-intensive, or apply annotation models to the entire 3D point cloud, which is computationally expensive and prone to errors due to the sparsity of LIDAR data. By using the 2D image annotation as a guide, ’434 effectively narrows down the search space in the 3D data, leading to more efficient and accurate 3D annotation for training autonomous vehicle systems.