IP Verse

Data Pipeline And Deep Learning System For Autonomous Driving

Patent No. US11215999 (titled "Data Pipeline And Deep Learning System For Autonomous Driving") was filed by Tesla Inc on Jun 20, 2018.

What is this patent about?

’999 is related to the field of autonomous vehicle control systems, specifically those employing deep learning. The background involves using sensor data, such as images, as input to deep learning networks for tasks like object detection and path planning. Traditional systems often compress or downsample this data to make it compatible with the network, potentially reducing signal fidelity and requiring new conversion processes for different sensors.

The underlying idea behind ’999 is to improve the efficiency and accuracy of deep learning in autonomous vehicles by decomposing sensor images into multiple component images and feeding each component to a different layer of the neural network. This allows the network to process different aspects of the image data in parallel and at the most appropriate layer, maximizing the signal information available for each task.

The claims of ’999 focus on a method, a computer program product, and a system for receiving an image from a vehicle sensor, decomposing it into component images (at least two), providing each component image as input to a different layer of an artificial neural network, and using the network's result to autonomously operate the vehicle. Crucially, a first component image goes to a first layer , while a second component image and the output of the first layer go to a second, subsequent layer .

In practice, the sensor on the vehicle captures an image, which is then processed by an image signal processor to decompose it into components. For example, a high-pass filter might extract edge and feature data, while a low-pass filter extracts global illumination data. The edge data is fed into the first layer of the neural network, which is optimized for feature detection. The global illumination data, possibly downsampled for efficiency, is then combined with the output of the first layer and fed into a subsequent layer for higher-level reasoning.

This approach differs from prior methods that feed the entire sensor image into the first layer of the network. By separating the image into components and feeding them into different layers, the network can focus its computational resources on the most relevant information at each stage. This can lead to improved accuracy, reduced computational cost, and greater flexibility in adapting to different sensor types and environmental conditions. The multi-stage processing allows for a more complete analysis of the captured sensor data.

How does this patent fit in bigger picture?

Technical landscape at the time

In the late 2010s when ’817 was filed, autonomous driving systems commonly relied on sensor data as input for deep learning networks, at a time when converting sensor data to a format compatible with the initial input layer of the learning system was typically implemented using compression and down-sampling. When hardware or software constraints made maximizing the signal information from captured sensor data non-trivial.

Novelty and Inventive Step

The examiner approved the application because the available prior art, taken individually or together, does not teach providing each component image as input to a different layer of a neural network. Specifically, the layers are sequential and form portions of the neural network, where the first component image is input to a first layer, and the second component image and an intermediate result output from a prior layer are provided as input to a second layer.

Claims

This patent includes 19 claims, with claims 1, 18, and 19 being independent. The independent claims are directed to a method, a computer program product, and a system for processing images captured by a vehicle's sensor using an artificial neural network to autonomously operate the vehicle. The dependent claims generally elaborate on and refine the specifics of the method described in independent claim 1, such as the type of sensor used, the image decomposition techniques, and the component images utilized.

Key Claim Terms New

Definitions of key terms used in the patent claims.

Term (Source)Support for SpecificationInterpretation
Artificial neural network
(Claim 1, Claim 18, Claim 19)		“In some embodiments, autonomous driving is implemented using a deep learning network and input data received from sensors. For example, sensors affixed to a vehicle provide real-time sensor data, such as vision, radar, and ultrasonic data, of the vehicle's surrounding environment to a neural network for determining vehicle control responses. In some embodiments, the network is implemented using multiple layers.”A neural network used to process component images and determine a result that is used to autonomously operate a vehicle.
Autonomously operate the vehicle
(Claim 1, Claim 18, Claim 19)		“For example, the result of deep learning analysis using the artificial neural network is used to control the steering, breaking, lighting, and/or warning systems of the vehicle. In some embodiments, the result is used to autonomously match the vehicle's speed to traffic conditions, steer the vehicle to follow a navigational path, avoid collisions when an object is detected, summon the vehicle to a desired location, and warn the user of potential collisions, among other autonomous driving applications.”Using the result of the artificial neural network to control the vehicle without direct human input.
Component images
(Claim 1, Claim 18, Claim 19)		“In some embodiments, the received image is decomposed into a plurality of component images. For example, feature data is extracted from a captured high dynamic range image. As another example, global illumination data is extracted from the captured high dynamic range image. As another example, the image may be decomposed using high-pass, low-pass, and/or band-pass filters.”A set of images derived from an original image captured by a sensor on a vehicle, where the set includes at least a first and a second component image.
Image signal processors
(Claim 19)		“A data pipeline that extracts and provides sensor data as separate components to a deep learning network for autonomous driving is disclosed. In some embodiments, autonomous driving is implemented using a deep learning network and input data received from sensors. The sensor data is extracted into two or more different data components based on the signal information of the data.”Processors configured to receive an image captured using a sensor and decompose the received image into a plurality of component images.
Vehicle control module
(Claim 19)		“Using data captured from sensors and analyzed using the disclosed deep learning system, a machine learning result is determined for autonomous driving. In various embodiments, the machine learning result is provided to a vehicle control module for implementing autonomous driving features. For example, a vehicle control module can be used to control the steering, braking, warning systems, and/or lighting of the vehicle.”A module that receives the result of the artificial neural network and uses it to at least in part autonomously operate the vehicle.

Litigation Cases New

US Latest litigation cases involving this patent.

Case NumberFiling DateTitle
2:25-cv-00742Jul 23, 2025Perceptive Automata Llc V. Tesla, Inc.

Patent Family

Patent Family

File Wrapper

The dossier documents provide a comprehensive record of the patent's prosecution history - including filings, correspondence, and decisions made by patent offices - and are crucial for understanding the patent's legal journey and any challenges it may have faced during examination.

  • Date

    Description

  • Get instant alerts for new documents

US11215999

TESLA INC
Application Number
US16013817
Filing Date
Jun 20, 2018
Status
Granted
Expiry Date
Mar 6, 2040
External Links
Slate, USPTO, Google Patents