Method for sub-pixel value interpolation

Patent No. US6950469 (titled "Method for sub-pixel value interpolation") on Sep 17, 2001. The application was issued on Sep 27, 2005.

What is this patent about?

’469 is related to the field of digital video coding and decoding, specifically focusing on motion-compensated temporal prediction. In modern video compression, motion vectors often point to sub-pixel locations to better model real-world movement. This requires interpolating values for pixels that do not exist in the original sampled grid, a process that traditionally forces a trade-off between high computational complexity and heavy memory consumption.

The underlying idea behind ’469 is a multi-stage interpolation strategy that balances precision and efficiency by using diagonal linear interpolation for the finest sub-pixel resolutions. Instead of deriving every fractional pixel through complex multi-tap filtering or relying on previously truncated values that lose accuracy, the invention calculates primary sub-pixels using weighted sums and then determines the remaining fractional positions through simple averages of these primary points. This approach maintains high visual quality while significantly reducing the mathematical burden on the decoder.

The claims of ’469 focus on a method for generating sub-pixel values at fractional locations defined by a power of two. The process involves directly interpolating half-unit horizontal and vertical positions using weighted sums of original pixels. For the most granular fractional locations, such as quarter-pixel or eighth-pixel positions, the claims specify taking a weighted average of two existing pixels or sub-pixels that are located diagonally relative to the target sub-pixel, ensuring a streamlined calculation path.

In practice, the system implements a hybrid of before-hand and on-demand interpolation. The encoder pre-calculates a subset of sub-pixels to save time during the exhaustive motion estimation search, while the decoder only calculates the specific sub-pixel values indicated by the received motion vectors. By using intermediate values with a dynamic range greater than the final pixel depth, the invention avoids the rounding errors that typically plague multi-stage interpolation schemes, ensuring the encoder and decoder remain perfectly synchronized.

This invention differs from prior approaches like TML5 and TML6 by eliminating the need for high-precision arithmetic across all sub-pixel calculations. Unlike TML5, which suffers from precision loss due to cumulative clipping, ’469 uses linear interpolation from primary sub-pixels to maintain accuracy. Compared to TML6, it drastically reduces memory requirements because it does not need to store massive arrays of high-bit-depth intermediate values, making it ideal for implementation in resource-constrained mobile telecommunications terminals.

How does this patent fit in bigger picture?

Technical Landscape

In the early 2000s when ’469 was filed, digital video compression was typically implemented using motion-compensated temporal prediction to manage the high data rates required for real-time transmission over bandwidth-constrained networks. At a time when systems commonly relied on integer-pixel or basic half-pixel motion estimation to reduce temporal redundancy, the computational overhead and memory bandwidth required for higher-precision spatial interpolation made fine-grained sub-pixel accuracy non-trivial. During this era, hardware and software constraints often forced a trade-off between the precision of motion vectors and the complexity of the interpolation filters used to generate fractional pixel values.

Prosecution Position

The disclosed invention achieves a technical advancement in video coding efficiency through a multi-stage architectural shift in sub-pixel value generation. By integrating a direct weighted sum approach for primary fractional locations with a diagonal weighted average for finer sub-pixel resolutions, the system enables high-precision motion compensation without the cumulative rounding errors or excessive computational load associated with traditional iterative filtering. This structural solution allows for the interpolation of quarter-pixel or finer values by selectively utilizing previously calculated intermediate values of higher dynamic range, thereby overcoming the technical constraint of precision loss during the prediction process and improving the accuracy of the motion-compensated temporal prediction.

Claims

This patent contains 51 claims, with claims 1, 18, 19, 23, 27, 31, 34, 35, 42, 46, 49, 50, and 51 serving as the independent claims. The independent claims focus on methods, video coders, codecs, and telecommunications systems designed for sub-pixel interpolation in video coding, specifically utilizing weighted sums and diagonal averages to generate values for fractional pixel locations. The dependent claims serve to further define the interpolation parameters, such as specific weighting factors, sub-pixel resolutions, and the integration of these methods into mobile networks and wireless communication hardware.

Key Claim Terms New

Definitions of key terms used in the patent claims.

Term (Source)Support for SpecificationInterpretation
Dynamic range
(Claim 1, Claim 18, Claim 19, Claim 23, Claim 27, Claim 31, Claim 34, Claim 35, Claim 42, Claim 46, Claim 49, Claim 50, Claim 51)
The term dynamic range refers to the range of values which the sub-pixel values and the weighted sums can take. Preferably changing the dynamic range, whether by extending it or reducing it, means changing the number of bits which are used to represent the dynamic range. The intermediate value for a sub-pixel having 1/2^(N-1) sub-pixel resolution may be used in the calculation of a sub-pixel value having 1/2^N sub-pixel resolution.The specific bit-depth or range of numerical values that represent pixel and sub-pixel data, which may be extended during intermediate interpolation steps.
Fractional horizontal and vertical locations
(Claim 1, Claim 18, Claim 19, Claim 23, Claim 27, Claim 31, Claim 34, Claim 35, Claim 42, Claim 46, Claim 49, Claim 50, Claim 51)
The pixels in the rows residing at unit horizontal locations and the pixels in the columns residing at unit vertical locations, is interpolated to generate values for sub-pixels at fractional horizontal and vertical locations. The fractional horizontal and vertical locations being defined according to 1/2^x, where x is a positive integer having a maximum value N. Sub-pixels at quarter unit horizontal location are to be interpreted as being sub-pixels having as their left-hand nearest neighbour a pixel at unit horizontal location and as their right-hand nearest neighbour a sub-pixel at half unit horizontal location.Spatial positions between the integer pixel grid coordinates, defined by a resolution of 1/2^x where x is a positive integer up to N, used for sub-pixel interpolation.
Located diagonally
(Claim 1, Claim 23, Claim 27, Claim 31, Claim 34, Claim 50)
The first and second sub-pixels or pixels are located diagonally with respect to the sub-pixel at 1/2^N unit horizontal and 1/2^N vertical location. Values for sub-pixels at 1/2^N unit horizontal and 1/2^N unit vertical locations may be interpolated by taking the average of values of a pixel located at a unit horizontal and unit vertical location, and a sub-pixel located at a 1/2^(N-1) unit horizontal and 1/2^(N-1) unit vertical location. Alternatively, values may be interpolated by taking the average of values of a sub-pixel located at a 1/2^(N-1) unit horizontal and unit vertical location, and a sub-pixel located at a unit horizontal and 1/2^(N-1) unit vertical location.The spatial relationship where two reference pixels or sub-pixels are positioned on a diagonal line passing through the target sub-pixel being interpolated.
Weighted average
(Claim 1, Claim 23, Claim 27, Claim 31, Claim 34, Claim 50)
Interpolating such a value by taking a weighted average of the value of a first sub-pixel or pixel and the value of a second sub-pixel or pixel located diagonally with respect to the sub-pixel at 1/2^N unit horizontal and 1/2^N vertical location. Preferably a first and a second weight are used in the weighted average, the relative magnitudes of the weights being inversely proportional to the proximity of the first and the second sub-pixel or pixel. In a situation where the first and the second sub-pixel or pixel are symmetrically located, the first and second weights may have equal values.A calculation performed by combining the values of two diagonally located pixels or sub-pixels to determine the value of a sub-pixel at a 1/2^N location.
Weighted sums of pixels
(Claim 1, Claim 18, Claim 19, Claim 23, Claim 27, Claim 31, Claim 34, Claim 35, Claim 42, Claim 46, Claim 49, Claim 50, Claim 51)
When values for sub-pixels at 1/2^(N-1) unit horizontal and unit vertical locations are required, interpolating such values directly using weighted sums of pixels residing at unit horizontal and unit vertical locations. At least one of step a) and step b) interpolating sub-pixel values directly using weighted sums may involve the calculation of an intermediate value for the sub-pixel values having a dynamic range greater than the specified dynamic range. The term dynamic range refers to the range of values which the sub-pixel values and the weighted sums can take.A calculation method where pixel values at integer locations are multiplied by specific weights and summed to directly generate sub-pixel values at 1/2^(N-1) resolutions.

Litigation Cases New

US Latest litigation cases involving this patent.

Case NumberFiling DateTitle
0:24-cv-04269Nov 25, 2024Element Television Company, Llc V. Nokia Corporation
1:23-cv-01237Oct 31, 2023Nokia Technologies Oy V. Hp, Inc.
1:23-cv-01236Oct 31, 2023Nokia Technologies Oy V. Amazon.Com, Inc.

Patent Family

Patent Family

File Wrapper

The dossier documents provide a comprehensive record of the patent's prosecution history - including filings, correspondence, and decisions made by patent offices - and are crucial for understanding the patent's legal journey and any challenges it may have faced during examination.

  • Get instant alerts for new documents

US6950469

SEP
Application Number
US09954608A
Filing Date
Sep 17, 2001
Status
Expired
Publication Date
Sep 27, 2005
External Links
Slate, USPTO , Google Patents