Motion prediction in video coding

Patent No. US9432693 (titled "Motion prediction in video coding") on Jan 6, 2012. The application was issued on Aug 30, 2016.

What is this patent about?

’693 is related to the field of video compression and decompression, specifically focusing on motion-compensated prediction. In modern video codecs, bi-directional prediction improves efficiency by averaging pixel values from different reference frames to predict a current block. However, standard methods often introduce rounding errors during the interpolation and averaging stages, which can accumulate and degrade the final image quality or require complex signaling to manage rounding directions.

The underlying idea behind ’693 is to preserve signal integrity by maintaining high-precision pixel data throughout the prediction calculation and only reducing the bit-depth at the very end of the process. Rather than rounding individual reference blocks to the standard pixel bit-depth before they are combined, the system performs fractional pixel sample interpolation and keeps the resulting values in a higher-precision format. By delaying the bit-reduction until after the multiple prediction signals are summed, the invention minimizes the loss of information caused by intermediate rounding steps.

The claims of ’693 focus on a specific sequence of operations for processing bi-predicted or multi-predicted blocks. The method identifies a block type requiring two or more reference blocks and performs interpolation to generate first and second predictions at a second precision that is higher than the original pixel bit-depth. These high-precision predictions are then added together along with a rounding value. Only after this summation is the precision decreased back to the original level using a right-shift operation.

In practice, this approach eliminates the need for the encoder to signal rounding offsets or alternate rounding directions between frames to cancel out errors. When the inter-predictor calculates sub-pixel values using a multi-tap filter, the intermediate results are stored in larger registers—for example, 16-bit registers for 8-bit video—to prevent truncation. The final combined prediction is then normalized to the target bit-depth in a single step, ensuring that the rounding error accumulation typically found in traditional bi-directional averaging is significantly reduced.

This invention differs from prior approaches by moving the rounding logic from the individual reference block level to the combined block level. Traditional codecs often round the results of each interpolation filter immediately to fit back into standard 8-bit or 10-bit formats before averaging them. By contrast, ’693 utilizes a high-precision intermediate state that acts as a buffer for mathematical accuracy, resulting in a more faithful reconstruction of the video signal without increasing the complexity of the bitstream signaling.

How does this patent fit in bigger picture?

Technical Landscape

In the early 2010s when ’693 was filed, video compression architectures were typically implemented using hybrid coding frameworks that relied on motion-compensated inter-prediction to reduce temporal redundancy. At a time when systems commonly relied on fixed-point arithmetic for pixel interpolation, intermediate prediction values were typically rounded or truncated to the standard bit-depth of the video signal immediately after each reference block was processed. When hardware constraints made high-precision memory buffers non-trivial, standard practice involved applying rounding offsets at each stage of multi-directional prediction to ensure that intermediate data remained within the native bit-depth of the codec, which often introduced cumulative rounding errors before the final combined prediction was generated.

Prosecution Position

The disclosed invention addresses the technical problem of precision loss and cumulative rounding errors in bi-directional or multi-directional video prediction. The architectural shift involves maintaining individual prediction signals at a higher precision than the original pixel bit-depth throughout the interpolation and combination stages, only decreasing the precision to the target bit-depth after the multiple reference signals have been combined. This integration enables a technical effect where the accuracy of the motion-compensated signal is preserved, effectively eliminating the need to signal rounding offsets or alternate rounding methods within the bitstream while improving the fidelity of the reconstructed video.

Claims

This patent contains 19 claims, with claims 1, 9, 17, 18, and 19 being independent. The independent claims focus on a method, apparatus, and computer program product for video processing that involves determining pixel blocks, performing fractional pixel sample interpolation to generate predictions at a higher precision than the original bitstream, combining these predictions with a rounding value, and subsequently decreasing the precision back to the original level via bit shifting. The dependent claims serve to further define the process by specifying rounding offsets, intermediate precision levels, specific bit depths, block types such as bi-directional blocks, and the use of filtering to obtain pixel predictions.

Key Claim Terms New

Definitions of key terms used in the patent claims.

Term (Source)Support for SpecificationInterpretation
Combined prediction
(Claim 1, Claim 9, Claim 17, Claim 18, Claim 19)
In some example embodiments prediction signals are maintained in higher accuracy until the prediction signals have been combined to obtain the bi-directional or multidirectional prediction signal. The accuracy of the bi-directional or multidirectional prediction signal can then be downshifted to an appropriate accuracy for post processing purposes.The result of adding multiple high-precision prediction signals and a rounding value before the final bit-depth reduction.
First precision
(Claim 1, Claim 9, Claim 17, Claim 18, Claim 19)
The first precision indicates the number of bits needed to represent values of said pixels. The encoder may discard some information in the original video sequence in order to represent the video in a more compact form, for example at a lower bit rate.The bit-depth or number of bits used to represent the pixel values of the original video representation and the reference blocks.
Fractional pixel sample interpolation process
(Claim 1, Claim 9, Claim 17, Claim 18, Claim 19)
This invention may keep the motion compensated prediction signal of each one of the predictions at highest precision possible after interpolation and perform the rounding to the bit-depth range of the video signal after both prediction signals are added.A mechanism used to obtain prediction values at non-integer pixel locations, resulting in values with higher precision than the source pixels.
Second precision
(Claim 1, Claim 9, Claim 17, Claim 18, Claim 19)
According to some embodiments of the invention prediction signals are maintained in a higher precision during the prediction calculation and the precision is reduced after the two or more prediction signals have been combined with each other. The second precision indicates the number of bits needed to represent values of said first prediction and values of said second prediction.A higher bit-depth accuracy maintained during intermediate prediction calculations to reduce rounding errors before final combination.
Shifting bits of the combined prediction to the right
(Claim 1, Claim 9, Claim 17, Claim 18, Claim 19)
The accuracy of the bi-directional or multidirectional prediction signal can then be downshifted to an appropriate accuracy for post processing purposes. Then, no rounding direction indicator need not be included in or read from the bitstream.A bitwise operation used to decrease the precision of the combined prediction back to the original bit-depth of the video representation.

Litigation Cases New

US Latest litigation cases involving this patent.

Case NumberFiling DateTitle
1:23-cv-01236Oct 31, 2023Nokia Technologies Oy V. Amazon.Com, Inc.

Patent Family

Patent Family

File Wrapper

The dossier documents provide a comprehensive record of the patent's prosecution history - including filings, correspondence, and decisions made by patent offices - and are crucial for understanding the patent's legal journey and any challenges it may have faced during examination.

  • Get instant alerts for new documents

US9432693

SEP
Application Number
US13344893A
Filing Date
Jan 6, 2012
Status
Granted
Publication Date
Aug 30, 2016
External Links
Slate, USPTO , Google Patents