Multi-level significance maps for encoding and decoding

Patent No. US10659782 (titled "Multi-level significance maps for encoding and decoding") on Dec 20, 2018. The application was issued on May 19, 2020.

What is this patent about?

’782 is related to the field of video data compression, specifically the efficient entropy encoding and decoding of quantized transform coefficients. In modern video standards like HEVC, significance maps are used to identify the locations of non-zero coefficients within a transform unit, but as transform sizes increase to 16x16 or 32x32, the computational cost of managing individual coefficient contexts and the bitstream overhead for sparse data become significant bottlenecks.

The underlying idea behind ’782 is the implementation of a multi-level significance map structure that partitions a transform unit into smaller, non-overlapping blocks or groups. By first signaling a high-level flag for each group to indicate whether it contains any non-zero data, the system can bypass the encoding of individual flags for entirely empty regions. This hierarchical approach reduces memory bandwidth requirements and improves compression efficiency by avoiding the processing of large clusters of zero-value coefficients.

The claims of ’782 focus on a specific encoding logic that utilizes these significant-coefficient-group flags to control the bitstream inclusion of individual coefficient flags. A critical aspect of the claimed mechanism is an inference rule for the flag at position (0,0) within a group: if a group is signaled as non-empty but all previously scanned flags in that group are zero, the final flag is inferred to be one. This eliminates the need to explicitly encode the last possible non-zero flag in a group, further tightening the bitstream.

In practice, the invention operates by scanning the transform unit in a prescribed order, such as a reverse diagonal scan, to determine which groups contain significant data. The encoder uses context-adaptive binary arithmetic coding (CABAC) with contexts derived from the status of neighboring groups, such as those to the right or below the current block. This spatial dependency allows the probability model to adapt to the typical clustering of energy in the low-frequency regions of a transform block.

This approach differentiates itself from prior methods by replacing computationally intensive neighbor-based context derivation for every single coefficient with a more streamlined hierarchical signaling strategy. By incorporating rate-distortion optimized quantization (RDOQ) at the group level, the encoder can strategically zero out entire blocks if the bit savings outweigh the visual distortion. This results in a high-performance coding engine that scales effectively with the larger transform units required for high-definition video content.

How does this patent fit in bigger picture?

Technical Landscape

In the early 2010s when ’782 was filed, video compression systems were transitioning toward higher resolution formats at a time when entropy encoding was typically implemented using single-level significance maps to identify non-zero transform coefficients. When systems commonly relied on large transform units, such as 16x16 or 32x32 blocks, to improve coding efficiency, the determination of context for each coefficient position required evaluating the status of multiple neighboring flags. Hardware and software constraints made this process non-trivial, as the high frequency of memory access operations required to retrieve neighboring flag values created significant computational bottlenecks during the real-time decoding of high-definition bitstreams.

Prosecution Position

The disclosed invention represents a technical advancement through an architectural shift from single-level to multi-level significance map encoding. By partitioning a large transform unit into smaller sub-blocks and generating a higher-level map to indicate which sub-blocks contain at least one non-zero coefficient, the system enables the decoder to bypass entire regions of zero-valued coefficients. This hierarchical structure overcomes the constraint of high computational complexity by reducing the number of context-model lookups and memory accesses required to reconstruct the coefficient matrix. The integration of this multi-level approach allows for more efficient processing of large transform units without the proportional increase in processing overhead typically associated with neighbor-based context derivation.

Claims

This patent contains 23 claims, with claims 1, 12, and 23 serving as the independent claims. The independent claims focus on a method, an encoder, and a processor-readable medium for encoding significant-coefficient flags within partitioned transform units by conditionally inferring flag values to avoid redundant bitstream encoding. The dependent claims serve to further define specific transform unit and block dimensions, establish criteria for setting flags based on neighbor group values or coefficient positions, and detail the determination of encoding contexts.

Key Claim Terms New

Definitions of key terms used in the patent claims.

Term (Source)Support for SpecificationInterpretation
Bitstream
(Claim 1, Claim 12, Claim 23)
The encoding of the quantized transform coefficients often occupies 30-80% of the encoded data in the bitstream. The invention allows for setting a flag to a value without encoding it into the bitstream, thereby reducing the total number of bits transmitted. This occurs when the value of a flag can be inferred by the decoder based on previously decoded information.The sequence of bits forming the encoded video data where certain flags may be explicitly written or omitted based on inference rules.
DC block
(Claim 1, Claim 12, Claim 23)
The DC block is the group at the top-left corner of the transform unit. In the context of the invention, the DC block is treated differently during encoding because the DC coefficient (position 0,0 of the TU) is often handled separately from the inference logic applied to other groups. The DC position (0,0) is the first position in the scan order for the entire transform unit.The specific sub-block or group within a partitioned transform unit that contains the DC (top-leftmost) coefficient of the entire transform unit.
Significant-coefficient flags
(Claim 1, Claim 12, Claim 23)
Let s(r, c) denote the significance flag of a coefficient at position (r, c), where s(r, c)=1 if the coefficient is not zero and s(r, c)=0 otherwise. A significance map indicates the positions in the block that contain non-zero coefficients. These flags are encoded to represent the sparsity of the transform unit.Individual bits within a transform unit that indicate whether a specific coefficient at a given position is non-zero (significant) or zero.
Significant-coefficient-group flags
(Claim 1, Claim 12, Claim 23)
The transform unit is partitioned into a plurality of non-overlapping blocks, and a significant-coefficient-group flag is a bit associated with each block to indicate whether the block contains any non-zero coefficients. If the flag is 0, all coefficients in that block are zero; if the flag is 1, at least one coefficient in the block is non-zero. This creates a multi-level significance map where the first level is the map of these group flags.Flags that indicate whether a specific sub-block or group within a partitioned transform unit contains any non-zero transform coefficients.
Transform unit
(Claim 1, Claim 12, Claim 23)
Transform units are typically N×N blocks, such as 4×4, 8×8, 16×16, and 32×32, although other sizes are possible. The block or matrix of quantized transform domain coefficients is sometimes referred to as a 'transform unit'. The entropy encoding of the symbols in the significance map is based upon a context model associated with the transform unit.A block or matrix of quantized transform domain coefficients, typically resulting from a spectral transform of residual video data.

Litigation Cases New

US Latest litigation cases involving this patent.

Case NumberFiling DateTitle
1:25-cv-00967Jun 23, 2025Velos Media, LLC v. ByteDance Ltd et al

Patent Family

Patent Family

File Wrapper

The dossier documents provide a comprehensive record of the patent's prosecution history - including filings, correspondence, and decisions made by patent offices - and are crucial for understanding the patent's legal journey and any challenges it may have faced during examination.

  • Get instant alerts for new documents

US10659782

SEP
Application Number
US16226870A
Filing Date
Dec 20, 2018
Status
Granted
Publication Date
May 19, 2020
External Links
Slate, USPTO , Google Patents