Grouping of image frames in video coding

What is this patent about?

’521 is related to the field of scalable video coding and streaming. It addresses the challenges of managing compressed video sequences where frames are interdependent, particularly when a streaming server or network element needs to adjust the bit rate by removing certain frames without fully decoding the stream. The background context involves the use of temporal prediction, where frames like P-frames and B-frames rely on reference images, making it difficult to scale the video without causing decoding errors or synchronization issues.

The underlying idea behind ’521 is the organization of video frames into logically grouped sub-sequences that explicitly signal their interdependencies. Rather than treating a video as a linear string of frames or simple hierarchical layers, the invention treats groups of frames as independent units that carry metadata identifying which other sub-sequences they rely on for prediction. This insight allows a network node to prune the video stream by removing entire sub-sequences that are not needed for the reconstruction of the remaining parts, effectively enabling bit rate control without the overhead of full video parsing.

The claims of ’521 focus on a method and apparatus for encoding and decoding a video sequence by defining at least two distinct sub-sequences. The first sub-sequence contains independent frames, such as I-frames, while the second sub-sequence contains predicted frames that rely on the first. Crucially, the independent claims require the determination and encoding of explicit dependency data into the video sequence, which identifies the relationship between the second sub-sequence and the first, separate from standard picture type information.

In practice, the invention works by assigning each frame a unique identifier that combines a scalability layer number, a sub-sequence ID, and an image number. This metadata is preferably placed in the header fields of the transport protocol or the video bitstream itself. When a streaming server detects a drop in available bandwidth, it can examine these headers to identify which sub-sequences can be safely discarded. Because the dependencies are clearly mapped, the server can ensure that if a reference sub-sequence is removed, all dependent sub-sequences are also removed, preventing the decoder from attempting to process broken links.

This approach differs from prior solutions by moving the intelligence of frame dependency from the deep video payload to the accessible metadata layer. Traditional methods often required a server to buffer and parse the video for a long period to detect dependencies, or they suffered from numbering discontinuities when frames were removed. By using sub-sequence-specific numbering and explicit dependency signaling, the invention allows for seamless bit rate adjustment and easier random access, as the decoder can immediately identify the start of a valid, decodable group of frames even after a stream has been modified.