US20060083300A1 - Video coding and decoding methods using interlayer filtering and video encoder and decoder using the same - Google Patents
Video coding and decoding methods using interlayer filtering and video encoder and decoder using the same Download PDFInfo
- Publication number
- US20060083300A1 US20060083300A1 US11/251,917 US25191705A US2006083300A1 US 20060083300 A1 US20060083300 A1 US 20060083300A1 US 25191705 A US25191705 A US 25191705A US 2006083300 A1 US2006083300 A1 US 2006083300A1
- Authority
- US
- United States
- Prior art keywords
- frames
- video
- video coding
- interlayer
- wavelet
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
- H04N19/615—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding using motion compensated temporal filtering [MCTF]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/115—Selection of the code volume for a coding unit prior to coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/117—Filters, e.g. for pre-processing or post-processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/127—Prioritisation of hardware or computational resources
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/149—Data rate or code amount at the encoder output by estimating the code amount by means of a model, e.g. mathematical model or statistical model
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/15—Data rate or code amount at the encoder output by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/146—Data rate or code amount at the encoder output
- H04N19/152—Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/162—User input
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/164—Feedback from the receiver or from the transmission channel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/187—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/31—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/34—Scalability techniques involving progressive bit-plane based encoding of the enhancement layer, e.g. fine granular scalability [FGS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/36—Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/513—Processing of motion vectors
- H04N19/517—Processing of motion vectors by encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
- H04N19/51—Motion estimation or motion compensation
- H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
- H04N19/63—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/70—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
- H04N19/86—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving reduction of coding artifacts, e.g. of blockiness
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
Definitions
- Apparatuses and methods consistent with the present invention relate to a multi-layer video coding technique, and more particularly, to a video coding technique using interlayer filtering.
- Multimedia data requires a large capacity storage medium and a wide bandwidth for transmission since the amount of data is usually large.
- a 24-bit true color image having a resolution of 640*480 needs a capacity of 640*480*24 bits, i.e., data of about 7.37 Mbits per frame.
- a bandwidth of 221 Mbits/sec is required.
- a 90-minute movie based on such an image is stored, a storage space of about 1200 Gbits is required. Accordingly, a compression coding method is a requisite for transmitting multimedia data.
- a basic principle of multimedia data compression is removing data redundancy.
- video data can be compressed by removing spatial, temporal and visual redundancy.
- Spatial redundancy occurs when the same color or object is repeated in an image.
- Temporal redundancy occurs when there is little change between adjacent frames in a moving image or the same sound is repeated in audio.
- the removal of visual redundancy takes into account the limitations of human eyesight and the limited perception of high frequency.
- FIG. 1 shows an environment in which video compression is applied.
- Video data is compressed by a video encoder 110 .
- DCT Discrete Cosine Transform
- Video data is sent to a video decoder 130 via a network 120 .
- the video decoder 130 decodes the compressed video data to reconstruct the original video data.
- the video encoder 110 compresses the original video data so it does not exceed the available bandwidth of the network 120 .
- communication bandwidth may vary depending on the type of the network 120 .
- the available communication bandwidth of Ethernet is different from that of a wireless local area network (WLAN).
- WLAN wireless local area network
- a cellular communication network may have a very narrow bandwidth.
- Scalable video coding is a video compression technique that enables video data to provide scalability. Scalability is the ability to generate video sequences at different resolutions, frame rates, and qualities from the same compressed bitstream.
- Temporal scalability can be provided using a Motion Compensation Temporal Filtering (MCTF), Unconstrained MCTF (UMCTF), or Successive Temporal Approximation and Referencing (STAR) algorithm. Spatial scalability can be achieved by a wavelet transform algorithm or multi-layer coding, which have been actively studied in recent years.
- SNR Signal-to-Noise Ratio
- EZW Embedded ZeroTrees Wavelet
- SPIHT Set Partitioning in Hierarchical Trees
- EZBC Embedded ZeroBlock Coding
- EBCOT Embedded Block Coding with Optimized Truncation
- Multi-layer video coding algorithms have recently been adopted for scalable video coding. While conventional multi-layer video coding usually uses a single video coding algorithm, increasing attention has been recently directed to multi-layer video coding using a plurality of video coding algorithms.
- FIGS. 2 and 3 illustrate examples of bitstreams generated by multi-layer video coding.
- a video encoder uses both a MPEG-4 Advanced Video Coding (AVC) algorithm offering excellent coding efficiency and a wavelet coding technique providing excellent scalability.
- AVC MPEG-4 Advanced Video Coding
- the bitstream shown in FIG. 2 contains AVC-coded lowest-resolution layer frames and highest-resolution layer frames wavelet-coded using the AVC-coded lowest-layer frames.
- the frame used as a reference during encoding is a frame reconstructed by decoding a frame encoded by AVC coding.
- a video encoder uses both wavelet coding offering the excellent scalability and AVC coding providing high coding efficiency. While the bitstream shown in FIG. 2 has only two wavelet-coded and AVC-coded layers, the bitstream shown in FIG. 3 has a wavelet-coded layer and an AVC-coded layer for each resolution.
- Multi-layer video coding has a problem in that the coding efficiency of an enhancement layer tends to be low due to quantization noise in the previously encoded layer (base layer).
- the problem is more severe when multi-layer video coding uses a plurality of video coding algorithms having different characteristics. For example, using both DCT-based AVC coding and wavelet-based coding as shown in FIG. 2 or 3 may degrade the coding efficiency in a wavelet layer.
- the present invention provides video coding and decoding methods using interlayer filtering, designed to improve the efficiency of multi-layer video coding, and video encoders and decoders.
- a video coding method including encoding video frames using a first video coding scheme, performing interlayer filtering on the frames encoded by the first video coding scheme, encoding the video frames using a second video coding scheme by referring the frames subjected to the interlayer filtering, and generating a bitstream containing the frames encoded by the first and second video coding schemes.
- a video coding method including downsampling video frames to generate video frames of low resolution, encoding the low-resolution video frames using a first video coding scheme, upsampling the frames encoded by the first video coding scheme to the resolution of the video frames, performing interlayer filtering on the upsampled frames, encoding the video frames using a second video coding scheme by referring the frames subjected to the interlayer filtering, and generating a bitstream containing the frames encoded by the first and second video coding schemes.
- a video encoder including a first video coding unit encoding video frames using a first video coding scheme, an interlayer filter performing interlayer filtering on the frames encoded by the first video coding scheme, a second video coding unit encoding the video frames using a second video coding scheme by referring the interlayer filtered frames, and a bitstream generator generating a bitstream containing the frames encoded by the first and second video coding schemes.
- a video encoder including a downsampler downsampling video frames to generate video frames of low resolution, a first video coding unit encoding the low-resolution video frames using a first video coding scheme, an upsampler upsampling the frames encoded by the first video coding scheme, an interlayer filter performing interlayer filtering on the upsampled frames, a second video coding unit encoding the video frames using a second video coding scheme by referring the interlayer filtered frames, and a bitstream generator generating a bitstream containing the frames encoded by the first and second video coding schemes.
- a video decoding method including extracting frames encoded by first and second video coding schemes from a bitstream, decoding the frames encoded by the first video coding scheme using a first video decoding scheme and reconstructing first layer frames, performing interlayer filtering on the reconstructed first layer frames, and decoding the frames encoded by the second video coding scheme using a second video decoding scheme by referring the interlayer filtered first layer frames and reconstructing second layer frames.
- a video decoding method including extracting frames encoded by first and second video coding schemes from a bitstream, decoding the frames encoded by the first video coding scheme using a first video decoding scheme and reconstructing first layer frames, upsampling the reconstructed first layer frames, performing interlayer filtering on the upsampled first layer frames, and decoding the frames encoded by the second video coding scheme using a second video decoding scheme by referring the interlayer filtered first layer frames and reconstructing second layer frames.
- a video decoder including a bitstream interpreter extracting frames encoded by first and second video coding schemes from a bitstream, a first video decoding unit decoding the frames encoded by the first video coding scheme by referring a first video decoding scheme and reconstructing first layer frames, an interlayer filter performing interlayer filtering on the reconstructed first layer frames, and a second video decoding unit decoding the frames encoded by the second video coding scheme using a second video decoding scheme by referring the interlayer filtered first layer frames and reconstructing second layer frames.
- a video decoder including a video decoder including a bitstream interpreter for extracting frames encoded by first and second video coding schemes from a bitstream, a first video decoding unit for decoding the frames encoded by the first video coding scheme using a first video decoding scheme and reconstructing first layer frames, an upsampler for upsampling the reconstructed first layer frames, an interlayer filter for performing interlayer filtering on the upsampled first layer frames, and a second video decoding unit for decoding the frames encoded by the second video coding scheme using a second video decoding scheme by referring to the interlayer filtered first layer frames and reconstructing second layer frames.
- FIG. 1 shows an environment in which video compression is applied
- FIGS. 2 and 3 show example structures of bitstreams generated by multi-layer video coding
- FIG. 4 is a block diagram of a video encoder according to a first exemplary embodiment of the present invention.
- FIG. 5 is a block diagram of a video encoder according to a second exemplary embodiment of the present invention.
- FIG. 6 is a block diagram of a temporal filter according to an exemplary embodiment of the present invention.
- FIG. 7 is a flowchart illustrating a video coding process according to an exemplary embodiment of the present invention.
- FIG. 8 is a flowchart illustrating a detailed process of encoding a second layer according to an exemplary embodiment of the present invention.
- FIG. 9 is a block diagram of a video decoder according to a first exemplary embodiment of the present invention.
- FIG. 10 is a block diagram of a video decoder according to a second exemplary embodiment of the present invention.
- FIG. 11 is a block diagram of an inverse temporal filter according to an exemplary embodiment of the present invention.
- FIG. 12 is a flowchart illustrating a video decoding process according to an exemplary embodiment of the present invention.
- FIG. 13 is a flowchart illustrating a detailed process of performing inverse temporal filtering on a second layer according to an exemplary embodiment of the present invention.
- FIG. 4 is a block diagram of a video encoder according to a first exemplary embodiment of the present invention.
- a video encoder includes a first video coding unit 410 , a second video coding unit 420 , a bitstream generator 430 , and an interlayer filter 440 .
- the first coding unit 410 includes a temporal filter 411 , a Discrete Cosine Transform (DCT) transformer 412 , and a quantizer 413 and encodes a video frame using advanced video coding (AVC).
- DCT Discrete Cosine Transform
- AVC advanced video coding
- the temporal filter 411 receives a video frame 400 and removes temporal redundancy that the video frame 400 has with adjacent frames.
- the temporal filter 411 may use a Motion Compensated Temporal Filtering (MCTF) algorithm to remove temporal redundancy between frames.
- MCTF Motion Compensated Temporal Filtering
- the MCTF algorithm supporting temporal scalability removes temporal redundancy between adjacent frames.
- a 5/3 filter is widely used for MCTF.
- Other temporal filtering algorithms supporting temporal scalability such as Unconstrained MCTF (UMCTF) or Successive Temporal Approximation and Referencing (STAR), may be used.
- UMCTF Unconstrained MCTF
- STAR Successive Temporal Approximation and Referencing
- the DCT transformer 412 performs DCT on the temporally filtered frame.
- the DCT is performed for each block of a predetermined size (8*8 or 4*4).
- the entropy of a block subjected to DCT is reduced compared with that of a block before DCT.
- the quantizer 413 quantizes the DCT-transformed frame.
- quantization is determined based on a quantization parameter (Qp).
- Qp quantization parameter
- the second video coding unit 420 includes a temporal filter 421 , a wavelet transformer 422 , and a quantizer 423 and encodes a video frame using wavelet coding.
- the temporal filter 421 receives a video frame 400 and removes temporal redundancy that the video frame 400 has with adjacent frames.
- the temporal filter 421 may use an MCTF algorithm to remove the temporal redundancy between frames.
- the MCTF algorithm supporting temporal scalability removes temporal redundancy between adjacent frames.
- Other temporal filtering algorithms supporting temporal scalability such as UMCTF or STAR, may be used.
- the wavelet transformer 422 performs a wavelet transform on the temporally filtered frame on a frame-by-frame basis.
- the wavelet transform algorithm supporting spatial scalability decomposes a frame into one low-pass subband (LL) and three high-pass subbands (LH, HL, and HH).
- the LL subband is a quarter size of and an approximation of the original frame before being subjected to wavelet transform.
- the wavelet transform is again performed to decompose the LL subband into one low-pass subband (LLLL) and three high-pass subbands (LLLH, LLHL, and LLHH).
- the LLLL subband is a quarter size of and an approximation of the LL subband.
- a 9/7 filter is commonly used for the wavelet transform.
- the quantizer 423 quantizes the wavelet-transformed frame.
- the quantization is performed using an embedded quantization algorithm such as Embedded ZeroTrees Wavelet (EZW), Set Partitioning in Hierarchical Trees (SPIHT), Embedded ZeroBlock Coding (EZBC), or Embedded Block Coding with Optimized Truncation (EBCOT) offering Signal-to-Noise Ratio (SNR) scalability.
- EZW Embedded ZeroTrees Wavelet
- SPIHT Set Partitioning in Hierarchical Trees
- EZBC Embedded ZeroBlock Coding
- EBCOT Embedded Block Coding with Optimized Truncation
- the temporal filter 421 removes temporal redundancy that exists in the video frame 400 using adjacent frames or the frame encoded by the first video coding unit 410 as a reference.
- Block-based DCT performed before quantization, may cause block artifacts, which degrade the efficiency of wavelet coding performed by the second video coding unit 420 . That is, a frame having block artifacts may degrade the wavelet coding efficiency since block artifacts such as noise propagate across the entire frame when the frame is subjected to a wavelet transform.
- the video encoder of FIG. 4 further includes the interlayer filter 440 to eliminate noise generated between layers.
- the interlayer filter 440 performs filtering in such a way that a frame encoded by a first video coding method is suitably used as a reference for a second video coding method. Interlayer filtering is needed when different video coding schemes are used for each layer as shown in the video encoder of FIG. 4 .
- the interlayer filter 440 performs filtering on a frame encoded by DCT-based AVC coding such that it can be suitably used as a reference for wavelet coding.
- the interlayer filter downsamples the AVC-coded frame with an MPEG (or other) filter after upsampling the same with a wavelet filter; however, this is merely exemplary. Rather, a downsampling filter in the interlayer filter 440 may be a low-pass filter showing a steep gradient at a cut-off frequency.
- the interlayer filter 440 may be any single or plurality of filters designed such that the frame subjected to interlayer filtering can be suitably used as a reference by the second video coding unit 420 .
- the bitstream generator 430 generates a bitstream containing an AVC-coded frame 431 , a wavelet-coded frame 432 , motion vectors, and other necessary information.
- FIG. 5 is a block diagram of a video encoder according to a second exemplary embodiment of the present invention.
- the video encoder includes a first video coding unit 510 , a second video coding unit 520 , a bitstream generator 530 , an interlayer filter 540 , an upsampler 550 , and a downsampler 560 .
- the video encoder encodes a video frame and a low-resolution video frame using different video coding schemes.
- the downsampler 560 downsamples a video frame 500 to generate a low-resolution video frame.
- the first video coding unit 510 includes a temporal filter 511 , a DCT transformer 512 , and a quantizer 513 and encodes the low-resolution video frame using AVC coding.
- the temporal filter 511 receives the low-resolution video frame and removes temporal redundancy that the low-resolution frame has with adjacent low-resolution frames.
- the temporal filter 511 uses an MCTF algorithm, which supports temporal scalability, to remove temporal redundancy between low-resolution frames.
- MCTF MCTF algorithm
- a 5/3 filter is widely used for MCTF, but other temporal filtering algorithms supporting temporal scalability such as UMCTF or STAR may be used.
- the DCT transformer 512 performs DCT on the temporally filtered frame.
- the DCT is performed for each block of a predetermined size (8*8 or 4*4).
- the entropy of a block subjected to DCT is reduced compared with that of a block before DCT.
- the quantizer 513 quantizes the DCT-transformed frame.
- quantization is determined based on a quantization parameter (Qp).
- Qp quantization parameter
- the upsampler 550 upsamples the AVC-coded frame to the resolution of the frame 500 .
- the interlayer filter 540 performs filtering in such a manner that an upsampled version of a frame can be suitably used as a reference for wavelet coding.
- the upsampled version of the frame may be upsampled with a wavelet filter, followed by downsampling using an MPEG filter; however, this is merely exemplary. Rather, a downsampling filter in the interlayer filter 540 may be a low-pass filter showing a steep gradient at a cut-off frequency.
- the interlayer filter 540 may be any single or plurality of filters designed such that the frame subjected to interlayer filtering can be suitably used as a reference by the second video coding unit 520 .
- the second video coding unit 520 includes a temporal filter 521 , a wavelet transformer 522 , and a quantizer 523 , and it encodes a video frame using wavelet coding.
- the temporal filter 521 receives the video frame 500 and removes temporal redundancy that the video frame 500 has with adjacent frames.
- the temporal filter 521 uses an MCTF algorithm to remove temporal redundancy between low-resolution frames.
- the MCTF algorithm supporting temporal scalability removes temporal redundancy between adjacent low-resolution frames.
- Other temporal filtering algorithms supporting temporal scalability such as UMCTF or STAR may be used.
- the wavelet transformer 522 performs a wavelet transform on the temporally filtered frame. Unlike the DCT transform that is performed in units of blocks, the wavelet transform is performed in units of frames.
- the wavelet transform algorithm supporting spatial scalability decomposes a frame into one low-pass subband (LL) and three high-pass subbands (LH, HL, and HH).
- the LL subband is a quarter of the size of and an approximation of the original frame before being subjected to wavelet transform.
- the wavelet transform is again performed to decompose the LL subband into one low-pass subband (LLLL) and three high-pass subbands (LLLH, LLHL, and LLHH).
- the LLLL subband is a quarter of the size of and an approximation of the LL subband.
- a 9/7 filter is commonly used for the wavelet transform.
- the quantizer 523 quantizes the wavelet-transformed frame.
- the quantization may be an embedded quantization algorithm, which provides SNR scalability, such as EZW, SPIHT, EZBC, or EBCOT.
- the temporal filter 521 removes temporal redundancy that exists in the video frame 500 using adjacent frames or the frame encoded by the first video coding unit 510 as a reference.
- the frame encoded by the first video coding unit is upsampled and subjected to interlayer filtering before being sent to the temporal filter 521 .
- the bitstream generator 530 generates a bitstream containing an AVC-coded frame 531 , a wavelet-coded frame 532 , motion vectors, and other necessary information.
- the temporal filter will be described in greater detail with reference to FIG. 6 .
- FIG. 6 is a block diagram of a temporal filter 600 according to an exemplary embodiment of the present invention.
- FIGS. 4 and 5 show that the first and second video coding units 410 ( 510 ) and 420 ( 520 ) include temporal filters 411 ( 511 ) and 421 ( 521 ), respectively, for convenience of explanation, it is assumed in an exemplary embodiment that the temporal filter 600 is employed in the second video coding unit 420 .
- the temporal filter 600 removes temporal redundancy between video frames using MCTF on a group-of-picture (GOP)-by-GOP basis.
- the temporal filter 600 includes a prediction frame generator 610 for generating a prediction frame, a prediction frame smoother 620 for smoothing the prediction frame, a residual frame generator 630 for generating a residual frame by comparing a smoothed prediction frame with a video frame, and an updater 640 for updating the video frame using the residual frame.
- the prediction frame generator 610 generates a prediction frame that will be compared with a video frame in order to generate a residual frame using video frames adjacent to the video frame and a frame subjected to interlayer filtering as a reference.
- the prediction frame generator 610 finds a matching block for each block in the video frame within reference frames (adjacent video frames and a frame subjected to interlayer filtering) (interceding) or within another block in the video frame (intracoding).
- the frame prediction smoother 620 smoothes a prediction frame since blocking artifacts are introduced at block boundaries in the prediction frame made up of blocks corresponding to blocks in the video frame. To accomplish this, the prediction frame smoother 620 may perform de-blocking on pixels at block boundaries in the prediction frame. Since a de-blocking algorithm is commonly used in the H.264 video coding scheme and is well known in the art, a detailed explanation thereof will not be given.
- the residual frame generator 630 compares the smoothed prediction frame with the video frame and generates a residual frame in which temporal redundancy has been removed.
- the updater 640 uses the residual frame to update other video frames.
- the updated video frames are then provided to the prediction frame generator 610 .
- the temporal filter 600 removes temporal redundancy in frames 1 , 3 , 5 , and 7 to generate residual frames 1 , 3 , 5 , and 7 , respectively.
- the residual frames 1 , 3 , 5 , and 7 are used to update frames 0 , 2 , 4 , and 6 .
- the temporal filter 600 removes the temporal redundancy from updated frames 2 and 6 to generate residual frames 2 and 6 .
- the residual frames 2 and 6 are used to update frames 0 and 4 .
- the temporal filter 600 removes temporal redundancy in the updated frame 4 to generate a residual frame 4 .
- the residual frame 4 is used to update the frame 0 .
- the temporal filter 600 performs temporal filtering on the eight video frames to obtain one low-pass frame (updated frame 0 ) and seven high-pass frames (residual frames 1 through 7 ).
- FIGS. 7 and 8 A video coding process and a temporal filtering will be described with reference to FIGS. 7 and 8 . It is assumed that video coding is performed on two layers.
- a video encoder receives a video frame in step S 710 .
- step S 720 the video encoder encodes the input video frame using AVC coding.
- a first layer is encoded using AVC coding because the AVC coding offers the highest coding efficiency currently available.
- the first layer may be encoded using another video coding algorithm.
- step S 730 the video encoder performs interlayer filtering on the AVC-coded frame so that it can be suitably used as a reference for encoding a second layer.
- the interlayer filtering involves upsampling the AVC-coded frame using a wavelet filter and downsampling an upsampled version of AVC-coded frame using an MPEG filter.
- step S 740 the video encoder performs wavelet coding on the video frame using the frame subjected to interlayer filtering as a reference.
- step S 750 the video encoder generates a bitstream containing the AVC-coded frame and the wavelet-coded frame.
- the video encoder uses a low-resolution frame obtained by downsampling the input video frame during the AVC coding.
- the video encoder changes the resolution of the AVC-coded frame. For example, when the resolution of the first layer is lower than that of the second layer, the video encoder upsamples the AVC-coded frame to the resolution of the second layer. Then, the video encoder performs interlayer filtering on the upsampled version of the AVC-coded frame.
- FIG. 8 is a flowchart illustrating a detailed process of encoding a second layer according to an exemplary embodiment of the present invention.
- a second video coding unit receives an encoded first layer video frame that has been subjected to interlayer filtering.
- step S 820 upon receipt of the video frame and the frame subjected to interlayer filtering, the second video coding unit performs motion estimation in order to generate a prediction frame that will be used in removing temporal redundancy in the video frame.
- Various well-known algorithms such as Block Matching and Hierarchical Variable Size Block Matching (HVSBM) may be used for motion estimation.
- HVSBM Hierarchical Variable Size Block Matching
- step S 830 after performing the motion estimation, the second video coding unit uses motion vectors obtained as a result of the motion estimation to generate the prediction frame.
- step S 840 the second video coding unit smoothes the prediction frame in order to reduce block artifacts in a residual frame. This is because an apparent block boundary degrades the coding efficiency during wavelet transform and quantization.
- step S 850 the second video coding unit compares the prediction frame with the video frame to generate a residual frame.
- the residual frame corresponds to a high-pass frame (H frame) generated through MCTF.
- a temporal filter uses a residual frame to update another video frame.
- An updated version of the video frame corresponds to a low-pass frame (L frame).
- the second video coding unit After an L frame and H frame are generated on a GOP basis through steps S 820 through S 860 , the second video coding unit performs wavelet transform on the temporally filtered frames (L and H frames) in step S 870 . While a 9/7 filter is commonly used for wavelet transform, an 11/9 or 13/11 filter may also be used.
- step S 880 the second video coding unit quantizes the wavelet-transformed frames using EZW, SPIHT, EZBC, or EBCOT.
- a video decoder and a decoding process will be described. While the decoding process is basically performed in reverse order to the encoding process, layers are encoded and decoded in the same order. For example, when a video encoder sequentially encodes first and second layers, a video decoder decodes the first and second layers in the same order. For convenience of explanation, it is assumed that a video frame is reconstructed from a bitstream having two layers.
- FIG. 9 is a block diagram of a video decoder according to a first exemplary embodiment of the present invention used when first and second layers have the same resolution.
- the video decoder includes a bitstream interpreter 900 , a first video decoding unit 910 , a second video decoding unit 920 , and an interlayer filter 940 .
- the bitstream interpreter 900 interprets an input bitstream and extracts frames encoded by a first video coding and frames encoded by second video coding.
- the frames encoded by the first video coding are then provided to the first video decoding unit 910 while the frames encoded by the second video coding are provided to the second video decoding unit 920 .
- the first video decoding unit 910 includes an inverse quantizer 911 , an inverse DCT transformer 912 , and an inverse temporal filter 913 .
- the inverse quantizer 911 inversely quantizes the frames encoded by the first video coding.
- the inverse quantization may involve entropy decoding, inverse scanning, and a process of reconstructing DCT-transformed frames using a quantization table.
- the inverse DCT transformer 912 performs an inverse DCT transform on the inversely quantized frames.
- the inverse temporal filter 913 reconstructs first layer video frames from the inversely DCT-transformed frames and outputs a decoded frame ( 931 ).
- the reconstructed first layer video frames are obtained by encoding original video frames at a low bit-rate and decoding the encoded frames.
- the interlayer filter 940 performs interlayer filtering on the reconstructed first layer frame.
- the interlayer filtering may be performed using a deblocking algorithm.
- the second video decoding unit 920 includes an inverse quantizer 921 , an inverse wavelet transformer 922 , and an inverse temporal filter 923 .
- the inverse quantizer 921 applies inverse quantization to the frames encoded by the second video coding.
- the inverse quantization may involve entropy decoding, inverse scanning, and a process of reconstructing wavelet-transformed frames using a quantization table.
- the inverse wavelet transformer 922 performs an inverse wavelet transform on the inversely quantized frames.
- the inverse temporal filter 923 reconstructs second layer video frames from the inversely wavelet-transformed frames using frames subjected to the interlayer filtering as a reference and outputs a decoded frame ( 932 ).
- the reconstructed second layer video frames are obtained by encoding original video frames at a high bit-rate and decoding the encoded frames.
- FIG. 10 is a block diagram of a video decoder according to a second exemplary embodiment of the present invention used when a first layer has lower resolution than a second layer.
- the video decoder includes a bitstream interpreter 1000 , a first video decoding unit 1010 , a second video decoding unit 1020 , an interlayer filter 1040 , and an upsampler 1050 .
- the bitstream interpreter 1000 interprets an input bitstream and extracts frames encoded by a first video coding and frames encoded by second video coding.
- the frames encoded by the first video coding have lower resolution than those encoded by the second video coding.
- the former is then provided to the first video decoding unit 1010 while the latter is provided to the second video decoding unit 1020 .
- the first video decoding unit 1010 includes an inverse quantizer 1011 , an inverse DCT transformer 1012 , and an inverse temporal filter 1013 .
- the inverse quantizer 1011 inversely quantizes the frames encoded by the first video coding.
- the inverse quantization may involve entropy decoding, inverse scanning, and a process of reconstructing DCT-transformed frames using a quantization table.
- the inverse DCT transformer 1012 performs an inverse DCT transform on the inversely quantized frames.
- the inverse temporal filter 1013 reconstructs first layer video frames from the inversely DCT-transformed frames and outputs a decoded frame ( 1031 ).
- the reconstructed first layer video frame is obtained by downsampling and encoding an original video frame and decoding the encoded frame.
- the upsampler 1050 upsamples the first layer frame to the resolution of a reconstructed second layer frame.
- the interlayer filter 1040 performs interlayer filtering on an upsampled version of the first layer frame.
- the interlayer filtering may be performed using a de-blocking algorithm.
- the second video decoding unit 1020 includes an inverse quantizer 1021 , an inverse wavelet transformer 1022 , and an inverse temporal filter 1023 .
- the inverse quantizer 1021 inversely quantizes the frames encoded by the second video coding.
- the inverse quantization may involve entropy decoding, inverse scanning, and a process of reconstructing wavelet-transformed frames using a quantization table.
- the inverse wavelet transformer 1022 performs an inverse wavelet transform on the inversely quantized frames.
- the inverse temporal filter 1023 reconstructs second layer video frames from the inversely wavelet-transformed frames using frames that have been subjected to the interlayer filtering as a reference and outputs a decoded frame ( 1032 ).
- the reconstructed second layer video frames are obtained by encoding original video frames and decoding the encoded frames.
- FIG. 11 is a block diagram of an inverse temporal filter 1100 according to an exemplary embodiment of the present invention.
- FIGS. 9 and 10 show that the first and second video decoding units 910 ( 1010 ) and 920 ( 1020 ) include the inverse temporal filters 913 ( 1013 ) and 923 ( 1023 ), respectively, for convenience of explanation, it is assumed in an exemplary embodiment that the temporal filter 1100 is employed in the second video decoding unit 920 shown in FIG. 9 .
- the inverse temporal filter 1100 reconstructs video frames from inversely wavelet-transformed frames on a GOP-by-GOP basis using MCTF.
- the inversely wavelet-transformed frames fed into the inverse temporal filter 1100 consist of low- and high-pass frames in which temporal redundancies have been removed during video coding.
- the inversely wavelet-transformed frames may include one low-pass frame (updated frame 0 ) and seven high-pass frames (residual frames 1 through 7 ) obtained as a result of video coding.
- the inverse temporal filter 1100 includes an inverse updater 1110 , a prediction frame generator 1120 , a prediction frame smoother 1130 , and a frame reconstructor 1140 .
- the inverse updater 1110 updates the inversely wavelet-transformed frames in the reverse order that video coding is performed.
- the prediction frame generator 1120 generates a prediction frame that will be used in reconstructing a low-pass frame or video frame from a residual frame using an interlayer filtered frame.
- the prediction frame smoother 1130 smoothes a prediction frame.
- the frame reconstructor 1140 reconstructs a low-pass frame or video frame from a high-pass frame using a smoothed prediction frame.
- the inverse temporal filter 1100 uses residual frame 4 and interlayer filtered frame 0 to inversely update a low-pass frame 0 .
- the low-pass frame 0 is used to generate a prediction frame for the residual frame 4 , and the prediction frame is used to obtain low-pass frame 4 from the residual frame 4 .
- the inverse temporal filter 1100 uses residual frames 2 and 6 and interlayer filtered frames 0 and 4 to inversely update low-pass frames 0 and 4 , and then uses the low-pass frames 0 and 4 and interlayer filtered frames 2 and 6 to generate prediction frames for the residual frames 2 and 6 .
- the prediction frames are then used to obtain low-pass frames 2 and 6 from the residual frames 2 and 6 .
- the inverse temporal filter 1100 uses residual frames 1 , 3 , 5 , and 7 and interlayer filtered frames 0 , 2 , 4 , and 6 to inversely update the low-pass frames 0 , 2 , 4 , and 6 , thereby reconstructing video frames 0 , 2 , 4 , and 6 .
- the inverse temporal filter 1100 uses the reconstructed frames 0 , 2 , 4 , and 6 and inter-layer filtered frames 1 , 3 , 5 , and 7 to generate prediction frames for the residual frames 1 , 3 , 5 , and 7 .
- the prediction frames are then used to reconstruct video frames 1 , 3 , 5 , and 7 .
- FIG. 12 is a flowchart illustrating a video decoding process according to an exemplary embodiment of the present invention.
- a video decoder receives a bitstream and extracts frames encoded by a first video coding and frames encoded by a second video coding.
- the first and second video coding schemes may be AVC coding and wavelet coding, respectively.
- step S 1220 the video decoder performs AVC decoding on the extracted AVC-coded frames.
- step S 1230 the video decoder performs interlayer filtering on the frames decoded by the AVC decoding.
- step S 1240 the video decoder performs wavelet decoding on the wavelet-coded frames by referring to the interlayer filtered frames.
- step S 1250 after wavelet decoding is finished, the video decoder uses reconstructed video frames to generate a video signal. That is, the video decoder converts luminance (Y) and chrominance (UV) color components of a reconstructed pixel into red (R), green (G), and blue (B) color components.
- Y luminance
- UV chrominance
- FIG. 13 is a flowchart illustrating a detailed process of performing inverse temporal filtering on a second layer according to an exemplary embodiment of the present invention.
- an inverse temporal filter receives inversely wavelet-transformed frames and interlayer filtered frames.
- the inversely wavelet-transformed frames are reconstructed on a GOP basis and consist of low- and high-pass frames.
- step S 1320 the inverse temporal filter uses a high-pass frame among the inversely wavelet-transformed frames and an interlayer filtered frame to inversely update a low-pass frame.
- step S 1330 the inverse temporal filter uses an inversely updated low-pass frame and an inter layer filtered frame to generate a prediction frame.
- step S 1340 the inverse temporal filter smoothes the prediction frame.
- step S 1350 after smoothing of the prediction frame is finished, the inverse temporal filter uses the smoothed prediction frame and a high-pass frame to reconstruct a low-pass frame or a video frame.
- the present invention provides multi-layer video coding and decoding methods employing a plurality of video coding algorithms that can improve the coding efficiency using interlayer filtering.
- the present invention can prevent the degradation of coding efficiency that may occur when a video coding scheme with a block-based transform algorithm and a video coding scheme with a frame-based transform algorithm are used together.
Abstract
Video coding and decoding methods using interlayer filtering and a video encoder and decoder using the same are provided. Multi-layer video coding employing a plurality of video coding algorithms can improve video coding efficiency by using interlayer filtering. The video coding method includes encoding video frames using a first video coding scheme, performing interlayer filtering on the frames encoded by the first video coding scheme, encoding the video frames using a second video coding scheme by referring to the frames subjected to the interlayer filtering, and generating a bitstream containing the frames encoded by the first and second video coding schemes.
Description
- This application claims priority from Korean Patent a Application No. 10-2004-0094597 filed on Nov. 18, 2004 in the Korean Intellectual Property Office, and U.S. Provisional Patent Application No. 60/619,023 filed on Oct. 18, 2004 in the United States Patent and Trademark Office, the disclosures of which are incorporated herein by reference in their entirety.
- 1. Field of the Invention
- Apparatuses and methods consistent with the present invention relate to a multi-layer video coding technique, and more particularly, to a video coding technique using interlayer filtering.
- 2. Description of the Related Art
- With the development of communication technology including Internet, video communication as well as text and voice communication has increased. Conventional text communication cannot satisfy the various demands of users, and thus multimedia services that can provide various types of information such as text, pictures, and music have increased. Multimedia data requires a large capacity storage medium and a wide bandwidth for transmission since the amount of data is usually large. For example, a 24-bit true color image having a resolution of 640*480 needs a capacity of 640*480*24 bits, i.e., data of about 7.37 Mbits per frame. When a video composed of such images is transmitted at a speed of 30 frames per second, a bandwidth of 221 Mbits/sec is required. When a 90-minute movie based on such an image is stored, a storage space of about 1200 Gbits is required. Accordingly, a compression coding method is a requisite for transmitting multimedia data.
- A basic principle of multimedia data compression is removing data redundancy. In other words, video data can be compressed by removing spatial, temporal and visual redundancy. Spatial redundancy occurs when the same color or object is repeated in an image. Temporal redundancy occurs when there is little change between adjacent frames in a moving image or the same sound is repeated in audio. The removal of visual redundancy takes into account the limitations of human eyesight and the limited perception of high frequency.
-
FIG. 1 shows an environment in which video compression is applied. - Video data is compressed by a
video encoder 110. Currently known Discrete Cosine Transform (DCT)-based video compression algorithms are MPEG-2, MPEG-4, H.263, and H.264. In recent years, research into wavelet-based scalable video coding has been actively conducted. Compressed video data is sent to avideo decoder 130 via anetwork 120. Thevideo decoder 130 decodes the compressed video data to reconstruct the original video data. - The
video encoder 110 compresses the original video data so it does not exceed the available bandwidth of thenetwork 120. However, communication bandwidth may vary depending on the type of thenetwork 120. For example, the available communication bandwidth of Ethernet is different from that of a wireless local area network (WLAN). A cellular communication network may have a very narrow bandwidth. Thus, research is being actively conducted into a method for generating video data compressed at various bit-rates from the same compressed video data, in particular, scalable video coding. - Scalable video coding is a video compression technique that enables video data to provide scalability. Scalability is the ability to generate video sequences at different resolutions, frame rates, and qualities from the same compressed bitstream. Temporal scalability can be provided using a Motion Compensation Temporal Filtering (MCTF), Unconstrained MCTF (UMCTF), or Successive Temporal Approximation and Referencing (STAR) algorithm. Spatial scalability can be achieved by a wavelet transform algorithm or multi-layer coding, which have been actively studied in recent years. Signal-to-Noise Ratio (SNR) scalability can be obtained using Embedded ZeroTrees Wavelet (EZW), Set Partitioning in Hierarchical Trees (SPIHT), Embedded ZeroBlock Coding (EZBC), or Embedded Block Coding with Optimized Truncation (EBCOT).
- Multi-layer video coding algorithms have recently been adopted for scalable video coding. While conventional multi-layer video coding usually uses a single video coding algorithm, increasing attention has been recently directed to multi-layer video coding using a plurality of video coding algorithms.
-
FIGS. 2 and 3 illustrate examples of bitstreams generated by multi-layer video coding. - Referring to
FIG. 2 , a video encoder uses both a MPEG-4 Advanced Video Coding (AVC) algorithm offering excellent coding efficiency and a wavelet coding technique providing excellent scalability. When a video encoder performs encoding using only wavelet coding, video quality tends to be significantly degraded at a low resolution. Thus, the bitstream shown inFIG. 2 contains AVC-coded lowest-resolution layer frames and highest-resolution layer frames wavelet-coded using the AVC-coded lowest-layer frames. The frame used as a reference during encoding is a frame reconstructed by decoding a frame encoded by AVC coding. - Referring to
FIG. 3 , a video encoder uses both wavelet coding offering the excellent scalability and AVC coding providing high coding efficiency. While the bitstream shown inFIG. 2 has only two wavelet-coded and AVC-coded layers, the bitstream shown inFIG. 3 has a wavelet-coded layer and an AVC-coded layer for each resolution. - Multi-layer video coding has a problem in that the coding efficiency of an enhancement layer tends to be low due to quantization noise in the previously encoded layer (base layer). In particular, the problem is more severe when multi-layer video coding uses a plurality of video coding algorithms having different characteristics. For example, using both DCT-based AVC coding and wavelet-based coding as shown in
FIG. 2 or 3 may degrade the coding efficiency in a wavelet layer. - The present invention provides video coding and decoding methods using interlayer filtering, designed to improve the efficiency of multi-layer video coding, and video encoders and decoders.
- The above stated object as well as other objects, features and advantages, of the present invention will become clear to those skilled in the art upon review of the following description.
- According to an aspect of the present invention, there is provided a video coding method including encoding video frames using a first video coding scheme, performing interlayer filtering on the frames encoded by the first video coding scheme, encoding the video frames using a second video coding scheme by referring the frames subjected to the interlayer filtering, and generating a bitstream containing the frames encoded by the first and second video coding schemes.
- According to another aspect of the present invention, there is provided a video coding method including downsampling video frames to generate video frames of low resolution, encoding the low-resolution video frames using a first video coding scheme, upsampling the frames encoded by the first video coding scheme to the resolution of the video frames, performing interlayer filtering on the upsampled frames, encoding the video frames using a second video coding scheme by referring the frames subjected to the interlayer filtering, and generating a bitstream containing the frames encoded by the first and second video coding schemes.
- According to still another aspect of the present invention, there is provided a video encoder including a first video coding unit encoding video frames using a first video coding scheme, an interlayer filter performing interlayer filtering on the frames encoded by the first video coding scheme, a second video coding unit encoding the video frames using a second video coding scheme by referring the interlayer filtered frames, and a bitstream generator generating a bitstream containing the frames encoded by the first and second video coding schemes.
- According to a further aspect of the present invention, there is provided a video encoder including a downsampler downsampling video frames to generate video frames of low resolution, a first video coding unit encoding the low-resolution video frames using a first video coding scheme, an upsampler upsampling the frames encoded by the first video coding scheme, an interlayer filter performing interlayer filtering on the upsampled frames, a second video coding unit encoding the video frames using a second video coding scheme by referring the interlayer filtered frames, and a bitstream generator generating a bitstream containing the frames encoded by the first and second video coding schemes.
- According to yet another aspect of the present invention, there is provided a video decoding method including extracting frames encoded by first and second video coding schemes from a bitstream, decoding the frames encoded by the first video coding scheme using a first video decoding scheme and reconstructing first layer frames, performing interlayer filtering on the reconstructed first layer frames, and decoding the frames encoded by the second video coding scheme using a second video decoding scheme by referring the interlayer filtered first layer frames and reconstructing second layer frames.
- According to yet another aspect of the present invention, there is provided a video decoding method including extracting frames encoded by first and second video coding schemes from a bitstream, decoding the frames encoded by the first video coding scheme using a first video decoding scheme and reconstructing first layer frames, upsampling the reconstructed first layer frames, performing interlayer filtering on the upsampled first layer frames, and decoding the frames encoded by the second video coding scheme using a second video decoding scheme by referring the interlayer filtered first layer frames and reconstructing second layer frames.
- According to yet another aspect of the present invention, there is provided a video decoder including a bitstream interpreter extracting frames encoded by first and second video coding schemes from a bitstream, a first video decoding unit decoding the frames encoded by the first video coding scheme by referring a first video decoding scheme and reconstructing first layer frames, an interlayer filter performing interlayer filtering on the reconstructed first layer frames, and a second video decoding unit decoding the frames encoded by the second video coding scheme using a second video decoding scheme by referring the interlayer filtered first layer frames and reconstructing second layer frames.
- According to yet another aspect of the present invention, there is provided a video decoder including a video decoder including a bitstream interpreter for extracting frames encoded by first and second video coding schemes from a bitstream, a first video decoding unit for decoding the frames encoded by the first video coding scheme using a first video decoding scheme and reconstructing first layer frames, an upsampler for upsampling the reconstructed first layer frames, an interlayer filter for performing interlayer filtering on the upsampled first layer frames, and a second video decoding unit for decoding the frames encoded by the second video coding scheme using a second video decoding scheme by referring to the interlayer filtered first layer frames and reconstructing second layer frames.
- The above and other aspects of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings in which:
-
FIG. 1 shows an environment in which video compression is applied; -
FIGS. 2 and 3 show example structures of bitstreams generated by multi-layer video coding; -
FIG. 4 is a block diagram of a video encoder according to a first exemplary embodiment of the present invention; -
FIG. 5 is a block diagram of a video encoder according to a second exemplary embodiment of the present invention; -
FIG. 6 is a block diagram of a temporal filter according to an exemplary embodiment of the present invention; -
FIG. 7 is a flowchart illustrating a video coding process according to an exemplary embodiment of the present invention; -
FIG. 8 is a flowchart illustrating a detailed process of encoding a second layer according to an exemplary embodiment of the present invention; -
FIG. 9 is a block diagram of a video decoder according to a first exemplary embodiment of the present invention; -
FIG. 10 is a block diagram of a video decoder according to a second exemplary embodiment of the present invention; -
FIG. 11 is a block diagram of an inverse temporal filter according to an exemplary embodiment of the present invention; -
FIG. 12 is a flowchart illustrating a video decoding process according to an exemplary embodiment of the present invention; and -
FIG. 13 is a flowchart illustrating a detailed process of performing inverse temporal filtering on a second layer according to an exemplary embodiment of the present invention. - Aspects of the present invention and methods of accomplishing the same may be understood more readily by reference to the following detailed description of exemplary embodiments and the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concept of the invention to those skilled in the art, and the present invention will only be defined by the appended claims. Like reference numerals refer to like elements throughout the specification.
- The present invention will now be described more fully with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown. For convenience of explanation, it is assumed that a video encoder has two coding units for two layers.
-
FIG. 4 is a block diagram of a video encoder according to a first exemplary embodiment of the present invention. - Referring to
FIG. 4 , a video encoder according to a first exemplary embodiment of the present invention includes a firstvideo coding unit 410, a secondvideo coding unit 420, abitstream generator 430, and aninterlayer filter 440. - The
first coding unit 410 includes atemporal filter 411, a Discrete Cosine Transform (DCT)transformer 412, and aquantizer 413 and encodes a video frame using advanced video coding (AVC). - The
temporal filter 411 receives avideo frame 400 and removes temporal redundancy that thevideo frame 400 has with adjacent frames. Thetemporal filter 411 may use a Motion Compensated Temporal Filtering (MCTF) algorithm to remove temporal redundancy between frames. The MCTF algorithm supporting temporal scalability removes temporal redundancy between adjacent frames. A 5/3 filter is widely used for MCTF. Other temporal filtering algorithms supporting temporal scalability, such as Unconstrained MCTF (UMCTF) or Successive Temporal Approximation and Referencing (STAR), may be used. - The
DCT transformer 412 performs DCT on the temporally filtered frame. The DCT is performed for each block of a predetermined size (8*8 or 4*4). The entropy of a block subjected to DCT is reduced compared with that of a block before DCT. - The
quantizer 413 quantizes the DCT-transformed frame. In AVC, quantization is determined based on a quantization parameter (Qp). The quantized frame is inserted into a bitstream after being subjected to scanning and entropy coding. - The second
video coding unit 420 includes atemporal filter 421, awavelet transformer 422, and aquantizer 423 and encodes a video frame using wavelet coding. - The
temporal filter 421 receives avideo frame 400 and removes temporal redundancy that thevideo frame 400 has with adjacent frames. Thetemporal filter 421 may use an MCTF algorithm to remove the temporal redundancy between frames. The MCTF algorithm supporting temporal scalability removes temporal redundancy between adjacent frames. Other temporal filtering algorithms supporting temporal scalability, such as UMCTF or STAR, may be used. - The
wavelet transformer 422 performs a wavelet transform on the temporally filtered frame on a frame-by-frame basis. The wavelet transform algorithm supporting spatial scalability decomposes a frame into one low-pass subband (LL) and three high-pass subbands (LH, HL, and HH). The LL subband is a quarter size of and an approximation of the original frame before being subjected to wavelet transform. The wavelet transform is again performed to decompose the LL subband into one low-pass subband (LLLL) and three high-pass subbands (LLLH, LLHL, and LLHH). The LLLL subband is a quarter size of and an approximation of the LL subband. A 9/7 filter is commonly used for the wavelet transform. - The
quantizer 423 quantizes the wavelet-transformed frame. The quantization is performed using an embedded quantization algorithm such as Embedded ZeroTrees Wavelet (EZW), Set Partitioning in Hierarchical Trees (SPIHT), Embedded ZeroBlock Coding (EZBC), or Embedded Block Coding with Optimized Truncation (EBCOT) offering Signal-to-Noise Ratio (SNR) scalability. - The
temporal filter 421 removes temporal redundancy that exists in thevideo frame 400 using adjacent frames or the frame encoded by the firstvideo coding unit 410 as a reference. Block-based DCT, performed before quantization, may cause block artifacts, which degrade the efficiency of wavelet coding performed by the secondvideo coding unit 420. That is, a frame having block artifacts may degrade the wavelet coding efficiency since block artifacts such as noise propagate across the entire frame when the frame is subjected to a wavelet transform. - Thus, the video encoder of
FIG. 4 further includes theinterlayer filter 440 to eliminate noise generated between layers. Theinterlayer filter 440 performs filtering in such a way that a frame encoded by a first video coding method is suitably used as a reference for a second video coding method. Interlayer filtering is needed when different video coding schemes are used for each layer as shown in the video encoder ofFIG. 4 . - The
interlayer filter 440 performs filtering on a frame encoded by DCT-based AVC coding such that it can be suitably used as a reference for wavelet coding. To accomplish this, the interlayer filter downsamples the AVC-coded frame with an MPEG (or other) filter after upsampling the same with a wavelet filter; however, this is merely exemplary. Rather, a downsampling filter in theinterlayer filter 440 may be a low-pass filter showing a steep gradient at a cut-off frequency. Theinterlayer filter 440 may be any single or plurality of filters designed such that the frame subjected to interlayer filtering can be suitably used as a reference by the secondvideo coding unit 420. - The
bitstream generator 430 generates a bitstream containing an AVC-codedframe 431, a wavelet-codedframe 432, motion vectors, and other necessary information. -
FIG. 5 is a block diagram of a video encoder according to a second exemplary embodiment of the present invention. - Referring to
FIG. 5 , the video encoder includes a firstvideo coding unit 510, a secondvideo coding unit 520, abitstream generator 530, an interlayer filter 540, anupsampler 550, and adownsampler 560. The video encoder encodes a video frame and a low-resolution video frame using different video coding schemes. Specifically, thedownsampler 560 downsamples avideo frame 500 to generate a low-resolution video frame. - The first
video coding unit 510 includes atemporal filter 511, aDCT transformer 512, and aquantizer 513 and encodes the low-resolution video frame using AVC coding. - The
temporal filter 511 receives the low-resolution video frame and removes temporal redundancy that the low-resolution frame has with adjacent low-resolution frames. Thetemporal filter 511 uses an MCTF algorithm, which supports temporal scalability, to remove temporal redundancy between low-resolution frames. A 5/3 filter is widely used for MCTF, but other temporal filtering algorithms supporting temporal scalability such as UMCTF or STAR may be used. - The
DCT transformer 512 performs DCT on the temporally filtered frame. The DCT is performed for each block of a predetermined size (8*8 or 4*4). The entropy of a block subjected to DCT is reduced compared with that of a block before DCT. - The
quantizer 513 quantizes the DCT-transformed frame. In AVC, quantization is determined based on a quantization parameter (Qp). The quantized frame is inserted into a bitstream after being subjected to reordering and entropy coding. - The
upsampler 550 upsamples the AVC-coded frame to the resolution of theframe 500. - The interlayer filter 540 performs filtering in such a manner that an upsampled version of a frame can be suitably used as a reference for wavelet coding. In an exemplary embodiment of the present invention, the upsampled version of the frame may be upsampled with a wavelet filter, followed by downsampling using an MPEG filter; however, this is merely exemplary. Rather, a downsampling filter in the interlayer filter 540 may be a low-pass filter showing a steep gradient at a cut-off frequency. The interlayer filter 540 may be any single or plurality of filters designed such that the frame subjected to interlayer filtering can be suitably used as a reference by the second
video coding unit 520. - The second
video coding unit 520 includes atemporal filter 521, awavelet transformer 522, and aquantizer 523, and it encodes a video frame using wavelet coding. - The
temporal filter 521 receives thevideo frame 500 and removes temporal redundancy that thevideo frame 500 has with adjacent frames. In an exemplary embodiment of the present invention, thetemporal filter 521 uses an MCTF algorithm to remove temporal redundancy between low-resolution frames. The MCTF algorithm supporting temporal scalability removes temporal redundancy between adjacent low-resolution frames. Other temporal filtering algorithms supporting temporal scalability such as UMCTF or STAR may be used. - The
wavelet transformer 522 performs a wavelet transform on the temporally filtered frame. Unlike the DCT transform that is performed in units of blocks, the wavelet transform is performed in units of frames. The wavelet transform algorithm supporting spatial scalability decomposes a frame into one low-pass subband (LL) and three high-pass subbands (LH, HL, and HH). The LL subband is a quarter of the size of and an approximation of the original frame before being subjected to wavelet transform. The wavelet transform is again performed to decompose the LL subband into one low-pass subband (LLLL) and three high-pass subbands (LLLH, LLHL, and LLHH). The LLLL subband is a quarter of the size of and an approximation of the LL subband. A 9/7 filter is commonly used for the wavelet transform. - The
quantizer 523 quantizes the wavelet-transformed frame. The quantization may be an embedded quantization algorithm, which provides SNR scalability, such as EZW, SPIHT, EZBC, or EBCOT. - The
temporal filter 521 removes temporal redundancy that exists in thevideo frame 500 using adjacent frames or the frame encoded by the firstvideo coding unit 510 as a reference. The frame encoded by the first video coding unit is upsampled and subjected to interlayer filtering before being sent to thetemporal filter 521. - The
bitstream generator 530 generates a bitstream containing an AVC-codedframe 531, a wavelet-codedframe 532, motion vectors, and other necessary information. - The temporal filter will be described in greater detail with reference to
FIG. 6 . -
FIG. 6 is a block diagram of atemporal filter 600 according to an exemplary embodiment of the present invention. - While
FIGS. 4 and 5 show that the first and second video coding units 410 (510) and 420 (520) include temporal filters 411 (511) and 421 (521), respectively, for convenience of explanation, it is assumed in an exemplary embodiment that thetemporal filter 600 is employed in the secondvideo coding unit 420. - The
temporal filter 600 removes temporal redundancy between video frames using MCTF on a group-of-picture (GOP)-by-GOP basis. To accomplish this function, thetemporal filter 600 includes aprediction frame generator 610 for generating a prediction frame, a prediction frame smoother 620 for smoothing the prediction frame, aresidual frame generator 630 for generating a residual frame by comparing a smoothed prediction frame with a video frame, and anupdater 640 for updating the video frame using the residual frame. - The
prediction frame generator 610 generates a prediction frame that will be compared with a video frame in order to generate a residual frame using video frames adjacent to the video frame and a frame subjected to interlayer filtering as a reference. Theprediction frame generator 610 finds a matching block for each block in the video frame within reference frames (adjacent video frames and a frame subjected to interlayer filtering) (interceding) or within another block in the video frame (intracoding). - The frame prediction smoother 620 smoothes a prediction frame since blocking artifacts are introduced at block boundaries in the prediction frame made up of blocks corresponding to blocks in the video frame. To accomplish this, the prediction frame smoother 620 may perform de-blocking on pixels at block boundaries in the prediction frame. Since a de-blocking algorithm is commonly used in the H.264 video coding scheme and is well known in the art, a detailed explanation thereof will not be given.
- The
residual frame generator 630 compares the smoothed prediction frame with the video frame and generates a residual frame in which temporal redundancy has been removed. - The
updater 640 uses the residual frame to update other video frames. The updated video frames are then provided to theprediction frame generator 610. - For example, when each GOP consists of eight video frames, the
temporal filter 600 removes temporal redundancy inframes 1, 3, 5, and 7 to generateresidual frames 1, 3, 5, and 7, respectively. Theresidual frames 1, 3, 5, and 7 are used to update frames 0, 2, 4, and 6. Thetemporal filter 600 removes the temporal redundancy from updated frames 2 and 6 to generate residual frames 2 and 6. The residual frames 2 and 6 are used to update frames 0 and 4. Then, thetemporal filter 600 removes temporal redundancy in the updated frame 4 to generate a residual frame 4. The residual frame 4 is used to update the frame 0. Through the above process, thetemporal filter 600 performs temporal filtering on the eight video frames to obtain one low-pass frame (updated frame 0) and seven high-pass frames (residual frames 1 through 7). - A video coding process and a temporal filtering will be described with reference to
FIGS. 7 and 8 . It is assumed that video coding is performed on two layers. - Referring first to
FIG. 7 , a video encoder receives a video frame in step S710. - In step S720, the video encoder encodes the input video frame using AVC coding. In the present exemplary embodiment, a first layer is encoded using AVC coding because the AVC coding offers the highest coding efficiency currently available. However, the first layer may be encoded using another video coding algorithm.
- After the first layer is encoded, in step S730 the video encoder performs interlayer filtering on the AVC-coded frame so that it can be suitably used as a reference for encoding a second layer. The interlayer filtering involves upsampling the AVC-coded frame using a wavelet filter and downsampling an upsampled version of AVC-coded frame using an MPEG filter.
- Following the interlayer filtering, in step S740 the video encoder performs wavelet coding on the video frame using the frame subjected to interlayer filtering as a reference.
- After the wavelet coding is finished, in step S750 the video encoder generates a bitstream containing the AVC-coded frame and the wavelet-coded frame. When the first and second layers have different resolutions, the video encoder uses a low-resolution frame obtained by downsampling the input video frame during the AVC coding. After the AVC coding is finished, the video encoder changes the resolution of the AVC-coded frame. For example, when the resolution of the first layer is lower than that of the second layer, the video encoder upsamples the AVC-coded frame to the resolution of the second layer. Then, the video encoder performs interlayer filtering on the upsampled version of the AVC-coded frame.
-
FIG. 8 is a flowchart illustrating a detailed process of encoding a second layer according to an exemplary embodiment of the present invention. - Referring to
FIG. 8 , in step S810 a second video coding unit receives an encoded first layer video frame that has been subjected to interlayer filtering. - In step S820, upon receipt of the video frame and the frame subjected to interlayer filtering, the second video coding unit performs motion estimation in order to generate a prediction frame that will be used in removing temporal redundancy in the video frame. Various well-known algorithms such as Block Matching and Hierarchical Variable Size Block Matching (HVSBM) may be used for motion estimation.
- In step S830, after performing the motion estimation, the second video coding unit uses motion vectors obtained as a result of the motion estimation to generate the prediction frame.
- In step S840, the second video coding unit smoothes the prediction frame in order to reduce block artifacts in a residual frame. This is because an apparent block boundary degrades the coding efficiency during wavelet transform and quantization.
- In step S850, the second video coding unit compares the prediction frame with the video frame to generate a residual frame. The residual frame corresponds to a high-pass frame (H frame) generated through MCTF.
- In step S860, a temporal filter uses a residual frame to update another video frame. An updated version of the video frame corresponds to a low-pass frame (L frame).
- After an L frame and H frame are generated on a GOP basis through steps S820 through S860, the second video coding unit performs wavelet transform on the temporally filtered frames (L and H frames) in step S870. While a 9/7 filter is commonly used for wavelet transform, an 11/9 or 13/11 filter may also be used.
- In step S880, the second video coding unit quantizes the wavelet-transformed frames using EZW, SPIHT, EZBC, or EBCOT.
- Next, a video decoder and a decoding process will be described. While the decoding process is basically performed in reverse order to the encoding process, layers are encoded and decoded in the same order. For example, when a video encoder sequentially encodes first and second layers, a video decoder decodes the first and second layers in the same order. For convenience of explanation, it is assumed that a video frame is reconstructed from a bitstream having two layers.
-
FIG. 9 is a block diagram of a video decoder according to a first exemplary embodiment of the present invention used when first and second layers have the same resolution. - Referring to
FIG. 9 , the video decoder includes abitstream interpreter 900, a firstvideo decoding unit 910, a secondvideo decoding unit 920, and aninterlayer filter 940. - The
bitstream interpreter 900 interprets an input bitstream and extracts frames encoded by a first video coding and frames encoded by second video coding. The frames encoded by the first video coding are then provided to the firstvideo decoding unit 910 while the frames encoded by the second video coding are provided to the secondvideo decoding unit 920. - The first
video decoding unit 910 includes aninverse quantizer 911, aninverse DCT transformer 912, and an inversetemporal filter 913. Theinverse quantizer 911 inversely quantizes the frames encoded by the first video coding. The inverse quantization may involve entropy decoding, inverse scanning, and a process of reconstructing DCT-transformed frames using a quantization table. - The
inverse DCT transformer 912 performs an inverse DCT transform on the inversely quantized frames. - The inverse
temporal filter 913 reconstructs first layer video frames from the inversely DCT-transformed frames and outputs a decoded frame (931). The reconstructed first layer video frames are obtained by encoding original video frames at a low bit-rate and decoding the encoded frames. - The
interlayer filter 940 performs interlayer filtering on the reconstructed first layer frame. The interlayer filtering may be performed using a deblocking algorithm. - The second
video decoding unit 920 includes aninverse quantizer 921, aninverse wavelet transformer 922, and an inversetemporal filter 923. - The
inverse quantizer 921 applies inverse quantization to the frames encoded by the second video coding. The inverse quantization may involve entropy decoding, inverse scanning, and a process of reconstructing wavelet-transformed frames using a quantization table. - The
inverse wavelet transformer 922 performs an inverse wavelet transform on the inversely quantized frames. - The inverse
temporal filter 923 reconstructs second layer video frames from the inversely wavelet-transformed frames using frames subjected to the interlayer filtering as a reference and outputs a decoded frame (932). The reconstructed second layer video frames are obtained by encoding original video frames at a high bit-rate and decoding the encoded frames. -
FIG. 10 is a block diagram of a video decoder according to a second exemplary embodiment of the present invention used when a first layer has lower resolution than a second layer. Referring toFIG. 10 , the video decoder includes a bitstream interpreter 1000, a firstvideo decoding unit 1010, a secondvideo decoding unit 1020, aninterlayer filter 1040, and anupsampler 1050. - The bitstream interpreter 1000 interprets an input bitstream and extracts frames encoded by a first video coding and frames encoded by second video coding. The frames encoded by the first video coding have lower resolution than those encoded by the second video coding. The former is then provided to the first
video decoding unit 1010 while the latter is provided to the secondvideo decoding unit 1020. - The first
video decoding unit 1010 includes aninverse quantizer 1011, aninverse DCT transformer 1012, and an inversetemporal filter 1013. - The
inverse quantizer 1011 inversely quantizes the frames encoded by the first video coding. The inverse quantization may involve entropy decoding, inverse scanning, and a process of reconstructing DCT-transformed frames using a quantization table. - The
inverse DCT transformer 1012 performs an inverse DCT transform on the inversely quantized frames. - The inverse
temporal filter 1013 reconstructs first layer video frames from the inversely DCT-transformed frames and outputs a decoded frame (1031). The reconstructed first layer video frame is obtained by downsampling and encoding an original video frame and decoding the encoded frame. - The
upsampler 1050 upsamples the first layer frame to the resolution of a reconstructed second layer frame. - The
interlayer filter 1040 performs interlayer filtering on an upsampled version of the first layer frame. The interlayer filtering may be performed using a de-blocking algorithm. - The second
video decoding unit 1020 includes aninverse quantizer 1021, aninverse wavelet transformer 1022, and an inversetemporal filter 1023. - The
inverse quantizer 1021 inversely quantizes the frames encoded by the second video coding. The inverse quantization may involve entropy decoding, inverse scanning, and a process of reconstructing wavelet-transformed frames using a quantization table. - The
inverse wavelet transformer 1022 performs an inverse wavelet transform on the inversely quantized frames. - The inverse
temporal filter 1023 reconstructs second layer video frames from the inversely wavelet-transformed frames using frames that have been subjected to the interlayer filtering as a reference and outputs a decoded frame (1032). The reconstructed second layer video frames are obtained by encoding original video frames and decoding the encoded frames. -
FIG. 11 is a block diagram of an inversetemporal filter 1100 according to an exemplary embodiment of the present invention. - While
FIGS. 9 and 10 show that the first and second video decoding units 910 (1010) and 920 (1020) include the inverse temporal filters 913 (1013) and 923 (1023), respectively, for convenience of explanation, it is assumed in an exemplary embodiment that thetemporal filter 1100 is employed in the secondvideo decoding unit 920 shown inFIG. 9 . - The inverse
temporal filter 1100 reconstructs video frames from inversely wavelet-transformed frames on a GOP-by-GOP basis using MCTF. The inversely wavelet-transformed frames fed into the inversetemporal filter 1100 consist of low- and high-pass frames in which temporal redundancies have been removed during video coding. For example, when each GOP is made up of eight frames, the inversely wavelet-transformed frames may include one low-pass frame (updated frame 0) and seven high-pass frames (residual frames 1 through 7) obtained as a result of video coding. - To accomplish this function, the inverse
temporal filter 1100 includes aninverse updater 1110, aprediction frame generator 1120, a prediction frame smoother 1130, and aframe reconstructor 1140. - The
inverse updater 1110 updates the inversely wavelet-transformed frames in the reverse order that video coding is performed. - The
prediction frame generator 1120 generates a prediction frame that will be used in reconstructing a low-pass frame or video frame from a residual frame using an interlayer filtered frame. - The prediction frame smoother 1130 smoothes a prediction frame.
- The
frame reconstructor 1140 reconstructs a low-pass frame or video frame from a high-pass frame using a smoothed prediction frame. - For example, when each GOP is made up of eight video frames, the inverse
temporal filter 1100 uses residual frame 4 and interlayer filtered frame 0 to inversely update a low-pass frame 0. - Then, the low-pass frame 0 is used to generate a prediction frame for the residual frame 4, and the prediction frame is used to obtain low-pass frame 4 from the residual frame 4.
- Then, the inverse
temporal filter 1100 uses residual frames 2 and 6 and interlayer filtered frames 0 and 4 to inversely update low-pass frames 0 and 4, and then uses the low-pass frames 0 and 4 and interlayer filtered frames 2 and 6 to generate prediction frames for the residual frames 2 and 6. The prediction frames are then used to obtain low-pass frames 2 and 6 from the residual frames 2 and 6. - Subsequently, the inverse
temporal filter 1100 usesresidual frames 1, 3, 5, and 7 and interlayer filtered frames 0, 2, 4, and 6 to inversely update the low-pass frames 0, 2, 4, and 6, thereby reconstructing video frames 0, 2, 4, and 6. - Lastly, the inverse
temporal filter 1100 uses the reconstructed frames 0, 2, 4, and 6 and inter-layer filteredframes 1, 3, 5, and 7 to generate prediction frames for theresidual frames 1, 3, 5, and 7. The prediction frames are then used to reconstructvideo frames 1, 3, 5, and 7. - A video decoding process will now be described with reference to
FIGS. 12 and 13 . -
FIG. 12 is a flowchart illustrating a video decoding process according to an exemplary embodiment of the present invention. - In step S1210, a video decoder receives a bitstream and extracts frames encoded by a first video coding and frames encoded by a second video coding. For example, the first and second video coding schemes may be AVC coding and wavelet coding, respectively.
- In step S1220, the video decoder performs AVC decoding on the extracted AVC-coded frames.
- In step S1230, the video decoder performs interlayer filtering on the frames decoded by the AVC decoding.
- In step S1240, the video decoder performs wavelet decoding on the wavelet-coded frames by referring to the interlayer filtered frames.
- In step S1250, after wavelet decoding is finished, the video decoder uses reconstructed video frames to generate a video signal. That is, the video decoder converts luminance (Y) and chrominance (UV) color components of a reconstructed pixel into red (R), green (G), and blue (B) color components.
-
FIG. 13 is a flowchart illustrating a detailed process of performing inverse temporal filtering on a second layer according to an exemplary embodiment of the present invention. - Referring to
FIG. 13 , in step S1310, an inverse temporal filter receives inversely wavelet-transformed frames and interlayer filtered frames. The inversely wavelet-transformed frames are reconstructed on a GOP basis and consist of low- and high-pass frames. - In step S1320, the inverse temporal filter uses a high-pass frame among the inversely wavelet-transformed frames and an interlayer filtered frame to inversely update a low-pass frame.
- In step S1330, the inverse temporal filter uses an inversely updated low-pass frame and an inter layer filtered frame to generate a prediction frame.
- In step S1340, the inverse temporal filter smoothes the prediction frame.
- In step S1350, after smoothing of the prediction frame is finished, the inverse temporal filter uses the smoothed prediction frame and a high-pass frame to reconstruct a low-pass frame or a video frame.
- The present invention provides multi-layer video coding and decoding methods employing a plurality of video coding algorithms that can improve the coding efficiency using interlayer filtering.
- In particular, the present invention can prevent the degradation of coding efficiency that may occur when a video coding scheme with a block-based transform algorithm and a video coding scheme with a frame-based transform algorithm are used together.
- While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the following claims. For example, while a video encoder that employs an AVC coding method and a wavelet coding method has been described, the video encoder may employ other coding methods. Therefore, the disclosed exemplary embodiments of the invention are used in a generic and descriptive sense only and not for purposes of limitation.
Claims (44)
1. A video coding method comprising:
encoding video frames using a first video coding scheme;
performing interlayer filtering on the frames encoded by the first video coding scheme;
encoding the video frames using a second video coding scheme by referring to the frames subjected to the interlayer filtering; and
generating a bitstream containing the frames encoded by the first and second video coding schemes.
2. The video coding method of claim 1 , wherein the first video coding scheme is an Advanced Video Coding (AVC) coding scheme and the second video coding scheme is a wavelet coding scheme.
3. The video coding method of claim 2 , wherein the interlayer filtering includes upsampling the AVC-coded frames using a wavelet filter and downsampling the upsampled frames using an MPEG filter.
4. The video coding method of claim 1 , wherein the encoding of the video frames using the second video coding scheme comprises:
performing Motion Compensation Temporal Filtering (MCTF) on the video frames by referring to the interlayer filtered frames;
performing wavelet transform on the temporally filtered frames; and
quantizing the wavelet-transformed frames.
5. The video coding method of claim 4 , wherein the performing of MCTF comprises:
generating prediction frames for the video frames by referring to the interlayer filtered frames;
smoothing the prediction frames;
generating residual frames for the video frames using the smoothed prediction frames; and
updating the video frames using the residual frames.
6. A video coding method comprising:
downsampling video frames to generate video frames of low resolution;
encoding the low-resolution video frames using a first video coding scheme;
upsampling the frames encoded by the first video coding scheme to the resolution of the video frames;
performing interlayer filtering on the upsampled frames;
encoding the video frames using a second video coding scheme by referring to the frames subjected to the interlayer filtering; and
generating a bitstream containing the frames encoded by the first and second video coding schemes.
7. The video coding method of claim 6 , wherein the first video coding scheme is an Advanced Video Coding (AVC) coding scheme and the second video coding scheme is a wavelet coding scheme.
8. The video coding method of claim 7 , wherein the interlayer filtering includes upsampling the AVC-coded frames using a wavelet filter and downsampling the upsampled frames using an MPEG filter.
9. The video coding method of claim 6 , wherein the encoding of the video frames using the second video coding scheme comprises:
performing Motion Compensation Temporal Filtering (MCTF) on the video frames by referring to the interlayer filtered frames;
performing wavelet transform on the temporally filtered frames; and
quantizing the wavelet-transformed frames.
10. The video coding method of claim 9 , wherein the performing of MCTF comprises:
generating prediction frames for the video frames by referring to the interlayer filtered frames;
smoothing the prediction frames;
generating residual frames for the video frames using the smoothed prediction frames; and
updating the video frames using the residual frames.
11. A video encoder comprising:
a first video coding unit which encodes video frames using a first video coding scheme;
an interlayer filter which performs interlayer filtering on the frames encoded by the first video coding scheme;
a second video coding unit which encodes the video frames using a second video coding scheme by referring to the interlayer filtered frames; and
a bitstream generator which generates a bitstream containing the frames encoded by the first and second video coding schemes.
12. The video encoder of claim 11 , wherein the first video coding scheme is an Advanced Video Coding (AVC) coding scheme and the second video coding scheme is a wavelet coding scheme.
13. The video encoder of claim 12 , wherein the interlayer filtering includes upsampling the AVC-coded frames using a wavelet filter and downsampling the upsampled frames using an MPEG filter.
14. The video encoder of claim 11 , wherein the second video coding unit comprises:
a temporal filter which performs Motion Compensation Temporal Filtering (MCTF) on the video frames by referring to the interlayer filtered frames;
a wavelet transformer which performs a wavelet transform on the temporally filtered frames; and
a quantizer which quantizes the wavelet-transformed frames.
15. The video encoder of claim 14 , wherein the temporal filter comprises:
a prediction frame generator which generates prediction frames for the video frames by referring to the interlayer filtered frames;
a prediction frame smoother which smoothes the prediction frames;
a residual frame generator which generates residual frames for the video frames using the smoothed prediction frames; arid an updater which updates the video frames using the residual frames.
16. A video encoder comprising:
a downsampler which downsamples video frames to generate video frames of a low resolution;
a first video coding unit which encodes the low-resolution video frames using a first video coding scheme;
an upsampler which upsamples the frames encoded by the first video coding scheme;
an interlayer filter which performs interlayer filtering on the upsampled frames;
a second video coding unit which encodes the video frames using a second video coding scheme by referring to the interlayer filtered frames; and
a bitstream generator which generates a bitstream containing the frames encoded by the first and second video coding schemes.
17. The video encoder of claim 16 , wherein the first a video coding scheme is an Advanced Video Coding (AVC) coding scheme and the second video coding scheme is a wavelet coding scheme.
18. The video encoder of claim 17 , wherein the interlayer filtering includes upsampling the AVC-coded frames using a wavelet filter and downsampling the upsampled frames using an MPEG filter.
19. The video encoder of claim 16 , wherein the second video coding unit comprises:
a temporal filter which performs Motion Compensation Temporal Filtering (MCTF) on the video frames by referring to the interlayer filtered frames;
a wavelet transformer which performs wavelet transform on the temporally filtered frames; and
a quantizer which quantizes the wavelet-transformed frames.
20. The video encoder of claim 19 , wherein the temporal filter comprises:
a prediction frame generator which generates prediction frames for the video frames by referring to the interlayer filtered frames;
a prediction frame smoother which smoothes the prediction frames;
a residual frame generator which generates residual frames for the video frames using the smoothed prediction frames; and
an updater which updates the video frames using the residual frames.
21. A video decoding method comprising:
extracting frames encoded by first and second video coding schemes from a bitstream;
decoding the frames encoded by the first video coding scheme using a first video decoding scheme and reconstructing first layer frames;
performing interlayer filtering on the reconstructed first layer frames; and
decoding the frames encoded by the second video coding scheme using a second video decoding scheme by referring to the interlayer filtered first layer frames and reconstructing second layer frames.
22. The video decoding method of claim 21 , wherein the first video coding and decoding scheme is the Advanced Video Coding (AVC) scheme and the second video coding and decoding scheme is the wavelet scheme.
23. The video decoding method of claim 22 , wherein the interlayer filtering includes upsampling the AVC-decoded frames using a wavelet filter and downsampling the upsampled frames using an MPEG filter.
24. The video decoding method of claim 21 , wherein the decoding of the frames using the second video decoding scheme comprises:
inversely quantizing the frames encoded by the second video coding scheme;
performing an inverse wavelet transform on the inversely quantized frames; and
performing inverse temporal filtering on the inversely wavelet-transformed frames using Motion Compensation Temporal Filtering (MCTF) by referring to the interlayer filtered first layer frames.
25. The video decoding method of claim 24 , wherein the performing of inverse temporal filtering on the inversely wavelet-transformed frames using MCTF comprises:
inversely updating the inversely wavelet-transformed frames;
generating prediction frames by referring to the inversely updated frames and the interlayer filtered first layer frames;
smoothing the prediction frames; and
reconstructing the second layer frames by referring to the inversely updated frames and the smoothed prediction frames.
26. A video decoding method comprising:
extracting frames encoded by the first and second video coding schemes from a bitstream;
decoding the frames encoded by the first video coding scheme using a first video decoding scheme and reconstructing first layer frames;
upsampling the reconstructed first layer frames;
performing interlayer filtering on the upsampled first layer frames; and
decoding the frames encoded by the second video coding scheme using a second video decoding scheme by referring to the interlayer filtered first layer frames and reconstructing second layer frames.
27. The video decoding method of claim 26 , wherein the first video coding and decoding scheme is the Advanced Video Coding (AVC) scheme and the second video coding and decoding scheme is the wavelet scheme.
28. The video decoding method of claim 27 , wherein the interlayer filtering includes upsampling the AVC-decoded frames using a wavelet filter and downsampling the upsampled frames using an MPEG filter.
29. The video decoding method of claim 26 , wherein the decoding the frames encoded by the second video coding scheme comprises:
inversely quantizing the frames encoded by the second video coding scheme;
performing an inverse wavelet transform on the inversely quantized frames; and
performing inverse temporal filtering on the inversely wavelet-transformed frames using Motion Compensation Temporal Filtering (MCTF) by referring to the interlayer filtered first layer frames.
30. The video decoding method of claim 29 , wherein the performing of inverse temporal filtering on the inversely wavelet-transformed frames using MCTF comprises:
inversely updating the inversely wavelet-transformed frames;
generating prediction frames by referring to the inversely updated frames and the interlayer filtered first layer frames;
smoothing the prediction frames; and
reconstructing the second layer frames by referring to the inversely updated frames and the smoothed prediction frames.
31. A video decoder comprising:
a bitstream interpreter which extracts from a bitstream frames encoded by first and second video coding schemes;
a first video decoding unit which decodes the frames encoded by the first video coding scheme using a first video decoding scheme and reconstructs first layer frames;
an interlayer filter which performs interlayer filtering on the reconstructed first layer frames; and
a second video decoding unit which decodes the frames encoded by the second video coding scheme using a second video decoding scheme by referring to the interlayer filtered first layer frames and reconstructs second layer frames.
32. The video decoder of claim 31 , wherein the first decoding unit reconstructs the first layer frames using Advanced Video Coding (AVC) decoding scheme and the second decoding unit reconstructs the second layer frames using wavelet decoding scheme.
33. The video decoder of claim 32 , wherein the interlayer filter upsamples the frames decoded by the AVC decoding scheme using a wavelet filter and downsamples the upsampled frames using an MPEG filter.
34. The video decoder of claim 31 , wherein the second video decoding unit comprises:
an inverse quantizer which inversely quantizes the frames encoded by the second video coding scheme;
an inverse wavelet transform unit which performs an inverse wavelet transform on the inversely quantized frames; and
an inverse temporal filtering unit which performs inverse temporal filtering on the inversely wavelet-transformed frames by referring to the interlayer filtered first layer frames using Motion Compensation Temporal Filtering (MCTF).
35. The video decoder of claim 34 , wherein the inverse temporal filtering unit comprises:
an inverse updater which inversely updates the inversely wavelet-transformed frames;
a prediction frame generator which generates prediction frames by referring to the inversely updated frames and the interlayer filtered first layer frames;
a prediction frame smoother which smoothes the prediction frames; and
a frame reconstructor which reconstructs the second layer frames by referring to the inversely updated frames and the smoothed prediction frames.
36. A video decoder comprising:
a bitstream interpreter which extracts frames encoded by first and second video coding schemes from a bitstream;
a first video decoding unit which decodes the frames encoded by the first video coding scheme using a first video decoding scheme and reconstructs first layer frames;
an upsampler which upsamples the reconstructed first layer frames;
an interlayer filter which performs interlayer filtering on the upsampled first layer frames; and
a second video decoding unit which decodes the frames encoded by the second video coding scheme using a second video decoding scheme by referring to the interlayer filtered first layer frames and reconstructs second layer frames.
37. The video decoder of claim 36 , wherein the first decoding unit reconstructs the first layer frames using Advanced Video Coding (AVC) decoding scheme and the second decoding unit reconstructs the second layer frames using wavelet decoding scheme.
38. The video decoder of claim 37 , wherein the interlayer filter upsamples the upsampled first layer frames using a wavelet filter and downsamples the upsampled frames using an MPEG filter.
39. The video decoder of claim 36 , wherein the second video decoding unit comprises:
an inverse quantizer which inversely quantizes the frames encoded by the second video coding scheme;
an inverse wavelet transform unit which performs inverse wavelet transform on the inversely quantized frames; and
an inverse temporal filtering unit which performs inverse temporal filtering on the inversely wavelet-transformed frames by referring to the interlayer filtered first layer frames using Motion Compensation Temporal Filtering (MCTF).
40. The video decoder of claim 39 , wherein the inverse temporal filtering unit comprises:
an inverse updater which inversely updates the inversely wavelet-transformed frames;
a prediction frame generator which generates prediction frames by referring to the inversely updated frames and the interlayer filtered first layer frames;
a prediction frame smoother which smoothes the prediction frames; and
a frame reconstructor which reconstructs the second layer frames by referring to the inversely updated frames and the smoothed prediction frames.
41. A recording medium having a computer readable program recorded therein, the program for executing a video coding method comprising:
encoding video frames using a first video coding scheme;
performing interlayer filtering on the frames encoded by the first video coding scheme;
encoding the video frames using a second video coding scheme by referring to the frames subjected to the interlayer filtering; and
generating a bitstream containing the frames encoded by the first and second video coding schemes.
42. A recording medium having a computer readable program recorded therein, the program for executing a video coding method comprising:
downsampling video frames to generate video frames of low resolution;
encoding the low-resolution video frames using a first video coding scheme;
upsampling the frames encoded by the first video coding scheme to the resolution of the video frames;
performing interlayer filtering on the upsampled frames;
encoding the video frames using a second video coding scheme by referring to the frames subjected to the interlayer filtering; and
generating a bitstream containing the frames encoded by the first and second video coding schemes.
43. A recording medium having a computer readable program recorded therein, the program for executing a video decoding method comprising:
extracting frames encoded by first and second video coding schemes from a bitstream;
decoding the frames encoded by the first video coding scheme using a first video decoding scheme and reconstructing first layer frames;
performing interlayer filtering on the reconstructed first layer frames; and
decoding the frames encoded by the second video coding scheme using a second video decoding scheme by referring to the interlayer filtered first layer frames and reconstructing second layer frames.
44. A recording medium having a computer readable program recorded therein, the program for executing a video decoding method comprising:
extracting frames encoded by the first and second video coding schemes from a bitstream;
decoding the frames encoded by the first video coding scheme using a first video decoding scheme and reconstructing first layer frames;
upsampling the reconstructed first layer frames;
performing interlayer filtering on the upsampled first layer frames; and
decoding the frames encoded by the second video coding scheme using a second video decoding scheme by referring to the interlayer filtered first layer frames and reconstructing second layer frames.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/251,917 US20060083300A1 (en) | 2004-10-18 | 2005-10-18 | Video coding and decoding methods using interlayer filtering and video encoder and decoder using the same |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US61902304P | 2004-10-18 | 2004-10-18 | |
KR1020040094597A KR100679022B1 (en) | 2004-10-18 | 2004-11-18 | Video coding and decoding method using inter-layer filtering, video ecoder and decoder |
KR10-2004-0094597 | 2004-11-18 | ||
US11/251,917 US20060083300A1 (en) | 2004-10-18 | 2005-10-18 | Video coding and decoding methods using interlayer filtering and video encoder and decoder using the same |
Publications (1)
Publication Number | Publication Date |
---|---|
US20060083300A1 true US20060083300A1 (en) | 2006-04-20 |
Family
ID=36748181
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/251,976 Expired - Fee Related US7881387B2 (en) | 2004-10-18 | 2005-10-18 | Apparatus and method for adjusting bitrate of coded scalable bitsteam based on multi-layer |
US11/251,917 Abandoned US20060083300A1 (en) | 2004-10-18 | 2005-10-18 | Video coding and decoding methods using interlayer filtering and video encoder and decoder using the same |
US11/251,869 Expired - Fee Related US7839929B2 (en) | 2004-10-18 | 2005-10-18 | Method and apparatus for predecoding hybrid bitstream |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/251,976 Expired - Fee Related US7881387B2 (en) | 2004-10-18 | 2005-10-18 | Apparatus and method for adjusting bitrate of coded scalable bitsteam based on multi-layer |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/251,869 Expired - Fee Related US7839929B2 (en) | 2004-10-18 | 2005-10-18 | Method and apparatus for predecoding hybrid bitstream |
Country Status (5)
Country | Link |
---|---|
US (3) | US7881387B2 (en) |
KR (3) | KR100679022B1 (en) |
CN (3) | CN101036388A (en) |
BR (1) | BRPI0516826A (en) |
RU (1) | RU2337503C1 (en) |
Cited By (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20080095235A1 (en) * | 2006-10-20 | 2008-04-24 | Motorola, Inc. | Method and apparatus for intra-frame spatial scalable video coding |
WO2008051755A2 (en) * | 2006-10-20 | 2008-05-02 | Motorola, Inc. | Method and apparatus for intra-frame spatial scalable video coding |
WO2008079508A1 (en) * | 2006-12-22 | 2008-07-03 | Motorola, Inc. | Method and system for adaptive coding of a video |
US20090180555A1 (en) * | 2008-01-10 | 2009-07-16 | Microsoft Corporation | Filtering and dithering as pre-processing before encoding |
US20090207912A1 (en) * | 2008-02-15 | 2009-08-20 | Microsoft Corporation | Reducing key picture popping effects in video |
US20100046612A1 (en) * | 2008-08-25 | 2010-02-25 | Microsoft Corporation | Conversion operations in scalable video encoding and decoding |
US20100215099A1 (en) * | 2007-10-23 | 2010-08-26 | Electronics And Telecommunications Research Institute | Multiple quality image contents service system and update method thereof |
US20110216821A1 (en) * | 2010-03-02 | 2011-09-08 | Samsung Electronics Co., Ltd. | Method and apparatus for adaptive streaming using scalable video coding scheme |
US8238424B2 (en) | 2007-02-09 | 2012-08-07 | Microsoft Corporation | Complexity-based adaptive preprocessing for multiple-pass video compression |
US20130044817A1 (en) * | 2008-09-11 | 2013-02-21 | James Bankoski | System and method for video encoding using constructed reference frame |
US20130251027A1 (en) * | 2012-03-20 | 2013-09-26 | Dolby Laboratories Licensing Corporation | Complexity Scalable Multilayer Video Coding |
CN103379319A (en) * | 2012-04-12 | 2013-10-30 | 中兴通讯股份有限公司 | Filtering method, filter and encoder and decoder including filter |
RU2503137C2 (en) * | 2008-01-09 | 2013-12-27 | Моторола Мобилити, Инк. | Method and apparatus for highly scalable intraframe video coding |
US20140161190A1 (en) * | 2006-01-09 | 2014-06-12 | Lg Electronics Inc. | Inter-layer prediction method for video signal |
CN104937941A (en) * | 2013-03-28 | 2015-09-23 | 华为技术有限公司 | Method for determining predictor blocks for a spatially scalable video codec |
US9392280B1 (en) | 2011-04-07 | 2016-07-12 | Google Inc. | Apparatus and method for using an alternate reference frame to decode a video frame |
US9426459B2 (en) | 2012-04-23 | 2016-08-23 | Google Inc. | Managing multi-reference picture buffers and identifiers to facilitate video data coding |
US9667964B2 (en) | 2011-09-29 | 2017-05-30 | Dolby Laboratories Licensing Corporation | Reduced complexity motion compensated temporal processing |
US10306227B2 (en) | 2008-06-03 | 2019-05-28 | Microsoft Technology Licensing, Llc | Adaptive quantization for enhancement layer video coding |
US10602146B2 (en) | 2006-05-05 | 2020-03-24 | Microsoft Technology Licensing, Llc | Flexible Quantization |
US20210329246A1 (en) * | 2018-08-03 | 2021-10-21 | V-Nova International Limited | Architecture for signal enhancement coding |
Families Citing this family (70)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8442108B2 (en) * | 2004-07-12 | 2013-05-14 | Microsoft Corporation | Adaptive updates in motion-compensated temporal filtering |
US8340177B2 (en) * | 2004-07-12 | 2012-12-25 | Microsoft Corporation | Embedded base layer codec for 3D sub-band coding |
US8374238B2 (en) | 2004-07-13 | 2013-02-12 | Microsoft Corporation | Spatial scalability in 3D sub-band decoding of SDMCTF-encoded video |
TWI301953B (en) * | 2005-03-14 | 2008-10-11 | Qisda Corp | Methods and apparatuses for video encoding |
JP4839035B2 (en) * | 2005-07-22 | 2011-12-14 | オリンパス株式会社 | Endoscopic treatment tool and endoscope system |
KR20070038396A (en) | 2005-10-05 | 2007-04-10 | 엘지전자 주식회사 | Method for encoding and decoding video signal |
KR100891663B1 (en) * | 2005-10-05 | 2009-04-02 | 엘지전자 주식회사 | Method for decoding and encoding a video signal |
KR100891662B1 (en) | 2005-10-05 | 2009-04-02 | 엘지전자 주식회사 | Method for decoding and encoding a video signal |
US20070160134A1 (en) * | 2006-01-10 | 2007-07-12 | Segall Christopher A | Methods and Systems for Filter Characterization |
US8619865B2 (en) | 2006-02-16 | 2013-12-31 | Vidyo, Inc. | System and method for thinning of scalable video coding bit-streams |
US8014445B2 (en) * | 2006-02-24 | 2011-09-06 | Sharp Laboratories Of America, Inc. | Methods and systems for high dynamic range video coding |
US8194997B2 (en) * | 2006-03-24 | 2012-06-05 | Sharp Laboratories Of America, Inc. | Methods and systems for tone mapping messaging |
KR100781524B1 (en) * | 2006-04-04 | 2007-12-03 | 삼성전자주식회사 | Method and apparatus for encoding/decoding using extended macroblock skip mode |
FR2903556B1 (en) * | 2006-07-04 | 2008-10-03 | Canon Kk | METHODS AND DEVICES FOR ENCODING AND DECODING IMAGES, A TELECOMMUNICATIONS SYSTEM COMPRISING SUCH DEVICES AND COMPUTER PROGRAMS USING SUCH METHODS |
US8422548B2 (en) * | 2006-07-10 | 2013-04-16 | Sharp Laboratories Of America, Inc. | Methods and systems for transform selection and management |
US8532176B2 (en) * | 2006-07-10 | 2013-09-10 | Sharp Laboratories Of America, Inc. | Methods and systems for combining layers in a multi-layer bitstream |
US7885471B2 (en) * | 2006-07-10 | 2011-02-08 | Sharp Laboratories Of America, Inc. | Methods and systems for maintenance and use of coded block pattern information |
US8130822B2 (en) * | 2006-07-10 | 2012-03-06 | Sharp Laboratories Of America, Inc. | Methods and systems for conditional transform-domain residual accumulation |
US7535383B2 (en) * | 2006-07-10 | 2009-05-19 | Sharp Laboratories Of America Inc. | Methods and systems for signaling multi-layer bitstream data |
US8059714B2 (en) * | 2006-07-10 | 2011-11-15 | Sharp Laboratories Of America, Inc. | Methods and systems for residual layer scaling |
US7840078B2 (en) * | 2006-07-10 | 2010-11-23 | Sharp Laboratories Of America, Inc. | Methods and systems for image processing control based on adjacent block characteristics |
KR100842544B1 (en) * | 2006-09-11 | 2008-07-01 | 삼성전자주식회사 | Method for Transmitting Scalable Video Coding in Using and Mobil Communication System Using The Same |
US8233536B2 (en) | 2007-01-23 | 2012-07-31 | Sharp Laboratories Of America, Inc. | Methods and systems for multiplication-free inter-layer image prediction |
US8503524B2 (en) * | 2007-01-23 | 2013-08-06 | Sharp Laboratories Of America, Inc. | Methods and systems for inter-layer image prediction |
US7826673B2 (en) * | 2007-01-23 | 2010-11-02 | Sharp Laboratories Of America, Inc. | Methods and systems for inter-layer image prediction with color-conversion |
US8665942B2 (en) * | 2007-01-23 | 2014-03-04 | Sharp Laboratories Of America, Inc. | Methods and systems for inter-layer image prediction signaling |
US8243789B2 (en) * | 2007-01-25 | 2012-08-14 | Sharp Laboratories Of America, Inc. | Methods and systems for rate-adaptive transmission of video |
US7760949B2 (en) | 2007-02-08 | 2010-07-20 | Sharp Laboratories Of America, Inc. | Methods and systems for coding multiple dynamic range images |
US8767834B2 (en) | 2007-03-09 | 2014-07-01 | Sharp Laboratories Of America, Inc. | Methods and systems for scalable-to-non-scalable bit-stream rewriting |
KR100918385B1 (en) * | 2007-08-10 | 2009-09-21 | 한국전자통신연구원 | Method and Apparatus for informing the information about the reference discardable slice in Scalable Video Coding |
JP5216303B2 (en) * | 2007-11-01 | 2013-06-19 | 株式会社東芝 | Composite video distribution apparatus and method and program thereof |
US8953673B2 (en) * | 2008-02-29 | 2015-02-10 | Microsoft Corporation | Scalable video coding and decoding with sample bit depth and chroma high-pass residual layers |
US8711948B2 (en) | 2008-03-21 | 2014-04-29 | Microsoft Corporation | Motion-compensated prediction of inter-layer residuals |
CN101557510A (en) * | 2008-04-09 | 2009-10-14 | 华为技术有限公司 | Method, system and device for processing video coding |
US8213503B2 (en) * | 2008-09-05 | 2012-07-03 | Microsoft Corporation | Skip modes for inter-layer residual video coding and decoding |
US8265140B2 (en) * | 2008-09-30 | 2012-09-11 | Microsoft Corporation | Fine-grained client-side control of scalable media delivery |
WO2010055399A1 (en) * | 2008-11-12 | 2010-05-20 | Nokia Corporation | Method and apparatus for representing and identifying feature descriptors utilizing a compressed histogram of gradients |
JP4947389B2 (en) * | 2009-04-03 | 2012-06-06 | ソニー株式会社 | Image signal decoding apparatus, image signal decoding method, and image signal encoding method |
KR101557504B1 (en) * | 2009-04-13 | 2015-10-07 | 삼성전자주식회사 | Method for transmitting adapted channel condition apparatus using the method and providing system |
KR101484280B1 (en) | 2009-12-08 | 2015-01-20 | 삼성전자주식회사 | Method and apparatus for video encoding by motion prediction using arbitrary partition, and method and apparatus for video decoding by motion compensation using arbitrary partition |
US8428364B2 (en) | 2010-01-15 | 2013-04-23 | Dolby Laboratories Licensing Corporation | Edge enhancement for temporal scaling with metadata |
JP5510012B2 (en) * | 2010-04-09 | 2014-06-04 | ソニー株式会社 | Image processing apparatus and method, and program |
US8392201B2 (en) * | 2010-07-30 | 2013-03-05 | Deutsche Telekom Ag | Method and system for distributed audio transcoding in peer-to-peer systems |
US20120076205A1 (en) * | 2010-09-29 | 2012-03-29 | Segall Christopher A | Methods and Systems for Capturing Wide Color-Gamut Video |
GB2487200A (en) * | 2011-01-12 | 2012-07-18 | Canon Kk | Video encoding and decoding with improved error resilience |
KR20120096863A (en) * | 2011-02-23 | 2012-08-31 | 한국전자통신연구원 | Rate control for hierarchical coding structure in high efficiency video coding(hevc) |
JP2012191465A (en) * | 2011-03-11 | 2012-10-04 | Sony Corp | Image processing apparatus, image processing method, and program |
CN102129838B (en) * | 2011-04-28 | 2013-01-23 | 深圳市天微电子有限公司 | LED (Light-Emitting Diode) display data processing method and device |
JP5989772B2 (en) * | 2011-07-01 | 2016-09-07 | サムスン エレクトロニクス カンパニー リミテッド | Entropy encoding method and apparatus using data unit of hierarchical structure, and decoding method and apparatus |
US9185424B2 (en) * | 2011-07-05 | 2015-11-10 | Qualcomm Incorporated | Image data compression |
US8948248B2 (en) | 2011-07-21 | 2015-02-03 | Luca Rossato | Tiered signal decoding and signal reconstruction |
US9271055B2 (en) * | 2011-08-23 | 2016-02-23 | Avaya Inc. | System and method for variable video degradation counter-measures |
US9467708B2 (en) | 2011-08-30 | 2016-10-11 | Sonic Ip, Inc. | Selection of resolutions for seamless resolution switching of multimedia content |
WO2013033458A2 (en) * | 2011-08-30 | 2013-03-07 | Divx, Llc | Systems and methods for encoding and streaming video encoded using a plurality of maximum bitrate levels |
US8976857B2 (en) * | 2011-09-23 | 2015-03-10 | Microsoft Technology Licensing, Llc | Quality-based video compression |
US20130318251A1 (en) * | 2012-05-22 | 2013-11-28 | Alimuddin Mohammad | Adaptive multipath content streaming |
US9313486B2 (en) * | 2012-06-20 | 2016-04-12 | Vidyo, Inc. | Hybrid video coding techniques |
US9900593B2 (en) | 2012-08-29 | 2018-02-20 | Vid Scale, Inc. | Method and apparatus of motion vector prediction for scalable video coding |
EP3661215A1 (en) * | 2012-09-28 | 2020-06-03 | Vid Scale, Inc. | Cross-plane filtering for chroma signal enhancement in video coding |
US10085017B2 (en) * | 2012-11-29 | 2018-09-25 | Advanced Micro Devices, Inc. | Bandwidth saving architecture for scalable video coding spatial mode |
TWI597968B (en) * | 2012-12-21 | 2017-09-01 | 杜比實驗室特許公司 | High precision up-sampling in scalable coding of high bit-depth video |
US9413951B2 (en) | 2014-07-31 | 2016-08-09 | Apple Inc. | Dynamic motion estimation and compensation for temporal filtering |
US9374526B2 (en) | 2014-07-31 | 2016-06-21 | Apple Inc. | Providing frame delay using a temporal filter |
US9479695B2 (en) | 2014-07-31 | 2016-10-25 | Apple Inc. | Generating a high dynamic range image using a temporal filter |
US9514525B2 (en) | 2014-07-31 | 2016-12-06 | Apple Inc. | Temporal filtering for image data using spatial filtering and noise history |
US10893266B2 (en) * | 2014-10-07 | 2021-01-12 | Disney Enterprises, Inc. | Method and system for optimizing bitrate selection |
KR102338980B1 (en) * | 2015-03-23 | 2021-12-13 | 삼성전자주식회사 | Encoder for adjusting quantization coefficient to remove flicker and device including the same |
TWI731063B (en) * | 2017-04-13 | 2021-06-21 | 物聯智慧科技(深圳)有限公司 | Processing and producing method for an av file |
US11265553B2 (en) * | 2017-09-22 | 2022-03-01 | V-Nova International Limited | Obtaining a target representation of a time sample of a signal |
US10728180B2 (en) * | 2018-08-21 | 2020-07-28 | At&T Intellectual Property I, L.P. | Apparatus, storage medium and method for adaptive bitrate streaming adaptation of variable bitrate encodings |
Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6263022B1 (en) * | 1999-07-06 | 2001-07-17 | Philips Electronics North America Corp. | System and method for fine granular scalable video with selective quality enhancement |
US20020118742A1 (en) * | 2001-02-26 | 2002-08-29 | Philips Electronics North America Corporation. | Prediction structures for enhancement layer in fine granular scalability video coding |
US20030002579A1 (en) * | 1998-07-06 | 2003-01-02 | U.S. Philips Corporation | Scalable video coding system |
US20030026341A1 (en) * | 2001-07-24 | 2003-02-06 | Sharp Laboratories Of America, Inc. | Resolution-scalable video compression |
US20030118099A1 (en) * | 2001-12-20 | 2003-06-26 | Comer Mary Lafuze | Fine-grain scalable video encoder with conditional replacement |
US20030138043A1 (en) * | 2002-01-23 | 2003-07-24 | Miska Hannuksela | Grouping of image frames in video coding |
US6625320B1 (en) * | 1997-11-27 | 2003-09-23 | British Telecommunications Public Limited Company | Transcoding |
US6639943B1 (en) * | 1999-11-23 | 2003-10-28 | Koninklijke Philips Electronics N.V. | Hybrid temporal-SNR fine granular scalability video coding |
US6728317B1 (en) * | 1996-01-30 | 2004-04-27 | Dolby Laboratories Licensing Corporation | Moving image compression quality enhancement using displacement filters with negative lobes |
US20050078750A1 (en) * | 2003-10-14 | 2005-04-14 | Matsushita Electric Industrial Co., Ltd. | De-blocking filter processing apparatus and de-blocking filter processing method |
US20060008003A1 (en) * | 2004-07-12 | 2006-01-12 | Microsoft Corporation | Embedded base layer codec for 3D sub-band coding |
US20060083308A1 (en) * | 2004-10-15 | 2006-04-20 | Heiko Schwarz | Apparatus and method for generating a coded video sequence and for decoding a coded video sequence by using an intermediate layer residual value prediction |
US20060114993A1 (en) * | 2004-07-13 | 2006-06-01 | Microsoft Corporation | Spatial scalability in 3D sub-band decoding of SDMCTF-encoded video |
Family Cites Families (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0336978A (en) * | 1989-06-30 | 1991-02-18 | Matsushita Electric Ind Co Ltd | Motor-speed controller |
JP3164647B2 (en) * | 1992-06-03 | 2001-05-08 | 株式会社東芝 | Video coding method and apparatus |
CA2126467A1 (en) * | 1993-07-13 | 1995-01-14 | Barin Geoffry Haskell | Scalable encoding and decoding of high-resolution progressive video |
US5754241A (en) * | 1994-11-18 | 1998-05-19 | Sanyo Electric Co., Ltd | Video decoder capable of controlling encoded video data |
KR100363162B1 (en) * | 1998-11-02 | 2003-01-24 | 삼성전자 주식회사 | Method and apparatus for transmitting and recovering video signal |
US6529552B1 (en) * | 1999-02-16 | 2003-03-04 | Packetvideo Corporation | Method and a device for transmission of a variable bit-rate compressed video bitstream over constant and variable capacity networks |
JP2000333163A (en) * | 1999-05-24 | 2000-11-30 | Sony Corp | Decoder, its method, coder, its method, image processing system and image processing method |
KR20020070362A (en) * | 1999-12-22 | 2002-09-06 | 제너럴 인스트루먼트 코포레이션 | Video compression for multicast environments using spatial scalability and simulcast coding |
US7095782B1 (en) * | 2000-03-01 | 2006-08-22 | Koninklijke Philips Electronics N.V. | Method and apparatus for streaming scalable video |
US6771703B1 (en) * | 2000-06-30 | 2004-08-03 | Emc Corporation | Efficient scaling of nonscalable MPEG-2 Video |
WO2002025925A2 (en) * | 2000-09-22 | 2002-03-28 | Koninklijke Philips Electronics Nv | Hybrid temporal-snr fine granular scalability video coding |
CN1636394A (en) * | 2000-10-11 | 2005-07-06 | 皇家菲利浦电子有限公司 | Spatial scalability for fine granular video encoding |
EP1217839A3 (en) * | 2000-12-08 | 2009-09-09 | Panasonic Corporation | Data conversion apparatus, data coding apparatus, and data recording apparatus |
US7203238B2 (en) * | 2000-12-11 | 2007-04-10 | Sony Corporation | 3:2 Pull-down detection |
KR100778471B1 (en) * | 2001-04-21 | 2007-11-21 | 엘지전자 주식회사 | Method for encoding or decoding of video signal |
CN100350488C (en) | 2001-09-06 | 2007-11-21 | 汤姆森特许公司 | Method and apparatus for elapsed playback timekeeping of variable bit-rate digitally encoded audio data files |
JP2005506815A (en) | 2001-10-26 | 2005-03-03 | コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ | Method and apparatus for spatially extensible compression |
US7317759B1 (en) * | 2002-02-28 | 2008-01-08 | Carnegie Mellon University | System and methods for video compression mode decisions |
US20030215011A1 (en) | 2002-05-17 | 2003-11-20 | General Instrument Corporation | Method and apparatus for transcoding compressed video bitstreams |
KR100484148B1 (en) * | 2002-07-27 | 2005-04-18 | 삼성전자주식회사 | Advanced method for rate control and apparatus thereof |
KR20040047010A (en) | 2002-11-28 | 2004-06-05 | 엘지전자 주식회사 | Bit rate control method based on gob of video telephony system |
KR100543608B1 (en) | 2003-01-03 | 2006-01-20 | 엘지전자 주식회사 | Bit rate control system based on object |
US20050008240A1 (en) * | 2003-05-02 | 2005-01-13 | Ashish Banerji | Stitching of video for continuous presence multipoint video conferencing |
KR100621581B1 (en) | 2004-07-15 | 2006-09-13 | 삼성전자주식회사 | Method for pre-decoding, decoding bit-stream including base-layer, and apparatus thereof |
KR200447010Y1 (en) | 2007-10-01 | 2009-12-17 | 이운영 | A electrical pot which boiled dishcloth |
-
2004
- 2004-11-18 KR KR1020040094597A patent/KR100679022B1/en not_active IP Right Cessation
- 2004-12-17 KR KR1020040107960A patent/KR100703724B1/en not_active IP Right Cessation
-
2005
- 2005-01-25 KR KR1020050006803A patent/KR100679030B1/en not_active IP Right Cessation
- 2005-09-13 CN CNA2005800340197A patent/CN101036388A/en active Pending
- 2005-09-16 BR BRPI0516826-0A patent/BRPI0516826A/en not_active IP Right Cessation
- 2005-09-16 RU RU2007114587/09A patent/RU2337503C1/en not_active IP Right Cessation
- 2005-09-28 CN CN2005800321957A patent/CN101027908B/en not_active Expired - Fee Related
- 2005-10-18 US US11/251,976 patent/US7881387B2/en not_active Expired - Fee Related
- 2005-10-18 US US11/251,917 patent/US20060083300A1/en not_active Abandoned
- 2005-10-18 CN CNB021604053A patent/CN100466738C/en not_active Expired - Fee Related
- 2005-10-18 US US11/251,869 patent/US7839929B2/en not_active Expired - Fee Related
Patent Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6728317B1 (en) * | 1996-01-30 | 2004-04-27 | Dolby Laboratories Licensing Corporation | Moving image compression quality enhancement using displacement filters with negative lobes |
US6625320B1 (en) * | 1997-11-27 | 2003-09-23 | British Telecommunications Public Limited Company | Transcoding |
US20030002579A1 (en) * | 1998-07-06 | 2003-01-02 | U.S. Philips Corporation | Scalable video coding system |
US6263022B1 (en) * | 1999-07-06 | 2001-07-17 | Philips Electronics North America Corp. | System and method for fine granular scalable video with selective quality enhancement |
US6639943B1 (en) * | 1999-11-23 | 2003-10-28 | Koninklijke Philips Electronics N.V. | Hybrid temporal-SNR fine granular scalability video coding |
US20020118742A1 (en) * | 2001-02-26 | 2002-08-29 | Philips Electronics North America Corporation. | Prediction structures for enhancement layer in fine granular scalability video coding |
US20030026341A1 (en) * | 2001-07-24 | 2003-02-06 | Sharp Laboratories Of America, Inc. | Resolution-scalable video compression |
US20030118099A1 (en) * | 2001-12-20 | 2003-06-26 | Comer Mary Lafuze | Fine-grain scalable video encoder with conditional replacement |
US20030138043A1 (en) * | 2002-01-23 | 2003-07-24 | Miska Hannuksela | Grouping of image frames in video coding |
US20050078750A1 (en) * | 2003-10-14 | 2005-04-14 | Matsushita Electric Industrial Co., Ltd. | De-blocking filter processing apparatus and de-blocking filter processing method |
US20060008003A1 (en) * | 2004-07-12 | 2006-01-12 | Microsoft Corporation | Embedded base layer codec for 3D sub-band coding |
US20060114993A1 (en) * | 2004-07-13 | 2006-06-01 | Microsoft Corporation | Spatial scalability in 3D sub-band decoding of SDMCTF-encoded video |
US20060083308A1 (en) * | 2004-10-15 | 2006-04-20 | Heiko Schwarz | Apparatus and method for generating a coded video sequence and for decoding a coded video sequence by using an intermediate layer residual value prediction |
Cited By (34)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140161190A1 (en) * | 2006-01-09 | 2014-06-12 | Lg Electronics Inc. | Inter-layer prediction method for video signal |
US9497453B2 (en) * | 2006-01-09 | 2016-11-15 | Lg Electronics Inc. | Inter-layer prediction method for video signal |
US10602146B2 (en) | 2006-05-05 | 2020-03-24 | Microsoft Technology Licensing, Llc | Flexible Quantization |
WO2008051755A2 (en) * | 2006-10-20 | 2008-05-02 | Motorola, Inc. | Method and apparatus for intra-frame spatial scalable video coding |
WO2008051755A3 (en) * | 2006-10-20 | 2008-09-04 | Motorola Inc | Method and apparatus for intra-frame spatial scalable video coding |
US20080095235A1 (en) * | 2006-10-20 | 2008-04-24 | Motorola, Inc. | Method and apparatus for intra-frame spatial scalable video coding |
WO2008079508A1 (en) * | 2006-12-22 | 2008-07-03 | Motorola, Inc. | Method and system for adaptive coding of a video |
US8238424B2 (en) | 2007-02-09 | 2012-08-07 | Microsoft Corporation | Complexity-based adaptive preprocessing for multiple-pass video compression |
US20100215099A1 (en) * | 2007-10-23 | 2010-08-26 | Electronics And Telecommunications Research Institute | Multiple quality image contents service system and update method thereof |
RU2503137C2 (en) * | 2008-01-09 | 2013-12-27 | Моторола Мобилити, Инк. | Method and apparatus for highly scalable intraframe video coding |
US8750390B2 (en) | 2008-01-10 | 2014-06-10 | Microsoft Corporation | Filtering and dithering as pre-processing before encoding |
US20090180555A1 (en) * | 2008-01-10 | 2009-07-16 | Microsoft Corporation | Filtering and dithering as pre-processing before encoding |
US8160132B2 (en) * | 2008-02-15 | 2012-04-17 | Microsoft Corporation | Reducing key picture popping effects in video |
US20090207912A1 (en) * | 2008-02-15 | 2009-08-20 | Microsoft Corporation | Reducing key picture popping effects in video |
US10306227B2 (en) | 2008-06-03 | 2019-05-28 | Microsoft Technology Licensing, Llc | Adaptive quantization for enhancement layer video coding |
US20100046612A1 (en) * | 2008-08-25 | 2010-02-25 | Microsoft Corporation | Conversion operations in scalable video encoding and decoding |
US10250905B2 (en) | 2008-08-25 | 2019-04-02 | Microsoft Technology Licensing, Llc | Conversion operations in scalable video encoding and decoding |
US9571856B2 (en) | 2008-08-25 | 2017-02-14 | Microsoft Technology Licensing, Llc | Conversion operations in scalable video encoding and decoding |
US20130044817A1 (en) * | 2008-09-11 | 2013-02-21 | James Bankoski | System and method for video encoding using constructed reference frame |
US11375240B2 (en) | 2008-09-11 | 2022-06-28 | Google Llc | Video coding using constructed reference frames |
US9374596B2 (en) * | 2008-09-11 | 2016-06-21 | Google Inc. | System and method for video encoding using constructed reference frame |
US10165306B2 (en) * | 2008-09-11 | 2018-12-25 | Google Llc | System and method for video encoding using constructed reference frame |
US20110216821A1 (en) * | 2010-03-02 | 2011-09-08 | Samsung Electronics Co., Ltd. | Method and apparatus for adaptive streaming using scalable video coding scheme |
US9392280B1 (en) | 2011-04-07 | 2016-07-12 | Google Inc. | Apparatus and method for using an alternate reference frame to decode a video frame |
US9667964B2 (en) | 2011-09-29 | 2017-05-30 | Dolby Laboratories Licensing Corporation | Reduced complexity motion compensated temporal processing |
US9641852B2 (en) | 2012-03-20 | 2017-05-02 | Dolby Laboratories Licensing Corporation | Complexity scalable multilayer video coding |
US9247246B2 (en) * | 2012-03-20 | 2016-01-26 | Dolby Laboratories Licensing Corporation | Complexity scalable multilayer video coding |
US20130251027A1 (en) * | 2012-03-20 | 2013-09-26 | Dolby Laboratories Licensing Corporation | Complexity Scalable Multilayer Video Coding |
CN103379319A (en) * | 2012-04-12 | 2013-10-30 | 中兴通讯股份有限公司 | Filtering method, filter and encoder and decoder including filter |
US9426459B2 (en) | 2012-04-23 | 2016-08-23 | Google Inc. | Managing multi-reference picture buffers and identifiers to facilitate video data coding |
US10250878B2 (en) | 2013-03-28 | 2019-04-02 | Huawei Technologies Co., Ltd. | Method for determining predictor blocks for a spatially scalable video codec |
CN104937941A (en) * | 2013-03-28 | 2015-09-23 | 华为技术有限公司 | Method for determining predictor blocks for a spatially scalable video codec |
US10944966B2 (en) | 2013-03-28 | 2021-03-09 | Huawei Technologies Co., Ltd. | Method for determining predictor blocks for a spatially scalable video codec |
US20210329246A1 (en) * | 2018-08-03 | 2021-10-21 | V-Nova International Limited | Architecture for signal enhancement coding |
Also Published As
Publication number | Publication date |
---|---|
KR20060034197A (en) | 2006-04-21 |
CN101036388A (en) | 2007-09-12 |
KR100703724B1 (en) | 2007-04-05 |
CN101027908B (en) | 2010-06-16 |
RU2337503C1 (en) | 2008-10-27 |
CN101027908A (en) | 2007-08-29 |
KR20060034195A (en) | 2006-04-21 |
KR20060034192A (en) | 2006-04-21 |
US7881387B2 (en) | 2011-02-01 |
US20060083303A1 (en) | 2006-04-20 |
KR100679022B1 (en) | 2007-02-05 |
BRPI0516826A (en) | 2008-09-23 |
US7839929B2 (en) | 2010-11-23 |
KR100679030B1 (en) | 2007-02-05 |
US20060083302A1 (en) | 2006-04-20 |
CN100466738C (en) | 2009-03-04 |
CN1764276A (en) | 2006-04-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20060083300A1 (en) | Video coding and decoding methods using interlayer filtering and video encoder and decoder using the same | |
US7933456B2 (en) | Multi-layer video coding and decoding methods and multi-layer video encoder and decoder | |
US8031776B2 (en) | Method and apparatus for predecoding and decoding bitstream including base layer | |
US20060013310A1 (en) | Temporal decomposition and inverse temporal decomposition methods for video encoding and decoding and video encoder and decoder | |
Andreopoulos et al. | In-band motion compensated temporal filtering | |
US8929436B2 (en) | Method and apparatus for video coding, predecoding, and video decoding for video streaming service, and image filtering method | |
KR100679011B1 (en) | Scalable video coding method using base-layer and apparatus thereof | |
US20060008006A1 (en) | Video encoding and decoding methods and video encoder and decoder | |
EP1538566A2 (en) | Method and apparatus for scalable video encoding and decoding | |
US20050163224A1 (en) | Device and method for playing back scalable video streams | |
US20060013312A1 (en) | Method and apparatus for scalable video coding and decoding | |
US20060013311A1 (en) | Video decoding method using smoothing filter and video decoder therefor | |
Andreopoulos et al. | Complete-to-overcomplete discrete wavelet transforms for scalable video coding with MCTF | |
EP1657932A1 (en) | Video coding and decoding methods using interlayer filtering and video encoder and decoder using the same | |
Sullivan | General characteristics and design considerations for temporal subband video coding | |
EP1766986A1 (en) | Temporal decomposition and inverse temporal decomposition methods for video encoding and decoding and video encoder and decoder | |
WO2006028330A1 (en) | Multi-layer video coding and decoding methods and multi-layer video encoder and decoder |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: SAMSUNG ELECTRONICS CO., LTD., KOREA, REPUBLIC OF Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HAN, WOO-JIN;LEE, BAE-KEUN;LEE, JAE-YOUNG;AND OTHERS;REEL/FRAME:017111/0909;SIGNING DATES FROM 20051006 TO 20051010 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |