WO2009047694A1

WO2009047694A1 - Method and system for managing the encoding of digital video content

Info

Publication number: WO2009047694A1
Application number: PCT/IB2008/054085
Authority: WO
Inventors: Hugues J. M. De Perthuis; Patrick E. Lejoly
Original assignee: Nxp B.V.
Priority date: 2007-10-08
Filing date: 2008-10-06
Publication date: 2009-04-16

Abstract

A technique for managing the encoding of digital video content that is organized into discrete units of pictures, slices, and macroblocks, involves identifying, at the slice or macroblock level,that a unit of digital video content from a stream of digital video content is redundant relative to another unit of the digital video content and adjusting encoding that is used to encode the stream of digital video content in response to the identified redundant unit. For example, the redundancy information comprises an identification of redundant slices or macroblocks. A redundant unit of digital video content can be identified by obtaining redundancy information from graphics hardware, obtaining redundancy information from an application, or hashing the unit of digital video content and comparing the hash value to a previously generated hash value.

Description

Method and system for managing the encoding of digital video content

FIELD OF THE INVENTION

The invention relates generally to wireless communications systems, and more particularly, to managing the encoding of digital video content that is to be wirelessly transmitted.

BACKGROUND OF THE INVENTION

Radio frequency wireless technologies are being developed to enable the wireless distribution of rich digital video content within a local environment such as a home or office. For example, the WiMedia Alliance has developed the WiMedia Ultra- Wideband (UWB) Common Radio Platform, which incorporates media access control (MAC) layer and physical (PHY) layer specifications based on Multi-band Orthogonal Frequency Division Multiplexing (MB-OFDM). The WiMedia UWB Common Radio Platform enables shortrange multimedia file transfers at data rates of 480 Mbit/s and beyond with low power consumption using the 3.1 to 10.6 GHz UWB spectrum. WiMedia UWB Common Radio Platform is optimized for the personal computers (PCs), consumer electronic (CE) devices, mobile devices, and automotive applications.

Although wireless technologies such as WiMedia UWB Common Radio Platform exist for distributing digital video content within a local environment, such technologies typically require extensive encoding (and decoding) of digital video content. The extensive encoding (and decoding) is typically accomplished using brut force processing power, large amounts of high-speed random access memory (RAM), and wide data buses, all of which add cost to the components that are required to support the wireless distribution of digital video content.

SUMMARY OF THE INVENTION A technique, in accordance with an embodiment of the invention, for managing the encoding of digital video content that is organized into discrete units of pictures, slices, and macroblocks, involves identifying, at the slice or macroblock level, that a unit of digital video content from a stream of digital video content is redundant relative to another unit of the digital video content and adjusting encoding that is used to encode the stream of digital video content in response to the identified redundant unit. For example, the redundancy information comprises an identification of redundant slices or macrob locks. A redundant unit of digital video content can be identified by obtaining redundancy information from graphics hardware, obtaining redundancy information from an application, or hashing the unit of digital video content and comparing the hash value to a previously generated hash value.

Once the stream of digital video content is encoded, the encoded stream is transmitted using a short range wireless transmission protocol. In an embodiment, only I- pictures are encoded. A system for managing a stream of digital video content that is organized into discrete units of pictures, slices, and macroblocks includes a redundancy identification unit configured to identify, at the slice or macroblock level, that a unit of digital video content from a stream of digital video content is redundant relative to another unit of the digital video content, an encoder configured to adjust encoding of the stream of digital video content in response to the identified redundant unit, and a wireless transmitter for transmitting the encoded stream of digital video content.

Because slice-level and macroblock-level redundancies are identified within a stream of digital video content, the redundant portions can be re-used for decoding and therefore do not need to be transmitted across the wireless link. Because resources are not being consumed to compress/decompress and wirelessly transmit redundant slices and/or macroblocks of digital video content, the compression/decompression and transmission resources can be put to other uses that will improve the overall performance of the wireless link.

BRIEF DESCRIPTION OF THE DRAWINGS

Other aspects and advantages of the present invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.

Fig. IA illustrates a time sequence of I-pictures identified as pictures N, N + 1, and N + 2, with picture N being first in time, I-picture N + 1 being second in time, and I- picture N + 2 being third in time.

Fig. IB illustrates a time sequence of slices that are transmitted across a wireless link to communicate the I-pictures from Fig. IA. Fig. 1C illustrates redundant slices that are used as reference slices to decode the I-pictures from Fig. IA.

Figs. 2A and 2B illustrate how macroblock-level redundancy can be exploited to improve wireless transmission of digital video content. Fig. 3 depicts a wireless media communications system that is configured to wirelessly transmit digital video content.

Fig. 4 depicts an embodiment of a source that includes graphics hardware and a redundancy information driver that provides redundancy information to an encoder.

Fig. 5 depicts an embodiment of a source that includes an application that provides redundancy information to an encoder.

Fig. 6 depicts an embodiment of a hash module that can be used to provide redundancy information to the encoder.

Fig. 7 depicts a process flow diagram of a method for managing a stream of digital video content that is organized into discrete units of pictures, slices, and macroblocks. Throughout the description, similar reference numbers may be used to identify similar elements.

DETAILED DESCRIPTION OF EMBODIMENTS

In an embodiment, wireless communication of digital video content over short ranges uses only intra (I) picture encoding. Using I-picture only encoding does not require any other pictures (e.g., I, P, or B pictures) to be stored by the encoder/decoder for encoding/decoding. In I-picture encoding, encoding (or decoding) a macroblock of 16 x 16 pixels requires the storage in memory of only the line of pixels located to the left and top boundaries of current macroblocks. In a typical implementation where pixels arrive in scan line order, only one line of a macroblock plus one line of pixels is required to be held in memory for encoding. The small amount of memory can easily and economically be provided by an internal buffer of an encoder or decoder.

Typical protocols for encoding digital video content take advantage of redundancies between I, B, and P frames. In I-picture only encoding, it is still possible to take advantage of redundancies that exist between slices of different pictures of the digital video content (i.e., slice-level redundancies) and/or redundancies that exist between macroblocks in the same slice (i.e., macroblock-level redundancies).

Fig. IA illustrates a time sequence of I-pictures 100 identified as pictures N, N

+ 1, and N + 2, with picture N being first in time, I-picture N + 1 being second in time, and I- picture N + 2 being third in time. Within the I-pictures, individual slices 102, 104, 106, 108, and 110 of digital video content are identified as well as areas of the pictures that have changed from one picture to the next (referred to herein as the "change zone"). For example, from picture N to picture N + 1, the change zone in picture N + 1 is identified by block 112 and from picture N + 1 to N + 2, the change zone in picture N + 2 is identified by block 114. Although the change zones are identified as contiguous blocks for description purposes, it should be understood that changes from picture to picture can occur non-contiguously. In an embodiment, slices that do not change over time are identified as containing redundant digital video content. For example, from picture N to picture N + 1 , there are no changes in slices 102 and 110 and from picture N + 1 to N + 2, there are no changes in slices 102 and 104. For slices that do not change from one picture to the next, there is an opportunity to save encoding and transmission resources by effectively re-using the redundant slices for decoding at the receive side of the wireless connection. Because redundant slices are re-used for decoding, the redundant slices do not have to be transmitted with every picture. This frees up resources, including encoding resources and transmission bandwidth, which can be put to other uses. For example, certain slices of a picture can be encoded with less compression and more transmission bandwidth can be used for transmission of the slices or more bandwidth can be used for error correction.

Fig. IB illustrates a time sequence of slices 102, 104, 106, 108, and 110 that are transmitted across a wireless link to communicate the I-pictures 100 from Fig. IA. In particular, the entire picture N is sent because it is the first picture in the series of pictures and so there is no redundancy with previous pictures. Picture N + 1 has two slices 102 and 110 that are redundant with slices of picture N. Because the two slices 102 and 110 are redundant with slices of the previous picture, these slices are not transmitted along with the other slices of picture N + 1. Picture N + 2 has two slices 102 and 104 that are redundant with slices of picture N + 1. Again, because the two slices 102 and 104 are redundant with slices of the previous picture, these slices are not transmitted.

At the receive side of the wireless link, when some slices of an I-picture are not transmitted, the corresponding redundant slices from a previously transmitted I-picture are needed to decode the subsequent pictures. Specifically, the redundant slices of a previous picture that were transmitted to the receive side are stored at the receive side and used as reference slices for the decoding of subsequent pictures. Fig. 1C illustrates redundant slices that are used as reference slices to decode I-pictures. In particular, redundant slices 102 and 110 from picture N are used to decode portions of picture N + 1, specifically to decode slice 102 and slice 110 of picture N + 1. Likewise, redundant slice 102 from picture N and redundant slice 104 from picture N + 1 are used to decode portions of picture N + 2, specifically to decode slice 102 and slice 104 of picture N + 2.

Figs. IA - 1C illustrate how slice-level redundancy (i.e., redundancies between slices) can be exploited to improve the wireless transmission of digital video content. Redundancies that exist at the macroblock level (i.e., redundancies between macroblocks within the same slice of different pictures) can also be exploited to improve the wireless transmission of digital video content. Figs. 2A and 2B illustrate how macroblock- level redundancy can be exploited to improve wireless transmission of digital video content. Assuming the same initial time sequence of I-pictures 100 as in Fig. IA, Fig.

2A illustrates a time sequence of slices 102, 104, 106, 108, and 110 that are transmitted across the wireless link to communicate the I-pictures from Fig. IA. Fig. 2A is similar to Fig. IB in that all of the slices in which image changes occur are sent to the receive side. However, in this case, redundancies in the digital video content are identified at the macroblock level. That is, redundant macroblocks within a slice are identified. Given that redundancies in the digital video content are identified on a per-macroblock basis, the encoding can be adjusted to take advantage of the identified redundancies. For example, redundant macroblocks within a slice can be encoded as "dummy" macroblocks, which are much more efficient to compress than normal macroblocks. Referring to Fig. 2A, in the slices that are transmitted for picture N + 1, all of the macroblocks that are in slices 102 and 108, but outside of the change zone 112 are fully coded while macroblocks that are in slice 108 but outside of the change zone 112 are coded as dummy blocks. In the slices that are transmitted for picture N + 2, all of the macroblocks that are outside of the change zone 114 are encoded as dummy macroblocks. Fig. 2B illustrates the slices 102, 104, 106, 108, and 110 that are used as reference slices to decode pictures N + 1 and N + 2. For example, portions of picture N + 1 that are outside of the change zone are decoded using slices 102, 106, and 110 from picture N as reference slices and portions of picture N + 2 that are outside of the change zoned are decoded using slices 102, 106, and 110 from picture N and slices 104 and 108 from picture N + 1. With reference to picture N + 1 in Fig. 2B, reference slice 106 from picture N is used to decode all of the dummy macroblocks that are outside of the change zone 112 in slice 106. Likewise, reference slice 106 from picture N, slice 108 from picture N + 1, and slice 110 from picture N are used to decode the corresponding dummy macroblocks in slice 106, 108, and l lO ofpicture N + 2. As illustrated above with reference to Figs. IA - 2B, slice-level and macroblock-level redundancy can be exploited in I-picture only encoding to identify portions of I-pictures that do not need to be transmitted across a wireless link. Because resources are not being consumed to encode and transmit redundant slices and/or macrob locks of digital video content, the encoding and transmission resources can be put to other uses that will improve the overall performance of the wireless link.

In order to take advantage of redundancies within a stream of digital video content that is being subjected to I-picture only encoding/encoding, redundant units (i.e., macroblocks, groups of macrob locks, slices) of digital vide content must be identified at the slice level and/or at the macrob lock level before the encoding process is complete. Redundant units of digital video content can be identified at the slice and/or macroblock level before the encoding process is complete using various different techniques. For example, redundancy information may be obtained directly from graphics hardware or from a higher level software or firmware based application, or redundancy information can be generated using a hashing operation. The technique that is used to identify redundant units of digital video content can be dependent on the environment in which the wireless link is employed. Fig. 3 depicts a wireless media communications system 200 that is configured to wirelessly transmit digital video content as described, for example, in the WiMedia UBW Common Radio Platform. The wireless media communications system includes a transmit side 202 and a receive side 204, with the transmit side having a source 206 of digital video content, an encoder 208, and a wireless transmitter 210 and the receive side having a wireless receiver 212, a decoder 214, and a playout device 216. On the transmit side, the source provides digital video content to the encoder, the encoder encodes the digital video content, and the wireless transmitter transmits the encoded digital video content. On the receive side, the wireless receiver receives the encoded digital video content, the decoder decodes the encoded digital video content, and the playout device plays out the decoded digital video content for viewing and/or recording. In accordance with an embodiment of the invention, the source provides redundancy information to a controller 218 of the encoder. The redundancy information identifies particular slices and/or macroblocks of the stream of I-pictures that are redundant. The encoder uses the redundancy information to adjust the encoding and/or transmission schemes that are used to encode and wirelessly transmit the digital video content to the receive side.

Fig. 4 depicts an embodiment of the source that includes graphics hardware 220. For example, graphics hardware may exist in a personal computer such as a desktop or laptop computer or in a game console. In addition to the graphics hardware, the source may include, for example, hardware devices such as a processor 222, memory 224, and an input/output (I/O) device 226 and software such as an operating system 228, device driver 280, graphics device interface (i.e. the part of the OS in charge of handling the generation of graphics) 284, and at least one software-based application 286. As illustrated in Fig. 4, the graphics hardware is the source of digital video content for the encoder. The graphics device interface has information that identifies redundant slices and/or macrob locks of digital video content. In accordance with an embodiment of the invention, the source includes a redundancy information driver 282 that is configured to collect redundancy information from the graphics device interface and to provide the redundancy information to the encoder.

In the case of a PC, the graphics device interface 284 has redundancy information before the I-pictures are ever generated. In particular, the graphics device interface knows precisely which part of the digital video content is changing. Because this redundancy information is available before encoding, the redundancy information can be obtained by the redundancy information driver 282, provided to the encoder 208, and used to optimize the consumption of resources. In particular, the redundancy information is used to adjust encoding algorithms and transmission protocols to take advantage of freed up bandwidth resources. For example, if it is known in advance of encoding that the second slice of an I-picture is redundant, resources normally allocated for the second slice can be allocated to another slice, even a slice that is earlier in time than the second slice. That is, the resources that were allocated to the second slice can instead be allocated to the first slice. As another example, if it is known in advance that a series of I-pictures are completely or mostly unchanged, the quality of the picture may be refined by encoding only one slice of a particular picture using the maximum available bandwidth. After a few static I-pictures, a lossless image will have been transmitted to the receive side. In still another embodiment, the redundancy information enables slices to be mapped differently, for example, to adjust the slices so that their boarders coincide with the limits of active portions of the picture.

Fig. 5 depicts an embodiment of a source 206 that includes an application 232. For example, an application may exist in a DVD player, a desktop or laptop computer, a game console, or some other media device. In general, applications are high-level entities that exist in software and/or firmware. In addition to the application, the source may include similar hardware and/or software resources to that of Fig. 4. In this embodiment, the application has information that identifies redundant slices and/or macrob locks of digital video content. For example, the application may know that a movie is to be played out in wide-screen format, with black boarders above and below the active portion of the video content. As indicated in Fig. 5, redundancy information is provided to the encoder from the application. The redundancy information can be used to allocate more resources (i.e., encoding and/or transmission bandwidth) to the active portion of the video content. In one embodiment, the source is a DVD player and the decoder and wireless transmitter are connected to the DVD player as an audio/visual (AJV) transmission module. The A/V transmission module includes a DVD decoder (e.g., embodied as a decoder integrated circuit (IC)). The DVD decoder includes a redundancy information application (e.g., a firmware application) and a communications channel. The redundancy information application identifies and transmits redundancy information to the encoder. Redundancy information identified by the application may include format information (e.g., wide-screen format), mode information (e.g., pause mode), and/or image type information (e.g., user interface images or video images).

In another embodiment, slice-level redundancy information is generated using a hashing operation. For example, a slice of an I-picture is hashed and a hash value is stored. In a subsequent picture, a corresponding slice is hashed and the current hash value is compared to the stored hash value. Matching hash values is an indication of redundancy between slices of digital video content. Fig. 6 depicts an embodiment of a hash module 260 that can be used to provide redundancy information to the encoder. The hash module includes a hasher 262, a hash compare unit 264, and a hash value store 266. In operation, a hash value is generated for a slice of a first picture. For example, a hash value is generated for slice 102 of picture N in Fig. IA. The hash value is stored in the hash store. A hash value is then generated for slice 102 of picture N + 1. The hash value of slice 102 of picture N + 1 is compared to the stored hash value to check for a match. Matching hash values indicates redundancy between the slices, for example, redundancy between corresponding slices of pictures, N and N + 1. The hashing operation can be repeated as necessary. An advantage of using a hashing operation is that it can be done "on the fly" as pixels are received at the encoder. However, because redundancy information is generated on the fly as pixels are received at the encoder, the redundancy information can only be applied to subsequent slices. For example, if the second slice of an I-picture is found to be redundant, it is too late to allocated freed resources to the first slice of the I-picture because the first slice has already been processed.

Although the transmit side and receive side are depicted as single units in Fig. 3, at the transmit side, the source, the encoder, and the wireless transmitter may be located in different physical devices. Similarly, at the receive side, the wireless receiver, the decoder, and the playout device may be located in different physical devices.

Fig. 7 depicts a process flow diagram of a method for managing a stream of digital video content that is organized into discrete units of pictures, slices, and macroblocks. At block 702, at the slice or macroblock level, a unit of digital video content from a stream of digital video content is identified as being redundant relative to another unit of the digital video content. At block 704, encoding that is used to encode the stream of digital video content is adjusted in response to the identified redundant unit. The stream of digital video content is then encoded and transmitted using a short range wireless transmission protocol. For example, the encoded digital video content is transmitted using as system that is compatible with the WiMedia UWB Radio Control Platform. In an embodiment, the digital video content is encoded using only I-pictures.

Although specific embodiments of the invention have been described and illustrated, the invention is not to be limited to the specific forms or arrangements of parts as described and illustrated herein. The invention is limited only by the claims.

Claims

CLAIMS:

1. A method for managing a stream of digital video content that is organized into discrete units of pictures, slices, and macroblocks, the method comprising: identifying, at the slice or macroblock level, that a unit of digital video content from a stream of digital video content is redundant relative to another unit of the digital video content; and adjusting encoding that is used to encode the stream of digital video content in response to the identified redundant unit.

2. The method of claim 1 wherein the redundancy information comprises an identification of redundant slices.

3. The method of claim 1 wherein identifying a redundant unit of digital video content comprises obtaining redundancy information from a graphics device interface.

4. The method of claim 1 wherein identifying a redundant unit of digital video content comprises obtaining redundancy information from an application.

5. The method of claim 1 wherein identifying a redundant unit of digital video content comprises hashing the unit of digital video content.

6. The method of claim 5 wherein identifying a redundant unit of digital video content further comprises comparing the hash value with a previously generated hash value for a unit of digital video content that is located in a similar location of a different picture.

7. The method of claim 1 further comprising encoding the stream of digital video content and transmitting the encoded stream of digital video content using a short range wireless transmission protocol. >. The method of claim 7 wherein only I-pictures are encoded.

9. A system for managing a stream of digital video content that is organized into discrete units of pictures, slices, and macroblocks, the system comprising: - a redundancy identification unit configured to identify, at the slice or macroblock level, that a unit of digital video content from a stream of digital video content is redundant relative to another unit of the digital video content; and an encoder configured to adjust encoding of the stream of digital video content in response to the identified redundant unit; and - a wireless transmitter for transmitting the encoded stream of digital video content.

10. The system of claim 9 wherein the redundancy identification unit comprises a redundancy information driver in communication with a graphics device interface that provides redundancy information to the encoder.

11. The system of claim 10 wherein the graphics hardware, the encoder, and the wireless transmitter are located in the same computer device.

12. The system of claim 9 wherein the redundancy identification unit comprises a software or firmware-based application that provides redundancy information to the encoder.

13. The system of claim 9 wherein the encoder is configured to implement I- picture only encoding.

14. The system of claim 9 wherein the redundancy identification unit comprises a hash module configured to hash the unit digital video content.

15. The system of claim 14 wherein the hash module comprises a hasher configured to hash the unit of digital video content, a hash value store configured to store previously generated hash values, and a hash compare unit configured to compare the hash value with a previously generated hash value and to output redundancy information in response to the comparison.

16. A method for managing a stream of digital video content that is organized into discrete units of pictures, slices, and macroblocks, the method comprising: receiving an indication, at the slice or macroblock level, that a unit of digital video content from a stream of digital video content is redundant relative to another unit of the digital video content; adjusting an I-picture only encoding algorithm in response to the identified redundant unit; encoding the digital video content using the adjusted I-picture only encoding algorithm; and - wirelessly transmitting the encoded digital video.

17. The method of claim 16 wherein identifying a redundant unit of digital video content comprises obtaining redundancy information from a graphics device interface.

18. The method of claim 16 wherein identifying a redundant unit of digital video content comprises obtaining redundancy information from a software or firmware-based application.

19. The method of claim 16 wherein identifying a redundant unit of digital video content comprises hashing the unit of digital video content and comparing the hash value with a previously generated hash value.

20. The method of claim 16 further comprising adjusting an allocation of wireless transmission bandwidth in response to the identified redundant unit of digital video content.