US8611433B2 - Information processing apparatus and video decoding method of information processing apparatus - Google Patents
Information processing apparatus and video decoding method of information processing apparatus Download PDFInfo
- Publication number
- US8611433B2 US8611433B2 US11/711,844 US71184407A US8611433B2 US 8611433 B2 US8611433 B2 US 8611433B2 US 71184407 A US71184407 A US 71184407A US 8611433 B2 US8611433 B2 US 8611433B2
- Authority
- US
- United States
- Prior art keywords
- unit
- decoding
- image
- information
- inter prediction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related, expires
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
- H04N21/25808—Management of client data
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/4424—Monitoring of the internal components or processes of the client device, e.g. CPU or memory load, processing speed, timer, counter or percentage of the hard disk space used
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
- H04N21/462—Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
- H04N21/4621—Controlling the complexity of the content stream or additional data, e.g. lowering the resolution or bit-rate of the video stream for a mobile client with a small screen
Definitions
- One embodiment of the invention relates to an information processing apparatus in which video decoding processing for decoding a compressed and encoded video stream is achieved by software, and a video decoding method thereof.
- H.261 and H.263 of the International Telecommunication Union Telecommunication Standardization Sector (ITU-T), Moving Picture Experts Group (MPEG)-1, MPEG-2, and MPEG-4 of the International Organization for Standardization (ISO), and the like, have been developed.
- H.264 As a next-generation video encoding method in which the technologies such as H.261 to H.263, MPEG-1 to MPEG-4, and the like have been succeeded, and further developed, there is the H.264 which has been standardized by the ISO and the ITU jointly (refer to ITU-T Recommendation H.264 (2003), “Advanced Video Coding for generic audiovisual services”
- the H.264 achieves a high compression ratio, it poses a serious problem because it is required a high computing processing capability when reproducing a video image with a high resolution, especially, like an HD-DVD.
- PCs personal computers having the same AV functions as those of audio-video (AV) equipment such as digital versatile disc (DVD) players and TV devices
- AV audio-video
- DVD digital versatile disc
- a software decoder which decodes a compressed and encoded video stream by software is used.
- a software decoder it is possible to decode a compressed and encoded video stream by a processor (CPU) without dedicated hardware being provided thereto.
- FIG. 1 is a perspective view showing an example of an outside of a computer according to one embodiment of the present invention
- FIG. 2 is a block diagram showing a system configuration example of the computer of FIG. 1 ;
- FIG. 3 is a block diagram showing a functional configuration example of a video playback application program for use in the computer of FIG. 1 ;
- FIG. 4 is a block diagram showing a configuration example in the case where the invention is applied to video decoding processing according to standardization specifications based on the H.264 as a software decoder achieved by the video playback application program of FIG. 3 ;
- FIG. 5 is a block diagram showing a structural example of a content information processing system including the video decoding apparatus shown in FIG. 4 as a video decoding unit;
- FIG. 6 is a flowchart showing a basic processing example of an inter prediction skip determining unit in FIG. 4 ;
- FIG. 7 is a flowchart showing a processing example for simply accomplishing skip determination in an inter prediction skip determining unit in FIG. 4 ;
- FIG. 8 is a flowchart showing a processing procedure further appropriately performing the skip determination by modifying the processing procedure shown in FIG. 7 in the inter prediction skip determining unit in FIG. 4 ;
- FIG. 9 is a flowchart showing a processing procedure of preferentially simplifying only B slices of B pictures which especially increase processing amounts at an inter prediction unit when loop processing in units of slices is performed in the inter prediction skip determining unit in FIG. 4 ;
- FIG. 10 is a flowchart showing a processing procedure which enables further appropriately simplifying the processing procedure shown in FIG. 5 by modifying the procedure when loop processing in units of pictures is performed in the inter prediction skip determining unit in FIG. 4 ;
- FIG. 11 is a flowchart showing a processing procedure of preferentially simplifying only bi-prediction portions of B pictures which especially increase processing amounts at an inter prediction unit when loop processing in units of macro-blocks is performed in the inter prediction skip determining unit in FIG. 4 ;
- FIG. 12 is a flowchart showing a processing procedure which enables further appropriately simplifying the processing procedure shown in FIG. 11 by modifying the procedure when loop processing in units of pictures (slices) is performed in the inter prediction skip determining unit in FIG. 4 ;
- FIG. 13 is a schematic view showing aspects of filtering processes in generating inter prediction signals
- FIG. 14 is a flowchart showing a processing procedure of a determination process of integer pixels (S 301 ), a 1 ⁇ 2 pixel filtering process 401 , and a 1 ⁇ 4 pixel filtering process 501 , as an ordinary decoding process for the processing contents shown in FIG. 13 ;
- FIG. 15 is a flowchart showing a first embodiment in decreasing filtering processes and the number of taps in response to load levels for the ordinary decoding process in FIG. 14 ;
- FIG. 16 is a flowchart showing a second example in decreasing the filtering processes and the number of the taps in response to load levels for the ordinary decoding process in FIG. 14 ;
- FIG. 17 is a flowchart showing a third example in decreasing the filtering processes and the number of the taps in response to load levels for the ordinary decoding process in FIG. 14 .
- an information processing apparatus which includes a video decoding processing to decode a compressed and encoded video image stream by software, and a control process based on load information and battery driving information, comprises: a prediction decoding unit which selectively generates one of an intra prediction image and an inter prediction image based on an encoding mode of a decoding object from the video stream and decoded images thereof; a residual decoding unit which generates a residual decoded image based on a quantization parameter of a decoding object from the video stream; an adding unit which generates a decoded image by adding one of an intra prediction image and an inter prediction image selectively generated by the prediction decoding unit, and a residual decoded image generated by the residual decoding unit; a filter process unit which applies deblocking filter process for reducing a block distortion onto a decoded image generated by the adding unit; a determining unit which extracts information on an en
- the information processing apparatus is realized as, for example, a notebook type personal computer 10 .
- FIG. 1 is a front view showing a state in which a display unit of the notebook type personal computer 10 is opened as seen from the front.
- the computer 10 is composed of a computer main body 11 and a display unit 12 .
- a display device composed of a liquid crystal display (LCD) 17 is built into the display unit 12 , and a display screen of the LCD 17 is positioned at substantially the center of the display unit 12 .
- LCD liquid crystal display
- the display unit 12 is attached so as to be freely rotatable between an open position and a closed position with respect to the computer main body 11 .
- the computer main body 11 has a thin box form case.
- a keyboard 18 , a power button 14 for turning the computer 10 power-on/off, an input operation panel 15 , a touch pad 16 , and the like are arranged on the upper surface of the computer main body 11 .
- the input operation panel 15 is an input device for inputting an event corresponding to a pushed button, and has a plurality of buttons for respectively starting a plurality of functions.
- a TV starting button 15 A and a digital versatile disc (DVD) starting button 15 B as well are included in the group of these buttons.
- the TV starting button 15 A is a button for starting a TV function for carrying out playback and recording of broadcast program data such as a digital TV broadcast program.
- broadcast program data such as a digital TV broadcast program.
- the DVD starting button 15 B is a button for playing back video contents recorded on a DVD. When the DVD starting button 15 B is pressed down by a user, an application program for playing back video contents is automatically started.
- the computer 10 has a CPU 111 , a north bridge 112 , a main memory 113 , a graphics controller 114 , a south bridge 119 , a BIOS-ROM 120 , a hard disk drive (HDD) 121 , an optical disk drive (ODD) 122 , a digital TV broadcasting tuner 123 , an embedded controller/keyboard controller IC (EC/KBC) 124 , a network controller 125 , and the like.
- a CPU 111 a north bridge 112 , a main memory 113 , a graphics controller 114 , a south bridge 119 , a BIOS-ROM 120 , a hard disk drive (HDD) 121 , an optical disk drive (ODD) 122 , a digital TV broadcasting tuner 123 , an embedded controller/keyboard controller IC (EC/KBC) 124 , a network controller 125 , and the like.
- HDD hard disk drive
- ODD optical disk drive
- the CPU 111 is a processor provided for controlling operations of the computer 10 , and executes various application programs such as an operating system (OS) and a video playback application program 201 which are loaded from the hard disk drive (HDD) 121 to the main memory 113 .
- OS operating system
- HDD hard disk drive
- the video playback application program 201 is software for decoding and playing back compressed and encoded video data.
- the video playback application program 201 is a software decoder according to the H.264/AVC standard.
- the video playback application program 201 has a function for decoding a video stream compressed and encoded in an encoding method defined by the H.264/AVC standard (for example, a digital TV broadcast program received by the digital TV broadcasting tuner 123 , vide contents according to the high definition (HD) standard read from the optical disk drive (ODD) 122 , and the like).
- the video playback application program 201 has a load detecting module 211 , a decode control module 212 , and a decode executing module 213 .
- the decode executing module 213 is a decoder for executing decode processing defined by the H.264/AVC standard.
- the load detecting module 211 is a module for detecting a load on the computer 10 .
- the load detecting module 211 detects a current loading dose of the computer 10 by inquiring of an operating system (OS) 200 about a current load of the computer 10 .
- a loading dose of the computer 10 is determined on the basis of, for example, a usage rate of the CPU 111 .
- a loading dose of the computer 10 can be determined on the basis of a combination of a usage rate of the CPU 111 and a usage rate of the memory 113 .
- a memory of a certain definite size or more is required for executing a software decoder smoothly.
- decoding performance of the software decoder is deteriorated due to paging of the OS.
- the decode control module 212 controls contents of decode processing executed by the decode executing module 213 in accordance with a load on the computer 10 detected by the load detecting module 211 .
- the decode control module 212 controls contents of decode processing to be executed by the decode executing module 213 such that the decode processing defined by the H.264/AVC standard is executed by the CPU 111 .
- the decode control module 212 controls contents of decode processing to be executed by the decode executing module 213 such that a part of the decode processing defined by the H.264/AVC standard is replaced with SKIPPED or simplified processing.
- Video data decoded by the video playback application program 201 are sequentially written into a video memory 114 A of the graphics controller 114 via a display driver 202 . Consequently, the decoded video data are displayed on the LCD 17 .
- the display driver 202 is software for controlling the graphics controller 114 .
- the CPU 111 executes a system BIOS (Basic Input Output System) stored in the BIOS-ROM 120 as well.
- the system BIOS is a program for controlling hardware.
- the north bridge 112 is a bridge device for connecting a local bus of the CPU 111 and the south bridge 119 .
- a memory controller for access-controlling the main memory 113 as well is built in the north bridge 112 .
- the north bridge 112 further has a function of executing communication with the graphics controller 114 via an AGP (accelerated graphics port) bus or the like.
- the graphics controller 114 is a display controller for controlling the LCD 17 used as a display monitor of the computer 10 .
- the graphics controller 114 generates a display signal to be transmitted to the LCD 17 from image data written in a VRAM (video random access memory) 114 A.
- VRAM video random access memory
- the south bridge 119 controls respective devices on a low pin count (LPC) bus, and respective devices on a peripheral component interconnect (PCI) bus. Further, the south bridge 119 has an integrated drive electronics (IDE) controller for controlling the HDD 121 and the ODD 122 built-in. Moreover, the south bridge 119 has a function for controlling the digital TV broadcasting tuner 123 and a function for access-controlling the BIOS-ROM 120 as well.
- LPC low pin count
- PCI peripheral component interconnect
- IDE integrated drive electronics
- the HDD 121 is a storage device which stores various software and data.
- the optical disk drive (ODD) 123 is a drive unit for driving a storage medium such as a DVD on which video contents are stored.
- the digital TV broadcasting tuner 123 is a receiving apparatus for receiving broadcast program data such as a digital TV broadcast program from the outside.
- the embedded controller/keyboard controller IC (EC/KBC) 124 is a one-chip microcomputer in which an embedded controller for managing electric power and a keyboard controller for controlling the keyboard (KB) 13 and the touch pad 16 are integrated.
- the embedded controller/keyboard controller IC (EC/KBC) 124 has a function of turning the computer 10 power-on/power-off in accordance with an operation of the power button 14 by a user.
- the embedded controller/keyboard controller IC (EC/KBC) 124 can turn the computer 10 power-on in accordance with an operation of the TV starting button 15 A or the DVD starting button 15 B by a user.
- the network controller 125 is a communication device which executes communication with an external network such as, for example, the Internet.
- FIG. 4 is a block diagram showing a structural example when the present invention is applied to a video decoding apparatus in accordance with standardization specifications based on the H.264, as one embodiment of a video decoding apparatus relating to the present invention.
- an input stream is a video stream which has been compressed and encoded in accordance with the H.264 standard, and is transmitted to a variable-length decoding unit (called an entropy decoding unit as well) 301 .
- the variable-length decoding unit 301 encodes an input stream so as to be a varying length, and generates syntax.
- An inverse quantization unit 302 and an inverse transform unit 303 generate a residual image from a result of encoding of a video encoded stream based on the generated syntax.
- An encoding mode control unit 304 discriminates an encoding mode based on the input stream from the variable-length decoding unit 301 , and selectively controls to drive a intra prediction unit 305 and a inter prediction unit 306 based on a result of discrimination.
- the intra prediction unit 305 and the inter prediction unit 306 respectively generate predicted images in a screen and between screens in accordance with an encoding mode designated by the encoding mode control unit 304 .
- Generated predicted images are selectively transmitted to a residual adding unit 307 .
- the residual adding unit 307 adds a predicted image from the intra prediction unit 305 or the inter prediction unit 306 , and a residual image from the inverse transform unit 303 to generate a decoded image.
- the generated decoded image is provided as a reference in the intra prediction unit 305 .
- the decoded image obtained from the residual adding unit 307 is transmitted to a deblocking filter unit 308 , and a reconstructed image is generated through the filter processing here.
- the reconstructed image is temporarily stored in a picture memory (frame memory) 309 , referred in the inter prediction unit 306 , then, output to a display system in turn as a decoding result.
- An inter prediction skip determining unit 310 extracts information related to an encoding mode of pictures, slices, or macro-blocks from the variable-length decoding unit 301 on the basis of the control information from a main control unit (not shown) to determine whether or not the prediction processing at the inter prediction unit 306 should be skipped. Its determining method will be described later.
- the decoded image is input to the deblocking filter unit 308 to generate the reconstructed image through the filer processing and stored in the picture memory 309 . If the deblocking filter processing is not applied, the decoded image is stored directly in the picture memory 309 as the reconstructed image. The reconstructed image stored in the picture memory 309 is output as an output image and also refereed in the inter prediction unit 306 .
- FIG. 5 shows a structural example of a content information processing system including the video decoding apparatus shown in FIG. 4 as a video decoding unit 401 .
- This system further includes a load detection unit 402 .
- the load detection unit 402 acquires information on processing load in video decoding processing from the video decoding unit 401 and information on the other processing load of the system in decoding processing for voice/audio signals, rendering processing, and the like.
- the load detection unit 402 calculates an entire load based on the input information on load and notifies the video decoding unit 401 of the information on load.
- the load detection unit 402 receives, for example, the remaining quantity notification of the battery, and if the notified quantity is less than the allowable remaining quantity, the detection unit 402 makes a video image decoding unit 401 execute skip control of its processing regardless of the state of the processing load. In AC power source driving, it is preferable to improve image quality by automatically turning off the skip of the processing.
- the load information is input to the inter prediction skip determining unit 310 in the video decoding unit 401 .
- a basic processing example of the determining unit 310 will be shown in GIG. 6 .
- the determining unit 310 inquires the load of a PC to an OS (S 101 ) to determine whether the load is a high or not (S 102 ). It is presumed that if it is not in a high-loaded state, the determining unit 310 performs ordinary decode processing (S 103 ), and if it is in the high-loaded state, the determining unit 310 performs decoding processing through special inter prediction (S 104 ). Finally, the determining unit 310 repeats the aforementioned processes until all video images decoded completely (S 105 ).
- the determining unit 310 detects various types of processing to be implemented in the entire system, and if the load is in the high-loaded state, the determining unit 310 reduces the decode processing amounts by adoptively skipping the processing in the inter prediction unit 306 .
- the determining unit 310 decreases the processing amounts by preferentially simplifying the B pictures (B slices, bi-prediction portions) which increase the processing amounts in the inter prediction unit 306 to reduce only the spots at which the processing amounts are specifically increased in response to the encoding mode information of the pictures, slices, or macro-blocks while suppressing the deterioration in image quality minimally.
- inter prediction skip determining unit 310 which accomplishes the foregoing processing will be described with reference to FIG. 7 through FIG. 12 .
- FIG. 7 is s flowchart showing a processing procedure for simply accomplishing skip determination.
- the determining unit 310 determines whether or not the loop processing is in the high-loaded state on the basis of the control information from a load detection unit 402 (S 201 ). If the loop processing is in the high-loaded state, the determining unit 310 determines whether or not it meets the skip conditions (S 202 ). If it does not meet the conditions, the determining unit 310 continues the loop processing as it is, and otherwise stated, the system issues an instruction so as to skip the inter prediction processing of the decoding objects (pictures, slices, or macro-blocks) (S 203 ).
- FIG. 8 is a flowchart illustrating a processing procedure for appropriately performing skip determination by modifying the processing procedure shown in FIG. 7 . Therefore, in FIG. 8 , the same components as those of FIG. 7 are designated by the identical symbols and the components to be characteristics will be described hereinafter.
- the determining unit 301 sets a threshold value for determining conditions in accordance with the level of the load (S 204 ) to determine whether or not the loop processing meets the skip conditions by using the threshold value (S 205 ).
- FIG. 9 is a flowchart illustrating a processing procedure for preferentially simplifying only B slices of the B pictures which specifically increases the processing amounts at the inter prediction unit when loop processing in units of slices is implemented. Also in FIG. 9 , the same components as those of FIG. 7 are designated by identical symbols, and explanation of those components will be omitted hereinafter.
- the determining unit 310 determines whether or not the decoding objects are B slices (S 206 ). If they are not the B slices, the determining unit 310 continues the loop processing as it is. If they are the B slices, the inter prediction processing for the decoding object slices is simplified (S 207 ).
- FIG. 10 is a flowchart showing a processing procedure enabling the processing procedure shown in FIG. 9 to be simplified further accurately by modifying the procedure when the loop processing in units of pictures is implemented. Therefore, in FIG. 10 , the same components as those of FIG. 7 and FIG. 9 are designated by identical symbols, and components to be characteristics will be described hereinafter.
- FIG. 11 is a flowchart showing a processing procedure in preferentially simplifying only the bi-prediction portions of the B pictures which especially increase the processing amounts of the inter prediction unit when the loop processing is performed in units of macro-blocks. Also in FIG. 11 , the same components as those of FIG. 7 are designated by identical symbols, and the explanation for those components will be omitted hereinafter.
- the determining unit 310 determines whether or not the objects are the bi-prediction macro-blocks (S 208 ). If they are not the bi-prediction macro-blocks, the loop processing is continued as it is. In the case of the bi-prediction macro-blocks, the determining unit 310 simplifies the inter prediction processing of the decoding object macro-blocks (S 209 ).
- FIG. 12 is a flowchart showing a processing procedure enabling the processing procedure shown in FIG. 11 to be simplified further accurately by modifying the procedure when the loop processing is implemented in units of pictures.
- FIG. 12 therefore, the same components as those of FIG. 7 and FIG. 11 are designated by the identical symbols, the components to be characteristics will be described hereinafter.
- the determining unit 310 reduces only the spots in which the processing amounts are especially large at the inter prediction unit 306 , such that only the B pictures (B slices, bi-prediction portions) which increase the processing amounts at the inter prediction unit 306 in response to the information on the encoding mode of the pictures, slices, or macro-blocks, so that it can reduce the processing amounts while suppressing the deterioration in image quality minimally.
- Simplification types are classified roughly into a process for interpolation filtering of a luminance signal and a process for that of a chrominance signal.
- the interpolation filtering of the luminance signal includes:
- interpolation image generation of the luminance signal of the inter prediction increases a processing amount because it generates compensated images by use of filtering processing with six taps, and the processing amount occupies around 1 ⁇ 3 extent of entire decode processing sometimes. Therefore, in the case of being in a high-load state, the decode processing amount is reduced by performing the inter prediction in a simple manner so as to reduce the decode processing amount.
- the processes are reduced by switching step-by-step to a no-filtering process, a filtering process only with a 1 ⁇ 2 pixel filter, a filtering process with a small number of taps, etc.
- the filtering processes will be mentioned by giving specific examples.
- FIG. 13 shows an aspect of a filtering process in generating the inter prediction signal.
- Provisional pixels b, h, m and s are generated from integer pixels (A, B, C, etc.) by applying direct 1 ⁇ 2 pixel filtering.
- a provisional pixel j of both of which horizontal and vertical directions are present at the 1 ⁇ 2 pixel, generates provisional pixels aa, bb, cc, etc., from the integer pixel by applying the 1 ⁇ 2 pixel filtering, and generates those pixels by further applying the 1 ⁇ 2 pixel filtering.
- Provisional pixels a, c, e, f, g, etc., present at the positions of the 1 ⁇ 4 pixels are generated by applying the 1 ⁇ 4 pixel filtering to the integer pixels and 1 ⁇ 2 pixels. That is, the values of the provisional pixels b, h, j, a, f, and r are typically obtained by the following equations.
- b ( E ⁇ 5 F+ 20 G+ 20 H ⁇ 5 I+J )/32 (1)
- h ( A ⁇ 5 C+ 20 G+ 20 M ⁇ 5 R+T )/32 (2)
- j ( aa ⁇ 5 bb+ 20 b+ 20 s ⁇ 5 gg+hh )/32 (3)
- a ( G+b )/2 (4)
- f ( b+j )/2 (5)
- r ( m+s )/2 (6)
- FIG. 14 is a flowchart illustrating a processing procedure of a determining process of an integer pixel (S 301 ), a 1 ⁇ 2 pixel filtering process S 401 , and a 1 ⁇ 4 pixel filtering process S 501 , as ordinary decode processing for processing contents shown in FIG. 13 .
- S 301 integer pixel
- S 401 1 ⁇ 2 pixel filtering process
- S 501 1 ⁇ 4 pixel filtering process
- the 1 ⁇ 2 pixel filtering process S 401 conducts processing by the use of the 1 ⁇ 2 pixel horizontal six taps filter for pixel positions present at b and s, by the use of the 1 ⁇ 2 pixel vertical six taps filter for pixel positions present at h and m, and by the use both 1 ⁇ 2 pixel horizontal and vertical six taps filters for pixel positions present at j.
- the 1 ⁇ 4 pixel filtering S 501 performs processes, without using any filter when the pixel positions are present at b, h and i, by using the 1 ⁇ 4 pixel horizontal two taps filter when the pixel positions are present at a, c, i, and k, by using the 1 ⁇ 4 pixel vertical two taps filter when the pixel positions are present at d, n, f, and q, and by using a 1 ⁇ 4 pixel slanted two taps filter when the pixel positions are present at e, g, p, and r.
- FIG. 15 is a flowchart showing a first embodiment when reducing the filtering processes and the number of taps in response to the load level to the foregoing ordinary decoding processes.
- the 1 ⁇ 2 filtering process and the 1 ⁇ 4 filtering process are performed through the 1 ⁇ 2 pixel six taps filter and 1 ⁇ 4 pixel two taps filter, respectively.
- the first embodiment skips only the 1 ⁇ 4 pixel filtering process by using only the 1 ⁇ 2 pixel six taps filter (the same processing for 1 ⁇ 2 pixel).
- the load level is load level 2 in a step 603
- the number of taps is decreased by the use of the 1 ⁇ 2 pixel two taps filter.
- determination of the load lever as load level 3 skips the filtering process (the same processing for integer pixel).
- FIG. 16 is a flowchart illustrating a second embodiment in the case of reducing filtering processes and the number of taps in response to load level to the aforementioned ordinary decoding processing.
- the second embodiment differs from the first embodiment in reducing the number of the taps by using the 1 ⁇ 2 pixel two taps filter and executing the 1 ⁇ 4 filtering process by using the 1 ⁇ 4 pixel two taps filter when the step S 602 determines that the load level is the load level 1 .
- FIG. 17 is a flowchart showing a third embodiment for reducing the filtering processes and the number of the taps in response to the load level to the foregoing ordinary decoding processing.
- the flowchart in this embodiment shows the processing procedure to skip the filtering process in a vertical direction only for the pixel position j to which six taps filtering processing is applied twice in a horizontal/vertical direction among 1 ⁇ 2 pixel interpolations in the load level 1 .
- the third embodiment can skip the 1 ⁇ 4 pixel filtering process like the case in which the load level is determined as the load level 1 in the step S 602 in FIG. 15 , by roughly calculating the accuracy of the motion vector into the 1 ⁇ 2 pixel accuracy. If the load level is determined as the load level 3 , roughly calculating the accuracy of the motion vector into the integer pixel accuracy enables the filtering process to be skipped like the case in which the load level has determined as the load level 3 in the step S 603 in FIG. 15 .
- the accuracy of the motion vector is the 1 ⁇ 2 pixel accuracy both in the horizontal and vertical components, roughly calculating the accuracy of the vertical component into the integer pixel accuracy allows only the pixel position j to be skipped its filtering process in the vertical direction like the case in which the load level is determined to be the load level 1 in FIG. 17 .
- the Rough calculation of the motion vector of a chrominance signal into the integer pixel accuracy makes it possible to skip the interpolation filtering of the chrominance signal.
- the interpolation filtering can be simplified. For instance, in performing the interpolation filtering processes among a plurality of sets of hardware such as a graphics controller, roughly calculating in advance the accuracy of the motion vector necessary for the interpolation filtering enables simplifying the interpolation filtering without changing the contents of the hardware.
- the present invention may be accomplished not only as the video decoding method mentioned above but also as a video decoding method including characteristic steps as means included in such a video decoding method.
- the invention may be also achieved as a program to make a computer execute these steps.
- Such a program can be distributed through a recording medium such as a CD-ROM and a transmission medium such as the Internet.
Abstract
Description
b=(E−5F+20G+20H−5I+J)/32 (1)
h=(A−5C+20G+20M−5R+T)/32 (2)
j=(aa−5bb+20b+20s−5gg+hh)/32 (3)
a=(G+b)/2 (4)
f=(b+j)/2 (5)
r=(m+s)/2 (6)
Claims (13)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2006-053694 | 2006-02-28 | ||
JP2006053694A JP4649347B2 (en) | 2006-02-28 | 2006-02-28 | Information processing apparatus and moving picture decoding method for information processing apparatus |
Publications (2)
Publication Number | Publication Date |
---|---|
US20070201555A1 US20070201555A1 (en) | 2007-08-30 |
US8611433B2 true US8611433B2 (en) | 2013-12-17 |
Family
ID=38443959
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/711,844 Expired - Fee Related US8611433B2 (en) | 2006-02-28 | 2007-02-28 | Information processing apparatus and video decoding method of information processing apparatus |
Country Status (2)
Country | Link |
---|---|
US (1) | US8611433B2 (en) |
JP (1) | JP4649347B2 (en) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4643437B2 (en) * | 2005-12-27 | 2011-03-02 | 株式会社東芝 | Information processing device |
JP4643453B2 (en) * | 2006-01-10 | 2011-03-02 | 株式会社東芝 | Information processing apparatus and moving picture decoding method for information processing apparatus |
JP5598540B2 (en) | 2010-04-28 | 2014-10-01 | 富士通株式会社 | Movie output device, movie output method, and movie output program |
CN108769693B (en) * | 2011-06-10 | 2022-07-08 | 拜特移动网络创新有限责任公司 | Macroblock-level adaptive quantization in quality-aware video optimization |
JP5653307B2 (en) * | 2011-06-27 | 2015-01-14 | 日本電信電話株式会社 | Image coding method, image coding apparatus and program thereof |
CN106941608B (en) | 2011-06-30 | 2021-01-15 | 三菱电机株式会社 | Image encoding device and method, image decoding device and method |
KR102290420B1 (en) * | 2013-04-05 | 2021-08-17 | 삼성전자주식회사 | Method and apparatus for video encoding, method and apparatus for video decoding |
Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11239352A (en) | 1997-12-10 | 1999-08-31 | Matsushita Electric Ind Co Ltd | Image processing method, image processing unit and data storage medium |
JPH11346368A (en) | 1998-04-03 | 1999-12-14 | Matsushita Electric Ind Co Ltd | Image processing method, image processing unit and data storage medium |
US20010017977A1 (en) | 2000-02-29 | 2001-08-30 | Kabushiki Kaisha Toshiba | Video reproducing method and video reproducing apparatus |
US6353683B1 (en) | 1997-12-10 | 2002-03-05 | Matsushita Electric Industrial Co., Ltd. | Method and apparatus of image processing, and data storage media |
US6393393B1 (en) * | 1998-06-15 | 2002-05-21 | Matsushita Electric Industrial Co., Ltd. | Audio coding method, audio coding apparatus, and data storage medium |
US20030206587A1 (en) * | 2002-05-01 | 2003-11-06 | Cristina Gomila | Chroma deblocking filter |
JP2005033724A (en) | 2003-07-11 | 2005-02-03 | Matsushita Electric Ind Co Ltd | Video decoding output device |
US20060126725A1 (en) * | 2004-12-10 | 2006-06-15 | Weimin Zeng | Automated test vector generation for complicated video system verification |
US7310371B2 (en) * | 2003-05-30 | 2007-12-18 | Lsi Corporation | Method and/or apparatus for reducing the complexity of H.264 B-frame encoding using selective reconstruction |
US7372999B2 (en) * | 2002-09-09 | 2008-05-13 | Ricoh Company, Ltd. | Image coder and image decoder capable of power-saving control in image compression and decompression |
-
2006
- 2006-02-28 JP JP2006053694A patent/JP4649347B2/en not_active Expired - Fee Related
-
2007
- 2007-02-28 US US11/711,844 patent/US8611433B2/en not_active Expired - Fee Related
Patent Citations (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11239352A (en) | 1997-12-10 | 1999-08-31 | Matsushita Electric Ind Co Ltd | Image processing method, image processing unit and data storage medium |
US6353683B1 (en) | 1997-12-10 | 2002-03-05 | Matsushita Electric Industrial Co., Ltd. | Method and apparatus of image processing, and data storage media |
JPH11346368A (en) | 1998-04-03 | 1999-12-14 | Matsushita Electric Ind Co Ltd | Image processing method, image processing unit and data storage medium |
US6393393B1 (en) * | 1998-06-15 | 2002-05-21 | Matsushita Electric Industrial Co., Ltd. | Audio coding method, audio coding apparatus, and data storage medium |
US20010017977A1 (en) | 2000-02-29 | 2001-08-30 | Kabushiki Kaisha Toshiba | Video reproducing method and video reproducing apparatus |
US20030206587A1 (en) * | 2002-05-01 | 2003-11-06 | Cristina Gomila | Chroma deblocking filter |
US7372999B2 (en) * | 2002-09-09 | 2008-05-13 | Ricoh Company, Ltd. | Image coder and image decoder capable of power-saving control in image compression and decompression |
US7310371B2 (en) * | 2003-05-30 | 2007-12-18 | Lsi Corporation | Method and/or apparatus for reducing the complexity of H.264 B-frame encoding using selective reconstruction |
JP2005033724A (en) | 2003-07-11 | 2005-02-03 | Matsushita Electric Ind Co Ltd | Video decoding output device |
US20060126725A1 (en) * | 2004-12-10 | 2006-06-15 | Weimin Zeng | Automated test vector generation for complicated video system verification |
Non-Patent Citations (3)
Title |
---|
ISO/IEC 14496-10 (2003); "Information Technology, Coding of Audio-Visual Objects-Part 10; Advanced Video Coding". |
ITU-T Recommendation H.264 (2003), "Advanced Video Coding for generic audiovisual services". |
Japanese Office Action dated Jun. 15, 2010 for Appln. No. 2006-053694. |
Also Published As
Publication number | Publication date |
---|---|
JP2007235447A (en) | 2007-09-13 |
JP4649347B2 (en) | 2011-03-09 |
US20070201555A1 (en) | 2007-08-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8625668B2 (en) | Information processing apparatus and video decoding method of information processing apparatus | |
US8040951B2 (en) | Information processing apparatus and program for use in the same | |
US8630347B2 (en) | Video decoding apparatus and video decoding method | |
US20060203917A1 (en) | Information processing apparatus with a decoder | |
US20070140355A1 (en) | Information processing apparatus, control method, and program | |
US20070189735A1 (en) | Moving image decoding apparatus and moving image decoding method | |
US8611433B2 (en) | Information processing apparatus and video decoding method of information processing apparatus | |
KR101375489B1 (en) | Reproduction device, reproduction method, and recording medium | |
US20060203910A1 (en) | Information processing apparatus and decoding method | |
US20060204221A1 (en) | Information processing apparatus and information processing program | |
US20060203909A1 (en) | Information processing apparatus and decoding method | |
US20090034615A1 (en) | Decoding device and decoding method | |
JP2007180767A (en) | Information processor | |
US8107536B2 (en) | Video decoder | |
JP2006101322A (en) | Information processing apparatus and program used for the processing apparatus | |
JP2006101405A (en) | Information processing apparatus and program used for the processing apparatus | |
JP2006101406A (en) | Information processing apparatus and program used for the processing apparatus | |
JP2006101321A (en) | Information processing apparatus and program used for the processing apparatus | |
JP5066232B2 (en) | Information processing apparatus and image processing method | |
JP4282582B2 (en) | Information processing apparatus and program used in the same apparatus | |
JP2006101404A (en) | Information processing apparatus and program used for the processing apparatus | |
JP2009182891A (en) | Information processor and program | |
JP2006101323A (en) | Information processing apparatus and program used for the processing apparatus | |
JP2006101402A (en) | Information processing apparatus and program used for the processing apparatus | |
US20120147968A1 (en) | Moving Picture Decoding Device and Moving Picture Decoding Method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: KABUSHIKI KAISHA TOSHIBA, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KIKUCHI, YOSHIHIRO;FUJISAWA, TATSURO;KAWASHIMA, YUJI;SIGNING DATES FROM 20070228 TO 20070301;REEL/FRAME:019205/0751 Owner name: KABUSHIKI KAISHA TOSHIBA, JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KIKUCHI, YOSHIHIRO;FUJISAWA, TATSURO;KAWASHIMA, YUJI;REEL/FRAME:019205/0751;SIGNING DATES FROM 20070228 TO 20070301 |
|
FEPP | Fee payment procedure |
Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
REMI | Maintenance fee reminder mailed | ||
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.) |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20171217 |