CA2154962A1 - Video decompression - Google Patents

Abstract

An MPEG video decompression method and apparatus utilizing a plurality of stages interconnected by a two-wire interface arranged as a pipeline process-ing machine. Control tokens and DATA Tokens pass over the single two-wire interface for carrying both control and data in token format. A token decode circuit is positioned in certain of the stages for recognizing certain of the tokens as control tokens pertinent to that stage and for passing unrecognized control tokens along the pipeline. Reconfiguration processing circuits are positioned in selected stages and are responsive to a recognized control token for reconfiguring such stage to handle an identified DATA Token. A wide variety of unique supporting subsystem circuitry and processing techniques are disclosed for implementing thesystem, including memory addressing, transforming data using a common processing block, time synchronization, asynchronous swing buffering, storing ofvideo information, a parallel Huffman decoder, and the like.

Description

215~962 I ~ 1~

VIDEO DECOMPRESSION

This application claims priorit,v under British Application Serial No. 9415413.5filed July 29, 1994.
INTRODUCTION
The prese, It invention relates generally to a new and improved system for clecoding a plurality of audio and video signals and, more particularly, to a new and improved system for decodi"y a plurality of MPEG audio and video siyrl~
A serial pipeline pr~cessi"g system of the present invention com-prises a single two-wire bus used for carrying unique and speci~ ~l interac-tive inte, rac;"g tokens, in the form of control tokens and data tokens, to a - plurality of adaptive decGmpressiol) circuits and the like positiGned as a reconrig.îrable pipeline p,ucessor.
PRIOR ART
United States Patent 5,111,292 r~isçloses an apparalus for encoding/decoding a HDTV signal for e.g. ter,est,ial transmission includes a priority selection processor for par~il)y cG",pr~:ssed video codewords bel~r~ee., high and low priority channels for transr"ission. A compressiGn circuit responsive to high defi~itiGn video source signals provides hierarch-.o~"y layered codewords CW
repr~sellti"g cGn,pr~ssed video data and ~ssoci~ted codewords T. defining the types of data repr~sented by codewords CW. The priority selection processor, responsive to the codewords CW and T, counts the number of bits in predeter-mined blocks of data and deter")i"es the number of bits in each block to be all o c~ted to the respective channels. Ther~aner the processor parses the codewords CW into high and low priority codeword sequences wherein the high and low priority codeword sequences cGI~espond to compressed video data of relatively greater and lesser importance to image reproduction respec-tively.
One prior art system is described in United States Patent No.
5,216,724. The appar~us cGr"prises a plurality of compute modules, in a preferred enlbGd;,~ent, for a total of four compute modules coupled in parallel.Each of the compute modules has a processor, dual port memory, scratch-pad ",er"ûly, and an arbitration ",echani ;.". A first bus co~lrl~s the compute modules and a host pr~cessor. The device comprises a shared ~llelllGIy which is coupled to the host processor and to the compute modules with a second bus.

``- 215~962 United States Patent No. 4,785,349 rliscloses a full motion color digital video signal that is co,npr~ssed, for")dlled for transmission, recorded on co"~pact disc media and decoded at convel ,liG"al video frame rates. During compr~ssio", regions of a frame are individually analyzed to select optimum fill5 coding Ill~lhods specific to each region. Region decoding time e~li",dtes are made to opli",i~a cGmpr~ssion lhresl,clds. Region desc,i~ /e codes convey-ing the size and lo~lions of the r~g;ons are grouped together in a first seg-ment of a data st,eal". Region fill codes conveying pixel amrlihlde indications for the r~yiol)s are grouped together accGrdi"g to fill code type and placed in 10 other seylnel~ts of the data stream. The data stream segments are individually varid~'~ length coded accon~i"g to their respective stalislical distributions and fol",dtleJ to form data r~dn'es. The number of bytes per frame is withered by - the ~dd;t;on of auxiliary data determined by a reverse frame sequence analysis to provide an average number selected to minimize pauses of the compact disc 15 during playback, thereby avoiding unpredictable seek mode latency periods charactëristic of compact discs. A decoder includes a variable length decoder respGnsive to st;-tislic~l infol " ,dlion in the code stream for separately variable length decoding individual sey,nellt:j of the data stream. Region location data is derived from region descri~ /e data and applied with region fill codes to a 20 plurality of region specific decoder~ s~l~cted by cletsction of the fill code type (e.g., relative, absolute, dyad and DPCM) and decoded region pixels are stored in a bit map for s~ ~hserluent display.
United States Patent No. 4,922,341 discloses a method for scene-model-~ssisted reduction of image data for digital television signals, whereby a25 picture signal supplied at time is to be coded, whereby a predecessor frame from a scene already coded at time t-1 is present in an image store as a ref~r~nce, and wllel~ the frame-to-frame infolllldlion is co,nposed of an ampli~i~tio" factor, a shift factor, and an adaptively acquired quad-tree division structure. Upon i"iti~ tiol, of the system, a uniform, prescribed gray scale 30 value or picture half-tone ex~.r~ssed as a defined luminance value is writteninto the image store of a coder at the transl"itler and in the image store of a decGder at the receiver store, in the same way for all picture elements (pixels).
Both the image store in the coder as well as the image store in the decoder are each operaled with feed back to themselves in a manner such that the content 35 of the image store in the coder and decoder can be read out in blocks of variable size, can be amplified with a factor gr~ater than or less than 1 of theluminance and can be written back into the image store with shifted addresses, .-- .

whereby the blocks of variable size are organized according to a known quad -tree data structure.
United States Patent No. 5,122,875 disclQses an apparalus for encoding/decoding an HD~V signal. The apparalus includes a cG",pression circuit responsive to high derilliliGn video source signals for providing hierarchi-cally layered codc~.~rds CW representing compressed video data and associ-ated codewords T, defining the types of data represented by the codewords CW. A priority s~le~tiGn circuit"espG,~sive to the codewords CW and T, parses the codewords CW into high and low priority codeword sequences wherein the high and low priority codeword sequences cGr,espond to com-pressed video data of relatively greater and lesser importance to image reproduction respectively. A transport pr-,cessor, respG"sive to the high and - low priority codeword sequences, forms high and low priority transport blocks of high and low priority codewords, respectively. Each transport block includes a header, codewords CW and error delecti~n check bits. The respective lldl~spG;l blocks are app' ed to a forward error check circuit for applying additional error check data. Therearler, the high and low priority data are applied to a modem wherein quadrature amplitude modulates respective carriers for l,anslnission.
United States Patent No. 5,146,325 discloses a video decGIll~Jr~ssiGn system for deco"" r~ssi"y compressed image data wherein odd and even fields of the video signal are independently cG",pressed in sequences of ir,l,dr,dl"e and intel rldllle cGr"pression modes and then interleaved for trans-mission. The odd and even fields are independently decompressed. During intervals when valid decompressed odd/even field data is not available, even/odd field data is s~ ~hstituted for the unavailable odd/even field data.
Independelltly decG"".r~ssing the even and odd fields of data and substitlJting the opposit~ field of data for unavailable data may be used to advantage to reduce image display latency during system start-up and channel changes.
United States Patent No. 5,168,356 disclQses a video signal encoding system that includes apparat"s for sey",enti"y encoded video data into transport blocks for signal lldlls,,,issioll. The transport block format enha,lces signal recovery at the receiver by virtue of providing header data from which a r~ceiver can determine re-entry points into the data stream on the occurrence of a loss or corruption of transmitted data. The re-entry points are "~axi",i~edby providing secGndal ~ transport headers ernhedclecl within encoded video data in respe~ /e llanspGI l blocks.

~`- . 21S~962 United States Patent No. 5,168,375 discloses a method for process-ing a field of image data samples to provide for one or more of the functions ofdeci,~-dliGn, interpol-tion, and sharpening. This is acco,nplished by an array transform processor such as that employed in a JPEG compression system.
5 Blocks of data samples are l,-dnsfor",ed by the discrete even cosine transform(DECT) in both the dec;.n-dlion and interpolation processes, after which the number of frequency terms is altered. In the case of decimation, the number of frequency terms is reduced, this being followed by inverse transformation to produce a reduced-size matrix of sample points representing the original block 10 of data. In the case of i,lter~,olalion, ~dditional frequency components of zero value are inserted into the array of frequency COI "~ onents after which inversel~-d~ rolmaliGI~ produces an enlalye.l data sampling set without an increase in - spectral bandwidth. In the case of sl,a" el)ing, accomplished by a convolution or filtering operation involving multiplication of l,ansfc,r"~s of data and filter 15 kemel in the frequency domain, there is provided an inverse transfor",-dtion resultin in a set of blocks of processecl data samples. The blocks are over-lapped followed by a savings of designated samples, and a discarding of excess san"~les from r~g;o,)s of overlap. The spatial representalion of the kernel is modified by reduction of the number of components, for a linear-20 phase filter, and zero-p~dcled to equal the number of samples of a data block, this being followed by forming the discrete odd cosine transform (DOCT) of the p~dded kernel matrix.
United States Patent No. 5,175,617 ~lisclQses a system and method for transmitting loy",ap video images through telephone line band-limited 25 analog channels. The pixel orgal,i~dlion in the loy",ap image is designed to match the sel1sor geometry of the human eye with a greater conce~lmdlion of pixels at the center. The l,d"s",itler divides the frequency band into channels,and assigns one or two pixels to each channel, for exdmple a 3 KHz voice quality tele"hone line is divided into 768 channels sp~ced about 3.9 Hz apart.
30 Each channel consisls of two carrier waves iri quadrature, so each channel can carry two pixels. Some cl~annels are reserved for special calibration signals enabling the receivcr to detect both the phase and magnitude of the received signal. If the sensor and pixels are connected directly to a bank of oscillatorsand the receiver can continuously receive each channel, then the receiver 35 need not be sy"chronized with the ll ansr"itler. An FFT algorithm implements a fast discrete approxi",-dlion to the continuous case in which the receiver synchro~ es to the first frame and then acquires sl ~t)secluent r, -alnes every frame period. The frame period is relatively low compared with the sampling period so the ,eceiver is unlikely to lose frame synchrony once the first frame is detected. An experi"~ental video telephone trans"litled 4 frames per second applied quadrature coding to 1440 pixel logmap images and obtained an 5 effective data l, dnsfer rate in excess of 40 000 bits per second.
United States Patent No. 5 185 819 discloses a video compression system having odd and even fields of video signal that are independently compressed in sequences of inl,dr,~me and interframe co",pression modes.
The odd and even fields of independently compressed data are interleaved for 10 tra"s"~ission such that the i"l~dr~dnle even field compressed data occurs midway between successive fields of int,dr,dl"e odd field compressed data.
The interleaved sequence provides r~ceivers with twice the number of entry points into the signal for decoding without i"cr~asi"g the amount of data transl"itled.
United States Patent No. 5 212 742 discloses an apparatus and ,netl,od-for processing video data for compression/decolnpression in real-time.
The apparatus col"p,ises a plurality of compute modules in a preferred embodi.nenl for a total of four compute modules coupled in parallel. Each of the compute modules has a processor dual port memory scralch-pad mem-ory and an arbitration ,nechani~l". A first bus couples the compute modules and host processor. Lastly the device comprises a shared memory which is courled to the host processor and to the compute modules with a second bus.
The method handles assigrii"y portions of the image for each of the proces-sors to operate upon.
United States Patent No. 5 231 484 discloses a system and method for i",plemel,li"g an encoder suitable for use with the proposed ISO/IEC MPEG
standards. Included are three cooper~li"g components or subsystems that operate to variously adaptively pre-,~.rocess the incoming digital motion video sequences allocate bits to the pictures in a sequence and adaptively quantize lldnsfor"~ coerricients in dirrt ~ leS~iOnS of a picture in a video sequence so as to provide optimal visual quality given the number of bits ~I!oc~ted to that picture.
United States Patent No. 5 267 334 discloses a method of removing frame redundancy in a computer system for a sequence of moving images.
The ",atl,od comprises dete~ti"g a first scene change in the sequence of moving i"~ages and gene,dti"g a first keyframe containing co""~lete scene i"fo""ation for a first image. The first keyframe is known in a preferred 215~962 embodiment, as a "forward-facing" keyframe or intraframe, and it is nor",ally pr~sent in CCITT co",pressed video data. The process then comprises ge"erali"g at least one intermediate compressed frame, the at least one intermediate compressed frame containing difference illfor",dlion from the firstimage for at least one image following the first image in time in the sequence of moving i,nayes. This at least one frame being known as an interframe.
Finally, detecting a second scene change in the sequence of moving images and generating a second keyframe cGnlai.lil)g complete scene inror",ation for an image displayed at the time just prior to the second scene change, known as a "backward-facing" keyframe. The first keyframe and the at least one inlen"ediate compressed frame are linked for forward play, and the second keyframe and the inter",ediate compressed frames are linked in reverse for reverse play. The int,dr dr"e may also be used for generation of col"plete scene infor"~dliol) when the images are played in the forward direction. When this sequence is played in reverse, the backward-facing keyframe is used for the gel,erdlion of complele scene i"r.r",alion.
United States Patent No. 5,276,513 ~liscl~ses a first circuit apparatus, COI ",urising a given number of prior-art image-pyramid stages, together with a second circuit apparatus, comprising the same given number of novel motion-vector stages, perform cost-effective hierarcl~ ' motion analysis (HMA) in real-time, with minimum system processing delay and/or employing minimum system processi"g delay and/or employing minimum hardware structure.
Specir,cally, the first and second circuit apparatus, in response to relatively high-,~solution image data from an ongoing input series of successive given pixel-density image-data frames that occur at a relatively high frame rate (e.g., 30 r~dr"es per second), derives, after a certain processi"g-system delay, an ongoing output series of successive given pixel-density vector-data r, dr"es that occur at the same given frame rate. Each vector-data frame is indicative of image motion occurring between each pair of successive image f,~mes.
United States Patent No. 5,283,646 disclQses a method and appara-tus for enabling a real-time video encoding system to accurately deliver the clesi,~d number of bits per frame, while coding the image only once, u~ dates the anquli~aliol1 step size used to quantize coerricients which describe, for example, an image to be transmitted over a communicdlions channel. The data is divided into sectors, each sector including a plurality of blocks. The blocks are encoded, for ex~r"plc, using DCT coding, to generate a sequence of coerr,cie.~t~ for each block. The coerricienls can be quantized, and depend-~_ 2154962 ing upon the quanti~dlion step the number of bits required to descril,e the datawill va~ signiricdlltly. At the end of the trans",ission of each sector of data the accumulated actual number of bits expended is compared with the accumu-lated desired number of bits expended for a selected number of sectors 5 associated with the particular group of data. The system then readjusts the qual,li~dlion step size to target a final desi,~d number of data bits for a pluralitv of sectors for exalnple descrilJing an image. Various ,nell,ods are described for updating the quanli~dlion step size and determining desired bit allocations.United States Patent 5 287 420 d;scl~ses a Ill~thod and apparatus for image 10 COI "pression suitabb for personal computer applicdlions which compresses and stores data in two steps. An image is captured in realtime and com-pressed using an erricie"t method and stored to a hard-disk. At some later time the data is further compressed in non-realtime using a computationally more i"te"se algG,itl"" that results in a higher compression ratio. The two-step15 approach allows the stolage reduction benefits of a highly sophislicated compression alsGIithl~ to be achieved without requiring the compuldlional resources to perform this algorithm in ,~alli",e. A compression algorithm suitable for performing the first compression step on a host processor in a per~onal computer is also descri~ed. The first compression step accepls 4:2:2 20 YCrCb data from the video digitizer. The two chrominance components are averaged and a pseudo-random number is added to all components. The resulting values are quanli~ed and packed into a single 32-bit word represent-ing a 2 x 2 array of pixels. The seed value for the pseudo-random number is re",embered so that the pseudo-random noise can be removed before per-25 forming the second cG",pr~ssion step .United States Patent 5 289,577 discloses a method and apparatus for a sequential pr~cess-pipeline which has a first processi"y stage coupled to a CODEC through a plurality of buffers including an image data input buffer an image data output buffer and an address buffer. The address buffer stores 30 addresses each of which idel,lifies an initial addr~ss of a block of addresses within an image melnG,y. Each block of addresses in the image ",el"oly stores a block of deco""~ressed image data. A local conl,uller is responsive to the writing of an addr~ss into the addr~ss buffer to initiate the Gperdlion of the CODEC to execute a Discr~le Cosine Transfol",dlion Process and a Discrete 35 Cosine Transfor,ndtiGn Qua"li~dlion P~ocess.
Thearticle Chong YongM. AData-FlowA~cl,ite~:tureforDigital Image P~ocessi-,g. Wescon Technical Paper~: No. 2 Oct./Nov. 1984 disclQses a real-time signal processi"g system specirically designed for image process-ing. More particularly, a token based data-flow architecture is disclosed v,/l,erei., the tokens are of a fixed one word width having a fixed width address field. The system cGnt~ins a plurality of identical flow processors connected in5 a ring fashion. The tokens contain a data field, a control field and a tag. The tag field of the token is further broken down into a processor address field andan identifier field. The processor address field is used to direct the tokens tothe correct data-flow processor, and the ider,li~ier field is used to label the data such that the data-flow processor knows what to do with the data. In this way, 10 the ide"tirier field acts as an instruction for the data-flow processor. The system directs each token to a specific data-flow processor using a module number (MN). If the MN matches the MN of the particular stage, then the appropriate operatiol)s are pe,rur,,,ed upon the data. If unrecognized, the token is directed to an output data bus.
The article, Kimori, S. et al. An Fl~ctic Pipeline Mechanism by Self-Timed Circuits, IEEE J. of Solid-State Circuits, Vol. 23, No1,. February 1988, discloses an elastic pipeline having self-timed circuits. The asynchronous pipeline comprises a plurality of pipeline stages. Each of the pipeline stages co"sists of a group of input data latches followed by a CGI I Ibinatorial logic 20 circuit that carries out logic operalions specific to the pipeline stages. The data latches are simultaneously supplied with a triggering signal generated by a data-l,ansfer control circuit ~ssoci ~lecl with that stage. The data-transfer control circuits are intercGnnected to form a chain through which send and ackl,ùJJlcdge signal lines control a hand-shake mode of data l,ansrer between 25 the successive pipeline stages. Furthermore, a decoder is generally provided in each stage to select operations to be done on the operands in the present stage. It is also possible to locate the decoder in the preceding stage in orderto pre-decode comp!ex decodi"g processi"g and to alleviate critical path problems in the logic circuit. The elastic nature of the pipeline eliminates any30 centralized control since all the interworkings between the submodules are determined by a completely localized decisiGn and, in addilioll, each submodule can autonomously pelrulll) data buffering and self-timed data-t,al-srer control at the same time. Finally, to increase the ~ s;ticily of the pipeline, empty stages are interleaved between the occupied stages in order to 35 ensure reliable data transfer between the stages.

_ 215~962 .

Accor~lingly, those skilled in the art have recognized a long felt need for a new and improved video decompression system obviating the deficiencies of the prior art systems. The present invention clearly fulfills this need.
BRIEF DESCRIPTION OF THE DRAWINGS
5 Figure 1 illu:jl,dtes data flow through a prerer~ed embodiment in the present invention;
Figure 2 shows an example of a 13 bit word used to address 8 bit data in a 64 X 32 RAM;
Figure 3 is a fu- ,~;tional block diagram of a Register file in the present 10 invention;
Figure 4 illusl,dtes data flow in a reg;ster file as shown in Figure 3;
Figure 5 is a block diagld", illuslldling register file address decoding, - in accordance with the present invention;
Figure 6 is a block diagram of a MicrocGdable State Machine, in 15 accordance with the present invention;
Figure 7 shows a fixed width word, in accordance with the present invention, used for add,essing and having an address field, a substitution fieldand a sl~hstitution header;
Figure 8 is a block diagram of one example of an A,itl"~elic Core in 20 accordance with the present invention;
Figure 9 illusl,ates the basis steps in a method, in accordance with the present invention, for pelru"~1ing an IDCT on input data;
Figure 10 is a block diagram illustrating the co",bined, simplified, two-stage architecture of an IDCT system, in accorddl,ce with the present inven-25 tion;
Figure 11 is a simplified block diagram of an integrated circuit thatcomprises the main system components of the IDCT shown in Figure 10;
Figure 12a and Figure 12b taken together are a block diagram of a pre-~,rucessi~g circuit cor,esponding to one of the main system component; for 30 ease of explandlion, these figures are refer,ed collectively as Figure 12;
Figure 13a, Figure 13b and Figure 13c depict timing diag,dms which illustrate the r~ldlionsl,i~.s between timing and control signals in the IDCT
system of a prefer,~d embodiment;
Figure 14a and Figure 14b taken together are a block diagram of a 35 co",rnon proc3ssinS~ circuit in the IDCT system;for ease of explal,alion, these figures are referred to collectively as Figure 14;

`_ ` 2154962 Figure 15a, Figure 15b, Figure 15c and Figure 15d taken together are a block diay~ of a post-processing circuit which cGnesponds to another main component of the system and are ,erer~d collectively as Figure 15;
Figure 16 is a block diagram, in accordance with the present inven-tion illusl,dling an IDCT having a twin data stream, a tra"spose RAM and an improved buffer;
Figure 17 is a block diagram showing in further detail the 1-D IDCT
system shown in Figure 16;
Figure 18 is a block diagram showing greater detail of the transform system as shown in Figure 17;
Figure 19 is a block diagram showing in greater detail the input buffer shown in Figure 18;
- Figure 20 is a simplified block diagram of a pre-processing circuit "PREC", in accordal1ce with the present invention;
Figure 21 is a block diagram illu~lldli"g a common processing circuit "CBLK" found in the IDCT;
Figure 22 is a block diagram of a post-processing circuit "POSTC";
Figure 23 is another illu~tldlion of the post-processi"g circuit shown in Figure 22;
Figure 24 is a block diagram depicting a round and saturate block, in accordance with the present invention;
Figure 25 is a block diagram of an output buffer in the present invention;
Figure 26 is a block diagram of a control shift register, in accordance with the present invention;
Figure 27 is a block diagrd", of a control shift register decode in the present invention;
Figure 28 depicts a control shift register and an input control buffer;
Figure 29 illust,dtes a control circuit for a T2 data stream;
Figure 30 shows data in a counter for a T1 data stream;
Figure 31 depicts data in a counter for a T2 data stream in the present invention;
Figure 32 is a timing dias~ra"~ showing the initi~ tion of the IDCT
and ~ssoci~tecl circuitry Figure 33 is a timing diagram showing the interleaving of T1 and T2 data;

Figure 34 is a timing diagla", illu:jlldlillg slippage and recovery of T2 data;
Figure 35 is a timing diagram depicting a flushing operation of the IDCT and ~ssoci ~t*cl circuitry in the present invention;
Figure 36 illusl,ates start-up of the system, in accordance with the present invention;
Figure 37 depicts slippage and recovery in the early stages of interleaving T1 and T2 data;
Figure 38 illusl,ales another preferred embodiment of the IDCT
system shown in Figures 16 through 37;
Figure 39 shows MPEG i~formaliGn sl~ea,ns being demulli~lexed in accordance with the pr~se~l invention into ele "enlary sl,eal1,s containing - data and li,nesta"~p i~fom~dliGn;
Figure 40 depicts a first embodiment of an elementary stream li",estan"~ error determination and time synchronization system in accordance with the-present invention;
Figure 41 illu~lldtes a second embodiment of an elementary stream li",estamp error determination and time synchronization system in accordance with the present invention;
Figure 42 shows a third embodiment of an elementary stream li",estamp error determination and time synchronization system in accordance with the present invention;
Figure 43 depicts a first embodiment of a video timestamp error determination and time syncl,roni,dlion system in accordance with the present invention;
Figure 44 illusl,dtes a second embodiment of a video timestamp error detellllindliGI) and time synchroni~dliGI) system in accorclal,ce with the present invention;
Figure 45 shows the second embodiment of a video limeslalnp error determination and time synchronization system as shown in Figure 44 and operating at 30Hz;
Figure 46 shows ti"leslamp illfor"~dliol, flow through the system of the present invention;
Figure 47 is a block diagram illustrating synchronization time i"for",a-tion being processed by a micropr~ al"l,lable state machine;
Figure 48 is a block .liagldm illustrating a first prefer,ed embodiment of the presel)t invention;

Figure 49 is another block diagram illu~lldli"g the first preferred embodi",ent of the present invention;
Figure 50 depicts a second p,efer,t:d embodiment of the present invention;
5Figure 51 illusl,dtes a detailed method of addressing used by the second prefer,ed embodiment in accGrda"ce with the present invention;
Figure 52 is a block diagram showing an ap,l)aral.ls for decoding Huffman VLCs in accordance with the present invention;
Figure 53 is a schellldlic diagram showing the overall structure of the 10parallel huffman decoder of the present invention;
Figure 54 is a schematic diagram illustrating a ROM adapted for decoding parallel huffman codes;
Figure 55 illu~ les a first embodiment of a ROM adapted for decod-ing parallel huffman codes;
15Figure 56 illu~l,ates a second embodiment of a ROM adapted for decodirig pa(allel huffman codes;
Figure 57 depicts a third embodiment of a ROM adapted for decoding parallel huffman codes;
Figure 58 is a block diagram illustldli"g the primary system compo-20nent of one e",bodi",ent of the present invention;
Figure 59 is a block diagram depicting the start code detector of the present invention;
Figure 60 is a block diagra", showing the parser of the present invention;
25Figure 61 is a block diagram depicting the primary components of the spatial pr~cessing circuitry of the present invention;
Figure 62 is a block diagram illustrating the display circuitry in accordance with the present invention;
Figure 63 illusl,dtes one embodiment of timestamp management in 30accordance with the present invention;
Figure 64 shows another embodiment of limestar"p management in the pr~sel,t invention;
Figure 65 is a block diay,d,n depi_tiny the hardware components of the system of the present invention;
35Figure 66 is a block diagram providing an overview of the system components of the microco"t,oller of the present invention;

21~4962 , , Figure 67 is a simplified diag,dr" illuslldling the A~itl""elic core of the prese"l invention;
Figure 68 illusllates the ALU of the present invention;
Figure 69 depicts a r~gister file in acc~r~lance with the present 5 invention;
Figure 70 illusl,ates the writing to independent bus registers in the present invention;
Figure 71 illusl,dles frame-based prediction wherein vector[1] = 0 and vector[0] = 0;
Figure 72 depi~1s frame-based predi~tion wherein vector[1] = O and vector[0] = 1;
Figure 73 shows frame-based prediction wherein vector[1~ = 1 and - vector[0] = 0;
Figure 74 illu:,tlates frame-based pr~diGtion wherein vector[1] = 1 and vector[0] = 1;
Figure 75 depicts field-based prediction wherein motion_vertical_field_select = 0 and vector[0] = 0;
Figure 76 Dluslldtes field-based pr~di~;tion wherein motion_vertical_field_select = 0 and vector[0] = 1;
Figure 77 similarly illuslldles field-based prediction wherein motion_vertical_field_select = 1 and vector[0] = 0;
Figure 78 shows field-based predi~tiGn wherein motion_vertical_field_select = 1 and vector[0] = 1;
Figure 79 shows field-based pledi~;tiGn in frame pictures wherein motion_vertical_field_select = 0 and vector[0] = 0;
Figure 80 Oluslldles the pr~ tiGn of Figure 79 wherein motion vertical field select = 0 and vector[0] = 1;
_ Figure 81 shows the prediction mode of Figure 79 wherein motion_vertical_field_select = 1 and vector[0] = 0;
Figure 82 shows the prediction mode of Figure 79 wherein both motion_vertical_field_select and vector[0] = 1;
Figure 83 illu~lldtes an ~dditiGnal mode of pr~di.;tiGn rilteri"g;
Figure 84 shows still anotl,er predi~ n mode;
Figure 85 illustrates yet an-tl,er predil;tiGn mode in accordance with the ,. r~sent invention;
Figure 86 shows another pred;ctiGn mode of the present invention;

21S~962 Figure 87 is a block diagram illusl,dling the Grga"i~dlion of the various system cG",ponents of the display system of the present invention;
Figure 88 depicts a 4:3 rillt:ring operation;
Figure 89 depicts a 3:2 filtering operation;
Figure 90 illusl,dtes a 2:1 filtering operation of the present invention;
Figure 91 shows a three tap filter used in the present invention;
Figure 92 illusl,ates the ,~pelilion of erroneous pels;
Figure 93 depicts the filed_id signal of the present invention;
Figure 94 shows the hori~onlal timing points (cycles), in accordance with the present invention;
Figure 95 illusl,dles the PAL vertical timing at 625 lines per field, in accordal)ce with the present invention;
- Figure 96 illusl,ales the NTSCV vertical timing at 525 lines per field, in accordance with the present invention;
Figure 97 shows a l ,Gri~o"lal counting machine, in accordance with the present invention;
Figure 98 illu~l,ates border generation in the present invention;
Figure 99 de,~ictc picture croppi"y, in accordance with the present invention;
Figure 100 is a block diagraln Rlu~lldlillg the present invention as a chip;
Figure 101 illusl,ales the sysclock requirements of the present invention;
Figure 102 depicts the two-wire protocol on a coded data interface, in accordance with the pr~se"l invention;
Figure 103 shows a DATA token of the present invention;
Figure 104 shows a FLUSH token of the present invention;
Figure 105 illusl,ates the timing of the coded data interface;
Figure 106 depicts using non-even mark-space ratio CDCLOCK, in accordance with the present invention;
Figure 107 shows output timing in 16 bit mode in the present inven-tion;
Figure 108 illu~l,ates outputtiming in 8 bit mode in the present invention;
Figure 109 shows the timing of the video output intel race in the present invention;

21~4962 .

Figure 110 depicts video output mode signals, in accordance with the present invention;
Figure 111 shows l,ori~ontal timing in the present invention;
Figure 112 shows the vertical timing for a 525 line system;
Figure 113 depicts the vertical timing for a 625 line system;
Figure 114 illustrates the sync and blanking signals for a 525 line system, in acconlance with the present invention;
Figure 115 shows the sync and blanking signals for a 625 line system, in accordance with the present invention;
Figure 116 illustrates a zero SDRAM connection configuration in the present invention;
Figure 117 shows one SDRAM connection configuration in the - present invention;
Figure 118 depicts a two SDRAM CGI ,, ,ection configuration, in accordance with the present invention;
Figure 119 illu~l,ales a three SDRAM connel;tion configuration Figure 120 is a flow chart depicting the flag_picture_end operation, in accordance with the present invention;
Figure 121 is a flow chart showing the start_code_search operation, in accordance with the present invention;
Figure 122 shows timeslar"p modification, in accordance with the present invention Figure 123 illusl~ales the read timing for the microprocessor interface;
and Figure 124 shows the write timing for the microprocessor interface.

` 2154962 16 In the ensuing desc,i~lion of the practice of the invention the following terms are frequently used and are ge,,erc.lly defined by the f~ ing glossary:
G L O S S A R Y
5 BLOCK: An 8-row by 8-column matrix of pels or 64 DCT coefficients (source quantized or dequanli,ed).
CHROMINANCE (COMPONENT): A matrix block or single pel representing one of the two color difference s4~, lals related to the primary colors in the manner defined in the bit stream. The symbols used for the color difference 10 signals are Cr and Cb.
CODED REPRESENTATION: A data cle "enl as represented in its encoded form.
- CODED VIDEO BIT STREAM: A coded rep,~sentalioil of a series of one or more pictures as d~:fined in this specilicalion.
15 CODED ORDER: The order in which the pictures are l,ansrr,itled and de-coded. This order is not necess~ily the same as the display order.
COMPONENT: A matrix block or single pel from one of the three mdl, ices (ll""i"ance and two chrominance) that make up a picture.
COMPRESSION: Reduction in the number of bits used to represent an item of 20 data.
DECODER: An embodi,nent of a decoding process.
DECODING (PROCESS): The process defined in this specification that reads an input coded bitstream and produces decoded pictures or audio sarnples.
DISPLAY ORDER: The order in which the decoded pictures are displayed.
25 Typically this is the same order in which they were presented at the input of the encoder.
ENCODING (PROCESS): A process not specified in this specification that reads a sl,eal" of input pictures or audio salllples and produces a valid coded bitstream as defined in this specifi~tiol-.
30 INTRA CODING: Coding of a macroblock or picture that uses info",)dtion only from that macroblock or picture.
LUMINANCE (COMPONENT): A matrix block or single pel represenli"sJ a monochr~",e representation of the signal and related to the primary colors in the manner defi"ed in the bit stream. The symbol used for luminance is Y.
35 MACROBLOCK: The four 8 by 8 blocks of luminance data and the two (for 4:2:0 chroma for~"dl) four (for 4:2:2 chroma format) or eight (for 4:4:4 chroma format) coll~spG"di"g 8 by 8 blocks of chrominance data coming from a 16 by ` 215~962 16 section of the luminance component of the picture. Macroblock is some-times used to refer to the pel data and so",eli",es to the coded representation of the pel values and other data elements defined in the macroblock header of the syntax d~r"ed in this part of this specificdtion. To one of ordinary skill in 5 the art the usage is clear from the context.
MOTION COMPENSATION: The use of motion vectors to improve the errciency of the prediction of pel values. The prediction uses motion vectors toprovide offsets into the past and/or future reference pictures containing previ-ously decoded pel values that are used to form the prediction error signal.
10 MOTION VECTOR: A two-dimensional vector used for motion compensation that provides an offset from the coordinate position in the current picture to the coordinates in a reference picture.
NON-INTRA CODING: Coding of a macroblock or picture that uses informa-tion both from itself and from macroblocks and pictures occurring at other 1 5 times.
PEL: Picture ele."ent.
PICTURE: Source coded or reconstructed image data. A source or recon-structed picture consisL~ of three rectangular n,dt,ices of 8-bit numbers repre-senting the luminance and two chrominance signals. For progressive video a 20 picture is identical to a frame while for inle, laced video a picture can refer to a frame or the top field or the bottom field of the frame depending on the context.
PREDICTION: The use of a predictor to provide an esli",ate of the pel value or data ele."e"t currently being decoded.
25 RECONFIGURABLE PROCESS STAGE (RPS): A stage which in response to a recogl,i~ed token r~col,r,yures itself to perform various operations.
SLICE: A series of ",acrol~locks.
TOKEN: A universal adaptatio" unit in the form of an interactive interfacing messenS;er package for control and/or data functions.
30 START CODES lSYSTEM AND VIDEO]: 32-bit codes embedded in a coded bitstream that are unique. They are used for several purposes including identifying some of the structures in the coding syntax.
VARIABLE LENGTH CODING; VLC: A reversible procedure for coding that assigns shorter code-words to frequent events and longer code-words to less 35 frequent events.
VIDEO SEQUENCE: A series of one or more pictures.

, Briefly, and in general terms, the present invention provides a new and improved ",eti,od and apparatus particularly adapted for use in a two-wire pipeline system having various control and DATA tokens. The major eleme~
of the system may include a Start Code Detector, a Video Parser incorporating 5 a Huffman Decoder and a Microprosiid"~rnable State Machine (MSM), an Inverse Disc(~:te Cosine Trallsro"" (IDCT), a synchronous DRAM cGr,l,ollar with an ~-ssoci~ted address generdlion unit, appropriate prediction circuitry and display circuitry which includes u ~,sampling and video timing generation.
More importantly, various embodiments of the invention may include 10 an MPEG video decompression method and apparatus utilizing a plurality of stages interconnected by a two-wire i"te, race arranged as a pipeline process-ing machine. Control tokens and DATA Tokens pass over the single two-wire i"lel race for carrying both control and data in token format. A token decoder circuit is positioned in certain of the stages for recognizing certain of the tokens 15 as control tokens pertinent to that stage and for passing unrecognized control tokens along the pipeline. Reconfiguration pr~cessing circuits are positioned inselected stages and are responsive to a recognized control token for reconfiguring such stage to handle an idellli~ied DATA Token. A wide variety of unique SUIJpGl lill9 subsystem circuitry and processing techniques are 20 disclosed for impleme"ting the system, including memory addressing, trans-forming data using a colnmon pr~cessinS~ block, time synchronization, asynchronous swing buffering, storing of video inror"~alion, a parallel Huffman decoder, and the like.
By way of example, and not necess~rily by way of limitation, the 25 present invention may include among its various features an apparatus for sy, Ichronizing time having, a time stamp for dete",li. li. ,g presentation time, a clock r~fer~nce for iniliali~i"g system time in a first circuit, a first time counter in communication with the clock reference for keeping system time in a first circuit and a second time counter initialized by the clock reference in a second circuit30 synchr~"i~ed with the first time counter, for keeping a local copy of the system time and for determining the presenlaliGI ~ timing error between the local copy of system time and system time by cGnl~aring the time stamp to the second time counter. It further includes an appardl,Js for sy, Ichr~nizing a system decoder and a video decoder using a time stamp for deter"lining display time, 35 a clock r~erence for initiali~il lg system time in the system decoder, a first time counter in communication with the clock reference for keeping system time in the system decoder and a second time counter inili~ ed by the clock refer-, -- , ence in the video decoder synchroni~ed with the first time counter, for keeping a local copy of system time and for determining the display timing error be-tween the local copy of system time and system time by comparing the time stamp to the second time counter.
Still another embodiment of the invention includes an appdldlus for synchronizing a first circuit and a second circuit using a clock reference for initializing system time in the first circuit, a first circuit having a time counter in communication with the clock refert:nce for keeping system time, a first ele-",entdly ~lledll, time counter in the first circuit for providing elementary stream 10 time. The first circuit is adapted to ~eceive a time stamp, and the first circuit generates synchrol ,i~dlion time by adding elementary stream time to the time stamp and sul 1l a.,til lg system time. The second circuit is adapted to receive synchroni~dliol) time from the first circuit and has a second elementary stream time counter in synchronization with the first elementary stream time counter 15 for providing a local copy of the elementary stream time and for deter" ,ining a timing error between the system time and the time stamp by comparing synchronization time to the local copy of elementary stream time. In this way, the clock re~rence signal does not have to be p~ssed directly to the second circuit in order to determine the timing error.
In another embodiment of the invention, an apparatus for synchroniz-ing a first circuit and a second circuit has a clock reference for initializing system time in the first circuit. The first circuit has a time counter in communi-cation with the clock rererel1ce for keeping system time, and a first video time counter for providing video decoding time. The first circuit is adapted to 25 receive a video time stamp and subtld-:ting system time. The second circuit is adapted to receivc synchro~ dliGI ~ time from the first circuit and has a second video time counter in synchronization with the first video time counter for providing a local copy of video decoding time and for determining a timing error between system time and the video time stamp by comparing synchrol,i~dlion 30 time to the local copy of video decoding time. Accordingly, the clock reference signal does not have to be passed directly to the second circuit in order to determine the timing error.
The present invention also includes a method for providing timing in~c r",aliGn by providing a video data stream having a time stamp carried in 35 packet header wherein the time stamp refers to the first picture in the packet of data. In the next step a register is provided having a flag used to i"dicate valid time stamp informdtiol) which is taken from the packet header and placed into the register. Next the time stamp is removed from the video data stream and placed in the register. Next the ",ell,od encounters a picture start and subse-quently examines the status of the r~g;sler to detel"~ine if valid time stamp infor",dlion is contai.)ed in the r~y;ster by checking the flag status. A time 5 stamp is generated in response to the picture start if the flag indic~tes valid time stamp i"fo""dtion is contained in the register and then the time stamp is insel led back into the data stream.
Another embodiment of the invention includes an apparatus described above wherein the ele.nel)la,~ stream time counters are re~ ;ted to 16 bits. Like-10 wise there is an apparalus as described above wherein the second elemen-tary stream time counter loc~ted in the elementary stream decoder is r~sl~icted to 16 bits. Fu,ll,el",or~ there is an a~.par~ s as described above wherein the - synchronization time is reslli~d to 16 bits for controlling the elementary sl,t:ar" decoder.
The present invention also has a process for decoding video and for determining display time errors agai"st a threshold value. It then parses video data into tokens for further pr.cessi"g determining if a time stamp token is indic~ted comparing the time stamp token to a video time and generates a compared value to determine an indicative of timing error. Next it determines 20 whether the COI "par~d value when compared against a threshold value is within acceptable parar"etels when a timing error is indic~ted and indicates when the compared value is outside acceptable parameters.
An aller"dti./e el"bodi",enl of the invention includes an apparatus for using a system decoder and a video decoder. The system decoder is adapted 25 to accept MPEG system sl,ea",s and demultiplexing video data and the video time stamp from the stream. The system decoder has a first time counter representative of system time. The video decoder accepts the video data and the video time stamp and has a second time counter in synchronization with the first time counter. The video decoder also has a decoder buffer for accept-30 ing the video data at a sul-st~nlially constanl rate and outputting the video data at a varying rate and for passi"g a video time stamp. The video decoder while decoding a picture from the video data also compares the video time stamp for the decoded picture with the second time counter to determine the appropriate display time. There is also a ",etl,od for deter",L,i"g a timing error between a35 first circuit and a second circuit by providing the first circuit with a system time (SY) a time stamp (TS) and an ele."entary stream time (ET) obtaining Sy~ cl ~rl)l li~dtiOn time (X) by using the elementary stream time (ET) the time 215~962 stamp (TS), and the system time (SY), in accordal ,ce with the equation X=ET +
TS-SY, providing sy"chr~"ization time (X) to the second circuit and generating a sy"chrol ,i~ed elel "e"ta, y stream time (ET2) and obtaining a timing error byusing synchronized time (X) and in accordance with the equation ET2-X.
5 Hence, the first circuit can be time synchronized with the second circuit without passing system time to the secGnd circuit.
Anothemll~lllGd for determining a timing error between a first circuit and a second circuit has the following steps: providing the first circuit with atime stamp (TS), and an initial time (IT), obtaining a synchronization time (X) by 10 using the time stamp (TS) and the initial time (IT), in accordance with the equation X=TS-1, providing synchr~n i~dLiGI) time (X) to the second circuit and generating a synchro~ ed elementary stream time (ET) and obtaining a timing error by using syncl,roni~ed time (X) and in accordance with the equation ET-X. In this way, the first circuit can be time synchronized with the second circuit 15 without ~ assi"g system time to the second circuit.
Still another method for determining a timing error between a first circuit and a second circuit includes the following steps: providing the first circuit with a system time (SY), a video time stamp (\/TS), and a video decod-ing time (\/T), obtaining synchroni~dlion time (X) by using the video decoding 20 time (VT), the video time stamp (\/TS) and the system time (SY), in accordance with the equation X=VT+VTS-SY, providing synchronization time (X) to the second circuit and gel1erdli,)s~ a video decoding time (VT2) in the second circuit which is sy, Icl ,runized to the video decoding time (VT) in the first circuit, and obtaining a time error by using synchroni~ed time (X) and in accordance with 25 the equation VT2-X. Accordingly, the first circuit can be time synchronized with the second circuit without passing system time to the second circuit.
In accordance with the present invention, the parallel Huffman decoder block will decode MPEG Huffman coded Variable Length Codes (\/LCs) and Fixed Length Codes (FLCs), and pass through tokens under the 30 control of the parser microproy,dlr""able state Machine ~MSM), and can sustain a high throughput.
In one embodiment of the invention a code lookup technique is employed to decode Huffman codes to a~:l,ieve performance requirements and to handle the second MPEG-2 l,ansform coeffficient table which is irregular or 35 non-canoni~' in nature. Practice of the invention also f~cilit~tes decoding certain more col"plex COmpGnentS from the stream in a single cycle without the ~ssist~nce of an external controller. Examples of such complex components _ 2154962 are Escape-coded coerricients, Intra-DC values and Motion Vector deltas, all of which are present in the stream as combined VLC/FLC components.
To decode a VLC, input is first loaded into the two input data registers handling most significant and least significant data. A selector is used to align 5 the beginning of the next VLC with the ROM input. Hence, for a very first VLC,the selector outputs the top 28 bits of its 59-bit input and the top 16 bits of these are p~ssed to a Huffman Code ROM. For subsequent VLCs, the s~le~,tor effectively shifts the input according to the total count of bits decoded thus far. the count is maintained by adding the size of each VLC, as it is 10 decoded, to a running total. The various word widths are a result of the ",d,~imum coded size which can be decoded, which is the 28-bit MPEG-1 Escape Coded Coerr,ciel1t, and the ,naxi",um VLC size which is 16 bits (DCT
coer~icient tables).
The "table select" input is used to select between the various different 15 Huffman code tables required by MPEG.
The ROM has addresses which are controlled with a selector/shifter.
The ROM performs a VLC table index c~lclJI~tion, followed by the index-to-data operation that yields decoded data.
The index c~'cul~tion is a conte"l addressable memory (CAM) 20 operation with "don't care" matching implemented to handle the Huffman codes which form the prese"led data. Since the index generation is performed in a look-up manner (rather than algorithmically) there is no rest, i.:tion to handling tables which are canonical.
The ROM addr~ss of the present invention is in two fields. The larger 25 field is the bit-palle", to be decoded, and the smaller field selects which Huffman code table is to be examined. In ~ddition to the complete MPEG code tables, the ROM also has entries to identify illegal VLC patterns, which exist for some code tables.
In anoll,er embodiment of the invention, a procedure is used for 30 providing a word with fixed width, having a fixed number of bits to be used for addressing v..riable width data, and having a width defining field and address field. There is also a procedure for add,essi"g ",emoly with a fixed width word, having a fixed number of bits, to be used for add,essL ,9 data and having a s~hstitution field and an addr~ss field, and an apparatus for addressing 35 memory, including a state machine and an aritl,rnelic core.
The procedure for addressing ",elno,y is characterized by providing a fixed width word having a predetel",i"ed fixed number of bits to be used for _ ` 2154962 addressi,)g variable width data, defining the fixed width word with a width defining field and an address field, providing the width defining field with at least one bit to serve as the ter",indlio" marker, defining the address field with a plurality of bits defining the address of data, varying the size of bits in the 5 address field in inverse relation to the size of the variable width data, varying the number of bits in the width defining field in direct r~ldtiG" to the size of the variable width data, and maintai"i,lg a fixed width word for addressing variablewidth data while varying the width of the width defining field and the address field.
The procedure for addressing memory may also include defining the address field with a plurality of bits defining the address of the data, defining a variable width substitution field with a least one s~hstitution bit, the substitution field having at least one bit to serve as a termination marker between the addr~ss field and the s~hstitlltion field, using the substitution field to indicate suhstitllted bits from a separate add,essing source, and maintaining a fixed width word for addr~ssi, Ig variable width data while inversely varying the width of the address field and the width of the substitution field.
In accordance with the invention, a process for addressing variable width data in a memory may be characterized by providing a memory having words of predetermined width and composed of partial words, rotating the partial word to be ~Gcessed to a least significant bit ju~lificalion, extending the remaining part of the word so that the ~ccessed word will be recognized as a partial word"eslolil)g the remaining part of the word, and rotating the word until the partial word is ~estored to its original position.
The invention may also include a method and apparatus for address-ing mell,G,y vvl lerein a word is provided with fixed width, having a fixed number of bits to be used for addressillg \/ariable width data, and having a width defining field and address field. In addition, a procedure for addressing Illell,o,y with a fixed width word, having a fixed number of bits, to be used for addressing data and having a substitution field and an address field, may be used.
The invention may also include a method of ~ccessi,lg from RAM a number M of words that is less than the predetermined fixed burst length N of the RAM, the RAM including an enable line that select~hly enables and disables reading from and writing to the RAM, the method comprising the steps of:
ordering N words to be read from or written to the RAM;

, determining when M words have been read from or written to the RAM, M being less than N; and disabling the RAM upon determining M words had been read from or written to the RAM.
The invention may also include a method of accessing Dynamic Random Access Memory (DRAM) to store and retrieve data words associated with a two d;."e"sional image, the DRAM including two separate banks, each bank being capable of operdli"g a page mode to read and write the data words, the two di",e"siGnal image being organized in a two dimensional grid pattem of cells, each cell cGnlaining an M by N matrix of pixels, and the words assocZ~led with each cell occupying one page or less of a bank, the method comprising the steps of:
(a) assigning each cell a particular one of the two banks so that all data words ~ssoci ~tecl with that particular cell are read from and written to one particular page of that particular bank, the assignment of banks to cells being done such that each cell is ~ssoci~ted with a different bank than any bordering cell which is also either in the same row or in the same column;
(b) (eadi"g the data words associated with a cell that is composed of a matrix of pixels, and that is not aligned with the two dimen-sional grid pattern, but that is aligned with pixels in cells in the two di",ensiGnal grid pattern.
(c) identifying which cells in the two dimensional grid pattern cGI)tain data words associ~tecl with the unaligned cell;
(d) reading, from the first bank of DRAM, the data words associ-ated with one of the cells in the grid pattern identified as con-taining data words associaled with the unaligned cell;
(e) reading, from the second bank of DRAM, the data words ~ssoci~ted with another of the cells in the grid pattem identi-fied as containing data words associated with the unaligned cell;
(fl repeating steps (d) and (e) until all the data words ~ssoci ~ted with the unaligned cell have been read.
The invention may also provide a RAM interface for connecting a bus 35 to RAM wherein a separate address generator generates the addresses the RAM i~telrdce needs to address the RAM. The address generator communi-cates with the RAM interface via a two-wire interface.

215~962 The invention may also include a method to control the buffering of encoded video data Grya~ ed as frames or fields. This method involves determining the picture number of each incoming decoded frame, determining the exrected pr~sen(alion number at any time and marking any buffer as ready 5 when its picture number is on or after the presentation number.
Accordingly, those cGnc~r"ed with the design, development, and utilization of systems for decoding video data have long recognized the need for enl~lanced pe,r~r",a"ce as accol"plished by the various features of the present invention. Other object~ and advantages of the present invention will 10 become apparent from the following more detailed desc, i~.lion taken in con-junction with the accG,npanying drawings.

~154962 .

DETAILED DESCRIPTIONS
The forthcoming "Detailed Desc,i,ulion of the Invention" contains the following S~iol~s:
1) Detailed Description of the Invention for Memory Address-5 ing Variable Length Fields Within a Fixed Width Word Using Fixed Width Word with Variable Length Fields to Perform Address S~lhstitlltion Addr~ssi,)g Variable V\lldth Data with a Fixed ~Idth Word MicrocGd~l~le State Machine Structure Alitll~ tic Core 2) Detailed Description of the ll~ve.lti o ., for Transforming Data using a Common Ploc~sing Block Tl,eort:tical Background of the 1 5 nvention ..

3) Detailed Description of Invention for Time Synchronization 20 4) Detailed Description of the Invention for Asynchronous Swing Buffering 5) Detailed Description of the Invention for Storing Video l~lfGr,,,a-tion 6) Detailed Description of the Invention for a Parallel Huffman Decoder The Huffman Code ROM
Maxill~ Throughput FLCs and Tokens 11lI~JI~.Ile~ iGn 7) MORE DETAILED DESCRIPTION

`~ 21~496Z

DETAILED DESCRIPTION OF THE INVENTION
As an introduction to the illustrative embodiment(s) of the most general fea-tures of the invention, and r~fer,ing more particularly to Figure 1 of the draw-ings, the data flow through the preferred embodiment 200 of the invention is 5 shown. The embodiment of the present invention is preferably implemented using a two-wire pipeline system having various control and DATA tokens. The major elements of the system are a Start Code Detector 201, a Video Parser 202 incorporating a Huffman Decoder 203 and a Microprogra",n)able State Machine (MSM) 204, an Inverse Discrete CosineTransform (IDCT) 205, a 10 synchronous DRAM controller 206 with an associated address generation unit 207, appropriate prediction circuitry 208 and display circuitry 209 which includes upsampling 210 and 211 and video timing generation 212.
This application relates to similar subject matter disclosed in British Patent Application number 9405914.4 entitled "Video Decompression" filed on March 24, 1994, by Discovision Acsociates~ and the latter application is specifically incorporated by reference in this application.
In accordance with the above, specific ~spects, features and subsys-tem areas of the present invention will be referred to in greater detail below. In the draw;"ys, like rerer~"ce numerals denote like or corresponding parts 20 throughout the various drawings and figures.

'' ' Zl5492682 Detailed Descri~tion of the Invention for Memory Ad.lre~sing In accordance with the present invention, a method and apparatus for address-ing memory is described herein. In particular, the present invention provides for derer~ i"g variable width bit fields with fixed width words. More particularly, 5 the present invention provides a method of addressing variable width data witha fixed width word. In various forms of the embodiment, variable bit field is used to specify bits to be substituted into the word or to specify an unused portion of the word in addressing variable width data with a fixed width word.
In addition, the system of the present invention includes a microcodable state 10 machine having an arill,l"elic core.
The microcodable state machine is intended to be used for solving design problems where there is a need for versatile and/or complicated - calcul-liol1s. Examples of such designs include address generation, stream parsing and decoding, and filter tap coerr,cie.1t c~lc~ tions. In this regard, the 15 addressing must cope with two different features: (1) variable length addresses to access varying width portions of words and (2) address substitution. In the present invention, a RAM having a 64 x 32 bit configuration can be addressed in partial words having 64 x 32 bit, 128 x 16 bit, 256 x 8 bit, 512 x 4 bit, 1024 x 2 bit, or 2048 x 1 bit for",ats.

Variable Length Fields Within a Fixed Width Word In many applications, it is useful to define varia~l- portions of a word (to be known as fields) for actions such as substitution, variable width data address-ing, or the con~ tion of other parts of the word. The conventional method for 5 defining variable poi lions of words is to have an additional word (or words) which specify the width of the field (or fields) within the word. In accordance with the present invention, a method for encoding this i,lforlndlion within the word itself is described. The present method has the advantages of savings bits in the overall definition of the word, simplifying decoding of the encoded 10 word and providing a more intuitive view of what has been encoded. Further-more, this encoding method is applicable if the variable width fields are most or least significant bit justified within the word.
Accordingly, Table 1 shows two examples of variable width fields (marked "F") that are least significant bit justified defined within an eight bit 15 word. A "w" marks other potential fields of these words.
. .
Table 1 Bit number (hex) 7 6 5 4 3 2 1 0 Fixed word w w w F F F F F
w w w w w w F F

Table 2 shows the conventional method of encoding the fields shown in Table 1 using sufficient ~dd~iol1al bits to specify the "laAimum width of the field in binary.
(Bits marked "x" are "don't care", i.e., their value is of no consequence. This 25 "lell,od is clearly i,lerriciant in its use of bits and, furthermore, provides a less intuitive form than that descril,ed in the present invention.
Table 2 Bit number (hex) 7 6 5 4 3 2 1 0 Field Define 30 Fixedword w w w x x x x x 1 0 wwwwwwxx010 The new method, in accor-lance with the present invention, defines the field within the word. This method defines the field by using a continua-35 tion marker and a termination marker. The field is specified, from oneend of the field, as a series of continuation markers followed by a ter"li.,alion marker. In the case of a zero length field, however, only a `_ 2154962 .

termination marker is provided at the end of the word. Both the continu-ation marker and the termination marker are single bits, and they must be comple"~e"lary. In ~ddition, the field must be justified to either end of the word. Accordingly, the method of the present invention for encod-5 ing fields requires a width of only one bit extra over the original wordwidth.As shown in Table 3, the encoding of the fields shown in the Table 1, in accGrdance with the new method, is depicted. In this exam-ple, the continuation marker is "1" and the termination marker is "0".
10 The field in this exdl",ol_ is least significant bit justified.

- ` 21S4962 Table 3 Bit number (hex) 7 6 5 4 3 2 1 0 Fixed word w w w 0 Continuation marker = 1;
Termination marker = O. w w w w w w 0 Therefore, the advantages of the encoding method, in accordance with the present invention, are:
1. A reduction in the number of bits needed in the encoding.
10 2. A simplification in the decoding process is required since the need for a "x to 1 of " decode of the "field define" shown in Table 1-2 that would normally be required is inherent in the encoding which is already in the form of 1 of 2X; and 3. The encoding is in a more intuitive form allowing the field defined to be more easily ide"lified.
Furthermore, the use of this encoding method of the present invention can also be used such that the termination marker and the continuation marker are inverted to provide that the encoding of Table 3 resembles that of Table 4.
Hence, the use of "1" or "0" is used i,lterchangeably throughout this applica-20 tion.
Table 4 Bit number (hex) 7 6 5 4 3 2 1 0 Fixed word w w w 1 0 0 0 0 0 Continuation marker = 1;
Termination marker = O. w w w w w w 1 0 0 As previously idel,liried, the field encoded must be justified to either end of the word. Table 5 illu~llales most significant justified fields, i.e., these are en-coded in a similar way to least significant bit justified fields except that the field 30 reaches from the most significant bit (hereinafter MSB) towards the least sig"ificanl bit (I,er~i.,drler "LSB") up to and including the first termination marker. The encodi"g of the fields shown in Table 5 are shown in Table 6.
Table 5 Bit number (hex) 7 6 5 4 3 2 1 0 35 Fixed word F F F F F w w w ¦ F ¦ F ¦ w ¦ w ¦ w ¦ w ¦ w ¦ w ¦

Table 6 Bit ~ ber (hex) 7 6 5 4 3 2 1 0 Fixed word 1 1 1 1 1 0 w w w Conffnuation marker = 1;
Termination marker = o. 1 1 0 w w w w w w Moreover, fields may be encoded from the least significant and most signifi-cant ends of the word simultaneously. For example, the two fields shown in 10 Table 7 may be encoded as in Table 8, with the addition of just one bit for each field as described previously.

-.` 21~4962 Table 7 Bit Iw~.,ber (hex) 7 6 5 4 3 2 1 0 Fixedword F F F F w w F F
w w w w F F F F

Table 8 Bit number (hex) 7 6 5 4 3 2 1 0 Fixed word 1 1 1 1 0 w w 0 Continuation marlcer = 1;
Tennination marker = O. O w w w w 0 Using a Fixed Width Word with Variable Length Fields to Perform Address Substitution There are sit~tions in which it is useful to substitute part of a memory address15 by anofher value. In this way it is possible to construct a data dependent address. The encoding Illetllod of the present invention can be applied to the addresses of a ",e,nG,y to specify what portion of the address is to be substi-tuted. If a least significant bit justified variable length field is used in theaddress, a s~bstitution field can be defined. For example, a 12 bit address 20 Ob~ el1coded to have its five least significant bit substituted by the 12 bit value ObCCCCCCCCCCCC would be Ob~a~ 011 11 1 and produce the address OIJ~a~ CCCCC Table 9 shows the encoding for substitution into a 12 bit address.
Table 9: Address substitution No Bitssubsti- B A 9 8 7 6 5 4 3 2 1 0 u O a a a a a a a a a a a a 1 1 a aaaaaaaaaaO1 2 aaaaaaaaaaO11 30 3 aaaaaaaaaO111 4 aaaaaaaaO1111 aaaaaaaO11111 6 aaaaaaO111111 7 aaaaaO1111111 35 8 aaaaO11111111 No.Bi~substi-BAgg7654321o t~ed 9 aaaO111111111 aaO1111111111 11 aO11111111111 Addr~ssing Variable Wldth Qata with a Fixed Width Word One embodi",el)t of the present invention is for add~ssi"g a memory which can be ~ccessed at its full width or in 2" widths up to its full width (these smaller words are called partial words). Hence, it will be shown how the variable field 10 encoding of the present invention can be used to address this memory and to index those acldr~sses into the ,ne",G,y.
To ~ccess a 64 x 32 bit Register file in widths of 32, 16, 8, 4, 2 and 1 bit requires dirfer~l)t lengths of add,~ss, i.e., the implemel)taliGn of this embodi-ment is a 64 x 32 bit Illemoly which can be ~Gcessed as 64 x 32 bits, 128 x 16 bits, 256 x 8 bits, 512 x 4 bits, 1024 x 2 bits, or 2048 x1 bit. It is seen that 5 bits are required to addless one of the 64 x 32 bit locations, while 12 bits arerequired to address one of the 2048 x 1 bit loc~tions. Hence, the addresses can be of variable length and, in fact, the width of the address specifies the address format of the n~e~GIy. AccGr~ingly, the address can be defined within 20 a fixed word width by using a most sigl)ifical)t justified variable width field which con~t,i~,~ the addless and defi"es its width. This is illustrated in Table 10.
Table10: Variablewidthadd~ssi"g Data~dth A9876543210 1 1aaaaaaaaaaa 2 01aaaaaaaaaa 4 001aaaaaaaaa 8 0001aaaaaaaa 16 00001aaaaaaa 32 000001aaaaaa To allow indexing of the addless, a portion of it can be suhstituted using the same method descril,ed previously for add,ess suhstitlJtion. The s~ ~hstitution portion (or field) of the address can be d~fined by a least significant bit justified variable length field (The continuation marker"1"; termination marker"0") that is superi",~.osed on top ~ ` 21~962 of those shown in Table 10. Using an address of an eight bit word, as an example, Table 11 shows how to define the number of the least significant bits to be substi-tuted. The least signi~icant bit added is the substitlJtion i"dic~tor (marked "w"). The general case of a Fixed width word for substitution is shown in Figure 2.
Table11: Addresss~hstitution Bi~tobesubs~ted A9876543210w 0 0001aaaaaaaaO
1 0001aaaaaaaO1 2 0001aaaaaaO11 3 0001aaaaaO111 4 0001aaaaO1111 0001aaaO11111 6 0001aaO111111 7..... 0001aO1111111 8 o o o 1 o 1 1 1 1 1 1 1 In effect the s~hstitlJte code is superi,-,posed on top of the address that is already coded. From this coding, it can be seen that there are illegal ad-dresses, most obviously OxO000 and Ox3ffl. In this case, a "0" must be in the 20 bollo", 9 bits to prevent substihltirlg more than 8 bits and a "1" in the top 6 bits speciries an allowable access width. If one of these errors is detected, the ~ccess is undefined, but the Register file contents will not be affected.
In accor-lance with the present invention, the system for add~essi"y and for ~ccessi- ,g partial words in a r~:gister file is discussed below.
25 The convelltiGllallllelnGl)/ circuitry diGtates that the ,ner"oly must always be accessed at it full width. To achieve variable width accesses, a full (32 bit) width word is read. This full word is ,ulaled until the partial word ~ccessed is justified in the LSB. The upper parts of the word are extended to the full width and then output. Extending may encGl "pass padding with zeros 30 or ones, sign extending, using the sign bit of a sign-",ayr,it.lde number as the new MSB or any similar conventional n l~ll lod. Extending is dependent on the mode of operdliGn. When the partial word is input to and written back into the men,G,y, it is multiplexed back into the r~taled full word, which is then rotated back and written into the array. Figure 3 shows these steps for the ~ccess of a 35 4 bit partial word in the fourth four bit word of the 32 bit word.

21~962 To ~Gcess or read partial words, such as the highlighted four bit word shown in row "1" 213 of Figure 3, the full width word must be rotated to place the partial word at the LSB, as shown in row "2" 214. As shown in row "3" 215, the four bit word is extended to create a full 32 bit word. This word can now be5 ~ccessed.
As shown in figure 3, a full width word that has been selected to be written back is tru"caled to the width of the original partial word which is multiplexed into the word shown in row "2" 214. At the LSB position, this is shown in row "4" 216. The resulting word is rotated back in its original signifi-10 cance in the read word, this is shown in row "5" 217. This full word can now bewritten back into the register file.
The following list, therefore, summarizes the steps numbered in Figure 3:
1. Full word read from IllelllGI~;
15 2. 12 bit ,ulated right puts partial word into the LSB;
3. E~xtended to full word, then passed to output;
4. The inputted partial word is multiplexed into rotated full word from (2);
and 5. 12 bit lot~led left puts full word back to original state to be written.
20 The above ~cesses s~ ~ggests the data flow structure of the ",em GI y that isshown in Figure 4. The numbers in the structure refer to the above text and to Figure 3.
The memory address must be decoded to control the above structure. It should be r~cog"i~ed that the MSB of any width of address is at the same 25 sigl,ir,cance with r~rerence to the memory. The top six bits of a decoded ad.lress are a 32 bit word address, whereas the remainder is a bit address.
Ther~rore, the stage of dçGoding (in parallel with the s~ ~bstitution) is to decode the address width defining vari~l,le field by detecting the position of the mostsiS~nir~cant termination marker. This allows the address to be MSB justified 30 (shming in zeros at the LSB). The top six bits can be used directly as a 32 bit word row address of the memory. The bottom five bits can be used to directly control both barrel shifler~ (as seen in Figure 4), because, for exa"~ple, an Griginal 32 bit address will always have a shift of ObûOOûO (these having been shifted when the address was MSB justified). Similarly, a 16 bit address can 35 have a shift of ObxOOOO, i.e., O or 16 bit shift and a 1 bit address can have a shift of Obxxxxx, i.e., O to 31 bit shifts. The extender and input multiplexer are controlled by the ~Gcess width decode to mask out the output words and 215~962 multiplex the input words to an appropriate significance, respectively. The block diagram of the decode is shown in Figure 5. It can be seen that the decode of the two va, iable width fields for width and substitution can be done in parallel and independently.
Figure 2 illu~l,ates an example of a fixed width word 13 bits long for add~ssi"g variable width data and s~hstitution as shown in the bottom two rows. For these exa,npl_s, an eight bit word would have been addressed at loc~tion Ob1101ssss, where "ssss" is substituted from another address source.
locoslabl~ State Machine Structure In accordance with the present invention, the substitution into a memory address and the variable width ~ccessinS~ of a memory have been brought together in the imple,nent~lion of a microco.l~ble state machine the structure - of which is shown in Figure 6. The structure is one of a state machine 218 providing control of an arithmetic core 219 by way of a wide word of control signals called a microcode instruction. The a,itl,r"etic core 219, in turn, p~sses status flags and some data to the state machine 218.
The state machine 218, in accordance with the present invention, includes a memory containing a list of the l "icrocode instructions. As with conver,lional microcod~ state macl,i,)es, it is capable of either proceedin through the list of microcode instructions contiguously or a jump can occur fromone instruction to another. The jump address is in the form shown in Figure 7.
The substituted value comes from the A,ill"nelic core 219 as shown in Figures 6 and 8. This allows the construction of ujump tables" within the microcode programs. Thus, if a jump is made with 3 bits substituted, for example, there are eight possible contiguous locations that may be jumped to, each depen-dent on the value from the a,ill,l"elic core, i.e., it has so become a program-mable jump.
A~itl"netic Core The alill""~lic core 219, as shown in Figure 8, includes a memory called a register file 221, an Al ithmelic and Logic unit (ALU) 222, an input port 223 and an output port 224. These components are connected via buses and multiplQxers. As previously stated, these components, and the multiplexers defining their cGnnel;tio,)s, are enlil~,ly controlled by the microcode instruction issued by the state machine 218. The ALU 222 and the ports 223 and 224 are conve~ ~liGnal, however, the register file 221 is a memory which allows variablewidth indexed ~ccesses. The addresses to the register file 221 is coded directly into the microcode instruction.

`~ 21S4962 .

There are many advanlages of using this method of addressi"g to the register file. First, many localioRs in an application do not need to be the full width of the l"elno,y (32 bits in this case). Whilst it will cause no effect on the operation of the device to use a full width loc~lion it is very wasteful of 5 mel"oly localiGns. Minimizing the number of memory IOCal;GnS will minimize the amount of space used by the mer"o, y and therefore minimize the capaci-tive loadil,g in the r~gister file. This n,axil"i~es the speed of the ,~gister file.
Second the indexing combined with the variable width of memory accessing allows the step,ci.,g through of loc~tions of variable width. In the one bit case 10 this allows an ele;ant imple",enldtiol, of long division and multiplication.In summary thererore, there is described a procedure for addressing memory having the following steps: (1) providing a fixed width word having a predetermined fixed number of bits to be used for addressing variable width data; (2) defining the fixed width word with a width defining field and an ad-15 dress field providing the width defining field with at least one bit to serve as atermination marker; (3) defining the address field with a plurality of bits defining the address of the data; and (4) varying the size of bits in the address field in inverse relation to the size of the variable width data varying the number of bits in the width defining field in direct relation to the size of the variable width data 20 and ",aintai.,i.,g a fixed width word for addlessi"g variable width data while varying the width of the width defining field and the address field. In addition a procedure for addressing ",elnG,y having the following steps is desc,ibed: (1) providing a fixed width word having a predeter"lined fixed number of bits to be used for add,t:ssi"g data; (2) dt:fining the fixed width word with an address 25 field and a s~ ~hstitution field; (3) defining the address field with a plurality of bits defining the address of the data; (4) defining a variable width substitution field with at least one substitution bit; (5) the s~ ~hstitlJtion field has at least one bit to serve as a termination marker between the address field and the s~ ~hstitution field; and (6) using the substitution field to indicate substituted bits from a 30 separ~te addr~ssi"g source and maintaining a fixed width word for addr~ssing variable width data while inversely varying the width of the address field and the width of the s~hstitlJtion field. In addition a process for addressi"g variable width data in a ll,~lnoly is described as having the following steps: (1) provid-ing a ",elnG,y having words of predetermined width and composed of partial 35 words; (2) r~tali,)g the partial word to be ~ccessed to a least significant bit justification; (3) extending the remaining part of the word so that the ~ccessedword will be recogrli~ed as the partial word; and (4) r~tori"g the remaining _ 215~962 part of the word and ,otati,lg the word until the partial word is restored to its original position.

Detailed Description of the Invention for Tra"sfc r."ing Data Using a Common rl-ces~ing Block This present embodiment in accordance with the present invention relates to a method for the tldl ,sfon"alion of signals from a frequency to a time 5 representaliol) as well as a digital circuit arrangement for implementing the transfor" ~aliGn.
It is a c~ mGI) goal in the area of telecG",r"unications to increase both i"rc,r"~dlion conte"l and l,d"smission speed. Each communications medium however i"~poses a limitation on l~dl~sm;ssiGn speed as does the hardware at 10 the tra"s",itlin$~ and ,eceiv;ng end that must process the tral,sn,illed signals. A
telegraph wire is for example typically a much faster medium for transl".:ling i"for",dlio" than the mail is, even though it might be faster to type and read a- mailed document than to tap out a telegraph key.
The method of encoding l,dnslr,itled i"ro""dlion also limits the speed at 15 which i~fom~dliGI~ can be conveyed. A long-winded telegraph message will for examplë take longer to convey than a succinct ",ess~ge with the same information content. The gr~atest l,ansr"ission and reception speed can ll ,er~fore be obtained by cGmpressi"g the data to be transmitted as much as possible and then using a high-speed llansln;ssiol, medium to process the 20 data at both ends as fast as possible which often means the reduction or elimination of 'bottlenecks' in the system.
One application in which it is essenlial to provide high-speed transmis-sion of large amounts of data is in the field of digital television. Whereas convel,lional television systems use analog radio and electrical signals to 25 control the luminance and color of picture elements ('pixels') in lines displayed on a television screen a digital televisiGn tra"s,nissiol- system generdtes a digital representatiGn of an image by conveying analog signals into binary 'numbers' c~r~espGnding to luminance and color values for the pixels. Modern digital encoding scl,elnes and hardware structures typically enable much 30 higher inr~r",ali~n l,dnsl"issiGn rates than do conventional analog transl"issio systems. As such digital tcl~ visions are able to achicvc much higher resolu-tion and much more life-like images than their conventional analog counter-parts. It is a, nic;l ated that digital television systems including so-called High-Definition TV (HDTV) systems will replace conventional analog television 35 technology within the next decade in much of in the indusl, ial;~ed world. The conversion from analog to digital imaging, for both transmission and storage `~ ` 2154962 will, thus, be similar to the change-over from analog audio records to the now ubiquitous compact discs (CD's).
In order to i"crease the general usefulness of digital image technology, standardized scl,e",es for encoding digital images have been adopted. Once 5 such sta"da,di~ed sche",e is known as the JPEG standard and is used for still pictures. For moving pictures, there are at present two standards, MPEG and H.261, both of which carry out JPEG-like procedures on each of the sequential fi~",es of the moving picture. To gain advantage over using JPEG repeatedly, MPEG and H.261 operate on the differences between subsequent fidrnes, 10 taking advantage of the well-known fact that the difference, that is, the move-ment between fid",es, is small. It, therefore, takes less time or space to transmit or store the illfo""dtion cûr,~sponding to the changes rather than to - transmit or store equivalent still-picture i~rom)dtiGn as if each frame in thesequence were cGIllFletely unlike the frames closest to it in the sequence.
For convenience, all the current standards operate by breaking an image o; picture into tiles or blocks, each block consisting of a piece of the picture eight pixels wide by eight pixels high. Each pixel is then represented by three (or more) digital numbers known as 'compGIlents' of that pixel. There are many different ways of breaki"g a csl3red pixel into components, for example, 20 using standard notalion, e.g., YUV, YCr, Cb, RGB, etc. All the conventional JPEG-like n,etl,ods operate on each component separately.
It is well known that the eye is insensitive to high-frequency components (or edges) in a picture. Inf~,r",aliGn concer"ing the highest frequencies can usually be omitted altogether without the human viewer noticing any significant 25 reduction in image quality. In order to achieve this ability to reduce the infor-mation conlenl in a picture by eliminating high-frequency infor"~dlion without the eye detect;ng any loss of i~rur~alion, the 8-by-8 pixel block conlc.i.~ing spatial info, Illdtion (for example, the actual values for luminance) must be llansfor",ed in some manner to obtain frequency information. The JPEG, 30 MPEG and H.261 standar~s all use the known Discrete Cosine Transfor", to operate on the 8-by-8 spatial matrix to obtain an 8-by-8 frequency matrix.
As described above, the input data represents a square area of the picture. In l,al-sro"ning the input data into the frequency representation, the transform that is applied must be two-dimensional, but such two-dimensional 35 t,ansf~,l")s are difficult to compute erficiel,lly. The known, two-dimensional Discrete Cosine T~ansfor", (DCT) and the associ~ted inverse DCT (IDCT), however, have the propel ly of being "'separable". This means that rather than ^

having to o,l~erate on all 64 pixels in the eight-by-eight pixel block at one time, the block can first be transfor",ed row-by-row into intermediate values, which are then bdnsr~llnecl column-by-column into the final transformed frequency values.
A one-di",e,~sio"al DCT of order N is ",dll~elndlically equivalent to multiplying two N-by-N ",al,ices. In order to perform the necess~ry matrix mullipl,~tion for an eight-by-eight pixel block, 512 multiplications and 448 additions are required, so that 1,024 multiplications and 896 additions are needed to perform the fu112 dimensi~nal DCT on the 8-by-8 pixel block. These a,ill,r"etic o~.eraliGI~s, and especially multiplication, are complex and slow and, therefore, limit the achievable transmission rate. They also require consider-able space on the silicon chip used to implement the DCT.
- The DCT prc,ce.lure can be rearranged to reduce the amount ofcomputation required. There are, at present, two main methods used for reducing the computation required for the DCT, both of which use "'binary deci",dl;onn. The term "binary decimation" means than an N-by-N l~ar)sform can be computed by using two N2-by-N2 l,ansf~"ndlions, plus some computa-tional overhead whilst arranging this. V\ll,er~as the eight-by-eight l,al,~form requires 512 multiplications and 448 additions, a four-by-four transform re-quires only 64 multi~,li r~tions and 48 additions. Binary dec;mdlion, thus, saves 284 multiplications and 352 additions and the overhead incurred in performing the dec;",dlion is typically i"s4"ilicant compared to the reduction in computa-tion.
At present, the two main mt:lllods for binary dec;",dliol~ were dcveloped by Eong Gi Lee ('A New AlgGrill,n, to Compute the DCT') IEEE Transac:tiol-s on Aco~lstics, Speech and Signal Processing, Vol. Assp 32, No 6, p 1243 Decem-ber 1984) and Wen-Hsiung Chen ('A Fast Computational Algorithm for the DCr, Wen-Hsiung Chen, C. Harrison Smith, S C Pralick, IEEE Transactions on Communications, Col. Com 25, No. 9 1004, Septel"ber 1977). Lee's method makes use of the sy~"",el,y inherent in the definition of the inverse DCT and, by using simple cosine idel ,lities, it defines a method for recursive binary deci, lldliGn. The Lee approach is only suitable for the IDCT.
The Chen method uses a recursive matrix identity that reduces the n)dt,ices into diag~l,als only. This method provides easy binary deci",alioR of the DCT using known idenlities for diagonal n~al,ices.
A serious disadvantage of the Lee and Chen methods is that they are unbalanced in respect of when mulli~,lic~tions and additions must be per-"- ` 21S4962 formed. Essentially, both of these Illetllods require that many additions be followed by many multiplications, or vice versa. When implementing the Lee or Chen ",etl,ods in hardware, it is, therefore, not possible to have parallel operation of adders and multipliers. This reduces their speed and efficiency 5 since the best utilization of hardware is when all adders and multipliers are used all the time.
An adJiliG"al disadvantage of such known methods and devises for performing DCT and IDCT operations is that it is usually difficult to handle theso-called nG~ tiGn coefficient, and known architectures require adding an 10 additional multiplication time when all the multipliers are being used.
Certain known ,netl,ods for applying the forward and inverse DCT to video data are very simple and highly erficic:nt for a so~ware designer who - need not be concerned with the layout of the ser"--ccnductor devices which perform the c~lc~ lions. Such mell,ods, however, often are far too slow or are too complex in semiconductor architecture and hardware interconnections to perforrn satisfactorily at the transll'issiGn rate desired for digital video.
Yet another sl,Gl lcoluing of exi~li"g methods and hardware structures for performing DCT and IDCT operalions on video data is that they require ncidtillg-point i"ter~,al re,cresentalion of numerical values. To illustrate this disadvantage, assume that one has a ca~cul~tor that is only able to deal with three - digit numbers, including digits to the right of the decimal point (if any).
Assume further that the c~lculqtor is to add the numbers 12.3 and 4.56 (Notice that the decimal point is not fixed relative to the position of the digits in these two numbers. In other words, the decimal point is allowed to 'float'). Since thec~cu~tor is not able to store the four digits required to fully represent the answer 16.86, the calculator must reduce the answer to three digits either by truncating the answer by dropping the right-most '6', yielding an answer of 16.8, or it must have the "ecess~ hardware to round the answer up to the closest three-digit appro~ aliGn 16.9.
As this very simple e~al"ple illustldtes, if nodling-point a,ill,melic is required, one must either accept a loss of precision or include highly compli-cated and space-wasting circuitry to minimize rounding error. Even with erficient rounding circuitry, however, the accumulation and propasidliGn of rounding or truncdliGn errors may lead to unacceptable distortion in the video signals. This problem is even greater when the methods for processi"g the video signals require several multiplications, since floating point rounding andtruncation errors are typically greater for mult;~,licdtiGn than for addition.

A much more errici~nt DCT/IDCT method and hardware structure would ensure that the numbers used in the method could be represented with a fixed decimal point, but in such a way that the full dynamic range of each number could be used. In such a system, truncalion and rounding errors would either be eliminated or, at least, greatly reduced.
In the above example, if the hardware can handle four digits, no number greater than 99.99 were ever needed, and every number had the decimal point between the second and third places, then the presence of the dec;",al point would not affect calculdtions at all. Accord;"gly, the arithmetic could be carried out just as if every number were an integer, e.g., the answer 1230+0456=1686 would be just as clear as 12.30+4.56=16.86, since one would always know that the '1686' should have a deci,oal point between the - middle '6' and '8'. Allt:r"dti~/ely, if numbers (constant or otherwise) are selec-tively scaled or adjusted so that they all fall within the same range, each number in the range couldls ao be accurately and unambiguously represented as a set of inleger~.
One way of reducing the number of multir' ers needed is simply to have a single multiplier that is able to accept input data from different sources. Inother words, certain architectures use a single multiplier to pe, run" the multipli-cdliGI)s required in dirrert "t steps of the DCT or IDCT calculations. Although such clossbar switching" may reduce the number of multipliers required, it means that large complicated multiplexer structures must be included i"slead to select the inputs to the multiplier, to isolate others from the multiplier, and to switch the appropridte signals from the s~lected sources to the inputs of the multiplier. Additional large-scale multiplexers are also required to switch the large number of outputs from the shared multipliers to the appropriate subse-quentcircuitry. C,ussbarswitchingormu~ ,gis, therefore, complex,is generally slow (bec~use of the extra storage needed) and costs are significant in a final semiconductor implementation.
Still anotl ,er drawback of exisli"g architectures, including the "crossbar switching is that they require general purpose multipliers. In other words, exisli"g systems require multipliers for which both inputs are variable. As is well known, implel"entatio"s of digital multipliers typically include rows of adders and sl,ifler~; such that, if the current bit of a m~ltir'.er word is a 'one' the value of the multiplicand is added into the partial result, but not if the current bit is a 'zero'. Since a general purpose multiplier must be able to deal with the _ case in which every bit is a '1', a row of adders must be provided for every bitof the multiplier word.
By way of exa""~lc, assume that data words are 8 bits wide and that one wishes to multiply single inputs by 5. An 9-bit representation of the number 5 is 00000101. In other words, digital multiplication by 5 requires only that the input value be shifted to the left two places (corresponding to multipli-cation by 4) and then added to its up-shifted value. The other six positions of the coerricients have bit values of '0', so they would not require any shifting or ~dditional steps.
A fixed-coefficient multiplier, that is, in this case, a multiplier capable of multiplying only by five, would require only a single shifter and a single adder in order to perforrn the multiplication (disregarding circuitry needed to handle - carry bits). A general purpose multiplier, in colllldsl, would require shifters and adder~ for each of the eight positions, even though six of them would never need to be used. As the example illustrates, fixed coefficients can simplify themultipliërs since they allow the designer to eliminate rows of adders that correspond to zeros in the coerricie.-t, thus saving silicon area.
In an IDCT method, in accordance with the present invention, a one-dimensional IDCT for each N-row and N-column of N-by-N pixel blocks is decimated and a 1-D IDCT is pe,ru,,,,ed separ~tely on the N-2 even-numbered pixel input words and the N-2 odd-numbered pixel input words.
In a p,~fer,ed embodiment, N=8 accGrding to the JPEG standard. The two-dimensiGnal IDCT result is then obtained by performing two one-dimen-sional IDCT o~,eraliGns in sequence (with an intermediate reordering-tra"s~ osition-of data).
In a colnmGn p~ocessing step, for N=8, a first pair of input values is passed without need for multi~ tion to output adders and subtractors. Each of a second pair of input values is multiplied by each of two constant-coerricie.,l values cGIlespGndill5a to two scaled cosine values. No other multiplications and only one sulltld~tiGn and one addition are required in the com",ol, pro-cessing step. The second pair is then added or dirrerenced pairwise with the first pair of input values to form even or odd resultant values.
In a pre~n~mGn pn~cessing stage, the lowest order odd input word is pre-multiplied by the square root of two and the odd input words are summed pailwise before processin~ in the common processing block. In a post-com-mon processing stage, i"tel",ediate values corresponding to the processed _ odd input words are multiplied by predetermined constant coefficients to form odd resultant values.
After calcul~tion of the even and odd resultant values, the N/2 high-order outputs are forlned by simple subtraction of the odd resultant values fromthe even resultant values, and the N/2 low-order outputs are formed by simple addition of the odd resultant values and the even resultant values.
For both the DCT (at the transn ,ission end of a video processing system) and the IDCT (at the receiving end, which incorporates one or more of the various aSpectC of the present invention), the values are pr~rerdbly and deliberately scaled downward by a factor of two by a simple binary right shift.
This deliberate, balanced, upward scaling eliminates several multiplication steps that are required according to convenlional methods.
- According to another aspect of the method, in accordance with the present invention, s~lected bits of constant coerr,c;ent or intermediate resulting data words are rounded or adjusted by predetermined setting of selected bits to either '1' or '0'.
Two-di",ensional transfo~ dliGn of pixel data is carried out by a second, identical 1-D Gperation on the output values from the first 1-D IDCT processing steps.
An IDCT system, accGr~Ji"g to yet another aspect of the present inven-tion, includes a pre-co",mol- processi"g circuit, and a coln",ol) processing circuit, in which the pre-cG"""on, common, and post-col"r"on processi"g calc~ lions are performed on input data words. A supervisory conl,oller generales control siynals to control the loading of various system latches;
preferably, to serially time-multiplex the application of the N/2 even and N/2 odd-nu"~ber~d input words to input latches of the pre-common block to direct ~ddition of the even and odd resultant values to form and latch low order output signals and to direct subt,d~;tion of the odd resultant values from the even resultant values to form and latch the high-order output signals and to sequenlially control internal multirlexers.
In the present invention, even and odd input words are preferably proGessed in separate p~sses through the same processi"g blocks. Input data words are pr~f~rably (but not necess~rily) latched, not in strictly ascending ordescending order, but rather in an order enabling an efficient 'butterfly' struc-ture for the data path.
Furthen"or~, at least the col"rnon processing circuit may be configured as a pre-logic circuit, with no clock or control signals required for its proper operation, as may be other processing blocks, depending on the particular applicalion.
No general-purpose multipliers (with two variable inputs) are required.
Rather, constant COt rricie,-t multir' ers are included throughout the preferredembodiment. Furthermore, fixed-point integer aritl"~,elic devices are included in the pre~"~d er"bodi,nent of the invention and can be so designed as to provide a method and system for pe,rur",i,lg IDCT transfol"~alion of video data with one or more of the fcllo~:;"y features:
1. Constant use of all costly arithmetic operaliûns;
10 2. In order to reduce the silicon area needed to implement the IDCT, there are a small number of storage elements (such as latches), preferably no more than required for erricient pipelining of the architecture, coupled with a small number of cG"stant co~rricient multipliers rather than general purpose multipliers that require extra storage elements;
15 3. Oper~tions are ar,dnged so that each arill,l"etic operation does not need to use sopl,ijlic~led designs, for example, if known 'ripple adders' are used, these would allow surriciant time to 'resolve' (see below) or produce their answers; if GperdliGns are arranged in such a way that other devises precede the rearranging operations so as to avoid delay and to allow greater throughput and err,ciency;
4. One is able to generale results in a natural order;
5. No costly, complex, crùssbar switching is required;
6. The architecture is able to support much faster operations; and 7. The circuitry used to control the flow of data through the transform hardware can be small in area.
Theor~tical Background of the Invention In order to understand the purpose and function of the various components and the advantages of the signal prucessing method used in the IDCT system according to the ~.resent invention, it is helpful to understand the system's theoretical basis.

Separability of a Two-DimensiGnal IDCT
The mathemdtical definition of a two-dimensional forward discrete cosine lldll~ro',ns (DCT) for an N x N block of pixels is as follows, where Ua,k) are the pixel frequency values cGr,esponding to the pixel absolute values X(m,n) Equation 1:

`- 2154962 Nc~ x(m~n)cos (2m~1)Jn cos (2n 1)krT rehe j,k=
5 0,1 ...,N-1 and c (1), c (k) ~ forj,kSO; otherwise The terms 2N govem the dc level of the transform, and the coer~icients ca), c(k) are known normalization factors.
The expressiGn forthe cGr,esponding inverse disclete cosine transform, that is for the IDCT, is as follows:
- 10 Equation 2:

x(m,n)-2~,~d~)C(k~Y~ cos (2m~1)~ cos~2N 1) wh ere 15 j,k=O,...,N-1 and c(j),c(~)~lt~) for j,k - O; vlh~ e 1 The for-ward DCT
20 is used to transform spatial values (whether representing characteristics such as luminance directly, or r~presenting differences, such as in the MPEG
standard) into their frequency represent~lion. The inverse DCT, as its name implies, Gperates the other 'dil~ctio"', that is, the IDCT l~nsforms the fre-quency values back into spatial values.
In the ex~,r~ssiGn, Equation 2, ( E2), note that the cosine functions each depend on only one of the sumr"~liGn indices.
The expr~ssion E2 can ll,erefore be rewritten as:
Equation 3:
x(m,n) . 2~c 0 cos (2m21~ c(1~1 Y a h~ cOS ~2n2 )xn This the equivalent of a first one-di"~ensional IDCT performed on the product of all terms that depend on k and n, followed, after a straighfforward standard data 215~962 .

transposition by a second one-dimensional IDCT using as inputs the outputs of the first IDCT operatiol).

21S496~
so Definition of the 1-D IDCT
A 1-dimensional N-point IDCT (where n is an even number) is defined by the following ex~r~ssiG".
Equation 4:
X (k) . ~, c (n) y (n) cos TT(22k+1) n k = ~0,1,...,N-1}

c (n) ~ ) for n=O; ulI,env;se 5 and where y(n) are the N inputs to the inverse tra"sfor",dliol) function and x(k) are its N outputs. As in the 2-D case, the fommula for the DCT has the same structure under the su"""dtion sign, but with the noll~ali~aliGI) constant outside the Sl mrrldtiOR sign and with the x and y vectors switching places in the e~u~tion.
Resolution of a 1-D IDCT
As is shown above, the 2-D IDCT can be ~ cl~qted using a sequence of 1-D
IDCT operations separated by a transpose. In accordance to one embodi-ment, each of these 1-D operdliGns is ,in tum, broken down into sub-proce-dures that are then exploited to reduce even further the required size and 15 cGIllplexity of the semiconductor imFle."entalion.

Normali~dtion of Coerficiel,l~
As is discussed above, an i"~pGIlant design goal for IDCT hardware is the reduction of the required number of multipliers that must be included in the circuitry. Most methods for c~lcu~ting the DCT of IDCT, there~re, alle,npt to 5 reduce the number of multiplications needed. According to this embodiment, however, all the input values are deliberately scaled upward by a factor of the square root of two. In other words, using the method according to this embodi-ment of the present invention, the right-hand side of the IDCT expression (E) isdeliberately multiplied by the square root of two.
According to this embodiment, two 1-D IDCT operations are performed in series (with an i"te",)ediate l,anspose) to yield the final 2-D IDCT result.
Each of these 1-D operalions includes a multiplication by the same square root of two factor. Since the inler" ,ediate transposition involves no scaling, the result of two multiplications by the square root of two in series is that the final 15 2-D results will be scaled upward by a factor two. To obtain the unscaled value, the circuitry need then only divide by two. Since the values are all represented digitally, this can be accomplished easily by a simple right shift of the data. As is made clearer below, the upward scaling by the square root of two in each 1-D IDCT stage and final down-scaling by 2 is accomplished by 20 adders, mlllt;, liers and shifters all within the system's hardware, so that the system places no requ;.~",er,l~ for scaled inputs on the other devises to which the system may be connected. Bec~use of this, the system is COI "pdliL,le with other conventional devises that operate according to the JPEG or MPEG
standards. Norm~ tion accor~li"g to this embodiment of the present inven-25 tion, ll,erefore, eliminates the need for hardware multipliers within the IDCTsel"icGnductor archite~,ture for at least two square root of two mulliplicdlionoperations. As is ex~Jlained below in greater detail, the single additional mul:ipli~ ~tion step (upward scaling by the square root of two) of the input data in each 1-D operdlion leads to the elimination of still other multiplication steps 30 that are required when using convel,liol)al methods.
Separation of the 1-D IDCT into High and Low-Order Outputs Expr~ssion E can now be ev~lu~ted separately for the N/2 low-order outputs (k=0, 1,...,N/2-1) and the N/2 high order outputs (k=N/2, k=N/2 + 1,...N). For N=8, this means that one can first l,ans~r"~ the inputs to c ~'cul~te y(0), y(1), y(2) and y(3), and then transform the inputs to c~lculate y(4), y(5), y(6) and y(7).

Introduce the variable k'=(N-1-k) for the high-order outputs (k=N/2+1,....
N), so that k' varies from (N/2-1) to N as k varies fm ro(N/2+1) to N. For N=8, this means that k'=(3,2,1,0) for k=(4,5,6,7). It can then be shown that expres-sion E can be divided into the following two suhe~ ressiGns E5 (which is the 5 same as E except for the interval of sulrlllldtiGn) and E6:

`- ` 2154962 Low order outputs:
Equation 5:

X (k'~ . ~ c(n) y(n) cos n(22kN1)n where k={0,1 ...,(N/2-1)}; and c (n) ~ for n sO; ~then~
High-order outputs:
Equation 6:

x (k) . x (N-1-k~) . ~ y (n) (-1)" cos n (22k ,1) where k={N,...,(N/2~ k'={0,1,...,(N/2-1)}
(Since c(n)=1 for all high-order terms, c(n) is not included in this expression)Note that both E5 and E6 have the same structure under the su",r"dliGn 20 sign except that the term (-1)n cha"yes the sign of the product under the summa-tion sign for the odd-number~d inputs (n odd) for the upper N2 output values andexcept that the y term will be multiplied by c (O) = 1/~

. ,~; .

215~962 Separation of the 1-D IDCT into Even and Odd Inputs Observe that the single sum in the 1-D IDCT exp~t:ssion E4 can also be separatedinto two sums: one for the even-numbered inputs (for N=8 y(O), y(2), y(4),and y(6) and one for the odd-numbered inputs (for n=8, y(1), y(3), y(5), and y(7). Let g(k) 5 represent the partial sum for the even-numbered inputs and h(k) ,~pr~sent the partial sum for the odd-n~ bered inputs.
Thus:
Equation 7.

( ~1 ( )-1 2~(2k1)2n~, ~ c(2n)y(2n)cos[ ( N

Where k={0,1,.. ,(N/2-1)}; and Equation 8.

h (k) ~ ~ y (2n+1) Cos[~(2k~1) (2n,1)]

where k=(0,1 ,...,(N/2-1)}.
For N=8, observe that the sums in E7 and E8 both are taken over n={0,1 ,2,3}.
25 Now recall the known cosine identity:
2.cosA.cosB=cos(A+B)+cos(A-B), and set A = TT (2k,1)/2N and B = n (2k~1)(2N~1)12N.
One can then multiply both sides of the ex~ r~ssio" E8 by:
2. cos A = 1/ {2 cos[ n (2k~1)/2Nl } =Ck.
30 Note that, since Ck does not depend on the summaliGn index n, it can be movedwHhin the su")mdtion sign. Assume then by derillitio,) that y(-1)=0, and note that the cosine function for the input y(7) is equal to zero. The ex~.ression for h(k) can then be rewritten in the following form:
- - 35~ - Equation 9.

215~962 ss h (k ) - ~, [y (2n~ 1 )t y (2n-1)]cosl n(2K, 1)nl 2 cos ( n (2k~ o 2(N) Where k=(0,1 ,...,(N/2-1)}.
Note that the inputs [y(2n+1)=y(2n-1)] imply that in c-'cvl~ting h(k), the odd input terms are paired to form N/2 paired inputs p(n)=[y(2n+1)=y(2n-1)].
For N=8 the values of p(n) are as follows:
n p(n) 0 y(-1) + Y(1) = Y(1) Y(-1) = 0 by defillilion y(1) + y(3) 2 y(3) + y(5) 3 y(5) +y(7) Expression E9 for h(k) can then be represented by the following:
Fqvation 10.

h (k) ~ Ck ~ p (n) cos[n(2k~1)n]
~ 2 (2) Where k=(0,1 ,...,(N/2-1)}.
Obselve now that the cosine term under the summation sign is the same for both g(k) and h(k) and that both have the structure of a 1-D IDCT (compared with e~-~,r~ssion E5). The result of the IDCT for the odd k tenns, that is, for h(k), however is multiplied by the factor 30 Ck=1/{2.cos 1 n (2kt1)12N.
In other words, g(k) is an n/2-point IDCT operali"g on even inputs y(2n) and h(k) is an n/2-point IDCT operdli"y on [y(2n+1)=y(2n-1)] where y(-1)=0 by defil ,ition.
_ - - Now introduce the f~ JJ;I l9 identities:

21~4962 Now introduce the following identities:
yn=y(n);
c1=cos(n8);
c2=cos(2~8)=cos(~4)=1.~ ;
c3-cos(3n8);
d 1=1 [2.cos(~1610)l;
d3=1 [2.cos(3~/16)];
d5=1 [2.cos(5~/16)]; and d7=1~2.cos(97n/16)].
Further introduce scaled cosincoe erricienls as follows:
c1& ~.cos(~/8);
c3s= ~.cos(3~8);
Using the evenness (cos(-O=cos(O)and periodicily (cos ( -q~ ))TT ( - O = - cos (q~) of the cosine function, expressions E7 and 15 E8 can then be e~.anded for N=8 to yield (recall also (O) is 1/~);

g(O)=1/~.y 0 ~ y2c1 , y4c2 t y6c3 = 1/~.(y ~ y2.cls ~ y4 ~ y6.c3s) g(1)=1/~.yO t y2c3 - y4c2 - y6c1 = 1/~. (yO t y2.c3s - y4 - y6.cls) ~ y6.cls) g(3) = 1/~ .yO - y2c1 t y4c2 - y6c3 = 1/~. (yO - y2.cls t y4 - y6.c3s) and h(O) = d1. {y1 ~ (y1 , y3) c1 ,(y3 t y5) c2 , (y5 ~ y7)c3}=
d11~.{ ~.y1 t (y1 ~ y3). cls ~ (y3 , y5) t (y5 t y7).c3s}
h(1) = d3.{y1 ~ (y1 t y3) c3 - (y3 ~ y5) c2 t (y5 ~ y7) c1} = d31~.
{ ~.y1 t (y1 t y3) c3s -(y3t y5) -(y5 ~ y7) c1s}
h(2) = d5.{y1 ~ y3) c3 - (y3 ~ y5) c2 t (y5 t y7) c1} = d51~.
{~.y1 - (y1 t y3). c3ys -(y3~ y5) -(y5 , y7) c1s}
h(3) = d7. { y1 - (y1 t y3) c1 ~ (y3 t y5) c2 - (y5 ~ y7) c3 } = d71~.

{ ~.y1 - (y1 ~ y3). c1s t (y3, y5)-(y5 t y7). c3s }
Now, recall that according to this embodiment of the present invention, all values are scaled upward by a factor of 2 for both the DCT and IDCT opera-tions. In other words, according to the embodiment, both h(k) and g(k) are multiplied by this scaling factor. The g(k) and h(k) expressions, erthefore, ~ecG".e:

, Equation 11. g(0) = yO ~ y2 ~ cls ~ y4 ~ y6 ~ c3s 9(1) = yO t y2 t c3s - y4 - y6 t c3s g(2) = yO - y2 ~ c3s - y4 t y6 ~ cls and g(3) = yO - y2 ~ cls t y4 - y6 ~ c3s Equation 12.
h(O) . d1 [~ ~ y1 ~ (y1 t y3) ~ c1s ~ (y3 ~ y5) t (y5, y7) ~ c3sl h(1) - d3 l~ ~ y1 ~ (y1 t y3) ~ c3s - (y3 + y5) - (y5, y7) ~ c1s]
h(2) - d5 [~ ~ y1 - (y1 t y3) ~ c3s - (y3 ~ y5) ~ (y5, y7) ~ c1s]
h(3) . d7 l~ ~ y1 - (y1 ~ y3) ~ c1s, (y3, y5) - (y5 ~ y7) ~ c3s]

Notice that since c2 = cos (n /4) = 1 /~ multiplication by ~ gives a scaled c2 value=1. By scaling the expressioi)s (conesponding to upward scaling of the values of the video absolute and frequency values) accordi"g to this embodi"~enl- it is therefore possible to eliminate the need to multiply and c3s both of which are constant coer~icie. ILs so that general utility multipliers are not needed. This, in turn cl;."inales the need for the cor,esponding hardware multiplier in the se,nicGn-ductor irnplelnentatiGn of the IDCT operations.
The similarity in structure of g(k) and h(k) can be illustrated by expressing these sets of equations in matrix form. Let C be the 4 x 4 cosine coer~ciel ,t matrix defined as follows:
Equation 13.
1 c1s 1 c3s 1 c3s -1 -c1s 1 -c3s -1 c1s 1 -c1s 1 -c3s ~.- 215~962 , _ Equation 14 g() yo ) y2 ~2~ ~ C ~ 4 g~3) y6 Equation 15.

h(O) 1~ ~ Yl ( ) . D ~ C ~ Y Y
h(2) y3 ~ y5 h(3) y5, y7 Where ~= diag[d1, d3, d5, d7]=the 4 x 4 matrix with d1, d3, d5, and d7 along the diagonal and with other elements equal to zero. As E14 and E15 show, the procedures for operdlins~ on even-numbered inputs to get g(k) and for operat-ing on the odd-numbered inputs to get h(k) both have the commol1 step of multiplication by the cosine cGe:rricie.-t matrix C. To get h(k), however, the inputs must first be pairwise summed (recalling that y(-1)=0 by deri"itiGn), y(1 ) must be premultiplied by 2, and the result of the multiplication by C must be multiplied by D.
As the expr~ssio"s above also indicale, the N-point, 1-D IDCT (see E4) can also be split into the two N/2-point, I-D IDCrs each involving cG"~mon core operaliG"s (under the summation sign) on the N/2 odd (grouped) and the N/2 even input values. The ex~.r~ssiG"s above yield the following simple structure for the IDCT as imple",ented in this embodiment:
Low-order outputs for (N=8, outputs k={0,1,2,3}):
Equation 16.
y(k)=g(k)~h(k) High-order outputs (for N=8, outputs k={4,5,6,7}):
Equation 17.
y(k)=y(N-1 -k')=g(k~)-h(k~) Note that g(k) operates directly on even input values to yield output 25 values directly, whereas h(k') involves grouping of input values, as well as multiplication by the values d1, d3, d5 and d7.

_ As always, thdee signer of an IDCT circuit is faced with a number of trade-offs, such as size versus speed and greater number of implemented devices versus reduced i"lercGi)nection complexity. For example, it is often possi~le to improve the speed of computation by including additional, or more 5 comr'ic~ted devices on the silicon chip, but this obviously makes the imple-mentation bigger or more complex. Also, what is available or desired on the IDCT chip may limit or preclude the use of so,~ licated, complic~ted, designs such as "look-ahead" adders.
Standar~ls of Accuracy 10 Assuming infinite p,ecision and accuracy of all c 'u~'~tions, and, thus, unlim-ited storage space and c~lc~ tion time, the image recreated by performing the IDCT and DCT-l,ans~r",ed image data would reproduce the oriy;nal image perfectly. Of course, such pe, rection is not to be had using existing technol-ogy.
In order to achieve some standardization, however, IDCT systems are at present measured accordil ,9 to a standardized method put forth by the Comite Consultatif International Tele~laphique et Telephonique ('CCIT') in 'Annex 1 of CCITT RecG"""endations H.261 - Inverse Transfo"" Accuracy Specification.' This test speciries that sets of 10,000 8-by-8 Blocks containing random inte-20 gers be generated. These blocks are then DCT and IDCT transformed (pre-ceded or followed by predefined rounding, clipping and a,itl,melic opei-aliGI)s)using predefined precision to produce 10,000 sets of 8-by-8 'reference' IDCT
output data.
When testing an IDCT imple",enlalion, the CCITT test blocks are used 25 as inputs. The actual IDCT l,ansfor",ed outputs are then compared statistic~ywith the known 'reference' IDCT output data. Maximum values are specified for the IDCT in terms of peak, mean, mean square, and mean mean error of blocks as a whole and as individual elements. Furthermore, the IDCT must produce all zeros output if the corresponding input block contains all zeros, and 30 the IDCT must meet the same standards when the sign of all input data is changed. Imple,nentalions of the IDCT are said to have acceptable accuracy only if their ",axi,num errors do not exceed the specified "laxi",um values when these tests are run.
Other known standards are those of the Institute of Elect,ical and 35 Electronic Engineers ('IEEE'), in 'IEEE Draft Stal-dard Speci~icalion for theImple",ent.lion of 8 by 8 Discl~te Cosine Transform', P1 180/D2, July 18, 1990; and Annex A of '8 by 8 Inverse Discrete Cosine Tl ansfor",',. ISO com-_ 21~962 mittee Draft CD 11172-2. These standards are essentially identical to the CCITT standard described above.
Hardware l~nple.nenldti~"
FIG 9 is a si""~lified block diagram illustrating the data flow of the IDCT method according to one elnbodi,~ent of the present invention (although the hardware structure as is illuslldted and explained below is made more compact and erricient). In FIG 9 the inputs to the system such as Y[O] and Y[4] and the outputs from the system such as X[3] and X[6] are shown as being conveyed on single lines. It is to be understood that each of the single-drawn lines in FIG
9 repr~sents several conductors in the form of data buses to convey, prefera-bly in parallel the several-bit wide data words to which each input and output cGr,asponds.
In FIG 9 the large open circles 225 and 226 represent two-input adders v,ll ,eraby a small circle 227 at the col ,nection point of an input with the adder indic~tes that the complement of the corlaspGnding input word is used. Adders with such a complementing input thus subtract the cor"plen,e"led input from the non comple."ented input. For excu,,ple although the output TO from the upper left adder will be equal to Y[O] + Yl4] that its TO = YO ~ Y4 the adder with the output T1 forms the value YO + (-1) * Y4=YO - W. Adders with a single comple. "entiny input can therefur~ be said to be differencing compo-nents.
Also in FIG 9 cGnstant-coerr,cienl multipliers are represented by solid l,ian$;1es 230 in the data path. For exal"ple the input Y1 p~sses through a square root of two multiplier before entering the adder to form BO. Conse-quently the inler",ediate value T3=Y2.T3=Y2.c1S + Y6.c3s and the i"terl"edi-ate value B2 = pl.c3s-p3.c1s=(Y1+Y3).c3s - (Y5+Y7).c1s. By performing the indic~ted additions, sul,t,actions and multiplications, one will see that the illusllated structure imple",enls the ex~ rassiGns E11 and E12 for 9(0) to 9(3) and h(O) to h(3).
FIG 9 illusl,dtas an important advantage of the embodi~ent in accor-dance with the present invention. As FIG 9 shows the structure is divided into four main regio"s: a pre-co",r"on block PREC 231 that forms the paired inputs p(k) and multiplies the input y(1) by the square root of two; a first post-col""~onblock POSTC1233thatincludesfourmu'tir'iersfortheconstants d1 d3 d5 d7 (see ex~.rassiol, E12); a second post-co",r"on block POSTC2 235 that sums the gO to 93 terms and the hO to h3 terms for the low order outputs and forms the ,lifrerance of the gO to 93 terms and the hO to h3 terms ~-.` 21S49~2 for the high-order outputs (See expressions E17 and E17); and a co",l"on block, CBLK 2 3 2,is included in both the even and odd data paths. In the processi"g circuitry according to the embodiment of the present invention, the co,n",on operalions performed on the odd and even numbered inputs are carried out by a single structure, rather than dupl ~ ~ted structure as illusllaled in FIG 9.
To under~tand the method of operation and the advantages of certain digital structures used in the embodiment, it is helpful to understand what "carry bits". As a simple example, note that the addition of two binary numbers is such that 1 + 1 = 0, with a carry of "1", which must be added into the next higher order bit to produce the correct result '"10"' (the binary repr~sel)talion of the dec;",al number n2"). In other words, 01 + 01 = 00 (the "sum" without carry)+ 10 (the carry word); adding the "sum" to the "carry word"' one gets the correct answer 00 + 10 = 10.
As a decimal exal,)ple, assume that one needs to add the numbers '4 36' and '825'. The common procedure for adding two numbers by hand typically proceeds as follows:
1. Units '6' plus '5' is '1' with a carr,v of '1' into the 'tens' position -Sum: 1, Carry-in: 0, Carry-Out: 0.
2. Tens: '3' plus '2'is '5', plus the '1' carried from the preceding step, gives'6' with no carry -Sum: 5, Carry-ln: 0, Carry-Out:0.
3. Hundreds: '4' plus'8' is '2' with a carry of 1 into the thousands, but with 2 5 no carry to be added in from the previous step;
Sum: 2, Carry-ln:), Carry-Out:1 4. Thousands: '0' plus '0', plus the '1' carried from the hundreds gives, '1' Sum: 0, Carry-ln: 1, Carry-Out: 0.
The answer, '1 2 61',is, thus, formed by adding the carry-in sum for each 3 0 position to the sum for the same position, with the carry-in to each position being the carry-out of the ~dj~cent lower-order position. (Note that this implies that the carry-in to the lowest order position is always a '0'). The problem, ofcourse, is that one must wait to add the '4' and '8' in the hundreds place untilone knows whether there will be a carry-in from the tens place. This illustrates3 5 a "ripple adder", which operates essentially in this way. A ripple adder, thus, achieves a 'final' answer without needing extra storage elements, but it is slower than some other designs.

_ 2154962 One such alter"dli./e design is known as 'carry-save', in which the sum of two numbers for each positiGn is for"~ed by storing a partial sum or result word (in this example, 0251) and the carry values in a different word (here, 1010). The full answer is then obtained by 'resolving' the sum and carry words in a following ~d~ition step, thus, 0251 ~ 1010 = 1261. Note that one can perform the addition for every position at the same time, without having to waitto determine whether a carry word can be added to the partial result at any time as long as it is saved.
Since the resolving operdlions typically require the largest proportion of the time required in each c~ 'qtion stage, speeding up these operaliol)s has a signilicant effect on the overall Opeldtil~ speed while requiring only a relatively small increase in the size of the transform. Carry-save multipliers, - therefore, are usually faster than those that use ripple adders in each row.
However, this gain in time comes at the cost of greater complexity, since the carry word for each zdditiGn in the multiplier must be either stored or passed down to the next addition. Furthermore, in order to obtain the final product of a multiplication, the final partial sum and final carry word will have to be resolved, normally by addition in a ripple adder. Note, however, that only one ripple adder will be needed, so that the time savings are normally proportional to the size of the multiplication that must be performed. Furthermore, note that a carry word may be t,eated as any other number to be added in and as long as it is added in at some time before the final multi~.licdliol, answer is needed, the actual ~dditioll can be delayed.
In this embocli",el,l of the present invention, this possibility of delaying r~sDI ~tion is used to simplify the design and to increase the throughout of theIDCT circuitry. Also, certain bits of preselected carry words are, Gplionally and deliber~tely forced to predelermined values before resolution in order to provide greater eYpected accuracy of the IDCT result based on a stalislical analysis of test runs of the invention on standard test data sets.
FIG 10 is a block diag~l" that illuslldtes a p(efel,ed structure, in accordance with the present invention. In this preferred embodiment of the present invention, the even and odd numbered inputs are time-multiplexed and are processed separately in the col"r"on block CBLK 232. The inputs may be processed in either order.
In FIG 10, the notation Y[1,0], Y[5,4], Y[3,2] and Y[7,6] is used to indicate that the odd nu,ober~d inputs Y1, Y3, Y5, Y7 preferably pass through the calcu'~tion drcuitry first, followed by the even numbered inputs Y0, Y2, Y4, _ 2154962 Y6. This order is not essential to the present embodiment; nonetheless, as is explained below, certain dow"~ a", arithmetic operations are performed only on the odd numbered inputs, and by entering the odd numbered input values first, these dov~"~ a", operaliGIls can be processi"y at the same time that S a,illu"etic operalions co",r"Gn to all inputs are performed u,u~ ari, on the even nu"~bert:d inputs. This reduces the time that several arithmetic devices would otherwise remain idle.
Similarly, the notation X[0,7], X[1,6], X[3,4], X[2,5] is used to indicate that the low order outputs X0, X1, X2, X3 are output first, followed by the highorder outputs X4, X5, X6, X7. As FIGS. 9 and 10 illustrate, the inputs are preferably initially not grouped in ascel)ding orde~, although this is not neces-sary since to odd numbered inputs are Y1, Y5, Y3, and Y7. Arranging the input signals in this order makes possible the simple 'butterfly' data path structure shown in FIGS. 9 and 10 and greatly increases the interconnection 15 erricie.-cy of the i",plen,e"ldlion of the present invention in silicon semiconduc-tor devices.
As shown in FIG 10, adders and subl,dctors are inditecad by circles either a '+' (adder ) 235, '-' (subtractor ) 236 which is an adder with one comple-menting input or '+' (resolving adder/subtractor, which is able to switch between 20 addition and su~l,a1iGn 237). The left most adders and subtractors in the cGmmol) block 232 of the two m-bit input words is the m-bit partial resulting parallel with the m-bit or (m-1) bit word containing the carry bits of the addi-tion/subtraction. In other words, the first additions and subtld~;tions in the cGInmon block CBLK 232 are prererably unresolved, meaning that the addition of the carry bits is delayed until a subseguent prc cessing stage. The advan-tage of this step is that such carry-save adder/subtractors since they do not need to perform the final ~ddition of the carry-bit word to the result. Resolving adders may, however, also be used in order to reduce the bus width at the outputs of the adders.
FIG 10 also illuslldtes the use of one and two input latches in the pr~r,ed embodiment of the present invention. In FIG 10, latches are illus-trated as rectangles 238 and are used in both the pre-common block PREC
231 and the post-common block POSTC 233. Single-input latches are used at the inputs of the multipliers D1, D3, D5 and D7, as well as to latch the inputs to the resolving adders/subtractors which are the computed g(k) and h(k) values cor,~spondi.,g to the respe~tive outputs from latches 9[0,7], 9[1,6], 9[3,4] and9[2,5] and h~0,7], h[1,6], h[3,4] and h[2,5]. As such, the resolving adders/

2 1 ~ 4 9 6 2 subtractors perform the addition or subtraction i"dicated in expressions E16 and E17 above.
As described previously, the even-numbered inputs Y0, Y2, Y4 and Y6 do not need to be paired before being plocessed in the common block CBLK
232. However, not only do the odd-numbered inputs require such pairing, but the input Y12 must also be multiplied by the square root of two in order to ensure that proper input values are presented to the co"""on block CBLK 232.
The pre-col""~ol~ block PREC 231, therefore, includes a 2-input multiplexing ('mux') latch C10, C54, C32 and C76 for each input value. One input to the 2-input mux latch is consequently tied directly to the unprocessed input values, whereas the other input is receivcd from the resolving adders and, for the inputY1, the resolving square root of two multiplier. The correct paired or unpaired inputs can, ther~for~, be easily pr~sented to the co",r"on block CBLK 232 easily by simple switching of the multiplexing latches between their two inputs.As FIG 10 illusl,ates, the square root of two multipliers D1, D3, D5, D7 prefera~ly resolvc their outputs, that is, they generate results in which the carry bits have been added in to generale a cor"plete sum. This ensures that the outputs from the multipliers have the same bus width as the un-multir'.ed inputs in the col,esponding parallel data paths.
The pr~fer,ed embodi",ent of the cGn,r"on block 232, in accordance with the presel,t invention, also includes one 'dummy' subtractor 240 in the forward data path for Yl1,0] and Y[5,4], respectively. These devices act to combine the two inputs (in the case of the dummy subtractor, after 2's-colllple-menting the one input) in such a way that they are passed as parallel outputs.
In each case, the one input is manipulated as if it contained carry bits, which are added on in the subsequent processing stage. The corresponding addition and sublld~;tion is, thus, performed, although it is delayed.
This technique reduces the resources required in the upper two data paths since a full-scale adder/~uL t,actor need not be implemented for these devices. Then2fore, the 'combiners' act as adders and subtractors and can be implemented for these devices and can be implemented either as simple conductors to the next device (for addiliGn), or as a row of inverters (for subtraction), either of which requires little or no additional circuitry.
The use of such combiners also means that the outputs from the initial adders and subtractors in the coll,r"on block CBLK 232 will all have the same width and will be co,npdlil,le with the outputs of the carry-save adder/subtractor 21S~962 found in the bottom two data paths, with which they form inputs to the subse-quent resolv;"g adders and subtldctor~ in the common block CBLK.
As desc,il,ad previously, the even-numbered inputs are processecl separately from the odd-numbered inputs in this preferred embodiment of the 5 present invention. Assume, further, that the odd-numbered inputs are to be processed first. Supervisory control circuitry (not shown in FIG 10) applies theodd-numbered input words to the pre-co"~l"on block PREC, and select~ the lower inputs (viewed as in FIG 10) of the multiplexing latches C10, C54, C32, C76 which then stores the paired values p0 to p3 (see FIG 9 and the definition of p(n) above). The latches 1 hO,1 h1, 1 h3 and 1 h2 are then activated to latchthe values H0, H1, H3 and H2, respectively.
The supervisory control circuitry latches and then selects the upper inputs of the two-input multiplexing latches C10, C54, C32 and C76 in the precommGn block PREC 231 and applies the even numbered input words to these lalches. Since the even-numbered inputs are used to form the values of g0 to 93, the supervisory control circuitry also opens the latches Lg0 to Lg3 inthe post-col~mGn block POSTC 233, to store the g(k) values.
Once the g(k) and h(k) values are latched, the post-common block POSTC 233 outputs the high-order signals X7, X6, X5 and X4 by switching the resolving adder subl~dctor~ to the sublldctiol) mode. The low order output signals X3, X2, X1 and X0 are then generated by switching the resolving adders/subtractors to the addition mode. Note that the output data can be presel)led in an arbitrary order, including natural order.
The p(efer,~d multiplexed imple",el1talion, in accordance with the present invention, is illusl,ated in greatly simplified, schematic form in FIG 10, performs the same calculdliol)s as the non-multiplexed structure illusl,dled in FIG 9. The number of adders, subtractors and multipliers in the co,nmon block CBLK 232 is, ho/:ever, cut in half and the use of dummy adder/subtractors 240 further reduces the complexity of the costly aritl"nelic circuitry.
FIG 11 illu~l,ates the main components and data lines of an actual implementatiGn of the IDCT circuit according to the embodi",ent of the present invention. The main components include the precol"rnon block circuit PREC
231, the cor""~Gn block circuit CBLK 232, and the post-col"",on block POSTC
233. The system also inclucles a conl,~'lEr CNTL 241 that either directly or indirectly applies input, timing and control signals to the p,t:con,r"on block PREC 231 and post-con~")on block POSTC 233.

`~ ` 21S4962 In the preferred embodiment of the present invention, the input and - output signals (Y0 to Y7 and X0 to X7, respectively) are 22 bits wide. Tests have indicated that this is the minimum width that is possiL,I~ which still yields acceptable accuracy as measured by existing industry standards. As is explained in yl~ater detail below, this minimum width in achieved in part by deliberately forcing certain carry words in selected a~ itl " "etic devices to be either a '1' or a '0'. This bit manipulation, corresponding to an adjustment of certain data words, is carried out as the result of a stali~lical analysis of the results of the IDCT system, in accGr~lal)ce with the present invention, to the after using the IDCT tra"~ror",dlion of known input test data. By forcing certain bits to pr~dete",lined values, it was discovered that the effects of rounding and truncation errors could be reduced, so that the spatial output datafrom the IDCT system could be made to deviate less from the known 'c~r,t:ct' spatial data. The pr~sent invention is equally applicable, however, to other data word lenyll,s since the cG,nponenls used in the circuit according to the present embodiment can all be adapted to dirrerent bus widths using known methods.
Although all four inputs that are processed together could be input simultaneously to the pre-cGI~rnGn block PREC along 88 parallel conductors (4 x 22), pixel words are typically converted one at a time from the transmission data. According to the present embodiment, input data words are, therefore, pr~ferdbly all conveyed serially over a single 22 bit input bus and each input word is sequentially latched at the proper input point in the data path. As shown in FIG 11, the 22 bit input data bus is labelled T_IN[21 :0] 242.
In the Figures and in the discussion below, the widths of multiple-bit signals are indic~d in brackets with the high-order bit to the left of a colon ':' and the least significant bit (LSB) to the right of the colon. For example, the input signal T IN[21 :0l 242 is 22 bits wide, with the bits being numbered from 0 to 21. A single bit is idel ,li~ied as a single number within square brackets, thus, T_IN[1] i"dicates the next to least significant bit of the signal T_IN.
The fcllDw;ng control signals are used to control the operaliol) of the pre-common block PREC 231 in the pr~fer,ed embo.li",e"l of the present inven-tion.
IN_CLK, OUT_CLK: The system, in accordance with the present invention, plefer~bly uses a non-overlappi.,5a two phase clock. The signals IN_CLK and OUT_CLK are accGrdi"yly columns of latches that hold the values of input, intermediate, and output signals.

215~962 LATCH10, LATCH54, LATCH32, LATCH76: Pleferably, one 22-bitword is input to the system at a time. On the other hand, four input signals are processed at a time. Each input signal must, therefore, be latched at its appropriate place in the archite~;t.lre before being processed with three other 5 input words. These latch signals are used to enable the respective input latches. The signal LATCH54, for example, is first used to latch input signal Y5and later to latch input signal Y4, which enters the pre-co",ri,ol1 block PREC
231 at the same point as the input signal Y5 (see FIG 10) but during a subse-quent processi"g stage.
LATCH: Once the four even or odd-numbered input signals are latched into the pre-cor",nol- block PREC 231, they are preferably shifted at the same time to a suhseguent column of latches. The signal LATCH is used to enable a second column of input latches that hold the four input values to be operated on by the aritl""etic devices in the pre-co",r"on block PREC 231.
SEL_BYP, SEL_P: As FIG 10 illustrates, the even-numbered input signals'~hat are latched into the latches C10, C54, C32 and C76 should be those that bypass the adders and the square root of two resolving multiplier.
The odd-numbered input signals, however, must first be paired to form the paired inputs p(n), and the signal Y1 must be multiplied by the square root of two. The control signal SEL_P is activated in order to select the paired input signals. Hence, these signals are used to control gates that act as multiplexersto let the correct s4anals pass to the output latches of the precommon block PREC 231.
As discussed previously, not having to arrange the inputs in strictly ascending order leads to a simplified 'butterfly' bus structure with high intercon-nection erficiency. As also desc,il,ed, the odd inputs are pr~ferably applied asa group to the pre-co~mGn block first, f~l'owod by the even-numbered inputs, but any order may be used within each odd or even group, i.e., any order of inputs may be used, however, su~t~hle latch arrangements as separalely provided to process the odd-numbered inputs, or at least are provided in separate regions of the circuit.
The supervisory control circuitry also generates timing and control signals for the post-colnmon block POSTC 233. These control signals are as follows:
EN_BH, EN_GH: Referring again to FIG 9, the outputs from the com-mon block CBLK 232, a~fter processi"g of the odd-numbered inputs, are shown as H0, H1, H3, and H2. These signals are then sent to the coerficient multipli-~,` 21S~962 ers, d1, d3, d7, d5, respectively, in the first post common block POSTC 1 233.
The signal EN_BH is used to enable latches that hold the 90 to g3 values, as well as to enable the latches that hold the hO to h3 values after they have beenmultiplied in the coerficie.,l multipliers.
ADD, SUB: As FIG 10 illusl,ates, the embodiment includes a bank of resolving adders/sul.l,actor~ that sum and difference(k) and h(k) values in order to form the low-order outputs, respectively. The signals ADD, SUB are used to set the resolving adders/subtlddor~ in the addition and subtraction modes, respectively.
EN_O: This signal is used to enable output latches that latch the results from the resolving adders/subt,~ctors.
MUX_OUT70, MUX_OUT61, MUX_OUT43, MUX_OUT52: In accor-dance with the present invention, the output data from the system is pr~ferably llansri,illed over a single 22-bit output bus, so that only one output value (X0 to X7) is transfer,ed at a time. These signals are activated sequentially to selectwhich of the four latched output values is to be latched into a final output latch.
Accordi,)gly, these signals thus act as the control signals for a 4-to-1 multiplexer.
T_OUT[21 :0]: This label indicates the 22-bit output signal from the post-common block POSTC 233.
The output signals from the pre-cGrr,r"o" block PREC 231 are latched to form the input siy"als to the coi "mol- block CBLK 232. As shown in FIG 11, the output sigl1als from the pre-co"~,non block PREC 231 are presented as the four 22-bit data words Cl10121:0], Cl54121:0], Cl32121:0], Cl76[21:0], which become the input signals IN[0], IN[1], IN[3], IN[2], respectively, to the cG,r,r"G, block CBLK 232.
As FIG 11 shows, the four 22-bit results from the common block CBLK
232 are tlansfer,ed in parallel as output signals OUT0[21:0], OUT1[21:0], OUT3[21 :0], OUT2[21 :0], which are then latched as the input signals of the pOSt-COI"I"O~ block POSTC 233 as CO70[20:1], CO61[21:0], CO43[21:0], CO52[21 :0].
One should take particular note that no control signals are required for the co",mGn block CBLK. Rec~use of the unique structure of the IDCT system in this exal"ple, the col"",on block of the system's operations can be per-ruI"~ed as pure logic operalioRs, with no need for clock, timing or control signals. This further reduces the complexity of the device. One should also note that in certain applications (particularly those in which there is plenty of 21~4962 time to perform all needed arill"~etic operations) the pre-cG,nrnGn and post-cGnllllGn blocks PREC 231, POSTC 233 may also be ar,dnged to operate without clock timing or control signals.
FIG 12 is a block diagram of the pre-col"mon block PREC 231 of the 5 present invention. In this and following Figures, the notation 'S1 [a], S2[b],...,SM[Z]', where S is an arbitrary signal label and a,b,..., z are integers within the range of the signal's bus width, i"dicates that the selected bits a, b, ..., z from the signals S1, S2, ..., SM are transferred in parallel over the same bus, with the most significant bits (MSBs) being the selected bits 'a' of the 10 signal S1, and the least significant bits (LSBs) being the selected 'z' of signal SM. The s~lected bits do not have to be individual bits, but rather, entire or partial multi-bit words may also be transmitted along with other single bits or complete or partial multi-bit words. In the Figures, the symbol S will be replaced by the cGr,espol1ding signal label.
For exal"~le, in FIG 12, a square root of two multiplier is shown as R2MUL The 'save'; or 'unresolved sum' output from this non-resolving multi-plier is indio~ed as the 21-bit word M5S[20:0], similarly, the 'carry' output from the mu4i, lier R2MUL is shown as the 22-bit word M5C[20:0], which is trans-ferred over the bus to the 'b' input of a carry-save resolving adder M5A.
(Recall that a '0' is i"se, led as an MSB to the least significant 21 bits of the save output, l,owevcr, this is accomplished before being applied to the 'a' input of the ,esolv;"g adder M5A. This is indicated in FIG 12 by the notdtiGn GND.M5S[20:0]). In other words the conductor cGr,~asponding to the MSB
input to the adder M5A is forced to be a '0' by tying it to ground GND.
In order to under:,tal1d why a '0' is inserted as the 22'nd bit of the 'sum', observe that if the partial sum of a multi~ tion is n places wide, the carry word is shifted one place to the left relative to the partial sum. The carr,v word, therefore, e)-lends to n + 1 places with a valid data bit in the n + 1'th position with an '0' in the least sigr,ificant position (since there is nothing before this ~ OSitiOI1 to produce a carry bit into the units posiliGn). If these two words are used as inputs to a resolving binary adder, care must be taken to ensure that the bits (digits) of the carry word are properly aligned with the cGr,esponding bits of the partial sum. This also ensures that the decimal point (even if only i"~plied, as in integer a,itl,l"etic) is kept 'aligned' in both words. Assuming the inputs to the adder are n + 1 bits wide, a '0' can then be inserted into the l,igl,est-order bit of all n-bit positive partial sum words to provide an n+ 1 bit input that is aligned with the carry word at the other input.

`~ 2154962 As is desc, il,ed above previously, the four inputs that are processed at a given time in the pre-co",mon block PREC 231 are transferred over the input bus T_IN(21 :0). This input bus is connected to the inputs of four input latchesIN10L, IN54L, IN32L, AND IN76L. Each respective latch is enabled only when 5 the input clock signal IN_CLK and the cGr"aspGnding latch selection signal LATCH10, LATCH54, LATCH32, LATCH76 are high. The four inputs can, ther~fore, be lalched into their respective input latches in four periods of theIN_CLK signal by sequential activation of the latch enabling signals LATCH10, LATCH54, LATCH32, and LATCH76. During this time, the LATCH signal 10 should be low (or on a dirrerent phase) to enable the input latches IN10L, IN54L, IN32L, and IN76L to stabilize and latch the four input values.
An example of the timing of the latches, in accordance with the present invention, is illu~tlated in FIG 13. Once the four input signals are latched in the pre~rled order, they are p~ssecJ to a second bank of latches L10L, L54L, 15 L32L, L76L. These second bank of latches are enabled when the signals OUT_CLK and LATCH are high. This signal timing is also illustrated in FIG 13.
Note that the system of the present invention does not have to delay recei"t of all eight input words. Once all the even or odd input words are received and lal~;l,e.l in IN10L, IN54L, IN32L and L76L, this frees the In 20 latcl ,es, which can then begin to receive the other four input signals without delay at the next rising edge of IN_CLK.
The 2-digit suffix notation [10, 54, 32, 76] used for the various compo-nents illustrated in the Figures indicates that odd-numbered signals are processed first, followed by the even-numbered signals on a subsecluent pass 25 through the structure. As is ",enlio"ed above, this order is not required by the present invention, and it will be appr~c; ~ted by one of ordinary skill in the art that ~dditiG"al orders may be used.
Once the four input signals are latched in proper order in the second set of latches L10L, L54L, L32L, L76L, the cor,esponding values are either ,uassed 30 . as inputs to output latches C10L, C54L, C32L and C76L on activation of the s~le~d bypass signal SEL_BYP, or they are passed as paired and multirlied inputs to the same output Idtches upon activation of the 'select p' signal SEL_P. In other words, all signals are passeJ, both directly and indirectly, viaarithl"~tic devices, to the output latches C10L, C54L, C32L, C76L of the pre-35 con""on block PREC 231. The proper values, however, are loaded into theselatches by activation of the 'select bypass' signal SEL_BYP (for even-num-bered inputs Y0, Y2, Y4, and Y6) or the "select p" signal SEL-P (for the odd-_ numbered inputs Y1, Y3, Y5 and Y7). As will be appreciated by one of ordi-nary skill in the art, the desired timing and order of these and other control signals is easily acco"~plished in a known manner by proper configuration and/or [micro-] progl~lnming of the controller CNTL 241.
The uppermosl input value at the output of latch L10L is passed first to the square root of two-multiplier R2MUL and then to the resolving adder M5A
as il Iclic~ted. The output from the resolving adder M5A is shown as an equiva-lent of the resolved multiplication of the output from the latch L10L by the square root of two. The outputs from the other three latches L54L, L32L, L76L
are also lldnsr~:rlt:d to cor,esponding output latches C54L, C32L and C76L
r~spectively, both directly via 22-bit latch buses LCH54[21 :0], LCH32[21 :0]
LCH76[21 :0] and indirectly to the output latches via resolving adders P2A, P1A
- and P3A, respectively.
In the present invention, each resolving adder P2A, P1A, P3A has two inputs "a" and "b". For adder, P2A, the one input is received from the latch L32L, and the other input is r aceived from the latch L54L. For input values Y5 (latched in L54L) and Y3 (latched in L32L), the output from the adder P2A will, therefore, be equal to Y5 + Y3, which, as is shown above, is equal to p(2).
Hence, the adders "pair" the odd-numbered inputs to form the paired input values p(1), p(2) and p(3). Of course, the even-numbered input signals latched in L54L, L32L, and L76L will also pass through the resolving adders P2A, P1A
and P3A, r~s~,e.;tively, however, the resulting p "values" will not be p~ssed tothe output lalches C54L, C32L and C76L because the "select p" signal SEL_P
will not be activated for even-numbered inputs.
The values that are latched in the output latches C10L, C54L, C32L and C76L upon activation of the input clock signal IN_CLK will therefora be equal toeither the even-nu",ber~d inputs Y0, Y2, Y4, Y6 or the paired input values P0, P1, P2, P3 for the odd-nu",bered inputs. One should recall that the input Y(1) is "pai.e.l with the value U(-1), which is assumed to be zero. As illu~l,alad inFIG 12, this assumption is implemented by not adding anything to the value Y1. Instead, Y1 is only multiplied by the square root of two as is shown in FIGS. 9 and 10.
FIG 14 illusl,ales the pr~1~r,ed architecture of the co,r",lon block CBLK
232, in accordance with the present invention. Because of the various multipli-calions and ~dditions in the different system blocks, it is necess~ry or advanta-geous to scale down the input values to the col"lnon block before performing the various calc~ tions. This ensures a uniform position for the decimal point ``- ` 2154962 (which is i,nFlied for integer a,itl""etic) for cor,esponding inputs to the various arithmetic devices in the system.
Accordi"gly, the input values IN0[21:0] AND IN1[21:0] are accordingly scaled down by a factor of four, which corresponds in digital a,ill"nelic to a 5 right shift of two bits. In order to preserve the sign of the number (keep positive values positive and negative values negative) in binary representaliGn,the most sig"ificant bit (MSB) must then be replicated in the two most signifi-cant bits of the resulting right-shifted word; this process is known as "sign exlensiGn". Hence, the input value IN0 is dow"sl,ifled by two bits with sign extension to form the shifted input value indic~ted as IN[21], IN0[21], IN0[21:2].
The input value IN1[21:0] is similarly sign-extended two places. The input IN2 is also shifted and extended to form IN2[21], IN2[21:1]. These one-position shifts cor,espond to truncated division by a factor of two.
As shown in FIG 10, the input IN2, IN3 are those which must be 15 multiplied by the scaled cOerricie~ c1s and c3s. Each input IN3 and IN2 must be multiplied by each of the scaled coefficients. As FIG 14 illusl,dles, this isimplemented by the four conslant-coer~icie.1t carry-save m~ltirli~rs MULC 1 S, MULNC1S, MULC3S3, and MULC2S2. One should note that the bottom multiplier for IN2 is an inverting multiplier MULCIS, that is, its output corre-20 sponds to the negative of the value of the input multiplied by the constant C1 S.Therefore, the value latchecl in C76 is subt,d.:ted from the value latched in C32 (after multiplication by C3S). By providing the inverting multiplier MULNC1S, SUbtla~tiGn iS i",ple.nel1ted by adding the negative of the corresponding value,which is equivalent to forming a difference. This allows the use of identical 25 circuitry for the sl ~hsequent adders, while allowing a non-inverting multiplier may be used with a following subt,dctor.
In the illustrated e,llbG~li,l,ent of the present invention, four cosine coefficient multipliers MULC1S, MULNC1S, MULC2S3, and MULC3S2 are included. If ar,dl1gemel,ls are made for signals to pass separately through the 30 multipliers, hoJ~evcr, the necessary multiplications can be implemented usingonly two multipliers, one for the c1 s coefficient and one for the c3s coerricient.
In accordance with the presel,l invention, the multipliers for MULC1S, MULNC1 S, MUL3S3 and MULC3S2 are preferably of the carry-save type, which means that they produce two output words, one cor,esponding to the 35 result of the various rows of additions pel rur")ed within a hardware multiplier, and another cor,esponding to the carry bits generated. The outputs from the `~ 215~962 multipliers are then connected as inputs to either of two 4-input resolving adders BT2, BT3.
For ease of illusl,aliGn only, five of the output buses from the multipliers are not drawn connected to the cGr,esponding input buses of the adders, as will be appl~ci~ted by one of ordi"aly skill in the art, these connections are to be under~tood, and are illu~llated by each respective output and input having the same label. Hence, the save output M1S[20:0] of the multiplier MULC1S is connected to the lower 21 bits of the "save-a" of the adder BT3.
As shown in FIG 14, five of the inputs to the adders BT2 and BT3 are shown as being "split". For exd""~le, the "ca" input of the adder BT2 is shown as having IN3[21] over M3C[20:0] being input as the least significant 21 bits.
Similarly, the "sa" (the "save-a" input) of the same adder is shown as being GND, GND over M3S[19:0]. This means that two zeros are appended as the two most significant bits of this input word. Such appended bits ensure that theproper 22-bit wide input words are formed with the proper sign.
The carry-save adders BT2 and BT3 add the carry and save words of two different 22-bit inputs to form a 22-bit output save word T3S[21 :0] and a 21-bit output carry word T3C[21:1]. Accordingly, the input to each adder is thus 88 bits wide and the output from each adder is 43 bits wide. As FIG 10 illu~t,ates, the output from the latch C10 is combined with the output from the latch C54 in the upper-most data path before addition with the output from the carry-save adder BT3. The "combination" is not, however, necess~,y until reaching the following adder in the upper data path. Consequently, as FIG 14 shows, the shifted and sign-exlended input value IN0 is connected to the upper carry input.
The upper carry input of adder CS0 is connected to the shifted and sign-extended input value IN0, and the shifted and sign-extended input IN1 is connected as the upper save input of the same adder. In other words, lN0 and IN1 are added later in the adder CS0.
The designdtion "dummy" adder/subtractor 240 used in FIG. 10, there-fore, indicates which operdliGn must be performed, although it does not ~,ecess~ily have to be pel ror" ,ed at the point indicated in FIG. 10. Similarly, the lower dummy sul,llactor 240 shown in FIG.10 requires that the output from latch C54 be sul,t,acted from the output from latch C10. This is the same as adding the output from C10 to the complement of the output of C54.
Ref~r,ing once again to FIG.14, the complement of the input IN1 (cor,espGnding to the output of latch C54 in FIG 10) is performed by a 22-bit ~ 2154962 input inverter IN1[21:03 (which generates the logical inverse of each bit of itsinput, bit-for-bit). The c~lllple.llent of IN1 value-NlN1[21:0]-is p~ssed to theupper "save" input of the adder CS1, with the c~"esponding upper "carry" input being the shmed and sign-e~lel,decl IN0. The upper portion of the adder CS1, 5 therefore, pel rur",s the subt,d~lion cGr,esponding to IN0 minus IN1.
In the lowertwo data paths shown in FIG. 10, resolving subt,dctors are used instead of the resolv;"y adders shown in the upper two data paths at the output of the c~" ,rnGn block CBLK 232. Each resolving adder or subtractor is equivalent to a carry-save adder or subt,~..1ùr followed by a r~:solv;"y adder.
This is shown in FIG. 14. Subtldct~l~ CS2 and CS3 have as their inputs the processed values of IN0 to IN3 accGrding to the c~n n ectiol, structure shown inFIG. 10.
- The 22-bit carry and save outputs from each of the adders/sul,l,actûr~
C20-CS3 are resolvcd in the resolving adders RES0-RES3. As will be appreci-ated by one of or~linal~ skill in the art, r~so' ~tisn of carry and save outputs is well under:jtood in the art of digital design and is, ll,er~fore, not descril,ed in y,eater detail here. As FIG. 14 illu~l,ales, the save outputs the carry-save adders/sul,l,actor:j CS0-CS3 are p~ssed directly as 22-bit inputs to the "a"-input of the c~r,esponding resolv;. ~9 adders RES0-RES3 As is also well known in the art, the 2's-cGI"plement of a binary number is formed by inverting each of its bits (chang;ng all "1's" to "0's" and vice versa) and then adding "1". Note that the "1" can be added i~,rnedia~ely after the bit inversion, or later. The LSB of a carry word will always be a "0" which is implemented in the illustrated embodiment of the present invention by tying the LSB of the carry words O0C and 01C to ground GND as they are input to the rt:solv;"y adders RES0 and RES1 "espe~ /ely. The addition of "1" to the carry outruts of the subt,dct~l~ CS2 and CS3 to form 2'S-compleloel)ted values, I ,ow_vcr, is imple,nel)ted by tying the LSB of these data words 02C
and 03C to supply voltage VDD, thus "leplac;,~y" the "0" LSB of the carry word by a "1", which is equivalent to ad~ition by "1".
For the reasGns provided above, a "0" is appended as the LSB to the 21-bit carry words from the carry-save adders Cs0 and CS1 (by tying the LSB
to ground GND) and the LSB of the carry words from the carry-save subt,d..~r:j CS2 and CS3 is set equal to "one" by tying the cGrlespGndilly data 35 line to the supply voltage VDD. The resolving adders RES0-RES3, therefore, resolvc the outputs from the adder/subL,dctur CS0-CS3 to form the 22-bit output signals OUT0[21:0] - OUT3[21:0].

`- ` 215~962 Two advantages of the IDCT circuitry according to the embodiment of the present invention can be seen in FIG 14. First, no control or timing signalsare required for the cG",r"on block CBLK 232. Rather, the input signals to the CGI,lnlGl, block are already processed in such a way they can be applied 5 i",mecli~tely to the pure-logic aritlll"elic devise in the common block 232.
Second, by proper scaling of the data words, integer a,ill,t"elic can be used throughout (or, at least, dec;",al point for all values will be fixed). This avoids the complexity and slowness of llGdling-point devices, with no unacceptable sacrifice of prëcision.
Yet anotl,er advantage of the embodiment of the present invention is that, by ordering the inputs as shown, and by using the balanced deci"~ated Illetllo.l in accor:lance with the presenl invention, similar design structures can be used at several points in the silicon i",,~le."entalion. For example, as shown in FIG.14, the constant cGerlicient multirliers MULC1S, MULC3S3, MULC3S2 and MULNC1S all have similar structures and receive data at the same point in the data path, so that all four multipliers can be working at the same time. This eliminates "bottlenecks" and the semiconductor imple",e"ld-tion is, ll,er~f~re, able to take full advantage of the d!~p'i~tive, parallel struc-ture. The carry-save adders BT2 and BT3 similarly will be able to work simulta-neously, as will the following carry-save adders and subtractors. This symme-try of design and efticie.lt simultaneous utilization of several devices is com-mon throughout the structure according to the embodiment of the present invention.
FIG 15 shows the prerér,èd arrangement of the post-cGi""~on block POSTC 233 in accGrdance with the present invention. As FIG. 10 shows, the primarv fun.;tions of the post-cGm~on POSTC 233 are to form the hO to h3 values by multiplying the outputs of the common block by the coef~cie~ d1, d3, d5 and d7; to add the g(k) and h(k) values to form the low order outputs;
and to subtract the h(k) values from the cor,espGnding g(k) values to form the high-order outputs. Referring now to both FIG. 10 and FIG. 15, the post-common block POSTC 233 latches the cGr,esponding outputs from the com-mon block CBLK 232 into latches BHOL, BH1L, BH3L and BH2L when the Bh latches are enabled, the control circuitry sets the EN_BH signal high, and the output clock signal OUTC_CLK signal goes high. The g(k), gO to g3 values are latcl ,ed into c~r"as,uG"ding latches GOL, G1 L, G3L and G2L when the control circuitrv enabl;~s these latches via the signal EN_GH and input clock signal IN_CLK goes high.

215~962 .

The processed odd-numbered inputs, that is, the values hO to h3, are latched into latches HOL, H1 L, H3L and H2L when the EN_GH and IN_CLK
signals are high, via the co"stant coefficient multipliers D1 MUL, D3MUL, D5MUL and D7MUL. These multipliers multiply, respectively by d1, d3, d5 and d7. In the preferred e"~bodi",el)l, these constant-coerFicie.,l multipliers are prererably carry-save multipliers in order to simplify the design and to i"c,ease r.~lCUlatjon speed. As FIG 15 illusl,dtes, the "carry" ("c") outputs from the constant co~rricie.-t multipliers are conne~led, with certain changes described below, to the a inputs of resolving adders HOA, H1A, H3A and H2A. The "save" ("s") outputs from the coerricie.)l multipliers are similarly, with certain forced chansJes des~,ibed below, connected to other input of the cor,espond-ing resolving adder.
As FIG 15 further illu~l,ates, the LSB of the HO signal is p~rer~bly forced to be a "1" by tying the cor,esponding "save" output for HO is set to O
(tied to ground GND), and the second bit (cGn~sponding to HOS[1]) is setto "1". The data words from the carry and save outputs of the cGnslanl-coer~icienl multiplier D3MUL are similarly manipulated an input to the resolving adder H1A. The advantage of these manipulaliGns and their input to the resolving adder H1A.
In accordance with the present invention, all 22-bits of the carry output from the coefFicient multipliers D7MUL and D5MUL are connected directly to the "a" input of cGr,esponding resolving adders H3A and H2A. The MSB of each multiplier's "save" output, howcvcr, is forced to "O" by tying the corre-sponding data line to ground GND.
The IDCT system desc,ibed was tested against the CCITT speciricdlion clesc, ibed above. Rec~use of the scaling and other well-known properties of digital adders and multipliers, some precision is typically lost in the 10,000 sample, but run that forcing the various bits described above to either "O" or "1"
reduced the exrected error of the digital lldnsr~Jr")dlion. As a result of the bit mar;ru'~tion of the data words, the embodiment of the present invention achieved acceptable accuracy under the CCITT slandard using only 22-bit wide data words, wl ,ereas 24 bits would normally be required to produce equivalent accuracy.
Rer~use of limited pr~cisiGn, and tru,)caliGn and rounding errors, there is typically some inaccuracy in every data word in an IDCT system. However, forcing ~ elected bits of a data word it was discovered that the error thereby s~stemdtically intro~ced into a particular data word at a particular point in the `- 2154962 .

hardware yielded sl~ lically better overall results. Bit-forcing may also be appl e:l "within" a mult;,~ tion, for exd"~ple, by selectively forcing one or more carry bits to predetermined values.
In the present invention, the bit-forcing scheme need not be static, with 5 certain bits always forced to take speci~ied values, but rather a dynamic sc~,el"e may also be used. For example, s~l~cted bits of a data word may be forced to "1" or "O" depending on whether the word (or even some other data) is even or odd, positive or negative, or above or below a pr~deter"lined ll,r~sh~ld, and the like.
NGr"~ally, only small syàte",dtic changes will be needed to improve overa!l st~ti~tic-~ pe,runnance. Consequently, according to this embodiment of the present invention, the LSB's of selected data words (,oreferdbly one bit andone data word at a time, although this is not "ecess~ry) are forced to be a "1"
or a "O". The CCITT test is run, and the CCITT stalialics for the run are col"~iled. The bit is then forced to the other of "1" or "O", and the test is rerun.
Then the LSB (or LSBs) of other data words are forced to "1" or "O", and similarstz.liYilics are compiled. By examining the st~lislics for various cGlnbi. ,aliGns of forced bits in various forced words, a best a~ lical performance can be determined.
If this st~tialic~lly based improvement is not required, however, the outputs from the constant-coefricient multipliers D1 MUL, D3MUL, D5MUL and D7MUL may be resolved in the conver,liol,al manner in the resolving adders HOA-H3A. The lower 21-bits of the input of the cor,esponding latches HOL-H3L, with the LSB of these inputs tied to ground.
The outputs from the H-latches (HOL-H3L) and the G-latches (GOL-G3L) pairwise form the respe~ /e a and b inputs to resolving adder-subtractors S70A, S61A, S43A and S52A. As was indic~ted above, these devise add their inputs when the ADD signal is high, and subtract the "b" input from the "a"
input when the subtraction enable signal SUB is high. The secGnd bits of the upper two latch pairs HOL, GOL,H1 L and G1 L are manipulated by multiplexing arrangell'enta in a manner described below.
The outputs from the resolving adder-suL l,actora S70A, S61A, S43A
and S52A are latched into result latched R70L, R61 L, R43L, R52L.
As depicted in FIG 15b, the input words to the adder/sul.lld..tor S70A
35 and dS61A, in accordance with the present invention, have the second bits of each input word manipulated. For example, the second bit of the input word to the "a"-input of the adder subt,a~tor S70A is GO[1M], GO[1M], GO[O]. In other words, the second bit is set to have the value G01 M. The secG"d bits of the other inputs to the adder/subtractors S70A and S61A are similarly manipu-lated. This bit manipulation is accomplished by four 2:1-bit multiplexers H01MUX, G01MUX, H11MUX and G11MUX (shown to the right in FIG 15b).
5 In the present invention, these multiplexers are controlled by the ADD and SUBsignc.ls such that the second bit (H01 M, G01 M, H11 M, and G11 M) is set to one if the respective adder sul,l,actor S70A, S61A is set to (ADD is high), and the second bit is set to its actual latch output value if the SUB signal is set too high. Setting of individual bits in this manner is an easily implel"enled high-10 speed operation. The prefer,ed elllbodiment, thererore, includesthis bit-forcing arrangel"enl since, as is desc,il,ed above, statistical analysis of a large number of tests pixel words has indicated that more accurate results are thereby obtai"e(J. It is not necess~y, however, to manipulate the second bits in this manner, although it gives the advantage of smaller word width.
The four high or low-order results are Idlched in the output latches R70L, F~61 L, R43L and R52L. The results are sequentially latched into the final output latched OUTF under the control of the multiplexing signals MUX_OUT70, MUX_OUT61, MUX_OUT43, MUX_OUT52. Hence, the order in which resulting signals are output can ll,erefor~ be controlled simply by 20 cl ,anging the sequence with which they are latched into the latch.
The reldtionship between the clock and control signals in the post-cGl"r"on block POSTC 233 is shown in FIGS. 13b and 13c.
As was disclJssed previously, two 1-dime"siGnal IDCT operations may be pel r~r",e.l in series, with an intervening transposition of data, in order to 25 perform a 2-D IDCT. The output signals from the post-cGmmol1 block POSTC
233, are ll,eref~re, according to this embodiment of the present invention, first sorted in a known manner column-wise (or row-wise) in a conver,liGnal storage unit, such as a RAM IllelllGly circuit (not shown), and are then read from the storage unit row-wise (column-wise) so as to be p~ssed as inputs to a subse-30 quent pre-cGmmGn block and for processi"g as described above in this block, and in a c~l"mon block CBLK 232, and a post-comr"on block POSTC 233.
Storing by row (column) and reading out by column (row) performs the required operation of transposing the data before the second 1-D IDCT. The output from the second POSTC 233 will be the desire, 2-D IDCT results and 35 can be scaled in a conventional ",anner by sl,i~i"g to offset the scaling shifts carried out in the various processi"S~ blocks. In particular, a right shift by one - ,` 21~49~2 position will perform the division by 2 necess~ry to offset the two square root of two multiplications pe,rol",ed in the 1-D IDCT operations.
Dependi"g on the applications, this second IDCT structure (which is pleferably ide"tical to that shown FIG 11) is preferably a separate semicGnduc-5 tor imple",entdlici,). This avoids the decrease in speed that would arise if thesame circuits were used for both transforms, although separate 1-D transforrn implementalicins are not necess~y if the pixel-clock rate is now surficienl suchthat a single imple."enlalion of the circuit will be able to handle two p~-sses in real time.
As shown in FIGS 16 through 38, a second preferred embodiment, in accordance with the present invention, uses a single one-dimensional trans-form. This elnbodi~el)l does not require a lowering of the pixel-clock rate as discl~ssed previously.
The existing "resolving-adders" in the first preferred embodi"~el1t have been changed to "fast-resolving-adders". As seen in FIG 38, these have been titled, "Fast Resolving Adders". This change has the effect of allowing more time for each cl~tar~tl, a,itl,l"elic block to act on its data inputs. The existing "latches" in the first pr~fer,ed elnbodi.~el1t have been changed to 2-phase "flip-flops" or 'lagisters".
The latching ",emo~ ele.nellts lo~led on the front and end of the existing 1D IDCT datap ~11l pipelines have been combined into single blocks, as shown particularly in particular in FIG 18. Additionally, the amount of ",el"olyele",e"ts present at the input and the output of the second prefer~ed embodi-ment has been i"cr~ased to allow variable amounts of T2 data to be buffered.
As shown in FIGS. 16 and 17, the two data sl,ear"s, stream "T1" (raw unoperdled upon data) and stream "T2" (data which has been through the ID
IDCT once and has been transposed in the TRAM), are introduced into the d~tap~lll pipeline in a time multiplexed fashion.
In the present invention, each stream takes its turn to introduce a group of data items into the d-~lap~lll pipeline. The data sllearns are "i"terleaved" as they pass sequentially down the cl~t.ap~ pipeline and are "de-interleaved" at the d ~ta,-~tl. output, as shown in FlGs. 17, 18 and 33. A group can vary in number, but in this example, they are eight bits.
In accor~lance with the present invention, T1 must not be stalled. If T2 arrives at the point of i"t~rleaving with T1, but the input buffer should not introduce its data into the pipeline bec~l~se this would clash with the T1 al~ean), then stream T2 provides an extra buffering so that T2 does not stall the ` 2154962 data stream, but inslead will buffer up data from its input stream until such a time as it may safely i"terleave with stream T1. This is shown in FlGs. 19 and 33 where the data from stream T1 is being loaded into the first transform in latches 0-7, using signals, "Latch 1(0) 'through' Latch 1(7)". Additionally, data from T2is being loaded in "Latch 2(0) 'through' Latch 2(15)", as shown in FIG.
19, using signals shown in FIG. 33.
The i"terleaving is controlled by "T1 OK2 insert" and "T2 OK2 insert"
signals. Under normal operaliGn, the interleaving will occur when the signals go high. However, if the appropriate amount of data in the latch for T2 has not yet been reached when "T2 OK insert" goes high, then the latch will miss its opportunity and must continue buffering data until the next opportunity to insert data occurs.
In summary, if the above described buffering, in accordance with the present invention, is to occur, co" ,parable "slippage" has to occur at the output of T2. T2slips when it misses its data insertion point and has to continue buffering in the latches shown in FIG. 19. If T2 slipped and did not introduce data into the pipeline there will be a corresponding gap in the T2 stream outputat the d~ta,-~lll output. This gap may be removed or "swallowed up" by use of the extra buffering at the T2 output. This process may be thought of as having a "fixed" T1 - 1 D IDCT llal)sfor", with a variable T2-1 D IDCT, where the data streams are interleaved in a time multiplex fashion such that they may use the same piece of aritl",~etic d~tap~lll pipeline.
In the present invention, "Recovery" takes place when non-data enters T1. It is an opportunity for the T2 buffer to catch up to T1 and the dataslleam.Non-data is a data type that byl,~sses the IDCT and is shown as a data spike in "Latch 2 1q~]" of FIG. 34. This eventually makes its way to T2 input, which allows the T2 buffering to fill up at the output. Recovery is shown in FIG. 33 and FIG. 25 when the "T2 dout" signal and the "out" signal are gapped by a number of cycles. The gap is used as a reference to fix the data stream. It should be noted that the gap in cycles between these two signals is the same as the gap of buffering when the latch for T2 was waiting to insert its data.
Following the TRANSFORM in POSTC 233 part B, the i"terleaved sl,ea", is de-interleaved into "T2 out", as shown in FlGs. 18 and 23. The "T2 out" data st,eal" has slip gaps in the data as described above. The T2 out [143: ~], shown in FIG. 17, enters a 16 to 1 multiplexor block, shown as block "IDDPMUX" in FIG. 17. This multiplexor block will select data from one of 16 positions in the output buffer block, as shown in FIG. 25. This position is s~le~,~3d by the control logic, shown in FIG. 29, which uses the gap by which T2 "buffered-up" at its input. This gap is used as a reference. The output stream, T2DOUT, from the multirlexer block is the "fixed" data stream.
In range tests carried out on an embodiment of the present invention for the IDCT arrangement descriLad above, it was found that all intermediate and final values were kept well within a known range at each point while still r"eeli"g the CCITT standards. Rec~use of this, it was possible to "adjust"
s~ ted values as descril,ed above by small amounts (for example, by forcing certain bKs of selected data words to desired values) without any fear of overflow or unde,llow in the aritl""etic c~lGIJl-~liolls.
The Illetllod and system, in accor~lance with the present invention, can be varied in numerous ways. For example, the structures used to resolve additions or multiplications may be altered using any known technology. Thus, it is possible to use resolving adders of subl,dctors where the preferred embodiment uses car~-save devices with separate resolving adders. Also, the preferred embodiment of the present invention uses down-scaling at various points to ensure that all values remain within their acceptable ranges. Down-scaling is not necess~y, however, bec~use other precautions may be taken to avoid over~ow or underflow.
In one e"lboJi,nent of the present invention, certain bits of various data words were manipulated to reduce the required word width within the system.
However, the various inte"nediale values may, of course, be passed without bit manipulation. Furthe""or~, although only data words were bit-manipulated in the illustrated example of the pr~sel)l invention, it is also possible to manipu-late the bits of constant coerficients as well and evaluate the results under the CCITT standard. If a comparison of the results showed that it would be advantas~eous to force a particular bit to a given value, in some cases, on might then be able to i"cr~ase the number of "zeros" in the binary representa-tion of these coerricienls in order to decr~ase further the silicon area required to i",ple."ent the col,espol)Jing multiplier. Once again, bit manipulation is not necess~y.
In summary of the above ~spects of the present invention, the following is disclosed: an apparal.ls for t~dnsfc,r"ling data having a first latch defining a first data stream source and a second latch defining a second data stream source. The first and second latches are in communicaliol- with a single a,itl"netic unit. The a,ill,r"~tic unit communi~ ~tes data to a transpose RAM, the l,al-spose RAM transposes the data and comm~"icdles it to the secol~d 21~4962 latch. The second latch is ~djustahle and can be varied in size to accol"l"o-date variable rates of data being received and transmitted. The second latch and first latch communicate 1st and 2nd data stream to the arithmetic unit sequentially however the sequential communication of the second latch does 5 not interrupt the communication from the first latch. In this manner common a,itl"nelic unit is used for a first and second data stream. Furthemmore a process for transforming data using a co",r"on arithmetic unit having the following steps is described. First loading the data into a first latch and uponreaching a pledt:rined number of cycles trans",itli"g the data to an arithmetic 10 unit and loadi"g a first marker bit into a control shift r~SJister. Next loading data into a secGnd latch the second latch is adjustable and can be varied in size to accolr,r"odate variable rate of data being received and transn,itled at Jirrer~nt rates. The next step is to transmit the data in the second latch to the aritl""~lic unit when the first control shift register reaches a predetermined 15 state and the second latch is filled with a predetermined amount of data. Next preventing l,ansr"issiGn of data from the second latch if the second latch is not filled with a predetermined amount of data and then recovering the second latch when the first latch is receiving non data.

- ~154962 Detailed Description of Invention for Time Synchronization In MPEG-2 video and audio data is synchronized using info""alion carried in the MPEG-2 systems stream. In this regard there are essentially two types of inron~aliGI) that deal with synchronization; clock references and 5 time star"ps. Clock referel)ces are used to inform the decoder what number is used to represent the time "now". This is used to initialize a counter that is incre" ,enled at regular intervals so that the decoder always knows what the current time is.
Time star"ps are carried in some of the streams of data that are used to 10 make up the proyrarr,r"e (typically video and audio). In the case of video a time stamp is ~ssosi ~ted with a picture and tells the decoder at what "time"
(defined by the counter that was initi~ ed by the clock reference) a picture should be displayed.
In MPEG multiplexed into the system stream are a series of clock 15 lefenj,)ces. These clock references define the "system time". There are two types of clock refer~nce; Plo5~ra"~ Clock References (PCRs) and System Clock References (SCRs). In the present invention, the distinction between PCRs and SCRs is not relevant since each of the clock references are used in the same manner by the decoder. PCRs and SCRs have timing infoll"dlion to a 20 resolution of 90 kHz with a further field e~lendi"g the resolution to 27 MHz (or 1/27 X 10e6 in seconds). Clock ,eferences are included in the data stream fairly often in order that "system time" may be reiniti~ ed after a random ~ccess or channel change. Accordingly it is important to appreciate that li,nesta,nps refer to a hy~,otl,~tical model of a decoder that can decode pictures 25 il,ata,ltly. As will be app,~ci~l.ed by one of ordinary skill in the art any real decoder cannot do this and must take steps to modify the theoretical time in which pictures should be displayed. Furth~r"~ore time alalll~,s and the clock rererences are used to determine display time and errors in display time. This modiri~lion depends upon the details of the architecture of the particular 30 decoder. Clearly any delay introduced by the video decoder must be matched by an equivalent delay in the audio decoder.
When decodi"~ MPEG discGI Itil ,uities in the concept of "system time"
may occur. For instance in an edited bitallealll each edit point will have discontinuous time. A similar situ~tion occurs at channel change. It will be 35 appr~ci '.ecl that care must be taken when using time stal"ps bec~use using atime stamp that was encoded in one time regime with ,t:spect to a "system `- 2154962 time" defined by a clock r~ference from another regime will clearly lead to incorrect results.
Figure 39 shows the demultiplexing of the MPEG systems stream into elementary sl,~aios 250. Each elementary stream will typically carries either 5 video or audio data although, in general, any form of data may be transported.Each ele",enlary stream is divided into a series of access units. In the case ofvideo, the aGcess unit is a picture. In the case of audio, it is a fixed number of samples of audio data.
Also multiplexed into the systems stream are a series of clock refer-10 ences. These clock ,~rerences define the "system time".
In accordance with the present invention, associated with each elemen-tary sl,ea", is a series of time stamps 251. The time stal"ps specify the "system time" at which the next access unit for the respective elementary sl,eal" is to be presel,led. These time stamps are referred to as presentation 15 time stamps, nPTS".
In the case of video data, a second type of time stamp is also defined is ,~fer,ed to as a deco~e time stamp, "DTS". Since the DTS is only present when a PTS is also present and there is a simple relalionship between them, the detailed d;rrerel)ces between these two types of limestamps can be ignored 20 since PTS/DTS differences have no bearing on the present invention.
The decode time slarl,ps (DTS) define the time at which an ~ccess unit (picture in the case of video) is to be decoded. The presentation time slar"ps (PTS) define the time at which an ~Gcess unit is to be presented (displayed).
However, the timing model used is a hypothetical model in which the decoder 25 is infinitely fast. In this case, the DTS and PTS would be identical to one another.
However, in MPEG video decoding, some of the pictures are reordered.
Ther~ror~, after decoding, the pictures are held in storage for a time period, e.g., several frame times, before they are displayed. During this time period, 30 other pictures that are decodecl s~ ~hsequent to the picture in question are displayed. Consequently, for these reordered pictures there is a difference between the DTS and PTS.
In accordance with the present invention, it will be appreciated that to properly synchronize time, it is "ecess~y to be consistent in the use of time 35 sta",ps. In one preferred embodiment, the time sy"chronizing circuitry is placed at a point in the decoding piipeline when the pictures occur in their clecoded order. AccGrdi"gly, this embodiment uses the DTS.

21~4962 Nevertheless, the circuitry could equally be moved to a point in the decGdi. ~y pipeline that occurs after the pictures are reordered and, ther~fore,the pictures would reach the synchro"i~i"g circuitry in their display order.
Hence, as will be appreci~tecl by one of ordinary skill in the art, PTS would beused in this embodiment.
In the pr~fer,ed embodiments of the present invention, the i"rur,.,dlion derived from the li,nestar"ps is transported through the various circuits by means of tokens. Tokens consist of a series of one or more words of informa-tion. The first word of the token contains a code which ider,li~ies the type of token and, hence, the type of inrom ~dliGn carried by that token. Associaled with each word of the token is an extension bit which is set to one to indicate that there are more words in the current token. Therefore, the last word of a token is indicated by the extension bit being zero. In the present invention, the code in the first word indicating the type of token may be of a variable number of bits so that some codes use a small number of bits (allowing the lemainder of the bits in the first word to be used to represent other information) while other codes use a larger number of bits.
Tokens may be cha,acteri~ed as being either control or DATA tokens.
For example, at the i"te, race between the system decoder and the video decoder, there are two types of information: (1) the coded video data and (2) the synchroni~dliol, time derived from the time stamp information. The coded video data is viewed as data and is carried in a DATA token (e.g., the token called DATA) while the sy"chr~l,ization time is viewed as control information and is carried in a control token (called SYNC_TIME). Additional control tokens may also be used from time to time in the present invention. For example, a FLUSH token that behaves in a manner similar to a reset signal may be required to initialize the video decoding circuitry before allel.l~ting to restart decod;. ,9 bec~use of an error.
In accor~ance with the present invention, it is an object of one preferred embodiment to time sy"chroi ,i~e two circuits and, more particularly, to time synchronize two circuits without directly communicdling system time from the first to the second circuit. In accor~lance with the invention, time sy"chr~nka-tion of two circuits is accol"plished without passing system time directly to the second circuit by providing synchror,i~ed time counters in each circuit.
The ~,r~sent invention also enables the system to time synchronize two circuits without communicating system time from the first to the second circuit by providing an ele..,enlary stream time counter in each circuit.

Accordingly, another object of the present invention is to time synchro-nize two circuits and to determine the ptesenlaliG,) time error, if any, of the object being pr~sented by using time stamp inrc,r")dlion, system time, and elementary stream time from the first circuit to generate synchroni~alion time p~ssed to the second circuit and co",pared to a copy of elementary stream time in the second circuit which is synchronized with the cle. "entary stream time in the first circuit. The system of the present invention can time synchro-nize a system decoder and a video decoder without directly communicdling system time from the system decoder to the video decoder, without passing system time directly to the video decoder by providing synchronized time counters in each circuit and without communicating system time from the system decoder to the video decoder by providing a video counter in each circuit.
The invention also enal)les the system to time synchronize a system decoder and a video decoder and to determine the display time error, if any, of the picture being displayed by using video time stamp infc,r",dlion, system time, and video decoding time from the system decoder to generate synchroni-zation time which is then p~ssed to the video decoder and compared to a copy of video decoding time in the video decoder which is synchronized with the video decoding time in the system decoder.
In accGr~lance with the present invention, information derived from the timesta."ps can be transpo,led through the system using a control token as previously described.
Figure 40 shows a first prefer,ed embodiment implementing elementary slream li",eslal"p r~alla9ament, in accGrdal1ce with the present invention. The clock r~fer~nces 253, which represent system time, are decoded by the system demultiplexer 254 and placed initially, and then as needed, into a time counter 255 within the system decoder 256, and are incr~l"ented at 90 kHz. A second copy of the clock refer~nce 253 is simultaneously loaded into the time counter 255 that is inside the ele."el1tary stream decoder 257, inc,~mented also at 90 kHz, and synchroni,ed to the time counter 255 in the system decoder 256.
The time stamps 251, in accordance with the present invention, flow from the system demux 254 through the elementary stream buffer 260 so that they are delayed by the same amount as the incon,i"g data. The time stalnps 251 may also have a co"t:ction added to compensate for the non-zero decode time of the ele.nentaly stream decoder 257. The corrected time stal"ps 251 are then compared with the copy of the time stored the time counter 258 inside 21~4962 the ele,nentary slledlll decoder 257 to determine whether the decoded infor-mation is presented too early or too late.
The above embodiment is better than merely passing system time directly to the ele."enta,y stream decoder 257 from the time counter 255 in the 5 system decoder 256 bec~use the counter in the system decoder changes 90,000 times a second. Ther~for~, system time would, in all essence, need to be continually passed to the elementary stream decoder 257. Passing system time continually would require clediG~tecl pins or the like. By using a time counter 255 lo~ted in the system decoder 256 and a time counter 256 located 10 in the elel"e"ldly stream decoder 252, system time can be passed in the form of clock r~ferences 253 a few times a second.
Another embodi",enl is shown in Figure 41. The embodiment shown in - Figure 41 avoids the need for the clock ~rerences 253 to be p~ssed to the ele,nentdl y stream decoder 257. This is achieved by using a second counter"
es_time" 262, containing illrol",dtiol- on ele,-,el,ldly stream time, which is maintaii~ed in both the system decoder 256and the elementary stream decoder 257. The two es_time counters 262 and 263 are reset at power on, and at other times such as channel change, and then they free run from there on.
Since this embodiment depends on the two es_time counters 262 and 263 staying in step, it will be appreci ~lecl that it is necess~ry to take measures to ensure the es_time counters do not get out of step. One way to ensure the es_time counters 262 and 263 stay in step is to use carry out of the es_time counter in the system decoder to reset the es_time counter in the ele. "enlary slledlll decoder 257 as shown in Figure 41.
As further shown in Figure 41, the clock references 253, which repre-sent system time, are decoded by the system demultiplexer 254 and placed into a time counter 255 within the system decoder 256 and incr~n,e"ted at 90 kHz. The es_time counter 262 in the system decoder 256 of the present invention and the es_time counter 263 in the elementary stream decoder 257 of the present invention are synchronized with each other and incremented at 90 kHz. Ele,nentdl y stream time star"ps are also decoded by the system demultiplexer 254. Accordi, l~ly, a synchroni~dliGI ~ value X is computed using the elementary stream li" ,estal"p, the system time contained in the time counter and the elel "el ,ldl y stream time contained in the es_time counter 262contained in the system decoder 256 according to the equations 3-1.
The following set of equations 3-1 (a - d) is illustrative of one method in accordance with the present invention, for time synchroni~dlion which avoids passing the clock lef~r~nces 253 to the elementary stream decoder 257.
Equation 3-1 (a) is the equation required for time synchronization. Since it is undesi,al~le to pass system time directly to the elementary stream decoder circuit 257, as shown in Figure 41, a s~"chror,i,dlion time representdlion X is 5 generdted, using Equation 3-1 (b-d), by the system decoder 256 and this value is p~ssed to the elementary stream decoder. Sy"chro,)ization time X is then compared to the elementary stream time contained within the es_time counter 263 located within the elen,e"laly stream decoder 257. Hence, the compared result is used to determine whether the decoded information is presented too 10 early or too late and then is further used in time synchronizing the system.

Equations 3-1:
a) Time Synchronization = (Ele",el1tary stream timestamp - system time) b) Time Sy"cl,r~"i~dli~n = (X - ele",e,)lary stream time) c) (X - elementary stream time) = (elementary stream timestamp - system time) d) X = (el~n,entd,y stream li")estamp - system time + elementary stream time) In the present invention, the synchrol ,i,dlion time, X, may have a correction added to compensate for the non-zero decode time of the elemel ,tary stream 10 decoder 257. The cGr,ected synch(oni~dlion time is then compared with the el~."el)tdly stream time contained in the es_time counter 263 loc~ted inside the el~.llentdly stream decoder 257 to determine whether the decoded infor-mation is presel1ted too early or too late and is further used to time synchronize the system. Note, the time c~r,~:ti~n could be suL,lla~;ted from elementary 15 sl,~d,n time contained in the es_time counter 263 loc~ted inside the elemen-tary stream decoder 257 i"stead of added to synchronization time X for the same result. The above embodiment is an example of a solution for generat-ing synchronization time X and determining whether the i"rur"~dliGn is pre-sented early or late. It will be apparent to those skilled in the art that there are 20 many other equivalent solutions for accG,nplishing the above.
For exal"ple, Figure 42 shows an aller"dli~/e method for determining the syllchrolli~dliGI) time, X, in acc~r~lance with the present invention. In this arrangement, the system decoder 256 does not maintain an elel "el ,tary stream time. Instead it r~cords, in the r~gisler initial_time 265, the value of system 25 time at the instant that the el~.nenlary stream time counter, es_time 263, loc~ted in the 01~.nenld,y stream decoder 257 is reset to zero. The value in es_time 263 can be computed by the system decoder 256 because it will be the difrerence between the current system time and the value recorded in initial_time.
30 The following equations 3-2 (a-c) is illustrative of this alternative method for time synchrol,i~dliGn. Equation 3-2 (a) is the equation represellti,.g the valueof the el~,nenld,y stream time stored in es_time 263 loc~ted in the elel"el,ldlysll~dln decoder 257. This is substitl~ted into equation 3-1 (d) to give equation3-2 (b) which is simpl;fied to derive equation 3-2 (c) providing the synchroniza-35 tion time, X, as a function of the system time and the value stored in theinitial_time register 265.
Equations 3-2:

a) ele.llellta~ stream time = system time - initial_time b) X = (ele."enlar~ stream li,nestan,p - system time + [system time -initial_time]) c) X = (elementary stream timeslal"p - initial_time) Two solutions for deriving the synchronization time X in accordance with the present invention have been illustrated. However it will be apparent to those skilled in the art that there are many other equivalent solutions.
Figure 43 shows another embodi~ent of the present invention imple-lllelllillg video li"~estari"~ managel"ent. The clock references 253 which represent system time are decoded by the system demultiplexer 254 and placed initially and then as needed into a time counter 255 within the system decoder 256 and are i"cre"~ented at 90 kHz. A second copy of the clock references 253 are simultaneously loaded into the time counter 258 that is inside the video decoder 270 and incremented at 90 kHz and synchronized to the time counter 255 in the system decoder 256.
The video time stal"ps flow from the system demux 254 through the video decocling buffer 271 so that they are delayed by the same amount as the incGI"i"y video data. The video time stari,ps may have a correction added to compensate for the non-zero decode time of the video decoder 270. The cGr,ected video time star"ps are than compared with the copy of the time in the time counter 258 inside the video decoder 270 to determine whether the decoded picture is displayed too early or too late.
The embodiment shown in Figure 43 is an improvement over the process of merely passi"g system time directly to the video decoder from the time counter in the system decoder because the counter in the system decoder changes 90 000 times a second. Therefor~ system time would in all essence need to be continually p~ssed to the video decoder. rassing system time continually would require dedicated pins or the like. By using a time counter located in the system decoder and a time counter loc~ted in the video decoder system time can be passed in the form of clock references a few times a second.
Refer,ing now to Figure 44 the clock references which represent system time are decoded by the system demultiplexer 254 and placed into a time counter 255 within the system clecoder 256 and incremented at 90 kHz.
The vid_time counter 272 in the system decoder 256 and the vid_time counter 273 in the video decoder 270 are synchronized with each other and incre-menled at 90 kHz. Video time sta,nps are also decoded by the system .

demultiplexer 254. ACCGrd;IIgIY~ a synch~nkdli~n value X is computed using a video li"~estar"p, the system time contained in the time counter 273 and the video decoding time contained in the vid_time counter 272 contained in the system decoder 256 accGrdi. ,g to the equations 3-3.
The following set of equations 3-3 (a - d) is illustrative of one method in accGrddnce with the pr~sent invention, for time synchronization which avoids passing the clock refer~nce 253 to the video decoder 270. Equation 3-3(a) is the equation required for time sy"chronization. Since it is undesirable to pass system time directly to the video decoder circuit 270 as shown in Figure 44, a 10 synchro~ dliGI ~ time r~presentation X is generated, using Equation 3-3 (b - d), by the system decoder 256 and p~ssed to the video decoder 270. Synchroni-zation time, X, is then compared to the video decoding time cGntdined within the vid_time counter 273 located within the video decoder 270. The compared result is used to determine whether the decoded picture is displayed too early 15 or too late and then further used in time syncl ,ro"i~i,)g the system.

`~ .` 215~962 Equations 3-3:
a) Time Synchronization = (Video timestamp - system time) b) Time Synchronization = (X - video clecodi"g time) c) (X - video decoding time) = (video li",estamp - system time) 5 d) X = (video timesta,np - system time + video decoding time) In the present invention, the synchronization time, X, may have a c~r,~..1i~n added to compensate for the non-zero decode time of the video decoder. The cGr,æted synchro~ dtiGn time is then compared with the video decGdi, lg time c~nlained in the vid_time counter 273 loc~ted inside the video decoder 270 to determine whether the decoded picture is displayed too early or too late and is also used to time synchronize the system. Note, the time cGr,e-,tion can be subtracted from the video decoding time contained in the vid_time counter 273 loc~l~d inside the video decoder 270 instead of added to synchronization time X for the same result. The above embodiment of the present invention is another example of a solution for generating synchroniza-tion time X and determining whether the picture is displayed early or late.
However, it will be apparent to those skilled in the art that there are many other equivalent solutions for accomplishing the above.
Another nice feature, in accGrdance with the present invention, is that there is no need to deal with the fu1133 bit time stamp number or 42 bit clock refer~nce number. The present invention restricts the counters to 16 bits to allow 16 bit handling on the video decoder 270. At first glance, it would appearthat 16 bits cannot represent a sur~icient number range at a resolution of 90 kHz (only 2/3 second to be used). However, there is no need for such high p~ecisiGn bec~use the time control on the video decoder 270 is only accurate to a field time (since the video timing generator VTG free-runs or is gen-lockedto sGIlletlling that has notl,in$~ to do with the MPEG stream being decoded) and, therefore, it is not related to li",esta",ps or presentation time in any way.
As shown in Figure 44 and Figure 45, the sy"chronization time X and the vid_time counter 273 within the video decoder 270 use only sixteen bits.
This is made possible by two factors. First, the difference between system time and the li",esla""~ (used to derive the synch~oni~dli~l) time; see Equation3-3) should always be small, thus allowing the more significant bits to be discarded. Second, it is only possible to control the presentaliGn of video to the nearest r,dl"etin,e. Therefore, the less significant bits are not required and are discar~led by s hini"9 right by four bits.

`- ` 2154962 Thus, the sixteen bits of time i,,rurrndlion used in the present invention are able to deal with timing errors of up to about 11.5 seconds with an accu-racy of about 180 ~s (about 1 % of a field time). A PAL or SECAM European 625 line TV system is, thus, 112.5 ticks of the 5625 Hz clock; a NTSC 525 line TV system is 93.84 ticks. Hence, using 16 bits allows timing c-~'c~ tions with an accuracy of about 1 % of a field time.
Figure 46 shows the pr~fer,ed process, in accordance with the present invention, of the moving the time stamp through the hardware. The preferred n~t:ll,od for communicating inrc,r",dliG" in this hardware is Tokens, but it will be appreciated that alternative methods may also be employed. The hardware is divided into two modules. The first module is added just after the Start Code Detector 201. This module is respGnsil,le for generali"y a token, SYNC_TIME
- CGI It'dil ~ing the sy, ,chruni~dlion time X, as discussed above, and this occurs just before an ~ssoci~ted PICTURE_START token. In the MPEG systems stream, the time stamp is carried in a packet header and refers to the first picture in the packet of data. Since the packets do not line up with the video data, there will, in general, be the end of the previous picture before the start of the picture to which the time stamp refers.
The synchronization time inf~m~dliGn may be supplied to the present invention either via a microprocessor inte, rdce or by using a Token. In either case, the synchr~ui~dtion time date (16 bits) is stored in the synchronization time register (divided into two parts to allow ~ccess to each byte individually), as further detailed in Table 12.

. ~ 21a49 62 Register Name Size/Dir Reset State Description ts_low 8/rw - The lower eight bits of the s~,,chruni~ation time value.
The ts_low register is slaved so that new values may be written into this register without affect-ing the value previously written (that will become part of a SYNC_TIME token).
Writes to ts_low register affect the master register whilst reads read-back the slave register.
Until a master-to-slave transfer has been effected using ts_valid the value written into ts_iow cannot be read back.
ts_high 8/rw - The upper eight bits of the s~"ch,uhi~dtion time value.
Slaved in the same way as ts_low.
ts_valid Urw O This bit controls the master-slave transfer of ts_iow and ts_high.
When values have been written into ts_low and ts_high the "1 up,ucessor should write the valu, one into this bit. It should then poll the bit unit it reads back the value one. At this point the values written into ts_low and ts_high will have been lldnaF~r,~d into the slave ,ey;_~ra (and can be read back) and ts_waiting will be set to one.
The "1 oprocessor should then write the value zero in prt:pa, dlion for the next access.

- '~15~962 ss Register Name Size/Dir Reset State Description ts-waiting Uro O When set to zero the ~eg;sl~r~ ts_low and ts_high do not contain valid sy,,~hruni~dliùn time i"rc"",alion.
When set to one the ,~y;~,t~,r:. ts_low and ts_high contain valid syllcl"uhi~dtion time illr~ dlion.
A SYNC_TIME token will be gen~,dl~d before the ne~tt PICTURE_START token and ts_waiting will then be-come zero.
This bit should be polled to ensure that it is zero before writing a one into ts_valid to ensure that the previous sy"~l~r~ni~dlion time value has been used before it is oven~ ~L i~l~n by the master-to-slave transfer.
. .
Table 12 Micropra~es~r registers for handling synchronization time In the present invention, a flag, ts_waiting, is set to indicate the fact that valid synchronization time information is in the timestamp register. If the data was supplied using the SYNC_TIME token, then that token is removed from the stream of tokens.
When a PICTURE_START token is encountered, the flag that indicates the status of the synchronization time register is examined. If the flag is not set, then no action is taken and the PICTURE_START token and all subse-quent data is unaffected. If, however, the flag is set, indicating that valid synchronization time inror",dlio" is available in the register, then a SYNC_TIME token is generated and placed in the data stream before th PICTURE _START token. The flag is then cleared and the synchronization time register is made available for the next time-stamp that occurs.
The second module as shown in Figure 46, consists of a prescaler clocked at 27 MHz and a vid_time counter clocked by the presc~ler 278 which are associated with the microprogrammable state machine, (MSM) 218.
There is a presc~ler 278 that divides the clock by 4800, as shown in Figure 44 and Figure 46. In other words, 4800 is 300 (27 MHz / 90 kHz) times 16. The 4804.8 option shown in Figure 45 and Figure 46 is discussed below.
In the NTSC color television, the frame rate is not 30 Hz but is, in fact, approxil"ately 29.94 Hz (precisely 30000/1001 Hz). [Before the advent of color ~ ` 2154962 television 30 Hz precisely was used.] There are precisely 1716, 27 MHz clock periods per NTSC line time (line time is 1/525 of frame time).
The American television standards body has e~c~ ressed an interest in returning to 30 Hz in the future (or more probably 60 Hz for HDT\/). As a result5 MPEG supports a frame rate of 30 Hz p,ecisely. However, since it is not possible to generate a stable 30 Hz television signal from a 27 MHz clock (there being 1714.29...cycles per line) it is difficult to generate a 30 Hz raster in the MPEG framework.
One possible solution is to "bend" the clock rate at the decoder so that 10i"stead of producing a 27 MHz clock, a 27.027 MHz clock is generated. This clock is generated using the MPEG clock lefere"ces with a divider of 300.3 (rather than 300) to yield the 90 kHz clock. This 27.027 MHz clock when clocking the identical video timing circuitry that provides a 29.94 Hz frame rate from 27 MHz will give a ~.reci~e 30 Hz rate.
15In the framework of the present invention, the 90 kHz is presc~led by a further factor of 16. Accordingly, division of the 27.027 MHz clock by 300.3x16=4804.8.
The Vid_time counter 273 (discussed above) contains the video decod-ing time and is incremented each time that the prescaler reaches its terminal 20 count. The vid_time counter 273 is reset by the reset-time pin.
The presc~ler and vid_time counter of the present invention can be implemented with fully clocked feed-back flip-flops which are much more resistant to a-pa, licll~ corruption than the resistivc fccdback or weak-feedb~ck latches used elsewhere. Using clocked feedback flip-flops for time counters 25 will help ensure that the time counter in the video decoder stays in step with the time counter in the system decoder.
Figure 47 illusl,ates the process the MSM 218 performs when it re-ceives the SYNC_TIME token. The MSM 218 is able to read the current time indicated by the video time counter and to then compare it with the value 30 supplied by the video SYNC_TIME token. It can, therefore, deter",i.,e whetherit is early or late, as co",pared to the time at which it should be decoding thepictures.
In the present invention, a 16 bit signed timestamp correction is added to the synchroni,dtion time X (discussed above) that was carried by the video 35 SYNC_TIME token. This correction is reset to zero by the MSM 218 at chip-reset, and if no action is taken, the synchronization time remains be unaltered.The controlling microprocessor can always write value into ts_correction to ~ 215~962 modify the synchr~ni~aliGn time and, ll ,erefore, compensate for differential delays through the video and audio decoders.
The current video decoding time cGntaL ~ed in the vid_time counter 273 is subtracted from the con~cted synchronization time. The sign of value gives 5 the direction of the error (and detem,i"es the error code, if any, generaled by the MSM 218 ). The absolute value of the difference is then taken and the result is cGI"par~d to a ll,r~sh~ld value to deter",ine whether the timing error is within acceptable limits. Since, at present, the video timing can only be controlled to an accuracy of plus or minus a frame time from the nominal time 10 (because the VrG 333 free-runs) this threshold is set at one frame time.
If the error exceeds a frame-time, then some correction must be made.
The MSM 218 of the present invention can correct the situation itself if the decoding is too early since the MSM can simply delay the decoding until the appropriate time. However, if the decoding is later than the intended time, then15 time correction is more difficult bec~use it is not possil,lo to discard pictures reliably at the output of the coded data buffer. Essentially, the decoding of the sequence is broken and the most reliable way to correct the situation is to restart the decoding process in a manner similar to random-~ccess or channel change. In order to facilitate this ~.rocess, the control register of the MSM 218 20 may be prog,all,llled to discar~J all data until a suitable start code or FLUSH
token is encountered. In addition, the error"ERR_TOO_EARLY" is not generated during start-up, irrespective of the setting of disable_too_early, bec~use at start up, the first picture is expected to be early.
Table 13 iS illustrative of how the MSM 218 registers work and details 25 some of the actions and error ",ess~ges inrom~atiGn p!aced in the registers can generate.

Re lister Name SizelDir Reset State Description ts_cG~e~,tiGn 16/rw zero Cor,t:~tion added to s~,lcl,~fii~dtion time before it is used.
3~frame_time 16/lw 226 or 188 Repr~sents the tolerance on the timing of decoding pictures. Reset state determined by the PAUNTSC pin.
vid_time 16/ro zero Resetbyeither reset orreset_time. Thecurrent value of video decoding time.

~_ I 21a4962 Register Name Size/Dir Reset State Description manual_startup Urw zero When set to one the start-up is to be per-formed manually using decode_disable.
In this case SEQUENCE_END and FLUSH tokens at the MSM cause decode_disable to be set to one.
decode_disable 1/rw zero When set to zero the decoding pn,ceeds normally.
At the start of each picture the MSM checks the status of decode_disable and will not proceed if it is set to one.
Note that if manual start-up is to be pe,f~,l ",ed (i.e. without the time-stamp " ,anager"ent hardware) then this bit should be set to one at the same time as manual_startup is set to one.
disable_too_early 1/rw zero When set to one the error "ERR_TOO_EARLY" indicaUng that the decoding is too early is su~,p,~ssed and the MSM simply waits to correct the situation.
NTSC_30 1/rw zero When set to one the pr~s~ r divides by 4804.8 rather than 4800. Set automatically when decoding 30 Hz frame rates.
~discard_if_late 1/nH zero This has no effect unless an "ERR_TOO_LATE" is gene,dtt:d (or would - be generdted if errors were not masked out). If it is set to one then data is dis-carded until the condition indicated by discard_until.

_.

R~ t~rname Size/Dir ResetState Description discard_until 2/rw zero Indicate the condition which causes time-stamp t, iyger~d discarding to be terminated.

1 - SEQUENCE_START
2 - GROUP_START

Note 1 - that discarding one picture may i"""e~i~t~ly be un~one if that picture is a field picture by the generation of a dummy field to preserve the all~:",ali"g top/bottom field structure. As a result if discard_until is set to "Next Picture" but the dummy field would be generdte:d one further picture is car ~ed.

Table 13 Timestamp MSM l~gist~rs As a result of the sy"chr~nization time handling of the present invention, it is possible that one of two errors will be generated.
ERR_TOO_EARLY is generdted if the decoding is taking place earlier than the time indicated by the time-stamp. ERR_TOO_EARLY may be suppressed, but ERR_TOO_LATE will always be generdted unless all errors are masked out.
In summary, the present invention includes an appdrdlUS for sy"chronizing time having, a li"~eslari,p for determining pr~sentdtiGn time, a clock reference for initializing system time in a first circuit, a first time counter in communication with the clock reference for keeping system time in a first circuit and a second timecounter initialized by the clock reference in a secGnd circuit sy"chr~nized with the first time counter, for keeping a local copy of the system time and for determining the presentation timing error between the local copy of system time and system time by CGlllpdlill!a the li"~estal"p to the second time counter. It further includes an apparatus for sy"chrol-i~i"g a system decoder and a video decoder using a li",estamp for determining display time, a clock reference for initializing system time in the system decoder, a first time counter in communication with the clockreference for keeping system time in the system decoder and a second time counter initialized by the clock rerere"ce in the video decoder synchronized with 21a4962 the first time counter for keeping a local copy of system time and for deter" ,ini"g the display timing error between the local copy of system time and system time by comparing the li",esta,np to the second time counter. A still another e"~bodiment includes an apparatus for synchronizing a first circuit and a second5 circuit using a clock reference for initializing system time in the first circuit a first circuit having a time counter in communication with the clock r~ference for keeping system time a first elementary stream time counter in the first circuit for providing ele,-,enla,y stream time. The first circuit is adapted to receive a time stamp and the first circuit generates sy"chro~ dlion time by adding elementary 10 stream time to the time stamp and sublldctil1g system time. The second circuit is adapted to r~:ceivc sy~ Icl ,roni~dtion time from the first circuit and has a second elementary sl,eam time counter in synchronization with the first elementary stream time counter for providing a local copy of the elementary stream time andfor detel..li.lillg a timing error between the system time and the time stamp by15 comparing sy"chro"i~alion time to the local copy of elementary stream time. In this way the clock reference signal does not have to be passed directly to the secol)d circuit in order to detemmine the timing error. In another embodiment anapparatus for sy"chr~ni~ing a first circuit and a second circuit has a clock reference for initializing system time in the first circuit. The first circuit has a time 20 counter in communication with the clock ,eferel)ce for keeping system time and a first video time counter for providing video decoding time. The first circuit is a.Japted to leceivc a video time stamp and generates synchronization time by adding video decoding time to the video time stamp and subtracting system time.
The second circuit is adapted to ~ceive synch~ul,i~dlion time from the first circuit 25 and has a second video time counter in synchronization with the first video time counter for providing a local copy of video decoding time and for deterl"ining atiming error l,et~ocn system time and the video time stamp by comparing s~" ,chr~nization time to the local copy of video decoding time. Accordingly theclock reference signal does not have to be p~ssed directly to the second circuit30 in order to determine the timing error. The present invention also includes a,.,etl,od for providing timing infc.m)dliGn by providing a video data stream having a time stamp carried in packet header wherein the time stamp refers to the firstpicture in the packet of data. In the next step a register is provided having a flag used to indicate valid time stamp information which is taken from the packet 35 header and placed into the register. Next the li",estarnp is removed from thevideo data stream and placed in the ~g;ster. Next the method encounters a picture start and sl ~hse~uently examines the status of the register to determine if 21~4962 valid time stamp i"f~"ndlion is contained in the register by checking the flag status. A time stamp is generated in response to the picture start if the flag i"di~les valid time stamp inror",dli~n is contai"ed in the register and then thetimestal"p is inserted back into the data stream. Another embodiment includes 5 an appa,dtus desc,ibed above wherein the elementary stream time counters are ,~sl,i-,ted to 16 bits. Likewise, there is an apparatus as desclibed above, wherein the second eler"e"laly stream time counter located in the cle,nel1tary stream decoder is re~tli~ d to 16 bits. Fu,ll,ell,1ore, there is an apparatus as described above wherein the synchroni~dliGn time is lesl, icted to 16 bits for controlling the 10 elementary stream decode. The present invention also has a process for decoding video and for determining display time errors against a ll ,reshold value.
It then parses video data into tokens for further processi"g, determining if a time stamp token is in~lic~ted~ cGnlparing the time stamp token to a video time, and generates a co",par~d value to determine an indicative of timing error. Next, it15 determines whether the compared value, when compared against a threshold value, is within acceplable parameters when a timing error is indic~ted and indicates when the compared value is outside acceptable parameters. An altemative embodiment includes an appar~t-Js for using a system decoder and a video decoder. The system decoder is adapted to accept MPEG system slleallls 20 and demultiplexing video data and the video time stamp from the stream. The system decoder has a first time counter ,~pr~sentative of system time. The videodecoder accepts the video data and the video time stamp, and has a second time counter in sy, Ichl ~ni~dtiGn with the first time counter. The video decoder also has a video dec~der buffer for accepti"g the video data at a sul ~st~nlially constant rate 25 and outputting the video data at a varying rate and for passing a video time stamp. The video decoder while decoding a picture from the video data also compares the video time stamp for the decoded picture with the second time counter to determine the appropriate display time. There is also a method for determining a time error between a first circuit and a second circuit by providing 30 the first circuit with a system time (SY), a time stamp (TS), and an elementary sl,eain time (ET), obldining synchronization time (X) by using the elementary stream time(ET), the time stamp (TS) and the system time (SY), in accordance with the equation; X=ET + TS-SY, providing synchronization time (X) to the second circuit and generdti"g a sy"~;l ,rol,i~ed elementary stream time (ET2) and 35 obtaining a time error by using synchr~nk,ad time (X) and in accordance with the equation ET2-X; hence, the first circuit can be time synchr~ni~ed with the secGnd circuit without passi"g system time to the second circuit. Another method for `_ ` 2154962 determining a time error between a first circuit and a second circuit has the following steps: providing the first circuit with a time stamp (TS), and an initial time(lT), obtaining sy"~;hr~"i~dtion time (X) by using the time stamp (TS) and the initial time (IT), in accor~ance with the eqU~tion X=TS-I, providing sy,Icl,r. nization time (X) to the second circuit and ge"eraling a synchronized elementary stream time (ET) and obtaining a time error by using synchronized time (X) and in accordance with the equation ET-X. In this way, the first circuit can be time synchronized with the seco"cJ circuit without passil Ig system time to the second circuit. Still anotller method for determining a time error between a first circuit and a second circuit includes the following steps: providing the first circuit with a system time (SY), a video time stamp (VTS), and a video decoding time (VT), obtaining synchronization time (X) by using the video decoding time (VT), the video time stamp (VTS) and the system time (SY), in accordance with the equation; X=VT I vrs-sY, providing syl ~chro, li'dliOI ~ time (X) to the second circuit and generating a video decoding time (VT2) in the second circuit which is synchronized to the video decoding time (\/T) in the first circuit, and obtaining a time error by using synchronized time (X) and in accordance with the equation VT2-X. Accordingly, the first circuit can be time synchronized with the second circuit without passing system time to the second circuit.
Detailed Description of the Invention for Asynchronous Swing Buffering For asyllchronous swing buffering, in accor lance with the present invention, two buffers are operated asynchronously; one is written while the other is read.
Accordirl!Jly, this allows for a data stream having a first rate of through-put to be resynchronized to another rate, while still maintaining a desired rate. In the invention, the write control and read control both have state indicators for communicating which buffer they are using and whether the conllols are waiting for ~ccess or are, in fact, accessing that buffer. Each side communicates to theother side a single bit to indicate which buffer it is using. This is the only signal that must be syllchronized between the two sides of asynchronous circuitry.
When one control circuit (read orwrite) finishes ~ccessing a buffer, then the invention will allow control to pass to the other circuit. If, after the control has swung, and two control circuits are trying to use the same buffer, then the later control circuit will begin waiting. The control circuit will wait until each side is using all_mdle buffers, i.e., the other side has swung. If, after it has swung, it finds that it is now using the altemate buffer to the other side, it will not wait, but il l ll l ,edi- tely co,llrllence accessing. This system of arbitration between the buffers is started up by both buffers using the same buffer, buffer 0, in this case. The read side starts up by waiting, while the write side is accesci"g, since there is nothing valid to read out of either buffer.
In one e"lbod;,l,ent, in accordance with the present invention, the swing 5 buffers are two discrete RAMS having all signals, such as enabling strobes, add~sses and data multiplexed from either the read or write side, dependent on which buffer is being ~Gcessed by each side. This structure has been shown to use a lot of area in the busing of a large number of signals between the two buffers.
Combining the two RAMs into a single structure saves much of the busing area while still maintaining pe,rur",ance to the same standard. This structure contains twice as many rows of cells as one of the discrete RAMs found in the first - embodiment of the present invention. I lo~l_vcr, the second el~bûdimenl must have two pairs of bit lines since the read and write to the disclete buffers is happening simullaneously and asynchronously. Each row will be of its original width (i;e., have the same number of cells) since ~ccesses are the same width asfor the d;scl~te RAMS. Each pair of rows are ~ccessed as if at the same add~ss, but from different buffers, so they conl-e~;t to a d~rrerent pair of bitlines. Using the same address, these pair of rows can be readily ~ccessed by one row decoder connected to the read address and one row decoder connected to the write address. Again, the read and write control never ~ccess the same buffer at the same time so there is no conflict as to which pair is ~ccessed by which row decoder.
In the same way in which each row decocler can ~ccess rows from each buffer, both the read and write circuHry within the structure of the present invention connect to each pair of bitlines, one pair from each buffer. The read and writesare then multiplexed into each of the buffers and, for the same reasons explained above, there will not be conflict.
As shown in Figure 48, a swing unit 1 includes swing buffers 10 with RAM
12 and 14 in accor~lance with the present invention. The swing unit 1 also includes a write control circuit and a read control circuit, which control the data into and out of the RAM 12 and 14. The read control circuit and the write control circuit accor"plish this by use of ~t,obes, data and addless control lines, 8. Lines 7 and 9 are control lines to indicate the RAM used by the write control circuit and the RAM used by the read control circuit. Line 7 functions to control the write control circuitry, i.e., when the read control circuitry is using, RAM 12 if low, RAM

215~962 14 if high. Similarly, Line 9 functions to inform the read control circuitry that the write control circuitry is using RAM 12 if low, RAM 14 if high.
In the present invention, swing buffer 10 has two RAM arrays, 12 and 14.
Swing Buffer 10 is capable of asy~ ~chr~"ous, altemative reading and writing to the 5 RAM area which enables the appardlus to achieve the necessary band width for high speed ~ccesci"g of the me",ory. The RAMs 12 and 14 require the following signals: write address 16, read address 18, data in 20, data out 22; and a read and write enable signal (not shown). See also Figure 49.
The write add~t:ss and read addless signals are multiplexed by mu~t;plexers 24. The RAM array 12 and 14 operate with the write circuitry, row decoder and read circuitry in a conventional sense.
In the first emboJ;."e"t of the present invention, during initiation of the - swing buffer 10, RAM 12 will be written to until the control circuitry s~:itches a write enable single to RAM 14.
Once the RAM array 12 has been written, it falls under the control of the read ci;cuitry 4, to be read. During this time, the RAM array 14 is also being written. It is important to note when the RAM array falls under the control of the read array control 2, or the write control circuit 4, the control is established until reading or writing is completed and then control is turned over. In the situation 20 where the read control circuit 4 is ~ccessi"g the RAM array, such as 12, and the write control circuitry 2 needs to ~ccess the same RAM array 12, then the write control circuit will begin waiting.
Thereror~, in accordance with the present invention, two control events are cr~aled. When a write control circuit or a read control circuit swings to a dirrer~l-t 25 RAM, it will either begin immedi~tely ~Gcessing the RAM since the RAM is freeand not under control of the alternative circuit, or it will begin to wait. During start up, the read side defers to the write side, since there is nothing valid to be read out of either buffer.
The second embodiment of the present invention is shown in Figure 50.
30 An i"tegr~ted swing buffer 30 includes a RAM array 32 having the logical size of RAM array 12 combined with RAM array 14. In other words, there is the same amount of RAM in both the first and second embodi",el)ls, however, it is combined in the second embodiment. Accordingly, the integlated swing buffer has the advantage of saving much of the busing area while still pe~rorl'';nS~ the 35 same swing buffer function.
In the second embodiment of the present invention, the write circuit and read circuit 34 and 36 ,espe~ /ely, are similar to those used in the swing buffer 21S~962 ;~

10. However, these circuits now include s~ ,tor~ which choose from the pairs of bit lines desc,iL,ed l,er,i.,dn~r. Likewise, the read access row decoder 38 and the write access row decoder 40 are similar to those contained in swing buffer 10, I)oJ~evcr, they are able to access a pair of rows as desc, ii-~ed her~,i,)dner in Figure 5 51.
As shown in Figure 51, the particular structure of the integrated swing buffer 30, in accGrclance with the present invention, is detailed. Individual cells 42 are contained in rows 44. The read row decoder 38 and write row decoder 40 ~ccess the rows 44 in pairs. A pair of rows have the same address provided by the address lines 16 and 18. The read buffer line 52 and write buffer line 54 provide the control infcn, ll~ion for sele~;til,g one of the paired rows 42. The buffer O bitlines 48 and buffer 1 bitlines 50 connect to alt~r"~ti~/e rows of cells and to the read and write circuitry 34 and 36. For clarity in depicting the addressing, the lighter shading illusl,ates the read row decoder 38 accessing a row in buffer 0.
15 Similarly, the darker shadi"g illustldtes the write row decoder 40 ~ccessi"g a row in buffe; 1. - -In summary, the present invention includes a swing buffer apparalus having at least two ~AM arrays, a write control circuit in communicaliGn with the RAM arrays for controlling data input into the RAM array, and a read control circuit 20 in communi~lion with the RAM arrays for controlling data output from the ~AM
arrays. F~"ll,er")ore, the write control circuit and read control circuit are in communication with one another to allow a synchronized control of the ~AM
arrays. There is also a swing buffer appardt-ls having a RAM array, a write control circuit in communication with the RAM array through a pair of bit lines, a read 25 control circuit in communication with the ~AM array through another pair of bit lines and a read row decoder and a write row decoder for addressing the RAM
through a pair of rows so that individual cells are read. The pr~sent invention also provides a method of asynchronously addressing RAM, by decoding at least a pair of cells in the RAM, using a row decoder to decode at least a pair of rows and 30 selecting one of the rows to be ~ssessed, using at least two pairs of bitlines co,)l)e.Aed to read a circuit and a write circuit and selecting the pair of bitlines to be used.
DETAILED DES~nlr I ION OF THE INVENTION FOR STORING VIDEO
INFORMATION
35 Video deco,npression systems contains three basic parts used to decode and display picture illfo",)dliGn. The three main parts of a video decG",pr~ssion system are the spatial decoder, temporal decoder and the video for",dller. The "~ ` 2154962 pr~se,lt invention involves the ~rnpGrdl clecoder and video fo",)dtler and the way in which the ten)pordl decoder and video ror"~dller manage their respective picture buffers, I,erei"drler the frame store buffer. In MPEG systems, the temporal decoder cG"lains two frame store buffers and the video formatter 5 contains two frame store buffers.
MPEG uses three different picture types: Intra (I), Predicted (P) and Bidirectionally interpolated (B). B pictures are based on predictions from two other pictures; one picture is from the future and one from the past. The I pictures require no further decoding by the temporal decoder, but must be stored in one 10 of the two frame store buffers for later use in decoding P and B pictures.
Decoding a P picture requires forming predi~,tiGns from a previously decoded P
or I picture. The decoded P picture is stored in a frame store buffer for use indecoding further P and B pictures. B pictures can require predi~:tions from bothof the frame store buffers. However, B pictures are not stored in the frame store 1 5 buffers.
ft will be appr~c; ~'sd that I and P pictures are not output from the temporal decoder as they are decoded Instead, I and P pictures are written into one of the frame store buffers, and they are read out only when a subse~uent I or P picturearrives for decodi~ ,9. In other words, the temporal decoder relies on suhse~uent 20 P or I pictures to flush previous pictures out of the two picture buffers. Accord-ingly, the spatial decoder of the pr~sent invention can provide a fake I or P picture when it is necessz~ ~ to flush the temporal decoder's two frame store buffers. In tum, this fake picture is flushed when a suhse~uent video sequence begins.
As shown in Table 14, the picture frames are displayed in n~""erical order.

Display Order 11 Be B3 P4 B5 B6 P7 B8 B9 110 TransmitOrder I P4 Be B3 P7 B5 B6 110 B8 B9 Table 14: Frame Stores 30 However, in orderto reduce the number of frames that must be stored in ",e",oly by the temporal decoder, the frames are transm.tted in a different order. It is useful to begin the analysis from an intra frame (I frame). The I frame is transmitted in the order it is to be displayed. The next predicted frame (P frame), P4 is then transmitted. Then, any bi-directionally interpolated frames (B r,~",es) 35 to be displayed between the I frame and P4 frame are l,d"s",illed, rep~sel)ted by Be and B3. This allows the transmitted B frames to ,ererel-ce a previous frame (forward p,~di~Aion) or a future frame (backward prediction). After transll,itli,19 all ~ ` 21S~962 the B frames to be displayed betvJccn I and P4, the P7 frame is transmitted. Next, all the B r,arnes to be displayed between the P4 and P7 frames are t,ansr"itted,i.e., co~aspondi"g to B5 and B6. Then, the next I frame, 110, is transr"illed.
Finally, all the B rl~",es to be displayed between the P7 and 110 r,ames are 5 tra"s"lilled, c~"t:sponding to B8 and B9. This ordering of l,dnsr"itled r,~",es requires only 2 frames to be kept in memory by the temporal decoder at any one time, and does not require the decoder to wait for the transmission of the next P
frame or I frame to display an interjecting B frame. As described above and shown in Table 14, the te"~pordl decoder of the present invention can be 10 configured to provide MPEG picture reordering. With this picture reordering, the output of P and I pictures is delayed until the next P or I picture in the data stream starts to be decoded by the tel"pGrdl decoder.
- As the P and I pictures are reordered, certain tokens, i.e. Picture_Start, Picture_Type, and Temporal_Reference, are stored ler"pGrdrily on the chip as the15 picture is written into the picture buffers. When the picture is read out for display, these stored tokens are retrieved. At the output of the temporal decocler, the DATA tokens of the newly clecodecl P or I picture are replaced with DATA tokens for the older P or I picture, and they are then sent to the video formatter. Note that the output from the temporal decoder is in tokenized macroblock format and 20 there is no block-to-raster conversion.
In brief, the video f~l"~dtler of the presenl invention stores two r,drnestores or pictures. In some video for",dtter:. three pictures or r,amestores are used to accGI~moddle such features as repeating or skipping pictures. The video f~l",dtter's off-chip DPAM holds three r,alne~lores. The use of three f,dr"esto(es 25 here allows frames to be either repe~ted or ski,uped in sit~ l~tions where the frame rates of the decoded video and the display are different.
All 1, B and P r,dmes are stored in the f,~l"estores of the video for",aller.
At any one time, there may be one frame store from which data is being displayed, one frame store into which data is being written, and in video fol",dll~r~
30 with three rldrnestores, one other frame may be being stored in the third frame store.
The presellt embodiment pelro~",s all the prediction, reorderi"g and block-to-raster tasks MPEG normally handles by using a temporal decoder with two r,dl"eslores and a video f~r",dll~r with two r,~l"estor~s, i.e., for a total of four 35 r,d,nesl~.r~s. This is accG",plished in the present invention by using a frame store sharing scheme that only uses three framestores. The present enlbodiment cannot, however, handle the repeat and skip frame tasks of a video fo""dller with only the three rl~,nestores.
The pr~sent invention stores I pictures in a first frame store and P pictures in a second frame store. Recause of the need to perform the block-to-raster conversion, B frames are stored in the manner detailed below in a third frame store. In order to minimize the amount of external DRAM required, a scheme is used where successive B frames share the same third frame store.
When a B frame is decoded, it may refer to the two previously decoded I
or P frames occupying the first and second f,~"~estores. The decoded B frame is written into the third frame store. The present embodiment allows the raster to c~m~ "ence prior to a frame store being completely filled. The raster is allowed to start before the frame store is filled so that the next B frame can be written into the same frame store to occupy the space vacated by the raster at the top of theprevious frame.
In order to keep a record of which parts of the frame store are occupied with pic~ure data, and which are available for new data, each frame store is split into sector~. In the present invention, each frame store is first split into two field stores, each of which cG,nprises N sectors, where N is the number of block rows in the field.
Frames coded as field pictures are straighfforward. Each successive macroblock row occupies two sectors in a field store. Once the write back has prog~3ssed far enough down the frame, the raster starts reading out each sector from the top. Once the write back of the first frame has been completed, the start of the next frame is written into the space left by the raster. Checks on the status of each of the sectors ensures that the sector to be rastered is indeed full, and that for write back, the two sectors required are empty.
Frames coded as frame pictures are more difficult. Unlike field pictures, the macroblock rows of data are not written to the DRAM in the same order as they are to be rastered. The field stores are written to in parallel, whereas the fields are ,astert:d in turn.
Consider a picture with 8 sectors per field store. That is, Field store 0 CO,15ist~ of 8 se..~r~ numbered 0 to 7, each of which contains one row of blocks(i.e., each 8 pixels deep by the width of the picture). Field store 1 cGnsi~ts of 8 sector~, numbered 8 to 15, each of which contai,)s one row of blocks (i.e., each35 8 pixels deep by the width of the picture).
The first macroblock row is written back into sector 0 in field store 0 and to sector 8 in field store 1. The field stores continue to be filed in parallel. At some 21S~962 point, the raster beings displaying sectors from field store 0, that point beingchosen so that the raster of field store 0 does not catch up with the write back.
However, the second frame cannot be written back in the same manner as the first. Re~use the secto,:j are written and read in a di~r~"l order, waiting for the 5 same two sectors to be free at the start of a frame would mean that write and read could not run continuously. This must be achieved in order to maintain the display and to l"ainlai" decoding at the necessa;y rate.
Accordingly, the second frame must be written into sectors of the frame store already freed by the raster. This is implemented by dividing the r,dr"es~ores 10 in two. Hence, for the secGnd frame, the meanings of the half field stores change.
Sectors 4-7 become the upper part of the second field store and sectors 8-11 become the lower part of the first field store, i.e., they swap over. The first macroblock row is written to se..~r~ 0 and 4, once they are freed, with S! ~hsequent rows written to 1 and 5, then 2 and 6, and then 3 and 7. The next row is writtento sectors 8 and 12, and so on through to 11 and 15. This reallocation to the mel~lolj is sufficient to allow the write back and raster to continue at the appropriate rate.
Should a third successive B frame arrive, the write back order reverts to that of the first frame.
In the shared B frame store, with FRAME pictures:
The FIRST picture is written back to -Sectors 0 and 8 [1st macroblock row=2 block rows]
Then 1 and 9, 2 and 10, 3 and 11, ... 7 and 15.
The FIRST picture is rastered from -Sector 0, Then1,2,3,4,5,6,7,8,9,10,11,12,13,14,15.
The SECOND frame is written to -Sectors 0 and 4, Then 1 and 5, 2 and 6, 3 and 7, 8 and 12, 9 and 13, 10 and 14, 11 and 15.
The SECOND frame is raslered from -Sector 0, Then1,2,38,9,10,11,4,5,6,7,12,13,14,15.
Note that, in accGr~ance with the pr~sent invention, the second frame, the first macroblock row is not written into seclor~ 0 and 1, which are, after all, the first two sectol~ to be freed by the raster. Instead, it waits for sector 4 to clear. This is done for two reasons: First, waiting for sector 4 to clear does not affect the system's ability to maintain continuous decod;ng and display, even in the sitll~tion _ of worst-case coded data, and it is easier to implement. Secondly, with picture sizes which divide into a number of sectors that are not a power of two, the sequence forwriting to and r~adi.lg from sector~ of ~an~ory does not repeat often (for example, NTSC format has 30 sectors per field and the sequence would 5 repeat every 58 r,dmes). This makes testability and recovery difficult.
As far as imple. ,)el)l~tion is concemed in the present invention, rather than keep a record of the status of each individual sector, each half field store is effectively implel"el)led as a fifo, with pointers to the next location to be written and to be read. Thus, each fifo being full or empt,v causes write back and raster, 10 r~specti~/ely to be disabled. This makes use of the knowledge that each half field store is itself written and read only one way, just like a fifo.
In summar,v, the pr~sent invention, provides method for storing video - i"rc"",atio" by providing video illf(~lmdliol~ in the form of an I Frame, a P Frame, a B, Frame and a B2 Frame, storing the I Frame in a first Frame store, storing the 15 P frame in a second frame store; providing a third Frame store having a first and second field store, the first and second field store being split into at least two mel"ory areas ,espe~ti~/ely, storing the B, Frame in the third register, reading the B, Frame from a selected portion of the "~emGIy area in the first or second field store; writing a portion of the B2 Frame into the selected portion of the memory 20 area from which the B, Frame was read; whereby a reduced amount of memor,v can be used to store video inror",dlion.
The two prograr"s found herein below col,tai" code to be used in the preferred elnbo~ ent of the invention.
Detailed Descr~iption of the Invention for a Parallel Huffman Decoder 25 In acco,dance with the pr~sent invention, the Parallel Huffman Decoder block will decode Huffman coded Variable Length Codes (VLCs) and Fixed Length Codes (FLCs), and pass through tokens underthe control of the parser microprog,an~
ble state machine (MSM).
This embodiment of the pr~sent invention handles both MPEG-2 as well as 30 MPEG-1 Huffman codes. An i"~pGIlanl aspect of this er,lbodi",ent of the invention is that it can sustain a high through-put due to the fact that it is a parallel decoder rather than a serial one.
This embodiment of the present invention uses a code lookup technique to decode Huffman codes. This is done to achieve the pelrul",al1ce requir~mel1ts 35 and also to handle the second MPEG-2 transform coefricie.1t table which is irregular or non-canonical in nature.

_ Fu,ll,e""ore, this embodirnent of the invention has some features that allow it to decode certain more cor"~,'ex components from the stream in a single cycle without the ~sisl~nce of an exle",al controller. Examples of such complex components are Fscare-coded coerric;ents, Intra-DC values and Motion Vector 5 deltas, all of which are pr~sent in the stream as combined VLC/FLC componer,l:~.
Referring now to Figure 52, there is shown how the Parallel Huffman Decoder 300 deals with variable length codes (VLCs). FLCs require a bypass mecha"isn, which uses the s~lector 301 output to generate data and an input field to spec fy the length of the FLC. Thus, the ROM 302 is not required at all during 10 FLC deco~li"g.
However, to clecode a VLC, input is first loaded into the two input data r~ ler:i, 'MSReg' and 'LSReg' as shown in Figure 52. As the names imply, the "earlie~' or most significant data is stored in MSReg. The s~lectl)r is used to align the beginning of the next VLC with the ROM input. Thus, to decode the very first VLC, the selector outputs the top 28 bits of its 59-bit input and the top 16 bits of these a;e passed to the Huffman Code ROM 302. For subsequent VLCs, the selector effectively shifts the input according to the total count of bits decoded thus far. The count is maintai,,ed by adding the size of each VLC, as it is decoded, to a running total. The various word widths are a result of the maxi"~um 20 coded size which can be decoded, which is the 28-bit MPEG-1 Escape Coded C~rric;elll, and the ",~i",um VLC size which is 16 bits (DCT coerficient tables).
The "table select" input is used to select between the various different Huffman code tables required by MPEG.
The Huffman Code ROM
25 The core of the imple",enlatio" of the present invention, used to decosle all the VLCs is a specia' ROM 302 whose addresses are conl~l'e~ with a s~le~lor/shirler 301 as shown in Figures 52 and 53. The ROM 302 has the job of performing a VLC table index ca'cl~'~tion, followed by the index-to-data operation that yields decoded data.
The index calculation can be thought of as a conlerlt addressa~le memory (CAM) oper~liol- with "don't care" matching implel"ented to handle the Huffman codes which form the pr~sented data. Since all the VLC code tables are fixed, a CAM-ROM will suffice and this is the job of the ROM AND-plane shown in Figures 54 through 57. Since the index generation is performed in a look-up manner 35 (rather than algorithmically) there is no resl,i~;tion to handling tables which are canonical.

The ROM Or-plane converts the "index" (an activated word-line) into the decoded data and the size (or length) of the code. The data forms the decoded output (subject to error checking) and the size information is fed back to allow a calculation to be perfomled which cont~ols the s~lector and thus pr~sents the 5 decoder ROM 302 with the correct data to perfoml the decoding of the next VLC
in the suhse~uent cycle.
The ROM 302 addless of the pr~sent invention is in two fields. The larger field is the bit-pattem to be decoded and the smaller field selects which Huffman code table is to be examined. The bit-paller" which must be examined is quite 10 long 16 bits co"esponding to the lo"ge~l VLC code and there is an additional 4 bits of table select. Thus, there is a total addrt:ss space of 20 bits (approxi",ately one million add~esses) although there are only in 450 entries in the ROM 302.
- The reason for the d~ference is due to the existence of "don't care" bits.
In order to decode VLCs the AND-plane must be able to decode "don't 15 care" bits in the VLC bit-pattem. This is be~use all VLCs which are shorter than the ",a,.i,num 18 bits will be followed by additional bits which form no part of the decoding of that VLC. Because of the wide address the AND-plane is predecoded (2-~4) and the ROM 302 must combine "don't care" handling with this pr~decode. F~" ll ,el ",ore in ~dditiol) to the complete MPEG code tables the 20 ROM 302 also has entries to identify illegal VLC patterns which exist for some code tables.

215~962 . -- , Mahil"i~i"~ Throu~hput In order to sustain output of one decoded item every cycle, some care must be taken to control the decoder input and special handling must be used for some "complex" symbols (i.e., ones which are not single FLCs or VLCs).
In order to sustain peak throughput of Escape-coded coefficients it must be possible to input at least one complete code per cycle. Since the "~axin,um length required is 28 bits in MPEG-1 this dicl~les the input word width of 32 bits (being the next sensible size greater than 28).
Normal lldn:irCJIIII cOe~ricie. ~t~ are also "complex" symbols, in the sense that 10 they consist of a VLC f~llowod by a 1-bit FLC which gives the sign of the level value and are handled in a similar manner to the other complex symbols (e.g.
motion Vectors, Intra DC and Escape coded coerficients). Peak throughput - cannot be achieved if coerricie.,l~ are decoded as a VLC f~llDwcd by an FLC (in separat~ cycles) and the altemative of allowing the ROM 302 to decode the sign 15 bit would double the size of the two largest tables in the ROM. Thus in the present invention, speci~l handling is used for various symbols so that a singlecycle can produce the "final" required result.
FLCs and Tokens The basis of FLC handling is to control the selector with the required length of the 20 FLC and to bypass the ROM 302 and simply output the correctly selected FLC.
Thus, simple FLCs are handled fairly naturally by the decoder, without significant extra hardware. Furthe,l"Gre, tokens are not manipulated, but simply p~ssed directly to the output of the decoder.
Imple",entalion 25 This se~;tion descril,es several important features of the imple",enlalion of the decoder., in accordal)ce with the present invention. The implementaliG,) includes the alldnyelllelll of l~:y;slel~ with the counter 303 and selector 301, as shown in Figure 52, and the actual code ROM.
The schel"dlic of Figure 53 shows how the core components are 30 interconnected to implement the main Huffman decoding core section of the pr~senl invention. The r~yi:jlers msl31:0] and Is[31 :0l are MSReg and LSReg, respectively, and the block phselect is the selector. The counter logic is contained in the block phcclog (together with various other logic) and the countlatch is called cntll4:0l. The other logic on this schematic deals with handling35 cGI"n~d"ds, data and cG"""and dynamics, tokens, and the manipulation of the more "complex" symbols (pe, r~,l "~ed in block phcop).

_ The sche",dlic shown in Figure 54 illustrates a very small sample ROM
design of the type used to implement the Huffman code ROM 302 in accordance with the pr~:sent invention. The unusual features of this ROM 302 lie in the AND-plane where pr~decode and "don't care" handling are used to implement a ",ethod 5 of decoding variable length Huffman codes.
Re~ni"y now to Figures 55, 56 and 57 and, more particularly to Figure 55, there is shown a first embodiment of a ROM AND-plane capable of "don't care" handling. In this embodi,nent, each address line (a[3], a[2], a[1] and a~0]) is driven across the AND-plane in both its true and inverted directions. To decode 10 a "one" or a "zero" on a given address line, a transistor is connected to either the true or inverted address line in the conventional manner. In order to decode a "don't care" (denoted by x) a l,al1sistor is not connected to either the true or the - inverted line.
Figures 56 and 57 show altemative embo.li"~e~ that utilize pre-decoding 15 to reduce worst-case number of series lldnsistor:j in the decoding logic. In these examplës, two add~ss bits are combined together in predecoding such that one of four lines is driven high for each of the four possible numbers that can be repr~sent~J with the two add~ss bits. It will be appreciated by one of ordinary skill in the art that the pr~sent invention would work equally well with higher levels 20 of predecGdi"g in which more than two bits are cGmb.. ~ed together. If the two address bits that are grouped together in the predecoding have defined values (either 1 or zero, but not the "don't care") then a transistor is connected to the appropriate pr~decoded add,~ss line in the conventional manner. Similarly, if both of the add~ess bits have a "don't care", then no transistor is used as before.
25 However, if one of the address bits needs to have a defined value (1 or zero)whilst the other addr~ss bit requires "don't care", then the decoding requires that the wordline driven across the Or-plane be selected when either of two of the pr~deco-Jecl add~ess lines is active. In the embodiment shown in Figure 56, thisis achieved by placing two llansistor:~, one on each of the relevant predecode 30 address lines, in parallel as shown in the case for the code; 001x. In the embodi",e"l shown in Figure 57 the required decodi"g is achieved without using a parallel col",e tion of l~dnsistors. In this case, two separate decodes are pe, run"ed both of which must be selected. They are combined together using a NOR gate in the wordline driver such that the wordline is only activated if both of 35 the sele~c are active.
The ~or~go.ng descri~.liol, is believed to adequately describe the overall concepts, system imple"~entalion and operdliGn of the various aspects of the -~ 21S4~62 invention in surticienl detail to enable one of ordinary skill in the art to make and practice the invention with all of its allenda"t features, objects and advantages.
I lo~/evcr, in order to facilitate a further, more detailed in depth unde,standing of the invention, and additional details in connection with even more specific, 5 co"~,ner~;al imple,nenldtiol) of various embodiments of the invention, the following further descri~,lion and explanation is pl~frer~d.
Note that additional Figures, which are self ex~,lanalory to those of ordi"ary skill in the art, are included with this ap~l c~tion for providing further insight into the detailed structure and operation of the environment in which the present 10 invention is i"lended to function.
The aror~desc, il ed pipeline system of the presenl invention sdti~ries a long existing need for further improver"el,ls in various ~spects of video decoding - systems, including an MPEG video decol "p,t:ssion method and apparatus utilizing a plurality of stages i"ler~onoected by a two-wire inlel race arranged as a pipeline 15 pr~cessi,)s~ machine. Control tokens and DATA Tokens pass over the single two-wire i"le, rdce for carrying both control and data in token format. A token decode circuit is positioned in certain of the stages for recognizing certain of the tokens as control tokens pertinent to that stage and for passing unrecognized control tokens along the pipeline. RecGuri~auration processing circuits are positioned in 20 s~le~d stages and are ,esponsive to a ,~cGg"i~ed control token for reconfiguring such stage to handle an identi~ied DATA Token. A wide variety of unique supporting subsystem circuitry and processing techniques are disclosQd for implel"e"li"g the system, including ",er"oly addressing, transforming data usinga commGn processi"g block, time sy~cl-ro~ dliGn, asynchronous swing buffering, 25 storing of video infor"~dlion, a parallel Huffman decoder, and the like.
It will be apparent from the foregoing that, while particular forms of the invention have been illusl,dled and described, various Illoditic-dlions can be made without depa, lin~a from the spirit and scope of the invention. Accor-li,)gly, it is not intended that the invention be limited, except as by the appended claims.

Claims (74)

WE CLAIM:

1. An apparatus for synchronizing time, comprising:
a time stamp for determining presentation time;
a clock reference for initializing system time in a first circuit;
a first time counter in communication with said clock reference for keeping system time in a first circuit; and a second time counter initialized by said clock reference in a second circuit synchronized with said first time counter, for keeping a local copy of said system time and for determining the presentation timing error between said local copy of system time and said system time by comparing said time stamp to said second time counter.

2. An apparatus for synchronizing a system decoder and a video decoder, comprising:
a system decoder;
a time stamp for determining display time;
a clock reference for initializing system time in said system decoder;
a first time counter in communication with said clock reference for keeping system time in said system decoder; and a second time counter initialized by said clock reference in said video decoder synchronized with said first time counter, for keeping a local copy of system time and for determining the display timing error between said local copyof system time and said system time by comparing the time stamp to said second time counter.

3. An apparatus for synchronizing a first circuit and a second circuit, comprising:
a clock reference for initializing system time in a first circuit, said first circuit having a time counter in communication with said clock reference for keeping system time;
a first elementary stream time counter in said first circuit for providing elementary stream time;
said first circuit being adapted to receive a time stamp, and said first circuitadapted to generate synchronization time by adding elementary stream time to said time stamp and subtracting system time; and said second circuit being adapted to receive synchronization time from said first circuit and having a second elementary stream time counter in synchroniza-tion with said first elementary stream time counter for providing a local copy of said elementary stream time and for determining a timing error between said system time and said time stamp by comparing synchronization time to said local copy of said elementary stream time;
whereby said clock reference signal does not have to be passed directly to said second circuit in order to determine timing error.

4. An apparatus for synchronizing a first circuit and a second circuit comprising:
a clock reference for initializing system time in a first circuit;
said first circuit having a time counter in communication with said clock reference for keeping system time;
a first video time counter for providing video decoding time;
said first circuit being adapted to receive a video time stamp and generate synchronization time by adding video decoding time and video time stamp and subtracting system time; and said second circuit being adapted to receive synchronization time from said first circuit and having a second video time counter in synchronization with said first video time counter for providing a local copy of video decoding time and for determining a timing error between said system time and said video time stamp by comparing said synchronization time to said local copy of video decoding time;
whereby said clock reference signal does not have to be passed directly to said second circuit in order to determine timing error.

5. An apparatus as recited in any of claims 1-4, wherein said elementary stream time counters are restricted to 16 bits.

6. An apparatus as recited in any of claims 14 wherein said second elementary stream time counter located in said elementary stream decoder is restricted to 16 bits.

7. An apparatus as recited in any of claims 1-6 wherein said synchronization time is restricted to 16 bits for controlling said elementary stream decode.

8. A method for providing timing information, comprising the steps of:
providing a video data stream having a time stamp carried in packet header said time stamp referring to the first picture in a packet of data;
providing a register having a flag for indicating valid time stamp information which is taken from said packet header and placed into the register;
removing said time stamp from said video data stream and placing it in said register; and encountering a picture start and subsequently examining the status of said register to determine if valid time stamp information is contained in said register by checking flag status.

9. A method as recited in claim 8 and comprising the additional steps of:
generating a time stamp in response to the picture start if the flag indicates valid time stamp information is contained in the register; and inserting the time stamp back into the data stream.

10. A process for decoding video comprising the steps of:
determining display time errors against a threshold value;
parsing video data into tokens for further processing;
determining if a time stamp token is indicated;
comparing the time stamp token to a video time;
generating a compared value to determine an indication of timing error;
determining whether the compared value when compared against a threshold value, is within acceptatle parameters when a timing error is indicated;
and indicating when the compared value is outside acceptable parameters.

11. An apparatus for using a system decoder and a video decoder, comprising:
a system decoder adapted to accept MPEG system streams and demultiplexing video data and a video time stamp from a stream;
said system decoder having a first time counter representative of system time;
a video decoder for accepting said video data and said video time stamp;
said video system having a second time counter in synchronization with said first time counter; and said video decoder also having a video decoder buffer for accepting said video data at a substantially constant rate and outputting said video data at a varying rate and for passing a video time stamp.

12. Apparatus as recited in claim 11 wherein said video decoder while decoding a picture from said video data, also compares said video time stamp forthe decoded picture with said second time counter to determine the appropriate display time.

13. A method for determining a timing error between a first circuit and a second circuit, comprising the steps of:

providing the first circuit with a system time (SY), a time stamp (TS), and an elementary stream time (ET);
obtaining synchronization time (X) by using the elementary stream time (ET), the time stamp (TS) and the system time (SY), in accordance with the equation, X=ET + TS-SY;
providing synchronization time (X) to the second circuit; and generating a synchronized elementary stream time (ET2);
obtaining a timing error by using synchronization time (X) and in accor-dance with the equation ET2-X;
whereby the first circuit can be synchronized with the second circuit without passing system time to the second circuit.

14. A method for determining a timing error between a first circuit and a second circuit, comprising the steps of:
providing the first circuit with a time stamp (TS), and an initial time (IT;
obtaining synchronization time (X) by using the time stamp (TS) and the initial time (IT), in accordance with the equation X=TS-I;
providing synchronization time (X) to the second circuit;
generating a synchronized elementary stream time (ET); an obtaining a timing error by using synchronization time (X) and in accor-dance with the equation ET-X;
whereby the first circuit can be time synchronized with the second circuit without passing time to the second circuit.

15. A method for determining a timing error between a first circuit and a second circuit, comprising the steps of:
providing the first circuit with a system time (SY), a video time stamp (VTS), and a video decoding time (VT);
obtaining synchronization time (X) by using the video decoding time (VT), the video time stamp (VTS) and the system time (SY), in accordance with the equation, X=VT+VTS-SY;
providing synchronization time (X) to the second circuit;
generating a video decoding time (VT2) in the second circuit which is synchronized to the video decoding time (VT) in the first circuit; and obtaining a timing error by using synchronized time (X) and in accordance with the equation VT2-X;
whereby, the first circuit can be time synchronized with the second circuit without passing system time to the second circuit.

16. A method for addressing memory, comprising the steps of:

providing a fixed width word having a predetermined fixed number of bits to be used for addressing variable width data;
defining the fixed width word with a width defining field and an address field;
providing the width defining field with at least one bit to serve as the termination marker;
defining the address field with a plurality of bits defining the address of the data;
varying the size of bits in the address field in inverse relation to the size ofthe variable width data;
varying the number of bits in the width defining field in direct relation to thesize of the variable width data; and maintaining a fixed width word for addressing variable width data while varying the width of the width defining field and the address field.

17. A method for addressing memory, comprising the steps of:
providing a fixed width word having a predetermined fixed number of bits to be used for addressing data;
defining the fixed width word with an address field and a substitution field;
defining the address field with a plurality of bits defining the address of the data;
defining a variable width substitution field with at least one substitution bit;
the substitution field having at least one bit to serve as a termination marker between the address field and the substitution field;
using the substitution field to indicate substituted bits from a separate addressing source; and maintaining a fixed width word for addressing variable width data while inversely varying the width of the address field and the width of the substitution field.

18. A method for addressing variable width data in a memory, comprising the steps of:
providing memory having words of predetermined width and composed of partial words;
rotating the partial word to be accessed to at least significant bit justifica-tion;
extending remaining part of the word so that the accessed word will be recognized as the partial word;
restoring the remaining part of the word; and rotating the word until the partial word is restored to its original position.

19. A parallel Huffman decoder, comprising:
a selector;
a pair of input register s for receiving Huffman coded data, both of said registers directing input in parallel to said selector; and a Huffman Code ROM for receiving input from said selector and another ROM table select input; said ROM providing decoded data output.

20. A Huffman decoder as recited in claim 19, adapted to decode MPEG-1 data.

21. A Huffman decoder as recited in claim 19, adapted to decode MPEG-2 data.

22. A Huffman decoder as recited in claim 19, adapted to decode both MPEG-1 and MPEG-2 data.

23. A Huffman decoder, as recited in any of claims 19-22, wherein one of said pair of input registers handles most significant data and the other of said pair of registers handles least significant data.

24. A Huffman decoder, as recited in any of claims 19-23, adapted to decode variable length codes (VLC).

25. A Huffman decoder, as recited in any of claims 19-24, wherein said ROM performs a VLC table index calculation.

26. A Huffman decoder, as recited in claim 25, wherein an index-to-data operation follows said index calculation, to provide decoded data.

27. A Huffman decoder, as recited in any of claims 19-26, adapted to decode Escape-coded coefficients.

28. A Huffman decoder, as recited in any of claims 19-26, adapted to decode Intra-DC values.

29. A Huffman decoder, as recited in any of claims 19-26, adapted to decode Motion Vector deltas.

30. A RAM interface for connecting a bus to RAM, comprising:
means for receiving from a bus a plurality of data words, and buffering the received data words;
means for receiving from said bus an address associated with said plurality of data words;
means for generating a series of addresses in RAM into which the buffered data words will be written, the series of addresses being derived from the received address; and means for writing said buffered data words into RAM at the generated addresses.

31. The RAM interface of claim 30, wherein said data word receiving the buffering means includes a swing buffer.

32. The RAM interface of claim 30, wherein;
said RAM operates in page addressing mode; and said address generating means includes means for generating row addresses, and means for generating column addresses based on the received address.

33. The RAM interface of claim 32, wherein;
said RAM is a DRAM;
said bus includes a two wire interface;
said data word receiving and buffering means includes a two wire interface;
said address receiving means includes a two wire interface;
said plurality of data words are in the form of a token; and said received address is in the form of a token.

34. The RAM interface of claim 30, and further including:
means for determining whether said data word receiving means has received and buffered said plurality of data words.

35. A RAM interface for connecting a bus to RAM, comprising:
a plurality of data words stored in RAM at predetermined addresses;
means for receiving from a bus a RAM address associated with said plurality of data words;
means for generating a series of RAM addresses for addressing said plurality of data words in said RAM, the series of addresses being derived from the received address;
means for buffering data words read from said RAM; and means for reading from said RAM said plurality of data words, using said series of RAM addresses generated by said address generating means, and writing said data words into said buffer means.

36. The RAM interface of claim 35, wherein said data word buffering means includes a swing buffer.

37. The RAM interface of claim 35, wherein:
said RAM operates in page addressing mode; and said address generating means includes means for generating row addresses, and means for generating column addresses based on the received address.

38. The RAM interface of claim 37 wherein:
said RAM is a DRAM;
said bus includes a two wire interface;
said data word buffering means includes a two wire interface;
said address receiving means includes a two wire interface; and said received address is in the form of a token.

39. The RAM interface of claim 35 and further including:
means for determining whether said data word receiving means has received and buffered said plurality of data words.

40. A method for controlling the buffering of encoded video data organized as frames, comprising:
determining the picture number of a frame;
determining the desired presentation number of said frame; and marking a buffer as ready when said picture number is on or after said desired presentation number.

41. An apparatus for transforming data, comprising:
A first latch defining a first data stream source and a second latch defining a second data stream source;
said first and said second latches being in communication with an arithmetic unit;
said arithmetic unit communicating data to a transposer;
said transposer transposing and communicating said data to said second latch;
said second latch being arranged to absorb data; and said second latch and said first latch communicating said first and second data streams in an interleaved manner to said arithmetic unit, further defined that in said communication in the interleaved manner said second latch does not interrupt communication from said first latch;
whereby a common arithmetic unit is used for said first and said second data streams.

42. A process for transforming data using a common arithmetic unit, comprising the steps of:
loading the data in a first latch and upon reaching a predefined number of cycles transmitting the data to the arithmetic unit and loading a first maker bit into a control shift register, loading data into a second latch, the second latch being adapted to absorb data;

transmitting the data in the second latch to the arithmetic unit when the firstcontrol shift register reaches a predetermined state and the second latch is filled with a predetermined amount of data;
failing to transmit data from the second latch, if the second latch is not filled with a predetermined amount of data; and recovering the second latch when the first latch is receiving non-data.

43. A swing buffer apparatus, comprising:
at least two memory arrays;
a write control circuit in communication with said memory arrays for controlling data input into said memory arrays;
a read control circuit in communication with said memory arrays for controlling data output from said memory arrays; and said write control circuit and said read control circuit being in communica-tion to allow a synchronized control of said memory arrays.

44. A swing buffer as recited in claim 43, wherein:
the synchronization enables said read control circuit to read from one memory array to the exclusion of said write control circuit writing to that array.

45. A swing buffer as recited in claim 44, wherein said exclusion is temporary.

46. A swing buffer as recited in claim 43, wherein said synchronization enables said write control circuit to read from one memory array to the exclusion of said read control circuit reading from that array.

47. A swing buffer as recited in claim 44, wherein said exclusion is temporary.

48. A swing buffer as recited in claim 44, wherein said synchronization includes causing the excluded control unit to wait until the other control unit has operated on the array.

49. A swing buffer apparatus, comprising:
a memory array;
a write control circuit in communication with said memory array through bit lines;
a read control circuit in communication with said memory array through said bit lines; and a read row decoder and a write row decoder arranged to access said memory array through a pair of rows so that individual cells are read.

50. A swing buffer as recited in claim 49, wherein said read row decoder is able to select at least one row for accessing.

51. A swing buffer as recited in claim 49, wherein at least one of said read or write control circuits is arranged to select at least a pair of said bit lines for determining the rows to be accessed.

52. A swing buffer as recited in claim 49, wherein said read and write decoders are unable to select the same row simultaneously, and said read and write control circuits are unable to select the same bit lines simultaneously.

53. A method of asynchronously accessing cells in a memory, comprising the steps of:
using a decoder to decode at least a pair of cells; and reading one of the cells and writing to the other of the cells.

54. A method as recited in claim 53, wherein each dell is located in a separate row.

55. A method as recited in claim 53, wherein the cells are commonly accessed by bit line decoders and are accessed by at least a first group of saidbit lines or a second group of said bit lines.

56. A method as recited in claim 53, wherein the cells are arranged into first and second groups which are only accessed by the read and write decoders, the first group of cells being accessed exclusively by a first group of said bit lines.

57. A method for storing video information, comprising the steps of:
providing video information in the form of an I Frame, a P Frame, a B1 Frame and a B2 Frame;
storing the I Frame in the first Framestore; storing the P frame in a second framestore; providing a third Framestore having a first and a second field store, the first and second field store being split into at least two memory areas respectively; and storing the B1 Frame from a selected portion of the memory area in the first or second field store; writing a portion of the B2 Frame into the selected portion of the memory area from which the B1 Frame was read;
whereby a reduced amount of memory can be used to store video information.

58. A memory for "don't care" handling, comprising:
a set of memory address lines, inverted address lines and data lines, said address lines and inverted address lines being connected according to a decodingformat to access the addressed information in the form of a data word, "don't care"
address locations on the data lines being unconnected with the associated address lines and said inverted address lines.

59. A memory as recited in claim 58, wherein said memory address lines and said inverted address lines are predecoded into sub-address lines of groups of address lines and inverted address lines, such that a signal is present on one of said sub-address lines for an address, said memory having a by-pass activatable by one sub-address line to decode that sub-address line as a definedvalue and another sub-address line as a "don't care" in the same group.

60. A memory, as recited in claim 58, wherein a pair of data lines are associated with each address line for an address location representing data to be accessed, each pair of said data lines being connected to a NOR gate.

61. A method of accessing Dynamic Random Access Memory (DRAM) to store and retrieve data words associated with a two dimensional image, the DRAM including two separate banks, each bank being capable of operating in page mode to read and write the data words, the two dimensional image being organized in two dimensional grid pattern of calls, each cell containing an M by N
matrix of pixels, and the words associated with each cell occupying one page or less of a bank, the method comprising the steps of:
(a) assigning each cell a particular one of the two banks so that all data words associated with that particular cell are read from and written to one particular page of that particular bank, the assignment of banks to cells being done such that each cell is associated with a different bank than any bordering cell which is also either in the same row or in the same column; and (b) reading the data words associated with a cell that is composed of a matrix of pixels, and that is not aligned with the two dimensionalgrid pattern, but that is aligned with pixels in cells in the two dimensional grid pattern.

62. The method of claim 61, wherein the DRAM includes a first and a second bank, and the step (b) of reading the data words associated with the unaligned cell includes the steps of:
(c) identifying which cells in the two dimensional grid pattern contain data words associated with the unaligned cell;
(d) reading, from the first bank of DRAM, the data words associated with one of the cells in the grid pattern identified as containing data words associated with the unaligned cell;
(e) reading, from the second bank of DRAM, the data words associated with another of the cells in the grid pattern identified as containing data words associated with the unaligned cell;

(f) repeating steps (d) and (e) until all the data words associated with the unaligned cell have been read.

63. A method for accessing Dynamic Random Access Memory (DRAM) to store and retrieve data words associated with a two dimensional image, the DRAM including two separate banks, each bank being capable of operating in page mode to read and write the data words, the two dimensional grid pattern of cells, each cell containing an M by N matrix of pixels, and the words associatedwith each cell occupying one page or less of a bank, the method comprising the steps of:
(a) assigning each cell a particular one of the two banks so that all data words associated with that particular cell are read from and written to one particular page of that particular bank, the assignment of banks to cells being done such that each cell is associated with a different bank than any bordering cell which is also either in the same row or in the same column; and (b) reading the data words associated with a cell that is composed of an M by N matrix of pixels, and that is not aligned with the two dimensional grid pattern, but that is aligned with pixels in cells in the two dimensional grid pattern.

64. The method of claim 63, wherein the DRAM includes a first and second bank, and the step (b) of reading the data words associated with the unaligned cell includes the steps of:
(c) reading, from the first bank of DRAM, the data words associated with one of the cells in the grid pattern containing data words associated with the unaligned cell;
(d) reading, from the second bank of DRAM, the data words associated with another of the cells in the grid pattern containing data words associated with the unaligned cell;
(e) repeating steps (c) and (d) until all the data words associated with the unaligned cell have been read.

65. The method of claim 63, wherein the DRAM includes a first and a second bank, and the step (b) of reading the data words associated with the unaligned cell includes the steps of:
(d) reading, in a predetermined order of cells, the data words associated with each cell in the grid pattern containing data words associated with the unaligned cell, the predetermined order of cells resulting in data words read from succeeding cells being read from alternating banks.

66. The method of claim 65, wherein the predetermined order is a clockwise rotation of cells in the grid pattern containing data words associatedwith the unaligned cell.

67. The method of claim 65, wherein the predetermined order is a counterclockwise rotation of cells in the grid pattern containing data words associated with the unaligned cell.

68. The method of claim 63, wherein the DRAM includes a first and second bank, and the step (b) of reading the data words associated with the unaligned cell includes the steps of:
(c) identifying which dells in the two dimensional grid pattern contain data words associated with the unaligned cell;
(d) reading, from the first bank of DRAM, the data words associated with one of the cells in the grid pattern identified as containing data words associated with the unaligned cell;
(e) reading, from the second bank of DRAM, the data words associated with another of the cells in the grid pattern identified as containing data words associated with the unaligned cell;
(f) repeating steps (d) and (e) until all the data words associated with the unaligned cell have been read.

69. The method of claim 63, wherein the DRAM includes a first and a second bank, and the step (b) of reading the data words associated with the unaligned cell includes the steps of:
(c) identifying which cells in the two dimensional grid patter contain data words associated with the unaligned cell;
(d) reading, in a predetermined order of cells, the data words associated with each cell in the grid pattern identified as containing data words associated with the unaligned cell, the predetermined order of cells resulting in data words read from succeeding cells being read from alternating banks.

70. The method of claim 69, wherein the predetermined order is a clockwise rotation of cells in the grid pattern identified as containing data words associated with the unaligned cell.

71. The method of claim 69, wherein the predetermined order is counterclockwise rotation of cells in the grid pattern identified as containing data words associated with the unaligned cell.

72. A method of accessing from RAM a number M of words that is less than the predetermined fixed burst length N of the RAM, the RAM including an enable line that selectably enables and disables reading from and writing to theRAM, the method comprising the steps of:
ordering N words to be read from or written to the RAM;
determining when M words have been rad from or written to the RAM, M
being less than N; and disabling the RAM upon determining M words had been read from or written to the RAM.

73. A method of reading from RAM a number M of words that is less than the predetermined fixed burst length N of the RAM, the RAM including an enable line that selectably enables and disables reading from the RAM, the method comprising the steps of:
ordering N words to be read from the RAM;
determining when M words have been read from the RAM, M being less than N; and disabling the RAM upon determining M words had been read from the RAM.

74. A method of writing to RAM a number M of words that is less than the predetermined fixed burst length N of the RAM, the RAM including an enable line that selectably enables and disables writing to the RAM, the method comprising the steps of:
ordering N words to be written to the RAM;
determining when M words have been written to the RAM, M being less than N; and disabling the RAM upon determining M words had been written to the RAM.