APPARATUS AND METHOD FOR DYNAMIC MULTIPLEXING
BACKGROUND OF THE INVENTION
The International Organization for Standardization (ISO) has adopted a standard (ISO/IEC 13818-1) that addresses the combining of one or more "elementary streams" of video and audio, as well as other data, into single or multiple streams suitable for storage or transmission. The ISO/IEC 13818-1 standard, hereinafter referred to as the "MPEG-2 Systems" standard, is described in detail in the ISO draft document "Generic Coding of Moving Pictures and Associated Audio", ISO/IEC JTC1/SC29/ G11 N0801 (13 November 1994) , which is incorporated herein in its entirety by reference.
The MPEG-2 Systems standard defines an individual coded video, audio or other coded bitstream as an "elementary stream" . The contents of an elementary stream may be broken into a sequence of discrete units, in which case the elementary stream is structured as a Packetized Elementary Stream (PES) . The individual units, or packets, are known as PES packets, which can be of large and variable size The MPEG-2 Systems
standard defines generic structures for PES packet formats and specifies particular rules for creating PESs from digital video and audio elementary streams.
The MPEG-2 Systems standard defines two methods of creating a multiplex of PESs. In a Program Stream (PS), all components in the multiplex are assumed to belong to a single "Program", that is, a collection of elementary streams which may sensibly be presented as a unity to a user, all components being referenced to a common time base, together with certain coordinating control information. PES packets from component PESs are multiplexed by PES packet.
In a Transport Stream (TS) , the components of the multiplex may belong to many programs. Each transport stream packet is assigned a "packet identifier" (PID) . A sequence of packets identified by the same value of the PID field generally represents a single service component, typically a video or an audio component, or a user data component. The PES packets are broken into small, fixed-size units called transport packets, which may be multiplexed with transport packets from other PESs. The Transport Stream is transmitted at a transport rate which is sufficient to accommodate the bandwidth requirements of all components carried within the Transport Stream. Since the transport rate may, either momentarily or in aggregate, exceed the bandwidth requirements of the constituent components, the MPEG-2 Systems standard has reserved PID OxlFFF as the "null PID". Packets in this PID are "null packets" and do not
carry any component . An MPEG decoder may discard them with impunity.
Coordination and control of the PESs and the Programs in the TS is managed via control data called Program Specific Information (PSI) that is structured as a set of service control tables.
Thus, a Transport Stream can be understood as a multiplex of service components combined with descriptive service control information. However, the MPEG-2 Systems standard does not define how the Transport Stream multiplex is to be formed.
SUMMARY OF THE INVENTION
An object of the invention is to maximize channel utilization by minimizing the number of null packets in an output stream such as an MPEG standard Transport Stream.
The present invention provides a dynamic multiplexing approach that is appropriate to any buffer servicing policy. In accordance with the invention, the output rate of a multiplexer is updated such that the buffers are constantly being serviced. According to a preferred embodiment, the output rate is updated at the boundary of each serviced packet . In an alternative embodiment, the output rate is updated at the boundary of every L packets.
Accordingly, a method of multiplexing includes providing plural input data streams, each data stream comprising fixed length packets, and plural buffers,
each buffer associated with a respective input data stream and having a buffer depth. Packets for each stream are received in the associated buffers at respective variable buffer input rates. The buffers are serviced at a service rate to provide an output data stream by updating the service rate at an update time interval such that the buffers are constantly being serviced. The updating of the service rate includes determining a maximum input rate for each of the variable buffer input rates and a fullness indication for each buffer. The updated service rate is computed from the maximum input rates and the buffer fullness indications.
In the case where the input streams are MPEG transport streams that include transport packets and null packets, each packet having a sync byte, the maximum input rate is determined by ignoring or removing the null packets and computing the time between two consecutive sync bytes. While the preferred embodiments are described with reference to MPEG-2 Systems standard Transport Streams, the principles of the present invention are intended to encompass the multiplexing of any data streams that comprise fixed length packets and have a self-contained directory structure which describes the multiplex.
Thus, while the term "transport stream" is used herein to denote a Transport Stream as defined by the MPEG-2 Systems standard, the principles of the present invention are also applicable to systems using similar
transport structures such as Asynchronous Transfer Mode (ATM) systems.
BRIEF DESCRIPTION OF THE DRAWINGS
The foregoing and other objects, features and advantages of the invention will be apparent from the following more particular description of preferred embodiments of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
FIG. 1 is a schematic block diagram of an N-input stream packet multiplexing apparatus in accordance with the present invention.
FIG. 2 is a schematic block diagram of a series configuration of single-service encoder/multiplexers in accordance with the invention.
FIG. 3 is a schematic block diagram of a single- service encoder/multiplexer of FIG. 2.
DETAILED DESCRIPTION OF THE INVENTION 1. Introduction
The general principles of the present invention are described first, followed by a description of preferred embodiments.
Referring to FIG. 1, an N-input stream packet multiplexing apparatus 400 is shown which illustrates
the principles of the present invention. The multiplexing apparatus 400 includes N first-in, first- out (FIFO) buffers 302-1, 302-2, ..., 302-N, and a multiplexer 304. Each FIFO buffer receives a respective input packet stream 300-1, 300-2, ..., 300-N. The input data rates of the input packet streams are variable up to a maximum rate respectively indicated by pl r p2, ..., pN. The multiplexer 304 multiplexes the N input packet streams to provide an output packet stream 306 at an output rate r(t) .
It can be shown, as described further below, that the output rate r(t) can be dynamically adjusted such that the buffers 302-1, 302-2, ..., 302-N are constantly serviced, thereby maximizing the utilization of the output channel. This result is important for MPEG multiple-channel systems, described further herein, where it is desirable to minimize the number of null packets that are inserted into a multiplexed transport stream so that the channel utilization is maximized. It can also be shown, as further described herein, that regardless of the service policy, no overflow occurs when the size of the buffers 302-1, 302-2, ..., 302-N is at least NP, where N is the number of input streams and P is the packet size. In a preferred method, the output rate r(t) is updated at the boundary of each serviced packet . In an alternative method, the output rate is updated at the boundary of every L serviced packets.
The following table is a summary of notations and conventions used in the description herein.
The case of updating the output rate at each packet boundary is now described.
Consider a time tk as the start time at which the kth packet is being serviced from any of the buffers 302- 1, 302-2, ..., 302-N. A measure of the fullness of each buffer 302-i at time tk is denoted Bx (tk) . The buffer fullness B1(tk)for each input buffer is continuously measured and updated. An error ABX in measuring the buffer fullness is due to the latency τM to read a register that defines the buffer fullness. Thus, the buffer fullness can be computed as follows :
B^tk+τ = B,(tk)+ ΔB,(tk) = Bx(tk)+ p, τM
(Eq. 1)
Based on the input rates px and the buffer fullness, the output rate is derived in the following manner.
Initially, the output rate r(t0)is set equal to the sum of the maximum input rates . As soon as the first packet is being serviced from any of the buffers, computation of the output rate for the next packet to be serviced begins.
More particularly, at a time t
k+1-T, the output rate, r(t
k+1), for servicing the next packet is computed, where the extra time interval T is such that P/r(t)>> T
>(τ
DDS,τ
M). Time interval T counts for the time it takes to compute r(t
k+1) . The new output rate is set equal to the following:
(Eq . 2 )
where δskιl is equal to 1 if one of the buffers is serviced at time tk, otherwise it is equal to zero. The sum of the buffer fullness at time tk+1 is equal to
N
∑ Bi( = ∑ Bl(tk+1 - T) + ∑plT. i+l 1=1 (=1
(Eq. 3)
It can be shown, as described in Appendix A below, that regardless of the service policy, no overflow occurs when the buffer sizes are at least NP bytes. Moreover, the following inequality can be shown to be always true:
∑ B,(tM) ≤ NP
1=1
(Eq. 4)
The case wherein the output rate is updated at the L packet boundary is now described.
The minimum time interval between two consecutive packet services is given by P/R(tk), where P is the packet size. It is possible that this minimum time interval is such that the multiplexer 304 does not have enough time to compute and update the new output rate before the next packet is serviced.
A measure of the total number of full packets that are ready to be serviced at time tk is denoted D(tk) .
The value of D(tk) is read every time the output rate r is updated at time tk.
Based on the input rates and the buffer fullness, the output rate is derived in the following manner. Initially, the output rate r(t
0) is set equal to the sum of the maximum input rates . As soon as the first packet is being serviced from any of the buffers, computation of the output rate for the next packet to be serviced starts . At time t
k, computation of the new output rate r(t
k+L) that should take effect at time t
k+L begins. That is, time t
k+L is the start time at which the k+L
th packet is being serviced. The new multiplexer output rate is set equal to the following:
where N is the number of input streams, D(tk) is the total number of full packets in the buffers at time tk, L is the number of packets between two consecutive output rate updates and ∑p± is the sum of the maximum input rates. It is straightforward to show that
D(tk) < N-l (Eq. 6)
is always satisfied regardless of the service policy. This is shown in Appendix A.
2. Preferred Embodiments
Having described the general principles of the invention, preferred embodiments in the context of MPEG systems are now described.
In an MPEG multiple-channel system, it is desirable to multiplex two or more transport streams such that the multiplexed transport streams include a minimum number of null packets. In this manner, measurements of the data rates are not affected by the presence of null packets and channel utilization is maximized.
Referring now to FIG. 2, a daisy-chain or serial configuration of single service encoder/multiplexer devices is shown. Such a daisy-chain configuration is disclosed in U.S. Patent Application No. 08/944,902, and in U.S. Patent Application No. 09/145,118, the contents of which are incorporated herein by reference. In particular, four MPEG encoder/multiplexer devices 100A, 100B, 100C and 100D are daisy chained to output a transport stream 108 which contains the content of individual transport streams from each encoder/multiplexer device. In such a configuration, the output transport stream of each device 100A, 100B, 100C and 100D at respective rates Rj.(t), R2(t), R3(t) and R4(t) is provided as an input transport stream of the next. No input stream need be provided to the first device 100A. The output of the last device 100D is the final transport stream 108 which contains all of the individual transport streams is passed through a device (not shown) that fixes its data rate from R4(t) to Rc, where Rc is the fixed channel rate. It should be noted that the number of devices which can be so daisy-chained together is limited only by the capacity of the output transport stream.
An individual encoder/multiplexer 100 includes a single channel encoder 50 and an elementary multiplexer 10 as shown in FIG. 3. The encoder 50 accepts as inputs on lines 52, 54, 56 a set of components to be inserted into a transport stream. A typical component set consists of a video input, two stereo audio channels,
and two user data channels. Ordinarily, the set of components are intended to form a single service, although in some cases they may form more than one service. Service control data, describing and configuring the components and service (s) added to the multiplex by the encoder 50, is passed on line 58 to the encoder processor 60, which uses this information to create a set of tables to be inserted into a transport stream 62 generated by the encoder 50. The components are each encoded, in a fashion specific to each component type, into a format suitable for inclusion in a transport stream by component encoders 64, 66, 68. Typically, this format will be based on the Packetized Elementary Stream (PES) encoding defined by MPEG-2. The encoded components are then passed to a transport multiplexer 70, wherein they are combined with each other and with the control tables provided by the processor 60 into the transport stream 62. The transport stream 62 is then passed to the elementary multiplexer 10, which combines it with an externally supplied input transport stream 12, such as streams 102, 104, 106 (FIG. 2) , to create an output transport stream 26 which adds the contributions of the encoder 50 to the input transport stream 12.
The elementary multiplexer 10 comprises four processing elements: a pair of transport demultiplexers 18, 20; a processor 22; and a transport remultiplexer 24. The demultiplexers 18, 20 demultiplex the transport
streams to their constituent service components and service control information tables. The service control information tables are routed to the processor 22 which combines and reconfigures the information contained in the tables to provide a single set of output service control information tables. The demultiplexers 18, 20 filter the service components based on the combined and reconfigured information in the service control information tables. The demultiplexers 18, 20 and the processor 22 each have an associated buffer that is serviced by the transport remultiplexer 24 to multiplex the output service control information tables with the filtered streams to provide the output transport stream 26. A copy of the output transport stream may optionally be made available on line 28.
It should be noted that, in general, one or more of the PIDs of input transport stream 12 may conflict with the PIDs in the transport stream 62. To avoid this conflict, the multiplexer 10 preferably re-maps or changes the PIDs for every conflicting packet to a new value that avoids the conflict before it is passed on to the output stream 26. The details of PID re-mapping are provided in the above-referenced co-pending U.S. Patent Application No. 08/944,902. In the above-referenced co-pending U.S. Patent
Application No. 08/944,902, the elementary streams 52, 54, 56 and the transport streams 62, 12 for the daisy- chain embodiment disclosed therein were assumed to have fixed data rates. In the present invention, the data
rates of the elementary streams 52, 54, 56 and the transport streams 12, 62, 26 are allowed to be variable. As described above, the present invention provides a methodology whereby the output rate of a multiplexer is updated such that the buffers are constantly being serviced.
From the foregoing discussion of the general principles of the invention and FIG. 1, it can be seen that the single channel encoder 50 (FIG. 3) can be modeled as a multiplexer having N buffers where N=4 for input streams 52, 54, 56, 58 having corresponding maximum input data rates pv, pA, pD and ps and output stream 62 from multiplexer 70 having a service rate given by r (t) . Likewise, it can be seen that the elementary multiplexer 10 (FIG. 3) can be modeled as a multiplexer having N buffers where N=3 for input streams 62 , 12 having corresponding input data rates r^t) and r3 (t) and control stream 43 having input rate rc(t), and output stream 26 from remultiplexer 24 having a service rate given by R± (t) .
The following describes particular considerations in applying the general principles of the invention to the preferred embodiments described above and shown in FIGs. 2 and 3.
2.1 One Packet Boundary Update 2.1.1 Single Channel Encoder 50
The input data rates pλ of the elementary streams input to the single channel encoder 50 are set by the system application. These rates are the maximum possible data rates. In the case of statistical multiplexing, an upper bound on the video data rates is set. The single channel encoder 50 constantly monitors and updates the input rates . In the case the sum of the input rates increases, the single channel encoder 50 updates the output rate based on the new input rates at the next packet boundary. In the case the sum of input rates decreases, the single channel encoder 50 services N packets before it updates its output rate to avoid overflowing its buffers .
The buffer fullness B1(tk) for each stream is constantly measured and updated as described above in relation to (Eq. 1) . As described above, the output rate is derived from the input rates and the buffer fullness. Initially, the output rate r(t0)is set equal to the sum of the maximum input rates . As soon as the first packet is being serviced from any of the buffers, the single channel encoder 50 starts computing the output rate for the next packet to be serviced. The minimum time interval between the servicing of any two consecutive packets is P/r (tk) > (188) (8) /50Mbps=30μs, where P is the packet size and r(tk)=50 Mbps is the present output rate. Therefore, the single channel encoder 50 has enough time to compute and update its new output rate before the next packet is serviced. The
single channel encoder 50 also accounts for the latency, τDDS, before updating the output rate.
At time tk+1-T, the single channel encoder 50 computes the output rate, r(tk+1), for servicing the next packet, where the extra time interval T is such that P/r(t) >> T >(τDDSrτM). Time interval T counts for the time it takes to compute r(tk+1) . The new output rate r(t) is set according to (Eq. 2) above with N=4 , P replaced with P', and the result multiplied by a factor of 188/184 to account for overhead in the input packets.
2.1.2 Elementary Multiplexer 10
In a transport stream of transport packets, each packet includes a sync byte as the first byte . The elementary multiplexer 10 computes the input data rates by first computing the time between two consecutive sync bytes after ignoring or removing any null packets and routing the packets to the processor 22. The multiplexer 10 then uses the shortest time value between two consecutive sync bytes to compute the maximum input data rates of the input transport streams 62, 12. The control input rate is extracted from the processor 22.
The elementary multiplexer measures the buffer fullness associated with the buffers for each input stream 62, 12 and the control stream 43, and updates the output rate at the transport packet boundaries.
In particular, the elementary multiplexer 10 measures the maximum input data rates by computing the time interval λ (t ) between two consecutive sync bytes while ignoring or removing the null packets. The lowest λ1(t) value is used to compute the maximum input rates. The elementary multiplexer 10 updates the minimum value when a lower time interval is measured. The minimum time interval λf is used to derive the -maximum input rate of the input transport streams 62, 12. Using the p notation of (Eq. 3) in general for maximum of each input rate τ∑ (t) and r3(t), the maximum input rates are equal to:
p
P λ .
(Eq. 7)
The elementary multiplexer 10 keeps monitoring the maximum input rate of the transport streams 62, 12. In the case where this maximum decreases within some number of consecutive measurements (system dependent) of λx(t), the elementary multiplexer lowers the input rates according to the new measured maximum value. The buffer fullness Bx(tk) for each stream is constantly measured and updated also in accordance with (Eq. 1) where px is given in (Eq. 7) or extracted from the processor 22.
As described above, the output rate R(t) is derived from the input rates and the buffer fullness. Initially, the output rate R(t0) is set equal to the sum of the maximum input rates derived in (Eq. 7) . As soon as the first packet is being serviced from any of the N=3 buffers, the elementary multiplexer 10 starts computing the output rate for the next packet to be serviced. The minimum time interval between the servicing of any two consecutive packets is P/R(tk) > (188) (8) /70Mbps=21.5μs, where P is the packet size and 70Mbps is the present output rate. Therefore, the elementary multiplexer 10 has enough time to compute and update its new output rate before the next packet is serviced. The elementary multiplexer takes into account the latency, τDDS, which is the time needed to update the output rate .
At time tk+1-T, the elementary multiplexer 10 computes the output rate, R(tk+1), for servicing the next packet, where the extra time interval T is such that P/R(t) >>T> (τDDS/τM) . Time interval T counts for the time it takes to compute R(tk+1) . The new output rate is set according to (Eq. 2) above with N=3.
As noted above, it can be shown that, regardless of the service policy, no overflow occurs when the buffer sizes are at least of size equal to NP . For N=3 in this case, no overflow should occur when the buffer sizes in the elementary multiplexer 10 are at least of size 3P.
2.2 L Packet Boundary Update
2.2.1 Single channel encoder 50
The single channel encoder 50 constantly monitors and updates the input rates . In the case a request to increase the input rates is received, the encoder 50 waits for the output rate to be updated first before effectively increasing the input rates. In the case the sum of input rates decreases, the single channel encoder 50 services N packets before it updates its output rate to avoid overflowing its buffers. In general, the number N << L where L is the number of packets serviced before the output rate is updated.
The single channel encoder 50 should be capable of updating the total number D(tk) of full packets ready to be serviced every time a full packet is received. The single channel encoder 50 reads D(tk) every time the output rate is updated at time tk. Given that it takes at least τM to read D(tk), the single channel encoder 50 should start reading D(tk)at time tk-τM.
Based on the input rates and the buffer fullness the output rate is derived in the following manner.
Initially, the output rate r(t0) is set equal to the sum of the maximum input rates. As soon as the first packet is being serviced from any of the single channel encoder 50 buffers, the single channel encoder 50 starts computing the output rate for the next packet to be serviced. Where the single channel encoder 50 does not have enough time to compute and update the new output rate before the next packet is serviced, the output rate is updated every L packets.
The single channel encoder 50 counts for the latency, τDDs/ introduced by the DDS clock to update the output rate. The minimum value of this latency may be, for example, 395 ns to reload the new DDS clock value and 18 cycles of 20 ns each for the new rate to take effect. Therefore, τDDS=395 ns .
At time tk, the single channel encoder 50 starts computing the new output rate r(tk+L) that should take effect at time tk+L. The new single channel encoder 50 output rate is set according to the relation in (Eq. 5) with N=4. For N=4 , it is known from above (Eq. 6) that D(tk)< 3 is always satisfied regardless of the service policy.
2.2.2 Elementary Multiplexer 10 The elementary multiplexer constantly monitors and updates the input rates . In the case where the sum of input rates decreases, the elementary multiplexer services four packets before it updates its output rate to avoid overflowing its buffers. In general, 4 << L. Therefore, the elementary multiplexer 10 will not overflow when the input rates are decreased.
The elementary multiplexer 10 should be capable of updating the total number of full packets D(tk) ready to be serviced every time a null packet is received. The elementary multiplexer reads D(tk) every time the output rate is updated at time tk. Given that it takes the microprocessor at least τM to read D(tk), the elementary multiplexer should start reading D(tk) at time tk-τM.
Based on the input rates and the buffer fullness, the output rate is derived in the following manner. Initially, the output rate R(t0) is set equal to the sum of the maximum input transport stream rates . As soon as the first packet is being serviced from any of the elementary multiplexer buffers, the elementary multiplexer starts computing the output rate for the next packet to be serviced. Where the elementary multiplexer 10 does not have enough time to compute and update the new output rate before the next packet is serviced, the output rate is updated every L packets.
At time tk, the elementary multiplexer starts computing the new output rate R(tk+L) that should take effect at time tk+L. The new elementary multiplexer output rate is set equal to the relation in (Eq. 5) above with N=3. As in the single channel encoder 50 case, it is known from above (Eq. 6) that D(tk) < 2 is always satisfied regardless of the service policy.
Appendix A One Buffer Case: The minimum value of the output rate at time tk that prevents overflow and null packet insertion is equal to:
rP
*('*) = B - B{tk ) + P
(Eq. 8)
where r is the maximum input rate, B(tk) is the buffer fullness at time tk and B is the buffer size.
Practical Case: R(tk)< Rmax
In this case, the N input buffers are modeled as one buffer and (Eq. 8) is used. The output rate is given by the following equation:
Λ_„
*( =- ι = N
NP - ∑ B,(tk ) + δS ι P
(Eq. 9)
where δskjl is the Dirac delta function equal to 1 if one of the buffers is serviced at time tk, otherwise it is equal to zero. As in the previous case, this case results in a zero null packet insertion, and it can be shown that the output rate cannot exceed the maximum sum of the input rates, i.e. R(tk) < Rraax. First, consider that
ι=W ι=l
(Eq. 10)
is always satisfied regardless of the service policy. This
(Eq. 10) can be shown by induction. Initially, for k=0, the sum of the buffer fullness should be less than or equal to NP, i.e.,
ι=N
∑B,{tQ)≤NP
The initial buffer fullness is B1(t0)=P. Next, assume that (Eq. 10) holds up to time k and thus at time k+1 it is satisfied. The buffer fullness at time tk+1 is given as:
B,{tk+l) = B1(tk) + al(tktk+l)-δl k P
(Eq. 11)
where δ
1/Jk is the Dirac delta function that indicates if the i
th buffer is the one that is serviced at time t
k. The number of bytes added to the i
ch buffer during the interval [t
k, t
k+1] could be upper bounded by using the maximum input rate r
1# i.e.,
(Eq. 12)
Therefore, (Eq. 11) and (Eq. 12) yields the following relation:
(Eq. 13;
however
ι=N ι=N
∑r t=R mm and∑ l k =δι S„ 1
/=! (=1
therefore the sum of the buffer fullness at time tk+1 verifies (Eq. 10) . The inequality in (Eq. 10) implies that if the output rate in updated according to (Eq. 2)
then the buffer sizes are equal to NP regardless of the service policy.
The proof of (Eq. 6) is made by considering the N buffers as one buffer and showing that the maximum size of the buffer is NP . Initially, for k=0, the sum of the buffer fullness should be less than or equal to NP, i.e., D(t0)< N. Next, assume that (Eq. 6) holds up to time k and show that at time k+s, where s=l, ...,L, it is satisfied. The number of full packets at time tk+s satisfies the following relation:
D(tk+g) = D(tk) + α(tk,tk+s) - n(tk,tk+s) (Eq. 14)
where (tk,tk+g) and n(tk,tk+s) are the number of packets added to and extracted from the buffers during the interval [tk,tk+g] , respectively. The number of packets extracted from the buffers is at most s packets. Because an upper bound to D(tk+g) is sought, assume that the buffers always include full packets in order to reach the worst case at time tk+L. Therefore, n (tk, tk+s) =s . The number of packets added to the N buffers during the interval [tktk+3] could be upper bounded by using the maximum input rate px and (Eq. 5), i.e.,
a{tk,tk+s) = 1,. , L
(Eq. 15)
Notice that the sum is taken inside the integer part rather than outside the integer part. This case is worse than splitting the buffer into N separate buffers. Therefore, (Eq. 14) and (Eq. 15) yield the following relation
D(tk+g) < D(tk)+N-D(tk)+L-1-L = N-l (Eq. 16)
While this invention has been particularly shown and described with references to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.