Número de publicación  US7817644 B2 
Tipo de publicación  Concesión 
Número de solicitud  US 10/985,927 
Fecha de publicación  19 Oct 2010 
Fecha de presentación  12 Nov 2004 
Fecha de prioridad  15 Sep 1998 
Tarifa  Pagadas 
También publicado como  CA2281399A1, CA2281399C, DE69940592D1, EP1021060A2, EP1021060A3, EP1021060B1, US6829218, US20050083946 
Número de publicación  10985927, 985927, US 7817644 B2, US 7817644B2, USB27817644, US7817644 B2, US7817644B2 
Inventores  Chien Chen, Linhai He, Albert Kaisun Wong 
Cesionario original  AlcatelLucent Usa Inc. 
Exportar cita  BiBTeX, EndNote, RefMan 
Citas de patentes (6), Clasificaciones (29), Eventos legales (5)  
Enlaces externos: USPTO, Cesión de USPTO, Espacenet  
This is a Continuation Application of prior application Ser. No 09/153,352 filed Sep. 15, 1998, now U.S. Pat. No. 6,829,218 assigned to the same assignee as that of the present invention, and to which priority is claimed.
The invention relates generally to asynchronous transfer mode (ATM) networks and, more particularly, to high speed weighted queuing systems for ATM switches.
Weighted fair queuing, (also known as packetized general processor sharing (PGPS), may be come important in the next generation of ATM switching system and routers in order to fairly share bandwidth while allowing for a guaranteed minimum delay to individual connections sharing a particular transmission facility. However, to date, the implementation of weighted fair queuing algorithms have been problematic and difficult to scale to a large number of connections. For example, in conventional weighted fair queuing, the complexity of an ideal implementation is 0(N)+0(logN) where N is the number of circuits, 0(N) represents the recalculation for all headofline packets for all circuits, and 0(logN) is the amount of calculations involved in resorting all of the reference finishing times. The next generation of ATM switches is expected to include tens of thousands of connections and operate at multigigabit rates. Accordingly, a scalable, highly efficient implementation of a weighted fair queuing algorithm is necessary.
The present invention provides improved algorithms for performing queuing in an ATM switch. In particular, the invention provides a highly efficient implementation of a weighted fair queuing algorithm in an ATM switch where the packets are of a fixed size. Although some important approximations are made in the proposed implementation, all of the properties of an ideal weighted fair queuing algorithm are preserved. The sorting algorithms in accordance with the present invention are advantageous in that it is possible to maintain appropriate servicing of connections without sorting all of the individual connections. This may be accomplished by presorting each of the individual virtual circuit connections into a finite number of predetermined bins according to a weight associated with the connection. Thereafter, only the bins need be sorted without having to sort each of the individual connections. Accordingly, the invention is suitable for implementations having transmission speeds of multiple gigabitspersecond.
The foregoing summary of the invention, as well as the following detailed description of preferred embodiments, is better understood when read in conjunction with the accompanying drawings, which are included by way of example, and not by way of limitation with regard to the claimed invention.
Referring to
Each of the individual ATM switches ATM A, ATM B, ATM C, may include an output queue for storing received cells prior to transmission on one or a plurality of output connections. Each output queue may, from time to time, become backlogged due to congestion on a particular output connection. While the circuit is backlogged, cells cannot be sent out over the circuit without loss of the cell. Accordingly, cells for each of the various connections begin to buildup in the output queue. When capacity becomes available on the output connection, a choice must be made as to which cells to transmit. This choice may be made, for example, using one or more algorithms described below.
A first exemplary embodiment of the weighted fair queuing algorithm may store a connection potential Pi for each backlogged circuit. The connection potential Pi is the amount of service that the connection has received normalized to the weight of a particular connection. For any given instant in time for which service is available, the circuit with the minimum connection potential is serviced. When a circuit newly becomes backlogged, its potential must be initialized. This may be done by keeping track of a system potential Ps that is used for the initial value of Pi for any connection that newly becomes backlogged. To maintain fairness, the system potential should always be kept within the range of the connection potentials of the backlogged connections. In one exemplary embodiment, the connection potential of the circuit which newly becomes backlogged is set to be the connection potential of all other virtual circuits within the bin which have the same connection weight. Where the bin is empty, it may be desirable to set the connection potential of the virtual circuit to be: a) the average of the connection potentials of the two bins having the closest weight to the newly nonempty bin, b) the potential of the bin having the next lowest or next highest weight. The object is to service the connection with the lowest normalized past service (i.e., potential Pi). In this manner, the all connection are maintained to within a narrow normalized past service range and the system is fair.
An additional mechanism is needed for a new connection which has just been initiated. If the connection potential is set at zero, the new connection will receive all of the service. Accordingly, a weighting mechanism is required for new connections to assign an initialization potential to a new connection which is comparable to the existing connection potentials. Thereafter, the new connection is serviced like all other connections in accordance with its normalized connection potential. A parameter Ps may be utilized by the ATM switch to keep track of where the connection potentials are and to provide a reference for initializing the potentials of newly backlogged circuits.
Using the above described method, the complexity for sorting N connections is 0(logN) plus some overhead for updating the system potential. A further improvement may be made where there are only a finite number of discrete potentials such as when one cell is transmitted, the potential varies by a discrete amount.
Referring to
In the potentialbased approach, every time a connection receives service (1 cell transmitted), its potential may be increased by 1/Wi, where Wi is its weight (i.e., the potential varies by a discrete amount). Where there are only a finite set of values for Wi, then for all connections with the same weight, their potentials are always incremented in the same fashion and intuitively, it is not necessary to keep track of the individual potentials and to sort them separately. In this manner, once a bin with the lowest potential is determined, it may be desirable to service all nonempty virtual circuits within that bin before resorting to find the bin with the new lowest potential. Where a empty virtual circuit within a bin becomes nonempty during servicing of a particular bin, it may be desirable to not service that virtual circuit until a new potential is recalculated.
For example, where there are K different weights allowed, then the ATM switch may include K bins for each output connection 12. Within each bin, a linkedlist of all the Vcs with weight Wk that are backlogged may be maintained. When a VC with weight Wk becomes newly backlogged, it joins the weight bin with weight Wk, meaning that the potential of this VC is initialized to the potential of all other VCs in this bin. This method may be more desirable (providing a tighter fairness bound) than other methods for updating the system potential. In exemplary embodiments, a potential is determined for each bin. The bins are then sorted to find the bin having the minimum potential. Whenever the kth weight bin is found to have the minimum potential, all connections within the bin may be served once. In this manner, the problem is converted from one that depends on the number of connections to one that depends on the number of different weights that are allowed, and hence the efficiency of the sorting process is greatly enhanced. Additionally, it becomes possible for a switch with a given processing power to handle a much faster output connection with substantially more virtual circuits.
Sorting of LeadBins
Where we have a plurality of weighted bins, each being associated with one of a plurality of finite weights, it may be desirable to sort the bins to determine which order the bins should be serviced.
Since Bin 1 now has the smallest potential (potential=0 which is less than ⅛), Bin 1 is serviced next with one or more cells (e.g., VCa1, VCf1) from each nonempty queue in Bin 1 being sent to the output connection 12. Thereafter the potential for Bin 1 is recalculated as the current potential (0) plus the distance (¼) such that the new potential of Bin 1=¼.
Next the potentials among all nonempty bins are resorted. Since Bin 2 has the lowest potential ⅛ versus ¼, all nonempty queues in Bin 2 may be serviced next (e.g., VCb2). Note there is no service of VCc since the queue associated with this virtual connection has become empty. Next, the new potential for Bin 2 is calculated as the current potential (⅛) plus the distance (⅛) which is equal to ¼.
Again the potential for all nonempty bins are resorted. Since Bin 1 and Bin 2 now have the same potential ¼, it may be desirable to service all nonempty queues from both Bin 1 and Bin 2 prior to recalculating and resorting. For example, cells VCa2, VCf2, and VCb3 may be sent to the output connection 12. Thereafter, the potential for Bin 1 and Bin 2 may be recalculated. For Bin 1, the new potential is the current potential ¼ plus the distance ¼ which is equal to ½. For Bin 2, the new potential is the current potential ¼ plus the distance ⅛ which is equal to ⅜. Since Bin 2 now has the lowest potential, bin 2 is again serviced by sending VCb4. Once VCb4 has been sent, Bin 2 has become empty. Thereafter, Bin 1 will be serviced until Bin 2 again becomes nonempty.
As is apparent from the above operational description, since Bin 2 has the greater weight, Bin 2 receives the greatest amount of service with the service being proportional to the weight. Additionally, even though there may be four virtual connections with different numbers of cells waiting, the weighted fair queuing algorithm only needs to sort two different potentials to determine which virtual circuit queue to service next. Thus, a highly efficient and yet fair algorithm is made possible.
Storage of Pij
In yet a further aspect of the invention, it is possible to further increase the efficiency of the algorithm by organizing the data associated with the bins in memory in a predetermined arrangement. For example, the above method requires the storage of the potential values for each of the individual bins. An alternative is to keep track of a system virtual time which is a reference potential from which all of the potentials of the bins may be offset by one or more values. In this manner, all potential values associated with the bins can then be compared as offsets to the system virtual time. The value of the offsets can be deduced from the system virtual time since the progression of the potential value for each of the bin is known. Also, any potential values for two or more bins which are tied across a row can be determined by a simple operation. This alternative method of sorting requires an efficient and accurate division of the system time t. Second t may grow without bound as long as the system stays busy. (Only when the system goes) idle can t be reset to zero. When t becomes large, the division or multiplication (if high speed division is utilized to implement the table lookup) becomes complex. Also, straightforward wraparound of t is not acceptable and hence the values may need to be reset as the system approaches wraparound condition. However, for suitable environments, the table based sorting based on offset from a system time is a highly efficient solution for sorting the bins.
Although the sorting of lead bins in a matrix arrangement may be accomplished using any suitable sorting mechanism, a two dimensional structure and algorithm may be useful. Alternatively, a three dimensional structure and algorithm may be used to store bins which have a tied potential with bins having the tied potential stored in the third dimension. For simplicity, the example given below will be shown in a two dimensional format. In this example, d may be set to the reciprocal of the weight, i.e., d=1/w. It may also be desirable to represent d by a floating point number with an nbit mantissa and mbit exponent forming the two sides of the matirx. As shown below, the bins may be arranged as a twodimensional structure where d has a two bit mantissa and a two bit exponent:
Mantissa (i) 

Exponent 
In an exemplary embodiment, a processor in the ATM switch may be configured with a memory implementing the matrix arrangement described above. Although the matrix may be accessed and maintained in any suitable manner to accomplish the above function, one technique is described below.
If k > j, then 
Pij less than or equal to Pik at all times 
Where t is the potential of the bin being serviced, after all nonempty weight bins 
with potential t are served, the system must find the next (set of nonempty 
weight bin(s)) with the minimum potential to serve. 
For row i, consider 
k = ceiling (t/1.i) (Using the shorthand notation above (1.i)) 
if k is divisible by 2^{r}, where r is an integer, then 
Pir = Pi(r−1) 
otherwise, Pir > Pi(r−1) 
Since if k is divisible by 2^{r}, it is divisible by 2^{(r−1)}, consequently: 
Pir = Pi(r−1) = Pi(r−2) = .... = Pi0. 
In exemplary embodiments in accordance with this aspect of the invention, it is not necessary to sort all of the weighted bins. For example, it is sufficient to sort the first nonempty weight bin in each row. Where there are ties, it may be desirable to serve all of the tied bins. Further, it may be desirable to determine all nonempty bins in each winning row that are tied with the winning lead bin (i.e., the first nonempty bin on that row). Finding all the bins that are tied with the lead bin in a row is on the order of one operation since all that is required is to check the weight of the next nonempty bin Bim in row i to see if it is a tie with the first nonempty bin. If it is, then we serve bin Bim and check if Bim′ (where Bim′ is the next nonempty bin) is a tie with Bim. Otherwise, we are done and sort the lead bins of all rows again. Note that where the bins are presorted, since if Bim is not tied with Bij, no subsequent bin Bik, k>m can be tied with Bij.
In exemplary embodiments of the table based sorting method described above, the first step is to represent the potential for each bin as a floating point number having a mantissa and an exponent. The next step is to organize all of the bins in a memory of the ATM switch into rows (each having the same mantissa value) and columns (each having the same exponent value. In organizing the bins, it may be desirable to organize each row such that each adjacent column has a higher potential (or equal potential for a two dimensional matrix) than the preceding column in the same row. In this manner, only the lead bins in each row need be sorted. Bins with the largest weight (smallest distance) may be organized first in the row/column with bins with the smallest weight (largest distance) organized last in the rows/columns. The next step is to sort the first nonempty bin (lead bin) of each row in terms of potential. Once the table has been sorted as defined above, the next step is to service the lead bins in each of the rows and then to serve all of the nonempty bins with the same potential as the lead bin. Thereafter, the potential for each of the serviced bins is recalculated, and the serviced bins are inserted into the table in an appropriate location. In this manner, the amount of sorting is reduced by the square root of the number of bins and the number of register loads are also decreased.
PseudoCode
Exemplary embodiments of the present invention may be performed using methods and systems as described above. In one embodiment, the methods and systems of the present invention may be described using the following pseudo code:
Start: 
system_virtual_time t = potential of bin currently being served 
Queue_To_Serve = the VC queue to be served 
L(i,j) = the linked list of all nonempty VC queues in weight bin Bij 
(We assume that a single bit comparator is used to quickly 
determine the lead bin, i.e., the first nonempty bin of each row. 
Cell Arrival: 
Put Cell into VC in Bin Bij based on weight of VC 
If VC Queue has been empty link this VC Queue to Linked List L(i,j) 
If Bin Bij is Empty 
then initialize Pij to ceiling[t/Dij]*Dij 
(In an alternate embodiment, Pij may simply be initialized to t, in 
which case it may be desirable to test for Pim >= Pij, m>j, Bim, 
Bij both nonempty to decide if we need to serve the next bin in 
a row. This should simply introduce an additional inaccuracy in 
potential initialization. As discussed above, the division t/Dij may 
grow to be problematic where the queue remains full for 
extended periods of time and the division makes the wraparound 
of t may be come more complex such that the remainder will need 
to be tracked for each wraparound operation.) 
Cell Departure: 
Send a Cell from Queue_To_Serve in weight Bin being Served Bkl 
If VC Queue becomes empty remove it from Link List of VC Queues 
in Bkl 
If VC Queue is not tail of L(k,l) 
Queue_To_Serve = next VC Queue in linked list L(k,l) 
else 
find next nonempty bin Bkm on row k 
if Pkl >=Pkm 
then {Queue_To_Serve = Head of L(k,m); 
Pkl < Pkl + Dkl; 
return } 
else { Pkl < Pkl + Dkl; 
Sort lead bin of all rows; 
QueueToServe = Head of L of the winning bin } 
While exemplary systems and methods embodying the present invention are shown by way of example, it will be understood, of course, that the invention is not limited to these embodiments. Modifications may be made by those skilled in the art, particularly in light of the foregoing teachings. For example, it will be well known in the art that a processor in each of the ATM switches in
Patente citada  Fecha de presentación  Fecha de publicación  Solicitante  Título 

US5638359 *  14 Dic 1993  10 Jun 1997  Nokia Telecommunications Oy  Method for congestion management in a frame relay network and a node in a frame relay network 
US5748614 *  6 Jun 1996  5 May 1998  Siemens Aktiengesellschaft  Method for scheduling message cells leaving an ATM node 
US5859835 *  15 Abr 1996  12 Ene 1999  The Regents Of The University Of California  Traffic scheduling system and method for packetswitched networks 
US6167030 *  20 Mar 1997  26 Dic 2000  Nokia Telecommunications, Oy  Bufferbased traffic measurement system and method for nominal bit rate (NBR) service 
US6198723 *  14 Abr 1998  6 Mar 2001  Paxonet Communications, Inc.  Asynchronous transfer mode traffic shapers 
US6338072 *  22 Jul 1998  8 Ene 2002  Bull S.A.  Device and process for dynamically controlling the allocation of resources in a data processing system 
Clasificación de EE.UU.  370/395.42, 455/101, 375/354, 370/389, 375/295, 370/352, 455/39, 370/230, 375/309, 370/395.1, 455/449, 455/91 
Clasificación internacional  H04L12/56, G01R31/08, H04L12/28 
Clasificación cooperativa  H04L49/3027, H04L47/623, H04L47/522, H04L47/6265, H04L47/6215, H04L2012/5651, H04L2012/5679, H04L47/50 
Clasificación europea  H04L12/56K, H04L49/30C, H04L47/52B, H04L47/62D2, H04L47/62C, H04L47/62G3 
Fecha  Código  Evento  Descripción 

25 Ago 2010  AS  Assignment  Owner name: ALCATELLUCENT USA INC., NEW JERSEY Free format text: MERGER;ASSIGNOR:LUCENT TECHNOLOGIES INC.;REEL/FRAME:024882/0295 Effective date: 20081101 
7 Mar 2013  AS  Assignment  Owner name: CREDIT SUISSE AG, NEW YORK Free format text: SECURITY INTEREST;ASSIGNOR:ALCATELLUCENT USA INC.;REEL/FRAME:030510/0627 Effective date: 20130130 
10 Abr 2014  FPAY  Fee payment  Year of fee payment: 4 
9 Oct 2014  AS  Assignment  Owner name: ALCATELLUCENT USA INC., NEW JERSEY Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:CREDIT SUISSE AG;REEL/FRAME:033949/0531 Effective date: 20140819 
13 Sep 2017  AS  Assignment  Owner name: PROVENANCE ASSET GROUP LLC, CONNECTICUT Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NOKIA TECHNOLOGIES OY;NOKIA SOLUTIONS AND NETWORKS BV;ALCATEL LUCENT SAS;REEL/FRAME:043877/0001 Effective date: 20170912 Owner name: NOKIA USA INC., CALIFORNIA Free format text: SECURITY INTEREST;ASSIGNORS:PROVENANCE ASSET GROUP HOLDINGS, LLC;PROVENANCE ASSET GROUP LLC;REEL/FRAME:043879/0001 Effective date: 20170913 Owner name: CORTLAND CAPITAL MARKET SERVICES, LLC, ILLINOIS Free format text: SECURITY INTEREST;ASSIGNORS:PROVENANCE ASSET GROUP HOLDINGS, LLC;PROVENANCE ASSET GROUP, LLC;REEL/FRAME:043967/0001 Effective date: 20170913 