CA2073179A1

CA2073179A1 - Router chip with quad-crossbar and hyperbar personalities

Info

Publication number: CA2073179A1
Application number: CA002073179A
Authority: CA
Inventors: John Zapisek
Original assignee: Individual
Current assignee: MasPar Computer Corp
Priority date: 1990-01-05
Filing date: 1991-02-05
Publication date: 1991-07-06
Also published as: AU4876293A; US5434977A; JPH05506526A; AU7258691A; US5450330A; EP0509058A4; AU644141B2; WO1991010198A1; US5345556A; EP0509058A1

Abstract

A router circuit (130) is made configurable as either a crossbar switch or a hyperbar switch by dividing the router chip into sections (sections A-D), where each section is associated with a group of input terminals, and wherein various grant circuits (272) within each of the sections have enable/disable input terminals (506) connected by controllable switches (400) to either enable/disable output terminals of grant circuits in an adjacent section (hyperbar personaly) or to fixed logic levels (crossbar personality). In the crossbar configuration, for each input terminal of a section, only one grant circuit (272) per channel is enabled. Each of the sections has a different set of grant circuits enabled per channel so that two sections can not access a same output terminal of the router chip. In the hyperbar configuration, the sections are made transparent, and grant circuits are enabled or disabled depending on output wire availability. Additionally, the router circuit contains novel routing address bit and protocol bit circuitry (252, 258) as well as novel diagnostic circuitry (584) for detecting broken wires (550) between router circuits.

Description

W091/10198 PCTtUS91/0009q 1- 207317~
ROUTER CHIP WITH QUAD-CROSSB~R
AND HYPERB~R PERSONALITIES

CROSS-REFERENCE TO RELATED APPLICATIONS
The following co-pending applications are assigned to t.he same assignee of the present application and are related to the present application: ~Scalable Inter Processor and Processor to I/O Messaging System for Paxallel Processin~ Arrays" by John Nickolls et al., filed lO concurrently herewith and assigned Serial No. 07/461,492;
"Parallel Processor Memory System" by David Bulfer et al., filed concurrently herewith and assigned Serial No. --07/461,567; and "Method of Wiring Router to Reduce Internal Blockage" by Stuart Taylor, filed conc~rrently 15 herewith and assigned Serial No. 07/461,572. The disclosures of these concurrently filed applications are incorporated herein by referenceO

FIELD OF THE I~VENTION
This invéntion relates to switching systems, and 20 more specifically to a switching network used in a parallel data processing system for selectively connecting a plurality of input ports to a plurality of output ports.

BACRGROUND OF THE_INVENTION
- Parallel processors have been develQped that are 25 based on the concurrent ~xecution o~ the same instruction by a large number of:relatively si~ple processor elements operating on respective data streams. These processors, - known as single instruction, multiple data (SI~D) processors, are useful in such applications as image 30 processing, signal processing, arti~icial intelligence, data base operations, and simulations.
Typically, a SIMD proGessor includes an array o~
proces~or elements and a routing networX. The routing WO91/10198 PCT/U$91/00094 network selectively switches the respéctive outputs of the processor alements to khe inputs of other processor elements or to the inputs of input/output ~I/O) devices.
A typical routing network may be required to selectively 5 connect any of 1024 input terminals to any of 1024 output terminals.
The operations of the processor elements and of the routing network are controlled by a separate control processor in response to instructions and data furnished 10 from a computer subsystem.
One switching method to selectively connect N
input terminals to N output terminals uses an N X N
crossbar switch, in which there are N X N switches, ea~h . of which are controllable to connect one of N horizontal 15 input wires to one of N vertical output wires. For a 1024 x 1024 switching requirement, this would require over one million switches within the crossbar switch. A much more .
economical and reliable switching network, which is used in a SIMD proces-~or, is described in International 20 Publication Number W0 88/05764, dated September 7, 1988, based on Application Number PCT/US88/00456, filed February 16, 1988, which names Grondalski as the inventor. This publication is incorporated by reference. In the SI~D
processor described in the Grondalski publication, an 25 array of 32 processor elements are located on a chip and the chips are grouped in pairs. Each processor element includes a processor and an associated memory. Each processor element is capable of processing data one bit at a time (bit mode) or four bits at a time (nibble mode).
A local interconnect network implements a nearest nei~hhor mechanism whereby a mesh interconnects each processor element within an array with four other processor elements within the array in order to provide rapid communications with one of the four processor 35 elements at any given time.
In the Grondalski device, global routing network implements a random transfer mechanism whereby any processor element in a processor element array can send a message to any other processor element within another or the same array. Fig. 1 shows the routing network described in the Grondalski publication. The routing 5 network comprises three stages 10, 20, and 30, where Fig. 1 shows one of four identical groups of the three stage routing network which operate in parallel. Each of the four groups of the routing network has the capability of switching 1024 processor element chip pair outputs to lO lQ24 pxocessor element chip pair inputs. In the group of switching elements shown in Fig. l, the outputs of 1024 processor element chip pairs are applied to each of 1024 input terminals of the 16 chips comprising the first ... .. .. ..
stage lO.
In the Grondalski SIMD processor, the message data generated by a processor element within a processor element chip pair is routed through the routing network of Fig. l by the use of routing address bits also generated by the processor element. The routing address bits 20 control each of the routing chips in the three stages to route the message data following the router address bits to a specific processor element chip pair input. The format for the router addres5 bits and message bits is shown in Fig. 2.
In Fig. 2, a data stream begins with a header 40 of 23 bits, namely bits (0) through (22), which identify the intended recipient processor element within the processor element chip pair, followed by the message data bits beginning with the bit (23~. The header 40 includes 30 three router control fields identified by reference numerals 42, 44, and 46, which control the routing through the switching stages lO, 20, and 30, respectively. Each switching stage lO, 20, and 30 retires one router control field, that is, it does not pass the bits in that field on 35 to the next switching stage or to the recipient processor element chips.
Each router control field begins with protocol . : . . . : :

..
-~

bit P which, when asserted, indicates that message bits follow. Special circuitry is described in the Grondalski publication to process this protocol bit P. If the protocol bit P is not received at an input terminal, the 5 switching chips ignore succeeding signals at that input te~minal during the message transmit cycle. The four RTR
ADRS bits following each protocol bit are the router address bits which are used by the router to establish a switching path through the stage. As an example, assuming l0 the first four RTR ADRS bits of header 40 outputted by a processor element connected to an input of a router chip in the first stage lO was 0000, as shown in Fig. l, these four RTR ADRS bits would connect that particular input to ~ ~ the top chip (0) of the second stage 20.
If a switching path is established through all of the three switching stages, a final protocol bit (bit 15) is received by a processor element chip pair connected to an output line o~ stage 30. In response, the receiving p_ocessor element chip pair generates an ACK
20 acknowledgment signal tbit 16), which is transmitted back over the switching path established through the routing network and received by the processor element chip pair which originated the data stream. The processor element chip pair originating the data stream then uses the 25 received AC~ signal to clear a flag associated with the processor element chip pair which transmitted the ACK
signal so that the message data may then be transferred.
After the flag is cleared, a 6-bit PROC ID
processor identification is transmitted over the switching 30 path. The first bit identifies one of the two processor element arrays in the chip pair which contains the processor element to receive the message, and the last five bits identify one of the 32 processor elements on th~
identified chip. The next bits are the message bits, 35 which are coupled by the processor element chip's internal router control circuit to the receiving processor element.

;.. . ,~ .., , .. : ,, ~.., . ..;

.. . . . . .. ...

WO91/10198 PCT/USgl/OO~9~

2~7317~

Referring back t~ Fig. 1, each of the chips the first stage lO retires the first four R~R ADRS bits ~1-4) of header 40 to select one of 16 output channels.
Each of the 16 output channels, each comprising four 5 wires, is coupled to one of 16 chips comprising the second stage 20. ~hus, as many as four inputs to a particular chip in the first stage 10 ~ay.be connected to a single chip in the second ~tage 20.
The next four RTR ADRS bits (6-9) are retired by 10 the second stage 20 to determine to Which of the 16 output channels of the second stage chip to route the remaining data stream. Each of the 16 output channels of the second stage 20 is coupled to a 16 X 16 crossbar switch, a plurality of which comprise the third stage 30, shown in 15 Fig. 1. Each crossbar switch has 16 inputs connectable to 16 outputs. Each of the output terminals of the crossbar switches is coupled to a processor element chip pair containing a total of 64 processor elements. The Grondalski crossbar switching chips are identicai to the 2Q chips used in the first and second stages except that, in the crossbar chips, three wires in each channel are disabled. Thus, in the Grondalski crossbar switching chips, three-quarters o~ the chip is effectiv21y disabled and wast~d. This inef~icient use of the router chips in 25 the third stage requires Grondalski to use four times as many chips in the third stage as are used in the first or second stages.
It would be highly desirable in the field of switching networks for a switching network to not require ~0 additional circuitry for processing a protocol bit and for each router chip comprising the 8witching network to be ~f~iciently con~igurable as either a switch having 64 inputs and 16 output channels (each channel having 4 wires each) or a crossbar switch comprising four 16 x 16 35 crossbar switches so as to not require any more chips in the third stage as is used in either the ~irst stage or the second stage.

.: : . . . :
...... ,, . . : ,, . : , - : -. .: . . ., : .. , , . .. : . .
, :.- ; , . .,, : .
, . .. ; ~ .. ,,, . ~-: ... . .

SUMMARY OF T E I~NVEMTION
The present invention, in one embodiment, is a routing network capable of simultaneously connecting 1024 input ports to 1024 output ports. The routing network is 5 a three stage system with 16 router chips in each stage, each chip having 64 input terminals and 64 output terminals. Each of the 64 input terminals of the chips in the first stage is connected to receive an output from a respective processor element chip having multiplex data lO thereon.
Each of the chips in the first and second stages have 64 output terminals arranged in 16 channels ha~ing four wires per channel.! Each of the 16 output channels from a chlp in the:first stage is coupled to a separate 15 one of 16 chips comprising the second stage. Thus, any pxocessor element coupled to a chip in the first stage can be connected to any chip in the second stage. Four router address bits within a header preceding a message ~o be transmitted control logic means within each chip of ~he "
20 first stage to select one output channel of that chip so as to couple the remainder of the data stream to the selected chip in the second stage.
I The next four routing address bits in the header are decoded by the s~cond stage to determine to which of 25 sixteen 16 x 16 cross~ar switches in the third stage the message is to be routed. (There is a total of sixty-four 16 x 16 crossbar switches in the third stage; however, the selection of a particular one of the 16 second stage router chips also implicitly designates a subset of 30 sixte~n 16 x 16 crossbar switches to which the outputs of the selected second stage router chip are connected.) The crossbar switches connect a single inp-lt terminal ko a single output terminal. The 16 crossbar switching chips in the third stage used in the preferred embodiment are 35 identical to the chips used in the f irst and second stages, but are each configured prior to use to operate as four 16 x 16 crossbar switches. The novel cixcuitry , : ::.: : . . : ,, . ;: , . ..

207317~
internal to each router chip enables each chip to be configurable as a crossbar switch or a hyperbar SWitCh (i.e., a switch having output channels of multiple wires) in such a way that the same number of routing chips are 5 used in the third stage as is used in either the first stage or second stage.
The four remaining router address bits of the header select a single one of the 16 output terminals of each l6 X 16 crossbar switch. A further novel feature of lO the router chip is that the chips do not require any additional circuitry to process a protocol bit, which identifies the existence or a~sence of a message following the router address bits. This novel feature is accomplished by novel router address decoding circuitry in l~ the router chip which enables the protocol bit to be transmitted as a fifth router address bit so as to disable or not disable, as appropriate, the only remaining output channel selected by the preceding ~our router address bits.

20 BRIEF pESCRIPTION OF ~HE DRAWINGS
Fig. l shows a prior art routing network for a parallel processing sys~em. I
Fig. 2 shows a data stream used in the prior art parallel processing systems for routing a message through 25 the router network of Fig. l.
Fig. 3 is a basic block diagram of the routing ne~wor~ in accordance wikh the preferred embodiment of the invention.
Fig. 4 shows in greater detail the three stages 30 of the routing networX, and their interconnections, in accordance with the preferred e~bodiment of the invention.
Fig. 5 is a basic blocX diagram showing the internal circuitry of each of the routing chips in accordance with the preferred embodiment o~ the invention.
Fig. 6 shows in greater detail the internal circuitry o~ a single router chip in accordance with the WO91/10198 PC~/US91/00094 preferred embodiment o~ the invention.
Fig. 7 shows the ~ormat of the data stream containing routing and message data in accordance with the preferred embodiment of the invention.
~ig. 8 illustrates the operation o~ the address decoding circuitry in accordance with the preferred embodiment of the invention. . -:
Fig. 9 shows the internal circuitry o~ the granting circuitry used in the switching array of each 10 router chip in accordance with the present invention.
Fig. lO.illustrates the personality switches within each router chip used to convert a router chip from a hyperbar chip to a crossbar chip in accordance with the preferred embodiment of the invention.
Figs. lla-lld show the pre~erred circuitry for the personality switches illustrated in Fig. lo.
Fig. 12 shows the internal circuitry associated with the diagnostic testing of the router chip in accordance with the pre~erred embodiment of the invention.
Figs. 13-25 show actual circuitry use in the prefèrred embodiment of the router chip as follows: :
Figs. 13a and 13b, taken together, show circuitry within th~ router chip for generating the various control signals used to control the operation of 25 the router chip.
Fig. ~3c shows more detail of the router chip input latch pad (latpad), hown in blocX form in Fig. 13a.
Fig. 13d shows more detail of the router chip input buff~r pad (bufpad), shown in block ~orm in 30 Fig. 13a.
Fig. 13e shows more detail of the router chip control latch (clat), shown in block form in Fig. 13a.
Fig. 13f shows more detail o~ the router chip clock generator ~cgen2), shown in block form in Fig. 13b.
Fig. 14 shows circ,uitry for thP router chip input/output (I/0) pad (lopad) including a strong pull-up/down CMOS device and a weak pull-up/down CMOS device.

", :, ~ .,. ~ : .. . -: . . . . : . : . ................ :: :: ., . : , .
:, ' ',. : : :: :, ' :':, ': :

WO91t10198 PCT/US91/00094 Fig. 15 shows circuitry for the router chip I/O
buffer (iobuf) for providing signals from the router chip to an associated I/O pad. A hysteresis circuit is also shown coupling an input signal from the I/O pad to the S router chip.
Fig. 16 shows circuitry for the router chip input transceiver showing detail of the pipeline lakch and other circuitry associated with processing an input signal, Fig~ 17 shows timing diagrams ~or the clocX and data signals for operating the pipeline latch shown in Fig. 16.
Fig. l8 shows circuitry for the router chip input transceiver (ix) for even input bits having 15 associated input terminals on a first side of the router chip.
Fig. 19 shows circuitry for the router chip input transceiver for odd input bits having associated input terminals on a second side of the router chip.
~ig. 20 shows circuitry fox the router chip array buffer (abuf) ~or passing input signals to the data-in and data-out lines of the switch array internal to the router chip.
Fig. 21 shows circuitry for the router chip 25 address decoder latch (aIat) for processing routing address bits and for controlling the grant circuits within the switch array to request connection between the data~in lines and the data-out lines.
Fig. 22 shows circuitry in the router chip 30 address decoder cell (adcd) and shows grant circuits connected to the address decoder latch of Fig. 2l.
Fig. 23 shows circuitry for the router chip arbitration daisy chain (arb) used in the switching circuitry showing internal circuitry of the grant 35 circuits.
Fig. 24 shows circuitry for the router chip output transceivers (ox) connected to the data-out lines . ,: . .............. ;.. ; .. .. : " . ~. , ~ ;.
- . ............. ,: , ........... ........ . .
~ :,;: : : .: , ...................... . , , ,~:
. . . . -~ . . . .. .

wo 91~10198 PCr/US91/00094 2073i79 of Fig O 2 3 .
Fig. 25 shows circuitry for the router chip guadoutput transceivers for an entire channel comprising four output wires.

Overview of Preferred Embodiment o~_Router Network In the pref erred embodiment of the invention, a three-stage routing network is used to provide a selectable communications path between the output 10 terminals-of any o~ 1024 processor element (PE) clusters and the input terminals o~ any of the 1024 PE clusters, or the input terminals of any of up to 1024 input/output (I/O) device cluster equivalents.~`A plurality of identical router chips comprise each o~ the three stages, 15 wherein each router chip is fabricated using double-metal, N-well epitaxial CMOS technology, with CMOS switch devices having 1.6 micron channel lengths, 250 ~ gate oxide thicknesses, and polysilicon gates. The die size of each router chip is approximately 302 x 318 mils. Each router 20 chip has 117,624 logical (154,692 physical) transistors and is packaged in a 164-lead plastic quad flat-pack. The router chip will be described down to its logic gate level `~
with reference to the various figures.
Fig. 3 is a diagram showing the preferred 25 embodiment router networX 48 connected so as to provide a selectable co~munications path between the output terminals 50 of any of the 1024 PE clusters 100 and the input terminals 52 o~ any of the 1024 PE clusters 100, or the input terminals of any o~ 1024 I/O device cluster 30 equivalents 102.
In the preferred em~odiment, there is provided a parallel array of 16,384 processor elements 104 divided up into 1024 PE clusters 100 of 16 processor elements each.
A 16:1 multiplexer 106 is provided at the output of each 35 PE cluster 100 for outputting messages from each processor element 104 within PE cluster 100, and a 1:16 multiplexer , ,,. : .., !, , ,::: .:: : , :,.: , ,,, , ', ., '. : ,:. : ,'' :, ' ,.,,. ': ' ::.

wos~ 1s8 PcT/US91/~0094 207317~
112 is provided at the input of each PE cluster 100 for inputting messages to a selected one of the 16 processor elements 104 within PE cluster 100.
Each PE cluster 100 has a message originating 5 line 113 coupling the PE cluster 100 to a bidirectional first stage pipeline latch 116 located within each router chip and associated with a single input terminal of the router chip. Each pipeline latch 116 is a master-slave equivalent type flip-f lop which passes an input signal o applied to the pipeline latch 116 to its output on the next clock pulse. The purpose o~ these pipeline latches 116 is twofold. A ~irst purpose of the pipeline latches 116 is to segment the electrical connection between itsel~
~nd an identical pipeline latch located at the input of 15 the next stage. This acts to reduce the distortion of a signal being routed through the router network 48 by lowering the equivalent resistance, capacitance, and inductance of the circuit from what it would be if a completed circuit extended from the output of a PE cluster 20 100, through the router network 48, and to the input of a PE clust~r 100. A second purpose of the pipeline latches 116 is to store an initial state during diagnostic testing of the router network 48 for transmission over the wires between the router network stages to identify any ~roken 25 wires between the stages. This diagnostics operation will be discussed more fully with reference to Fig. 12.
~ here are 16 identical router chips in the first stage 120 of routar network 48, with each chip having 64 input te~minals a~d 64 output t~rminals. The outputs of 30 the 64 pipeline latches 116 o~ each chip in the first stage 120 are applied to a decoder/switch array within the first stage 120 for selectively coupling an input terminal to one of 16 output channels. The internal circuitry of each o~ these chips will be described later.
The router chips in the first stage 120 and in the second stage 126 are con~igured to act as hyperbar switches, where the 64 input terminal~ of each chip are - :

,, , . . ~; . , . ,; : : ,...

WO91~10198 PCT/US91/00094 2073~7~ - 12 - .
selectively coupled to any of 16 output channels of four wires each. Each of the 16 output channels of a single one of the 16 router chips in the first stage 120 is routed to a separate one of 16 router chips in the second 5 stage 126. Thus, each of the 16 router chips in the first stage 120 is connected to each of the 16 router chips in the second stage 126 by four wires. The router chips O~
the second stage 126 similarly selectively coupl~ their 64 input terminals to any of 16 output channels of four wires lO each. Each output channel is connected to a separate one of 16 router chips comprising the third stage 130. The router chips o~ the third stage 13 0 are each configured to act as four 16 X 16 crossbar switches, which will be discussed in detail with reference to Fig. 10. ~ ~
If a message is to be routed to an I/O device instead of a PE: cluster 100, third stage 130 is di~;abled, and selected crossbar router chips 131 ( f our per I /0 ~oard~ associated with the 1024 I/O device cluster equivalents 102 are enabled. The message is then routed 20 by one of the selected crossbar routing chips 131 to a particular one of 1024 I/O device cluster equivalPnts 102 located on one of 16 I/O boards.
Shown in Fig. 4 is a more detailed diagram of the connectio~s ~etween router chip stages 12 0, 12 6, and 25 13 0 . As shown in Fig. 4, 64 PE clusters, each cluster having an originating line 0~, are contained on a single board. On this-single board also resides a first stage router chip (one of chips lO-lF) having 64 input terminals coupled to the 64 originating lines OL of the 64 PE
30 clusters and a third stage router chip (one o~ chips 30-3F) having 64 output terminals coupled to the 64 target lines TL leading to the inputs of the 64 PE clusters. The 64 PE clusters are contained within 32 PE chips, with ~wo ` PE clusters per chip.
Also shown in Fig. 4, are second stage router chips 20-2F, each having 64 input terminals and 64 output t~rminals, connected to the ~irst and third stage ruuting 20~3~79 - 13 - ~
chips, respectively, as shown. Each second stage router chip 20-2F is located on one of the i6 boards containing 64 PE clusters, a first stage router chip, and a third stage router chip.
In Fig. 4, the ~irst stage router chip 10 mounted on PE board no. 0 has 64 output pins arranged in 16 channels of four wires each. The 64 wires emerge from the first stage routPr chip 10 to contact 64 wires of a ~irst inter-board router bus 150 of 1,024 wires.
10 Similarly, each of the remaining 15 first stage router chips ll-lF mounted on PE board nos. 1-15, respectively, are coupled to this first inter-board router bus 150 of l,024 wires. These first inter-board router bus wires 150 are arranged to couple the 102-4 output terminals of the -15 first stage router chips lO-lF to the 1024 input terminals of the 16 second stage router chips 20-2F, as shown in Fig. 4.
The 1024 output terminals of the 16 second stage router chips 20-2F mounted on PE board nos. 0-15, 20 respectively, are coupled to a second inter-hoard router bus 152, having 1,024 wires. These 1,024 wires of the second inter-board router bus 152 connect the outputs of the second sta~e router chlps 20 2F to the respective inputs of the third stage router chips 30-3F, as shown in 25 Fig. 4. The wires comprising inter-board buses 150 and 152 are formed on a printed circuit backplane on which the 16 PE boards are mounted.
Also indicatad in Fig. 4 is the tapping off of each output channel of the 16 second stage router chips 30 for coupling to IJ0 third stage router chips 131. There are 16 I/0 boards, each board containing four I/0 third stage router chips 131. Each of the 16 second stage router chips 20-2F has a first group of four ~hannels t256 wires total from second stage 126) connected to 35 respective inputs of four I/o third stage router chips ~31 located on a first I/o board. A similar connection o~ the first group of four channels per each of the 16 second , , : : : : :: :
s .,: . . . , . . :, ~-., :.,. . ~ . . : : , .
. - . . : . . :,.. . . .

-,. , . : ~ : . :: : , :

WO91/10198 PCT/US9l/00094 2~73179 stage router chips 20-2F is made to second, third, and fourth I/O boards. Thus, I/O de~ice cluster equivalents located on each of the ~irst through fourth I/O boards can be connected to receive the same data stream. Only one 5 set of I/O third stage router chips 131 on either the first, second, third, or fourth I/O boards is enabled so that I~O device cluster equivalents located on the enabled I/0 board may receive a data stream from a PE cluster.
A second group of four channels ~rom each second 10 stage router chip is similarly connected to respective inputs of four I/0 third stage router chips 131 located on fifth, sixth, seventh, and eighth I/0 boards. A third and fourth group of four channels is similarly connected to respective inputs of I/O third stage router chips 131 15 located on ninth through sixteenth I/o boards.
This routing to the I/o devices 102 is accomplished by extending the 1024 wires in the second inter-hoard router bus 152 for coupling to the various inp~ts of the I/o third stage router chips 131. A four-20 wire control bus is used for enabling either the third stage router chips 30-3F associated with the PE board nosO 0-15 or one of the four sets of third stage router chips 131 associated with each group of four channels from the second stage 126.
Thus, the interconnections o~ the various router chips comprising the ~irst, second, and third stages of router chips have been shown. The internal circuitry and operation of each of the router chips comprising the first, second, and third stages will now be explained.
Descri~tion of Internal Circuitry and O~eration of Router_ChiPs Although each of the router chips comprising the first, second, and third stages of router chips shown in Figs. 3 and 4 have identical structures, the router chips 35 comprising the third stage are configured, using internal switchi~g elements, to act as ~our 16x16 cross-bar switches, while the router chips comprising the ~irst and " "
:. . ' ', , ' ~ ''' ' '" ; , ' WO91/101~8 PCT/US91/00094 second stages are configured to operate as hyperbar switches, previously described, connecting 64 input terminals to one of 16 output channels.
Fig. 5 illustrates the general structure of each 5 of the router chips in the f irst and second stages as configured in their hyperbar mode of operation.
Each of the router chips has 64 input terminals shown as input terminals 00-63. These input terminals are arran~ed vertically along opposite sides of the router 10 chip. Each of the input terminals 00-63 is connectPd to an as~ociated one of 64 bus transceivers, shown collectively as bus transceivers 160. Each transceiver 160 basically comprises two tristate cMOS drivers, where one or the other driver is enabled depending~on whether 15 transceiver 160 is to receive a signal from input terminals 00-63 or transmit a signal to input terminals 00-63. The output of each of the 64 bus transceivers 160 is coupled to an input of an associated bidirectional pipeline latch, shown collectively as pipeline latches 20 164, which retains the incoming digital bit for a single clock pulse to limit distortion o~ the incoming signal, as previously described with respect to Fig. 3.
The output of each pipeline latch 164 is coupled to an associated input disable latch and an assoriated 25 bidirectional driver, shown collectively as input disable lat~hesldrivers 166. The input disable latches are loade~
during a diagnostic test, which identifies broken wires between boards and, in this capacity, store the contents of an associated pipeline latch 164 to enable or disable 30 the routing address decoding section o~ the router chip.
This feature will be discussed later with respect to Fig. 12.
Assuming the diagnostic test indicates that the particular wire coupled to an input terminal is 35 functional, the output state of the input-disable latch enables the decoder/switch array 180 to receive router . address bits to select one of 16 output channels o~ ~our .

~- ~. . , . . ; : . . -; :.::

wires each. Once such a channel is selected, the data stream then passes through bidirectional tristate drivers 166, and an attempt is made to couple the data stream to a wire within a selected channel. The particular wire of 5 the channel selected is then coupled to an associated tristate bus transceiver, shown collectively as bus transceivers 186. The nominal output of each bus transceiver is coupled to an associated output terminal 190 of the router chip.
Output disable latches 192 are used during diagnostic testing, to be discussed later, and are bypassed via 2:1 multiplexer 193 during a routing operation.`
Clock gates 196, controlied by a periodic clock 15 signal, control the switching of the pipeline latches 164, the decoder/switch array 180, and the output disable latches 192.
Fig. 6 shows in more detail the internal circuitry of the hyperbar router chip o~ Fig. 5, where one 20 input terminal 200 of the 64 input terminals of a router chip is shown along with its associated circuitry.
Further shown in Fig. 6 i5 the channel address decoder section 202 within the decode/switch array associated with input terminal 200, along with the internal cixcuitry 25 coupled between input terminal 200 and channel address decoder section 202. Also shown is the internal rircuitry of one channel within decoding section 202 and the output circuitry coupled between one channel within decoding section 202 and associated output terminals 292-295. The 30 internal channel circuitry is only shown for the first two rows (rows 00 and 01) of the chip, each row being associated with an input terminal. Each of the remainder of the 64 rows (02-3F) in the channel is connected to an adjacent row in the same way row 01 is connected to row : 35 o0. All 16 ahannels are identical, with an address-out line 206 o~ ane channel being coupled to an address in line 208 o~ the next channel in that row.

~ .
.. . , ;; , . , ' ' ~ ~ . .; . ~, .. ..

- 17 - 2~ 731 79 The following discussior~ o how route~ address bits applied to input terminal 200 cause a connection to be made between an input terminal and an output terminal assume the diagnostics test performed prior to the 5 operation of the router network resulted in an indication that all wires are functional.
Four routing address.bits are required to ~e first applied to input terminal 200 in order to select one of the 16 channels in the channel address decoder section 10 202.
Prior to a routing address bit being applied to input terminal 200, tristate buffer 226 (to be described later) and tristate buffer 240 are disabled by an external control signal so that a signal is transmitted in one -15 direction only in the router chip. All enabling/disablingcontrol signal~ are generated by an array control unit, and all rout2r chips within a stage receive identical control signals.
A hysteresis circuit 221 couples an incoming 20 signal applied to input terminal 200 to a first input o~
pipeline latch 224. Hysteresis circuit 221 is a bista~le device which regenerates at its output a logical O or logical'l signal applied to its input and also raises or lowers its internal threshold level based on its last 25 received 6ignal so that the output of hysteresis circuit 221 is only switched by a subsequent logical 1 or logical O signal applied to input tarminal 200 and not by random noîse on ~he line. For example, a logical o signal appli~d to input terminal 200 will raise the threshold 30 voltage so that only a logica} 1 signal applied ~o input : terminal 2~0 will cause a change in state o~ the output of hysteresis circuit 221. Conversely, a logical 1 signal - applied to input terminal 200 will lower the threshold ~: voltage of hysteresis circuit 221.
: 35 To prevent input terminal 200 ~rom floating when :. buf~er 226 is disabled and when no signal is being applied to input ter~inal 200, weak latch circuit 227 is used .. . . . ... . . ............. ... .. . .

. , ., . ,: . ..

,, : " ' ' : :. , . ., : ' :: ' ' ' .:, ' ' ,. : :, ' , : : ' .:: ': ' :

.:': : ' ' , :: . .: :. ' 2 07 3 i7 9 - 18 -(equivalent circuit shown in dashed line) to weakly maintain the voltage on input terminal 200 at its last voltage level. Floating of an input terminal is undesirable since it may cause both the N and P channel 5 transistors in the associated CMOS circuitry to turn on and draw a static current. To prevent this floating, the output of hysteresis circuit 221 is connected to the input of inverter 228, and the output of inverter 228 is connected to the input of inverter 229. The output of lO inverter 229 is coupled to the input of hysteresis circuit 221 via a high ohmage resistance Rl (e.g., lOK ohms~ to create a feedback circuit. Resistance Rl is supplied by an MOS device having a short channel width. This feedback circuit keeps the voltage at the input of hysteresis 15 circuit 221, and at input tarminal 200, at either a high voltage or a low voltage depending on the state of the last signal applied to input terminal 200. The weak feedback signal will easily be switched by a next signal applied to input terminal 200.
A weaX latch is also incorporated into the output circuitry (to be described later~ of each router chip. Since anly one weak latch need be associated with a single inter-board router bus, only the weak latches at the outputs of the second and third stage router chips are 25 enabled, while the weak latches at the inputs and outputs of the first stage router chips are both enabled.
Enabling of the weak latches, such as weak latch 227, is performed by enabling inverter 229.
During power-up initialization, a logical O is 30 àpplied to the first input of pipeline latch 224 to initialize the state of pipeline latch 224. Routing address bits are then applied to input terminal 200.
The ~irst router bit applied to input terminal - 200, such as router address bit no. O shown in the data 35 stream of Fig. 7, enters the ~irst input of pipeline latch 224 and is held ~or one clock pulse within pipeline latch 224. The clocking for the first input terminaI of :
-:-: . . . , ; , . . .. .

- 19 - 2~73~79 pipeline latch 224 is controlled by a clock pulse applied to terminal CLK1.

On the next clock pulse, the output o~ pipeline latch 224 is applied to input disable latch 230 and to one 5 input of AND gate 232. Assuming the previously conducted diagnostic test indicated the wire connected to input terminal 200 is functional, input disable latch 230 would have an output of logical 0. This logical 0 output of ~atch 230 is inverted at a second input of AND gate 232, 10 and, thus, a logical 1 is effectively applied to that input. This logical 0 output of input disable latch 230 is frozen after the diagnostic test by the cessation of clo¢k signals applied to input disable latch 230 from the array control unit (not shown) and thus remains at logical 15 o despite a subsequent change in the level of the output of pipeline latch 224. Since the output o~ pipeline latch 224 is applied to the oth~r input o~ AND gate 232, the output of AND gate 232 will be a logical 1 if the output of pipeline latch 224 is a logical 1.
Further details of the ~locXing and control of pipeline latch 224 are discussed later with respect to Figs. 16 and 17.
Let usl assume the output of AND gate 232 is coupled to the address-in line 208 applied to the input of 25 row 01 of the channel address decoder 202. Row 01 comprises 16 channels, each channel having four output wires A, B, C and D. Only channel 15 of channels 15-0 is : shown in detail in Fig. 6. Row 00 is associated with an ~; input terminal directly below input terminal 200 and is 30 the highest priority row, as will be seen.
In the channel selecting mode j switch 250, which is a simple transmission gate, is toggled to position ~ 50 as to bypass inverter 252. The position of switch 250 is controlled by the state of decoder latch 2 62, which i5 35 initially set so as to apply a logical 1 output on line 264. The operation of decoder latch 262 will be described - below. As pxeviously stated, a ~irst routing address bit , .

, : . . ~: ; :, ,., . . ,,; : , . , , , . .. .,, , ;
: , , - .:: , . . . .. : , , .~ : ., :

- : ' ~ . , ' ' ' '. .. ' . ., . , ,' ;'. . , , , ' . !
,; ' ' ., ' '" ' , ' .; ' ' ' ' ', ~, , ," ' ,,, 2~73179 ~ 20 -outputted ~rom AND gate 232 is applied to the address-in terminal 208 of row 01. since switch 250 bypasses inverter 252, the lev~l of this first address bit outputted by AND gate 232 is inverted by downstream 5 inverter 258.
The output of inverter 258 is inverted by inv~rter 259 and applied to one input of ~ND gate 260.
The output of AND gate 260 is applied to the input of decoder latch 262. As previously stated, decoder latch 10 262 is` initially set prior to receiving the first router bi' so as to apply a logical 1 output on line 264. Since the logical 1 output on line 264 is also coupled to an input of AND gate 260, AND gate 260 will pass the signal ~~ ~at~the-output of inverter 259 to the input of decoder 15 latch 262.
A logical 0 input into decoder latch 262 resets the output of decoder latch 262 to a logical 0. Since this logical 0 output of decoder latch 262 is applied to an input of AND gate 260, the output of AND gate 260, and 20 thus the output of decoder latch 262, is prevented from being cha~ged to a logical 1 by ~ubsequent routing address bits applied to address-in line 208 once the decoder latch 262 is reset.
The output of decoder latch 262 is applied to 25 the grant request input 270 of a first of four grant circuits 272-275 associated with that particular row and channel. A logical 1 output of decoder latch 262 signals to grant circuit 272 that the channel is to remain enabled for possibl~ selection. A logical 0 output of decoder ; 30 latch 262 signals that the channel is to be disabled for .".` this row as it will not be selected by the four routing address bits.
: After the ~irst routing address bit has been decoded by decoder latch 262, if the decoder latch is set 35 to a logical 0, switch 250 i5 toggled to position A so as not to bypass inverter 252~ Thus, th~ routing address bit applied to address-in line 208 will not be inverted at the . . .,: :
. ~ - . : ,: , : .. . .
- , ;. , . ~
' ' ` ', . ' ' ; ; ' ' ` ~' ' . ~ ~, ' , W091/10198 PCT/US9l/00094 output ~f inverter 258.
Each of the 16 channels uses internal circuitry identical to that shown in Fig. 6, wherein the address-out line 206 coupled to the output of inverter 258 is applied 5 to the addxess-in terminal of the next channel. Thus, initially, with switch 250 in posltion B bypassing inverter 252, the first router.address bit applied to address-in terminal 2p8 will be i~verted when applied to the address-in terminal of the next channel (i.e., 10 channel 14). Since all 16 columns are ide~ical, each having a switch identical to switch 250 in position B, the decoder latches in channels 15-0 will have outputs which are of an opposite state as the decoder latch output in an adjacent channel. This is ciearly illustrated in Fig. 8, ~~
15 where the states of the decoder latches are shown in channels 15-0.
Fig. 8 shows all 16 decoder latches 262 in a single row of 16 channels being initially set to a logical 1 state. Assuming a first router address bit outputted 20 from AND gate 232 in Fig. 6 is a logical 0 signal, the decoder latches 262 in channels 15-0 will have alternate output states as shown in Fig. 8 after the first routing address bit has been ret~red by the row. The channels : associated with the decoder latches having a logical 0 25 output are disabled due to a logical 0 being applied to the grant request terminal of the first grant circuit in each of the non-selected channels. The decoder ; latche~ having an output of a logi¢al 0 control associated switches, identical to switch 250 in ~ig. 6, to switch to '~0 position A so that the remaining routing address bits are not inverted by those channels which were not selectedO
Assuming the second router addrass bit is a lo~ical 1, th~a decoder latche~ in channels 14, 10, 6, and 2 would have a logical 1 output and those channels would 35 remain ~nabled.
Assuming the third router address bit i~ a logical 0, as shown in Fig. 8, only channels 10 and 2 will . - : : :: ,; . : ., .: :: ; :: -WO91/10198 PCT/U~1/0~9~
2~3179 - 22 -remain enabled.
Upon propagation through the row of a fourth rout~r address bit o~ a logical 1, only channel 19 will remain enabled to connect the particular input terminal 5 ~or that row to one of the ~our output wires A, B, C and D
associated ~ith the selected channel.
Referring back to Fig. 6, after a single channel in a row is selecte*, a logical 1 signal frsm the decoder latch associated with the selected channel will be applied 10 to the grant requ~st input of the first grant circui~
associated with the selected channel. Since there are only four wires per channel, no more than four of the 64 input terminals of the router chip may use the same channeI~at the same time. Those input terminals which 15 could not be connected to one of the four wires of a selected channel will not be connected in the present message cycle and will have to try again during the next message cycle.
: To determine the priority of which input 20 terminal gets connected to one of the four wires within a selected channel so as to connect an input terminal of a router chip to an output terminal, the below-described circuitry is used.
Each of the grant circuits 272-275 in row 01 of ;. 5 Fig. 6 controls associated switches 280-283 to connect data-in line 288, associated with a particular input terminal, to one of four data output lines A, B, C and D, each associated with a particular output terminal 292-295, respectiv~ly. Each o~ switches 280-283 comprise a P
30 channel transistor 284 and an N channel transistox 285 : connected in parallel, as shown within the dashed box : proximate to switch 283. A grant accept control signal generated by an associated grant circuit is applied to the control terminal of P channel transistor 284 while an 35 inverted grant accept control signal ~i.e., a grant accept control signal) is applied to the control terminal of N
channel transistor 285. As seen, a logical o grant accept .
- . , . ; ,. , , ,:
; . . .
- . , .

WO91/10198 PCTtUS91/00094 ~073179 signal causes an associat~d switch 280-283 to couple a vertical data-out line (e.g., output line A, B, C, or D) to a horizontal data-in line (e.g., data-in line 288).
Bus transceivers 300-303 are coupled between 5 data output lines A, B, C, and D and ou~put terminals 292-295, respectively. Each bus transceiver 300-303 comprises a multiplexer 310, a tristate buffer 314, a weak latch 318, a hysteresis circuit 322, and a strong pull-up/down tristate buffer 326, connected as shown in 10 Fig. 6. The construction of weak latch 318 and hysteresis circuit 322 is similar to the construction of weak latch 227 and hysteresis circuit 221, previously described.
While the router chip is in the forward ~ransmission mode, where a signal is to be communicated from input terminal 15 200 to any of the output terminals 292-295, tristate buffer 314 is disabled, while multiplexer 310 is controlled to regenerate a signal on the ~arious data-output lines A-D, and buffer 326 and weak latch 318 are : enabled for all output terminals.
Output disable latch 368 is used in diagnostic testing to test for broken wires between chips. Output ; disable latch 368 has an input terminal coupled an output -;
of hysteresis circuit 322 and ~as an output terminal coupled to one input o~ multiplexer 310 and to a grant-in 25 terminal of an associated gran$ circuit. The operation of ~ output disable latch 368 will be discussed later with : respect to Fig. 12.
When there is both a logical 1 signal applied to : the grant-in terminal 310 of grant circuit 272 and a 30 logical 1 signal applied to the grant request terminal 270 ~: of grant circuit 272, grant circuit 272 controls its : associated switch 280 to couple the data-in line 288 to ~ the vertical data output line A so as to connect the input :: terminal associated with row 01 ~o output terminal 292.
35 The logic means incorporated into each of the gr~nt : circuits prevents all other lower priority grant circuits (i.e., those grant circuits in rows 02-63) within the .. . , ,.. : , ,: :, . . : , : ,: :
. , . :: : . . , , ,.,: ,::, ,,: ; , . , , :
,- :. ~ . , : . . . ................. : . :. :

~. . , .. . . : ,:, :,: . :-:::: ,: . :

WO91~10198 PCT/U591/00094 2~73179 router chip associated with that particular output wire from also switching a data-in line to that particular output wire.
Fig~ 9 shows the logic means within each of the 5 grant circuits used in the router chips. The grant circuit logic means shown in Fig. 9 exhibits the ~ollowing truth table.
TABLE I
Grant Grant Grant lO Grant-In Request Accept Grant-Out Request' O (e~ublcd) O (c~ec~O (m~o con-I (di~lc I (no rvquc~t e~quc~~cclicn) ~iebcr for De~l ~ nnt r~v-) tircuit to m~lce COODCC~Cn) ;^' 15 1 (di~blo) I (aoc~ectic:~ I (doo~m~o O ¦ I O ~doDd di~-ble hi~her n7~-) 20 1 o I I o~
nl c~uit to m~ke cmcc~ion) .

In the logic m~ans of Fig. 9, the grant reguest 25 terminal 340 o~ a grant circuit, such as grant circuit 272 in Fig. 6, is coupled to the input of inverter 342 and ~o one input of NAND gate 344. The grant input terminal 346 of the grant circuit is coupled to the input of inverter 348 and to one input of N~ND gate 350, whose output is 30 coupled to a grant circuit associated with the next wire in the channel.
The outputs of inverters 342 and 348 are applied to the inputs of NAND gate 352, whose output i5 used to toggle a switch, such as switch 280 in Fig. 6, to connect 35 a data in line to a data-out line associated with a ::
particular output terminal of the router chip.
The output of inverter 348 and the signal applied to the grant request terminal 340 is coupled to the inputs o~ NAND gate 344, whose output at grant-out 40 ter~inal 354 is applied to the grant-in terminals o~ the grant circuit in the next higher (i.e., lower priority) , ''-, ,'''-' "' , , '" f` ~
, . :, : ; :. .
- , , row~
As seen from Table I, i~ the g~ant-in signal and gxant reguest signal are both logical 0, the grant is accepted (i.e., grant accept = O~, and the grant circuit 5 controls a switch, such as switch 2~0, to make a con~ection between an input terminal and an output terminal of the router chip. Since the grant is accepted, the grant request will not be passed on to the next grant circuit in the channel (i.e., grant request~ = 1). Also, lO since the grant is accepted, the grant-out signal on . terminal 354 in Fig. 9 will be a logical 1, disabling all : grant latches in the higher rows associated with that output wire.
Also as seen in Table I, i~ a grant-in signal .
15 into the grant circuit is a logical 1, the grant circuit i is disabled and will output a logical 1 grant-out signal, disabling the grant circuits in the higher rows associated with that output wire.
` A logical O gran~-out signal applied to the 20 grant-in terminal 346 indic:ates that no grant circuit in the lower rows associated with that wire has accepted a grant, and thus the grant circuit remains available to - accept a grant upon receiving a logical o gIant request signal.
If a grant circuit is disabled, it will pass a ~`~ logical 0 gIant r~uest~ signal to the next grant circuit in that row. If all grant latches in that row are :~
disabled, a connection will not be made, and the PE
cluster will have to wait for the next message cycle to 3~ request a connection.
Fig. 6 shows the bottom row 00 associated with a bottom input terminal o~ a routing chip, wherein this bottom input te~minal is given the highest priority due to its position. To the grant-in terminals of each o~ the 35 grant circuits in this bottom row (~our o~ which are ,, , : : , ., , . .. , , :, .
' ' ' , . i . , .,. ., :' , ,, , ; ', ', ,,' !: . ' : ', , , :', : . : ' ',: , ': ' ~ '. ', ': ' ' , ': ' , . : . . .':' ': ' :
: ' ' : '~ ' "', . ,' , ~ , ' . ' ' ': ' 'i ' ' ' - ' . " . ,' "' ' ' i: ' ' ',' ' :'' , ',' identified as grant circuits 360-363) is applied a logical o grant-in signal from associated output disable latches (four of which are identified as latches 368-37l) whose outputs remain constant for the entire message cycle.
5 Thus, the grant circuits in the bottom row OO have the highest priority, since they can be disabled only if the output o~ an output disable latch is a logical l ~ i~
signal in response to the detection of a broken wire connected to the output terminal associated with that lO output disable latch.
Thus, as seen, this granting technique is used . to enable a maximum of four input terminals $o be .: connected to the ~ame output channel. To prevent the higher priority input terminals from accessing higher 15 priority input terminals in subsequent stages, the inter-board wires between the ~irst and second stages shown in Fig. 4 are arranged so that the output terminals asso~iated with higher priority wires are coupled:to the lower priority input terminals in the next stage of router - 20 chips.
The timing o~ the router address bits is such as : to allow the various grant circuits in all the channels to switch prior to a next router addr2ss bit being received by the decoder latches. The data stream format of Fig. 7 25 shows bit positions 5, 6, 12, and 13 empty to al~ow the various grant latches in a router chip to settle after the channels are selected.
After the four router address bits, which select one of 16 channels in each row, are retired by the router 30 stage, a protocol bit, shown as protocol bit PB in Fig. 7, i5 transmitted to indicate the presence or absence of a message existing at the end of the router address header.
In the prior art, the presence or absence of a message is indicated by the value o~ a protocol bit whic~
35 precedes the routing address bits. In these prior art methods, additional circuitry is provided within the router chip for the detection of thi~ protocol bit and to . ..

. . : -- . ~

- 27 - 2 0 73 ~ 79 enable or inhibit the decoding means within the router chip as appropriate.
In the present invention, no additional circuitry is necessary for the handling of this protocol : 5 bit which follows the four routing address bits.
After the four address bits have been decoded, and only ona decoder latch per row remains set to provide a logical l input into the grant request terminal of a grant circuit, the reception of the protocol bit causes lO the remaining set decoder latch to either be cleared (i.e., output a logical 0) or remain set. I~ it remains set, this indicates the presence of a message, and the decoder latch will maintain its request for a connection between the row's data-in wlre and one of the output wires 15 in the selected channel. For example, if the protocol bit : is a logical 1 and only decoder latch 262 in Fig. 6 is set, back-to-bac~ inverters 258 and 259 will cause a logical l signal to be applied to the input of decoder latch 262, and the decoder latch 262 will thus continue to 20 provide an enabling logical l signal to grant circuit 272.
~- A~ter this protocol bit is sent and the :::
designated switches (e.g., switch 280 in Fig. 6) have connected the various data-in lines to the appropriate vertical data-out lines, tristate buffer driver 240 25 coupled to the output of pipeline latch 224, shown in Fig. 6, is enabled, and the remaining data stream is coupled ~rom the output of tristate buffer 240 to its associated data-in line 288. Extern~l control means are used to count the four router address bits and the 30 protocol bit so as to only enable tristate buffer ~river 240 after all the switches (e.y., switch 280~ in the router chip are properly set~ No further cloc~ pulses are sent to the decoder latches (e.g., decoder latch 262~
from the array control unit (not shown), so the outpu~ of 35 the decoder latches are frozen.
Tristate bu~fer 326 within the output bus transceivers (~our of which are identified as 300-303) is " , . . . . ..

WO91/10198 PCT/US91/0009~

enabled at this time for transmitting the data stream to the next stage or to the receiving PE cluster. Tristate buffer 326 comprises a CMOS driver having transistors with wide channels so a5 to ensure the logical signals at the 5 output terminals of the xouting chips will be of a full high or full low v91tage and Of sufficient current to drive circuitry coupled to the output terminals. Thus, the remainder of the data stream is applied to the appropriate output terminal of the router chip.
The remaining router address bits in the data stream are then applied to the selected input terminals of a router chip of the next stage of router chips to select a channel coupled to a selected one of the 16 router chips ~ in the third stage. Thus, four router address bits plus a 15 protocol bit are retired by the first stage of router chips, and four router address bits plus a protocol bit are retired by the second stage of router chips. The third stage, comprises 16 router chips, each conflgured to act as four 16 x 15 crossbar chips for connecting an input 20 terminal to a specific output terminal of the crossbar chip. This is determined by the resolution o~ four additional router address bits plus a protocol bit. The configuration o~ the crossbar switch in the router chips of the third stage will now be described.

25 Configuration of Cross-Bar Routinq Chi~
Fiy. 10 shows the pre~erred circuitry within the decoder/switch array of a routiny chip for configuring a routing chip to act as a hyperbar chip or a crossbar chip. Grant ircuits for only one channel are shown;
30 however, the same circuitry exists for all 16 channels within the router chip. Shown in Fig. 10 are various switches 400 connecting the grant-in terminals of grant circuits in the lowest rows of sections A, B, C and D
within ~he router chip to either a fixed logic level 35 (crossbar personality) or to the grant-out terminals o~
grant circuits in the highest rows of sections A, B, and C

, . " - :, . . ;: - - . :.

- . , , . : :." ~, . ;:.: , .:

207317~

(or to the outputs of output disable latches 404A-404D in the case of the lowest row of grant circuits in section A) (hyp~rbar personality). ~y toggling switches 400 to the right, the router chip is configured from a hyperbar chip 5 to a crossbar chip.
. The 64 row~ of grant circuits are arranged in four sections A, B, C, and D of 16 rows per section, ~ach row being associated with a single input terminal of the ~ router chip. The diffexence between the crossbar :~ lO personality o~ the router chip and the hyperbar personality described with respect to Fig. 6 is that, in the crossbar personality mode, only one out of the f our output wires of each channel is anabled for each o~
sections A,~B, C, and D, with each section A, B, C, and D ------15 having a different one of the four wires enabled. ~hus, ;~
in each of the four sections A, B, C, and D, only one wire -per channel is available for connection to one of the 16 input terminals associated with that section. :.
This crossbar personality con~iguration is :.
20 accomplished by switching the grant-in terminals of three of the four bottom grant circuits associated with each channel in sections A, B, C, and D to a logical 0 signal - (i.e., grant-in = l in Table I) to disable these three grant circuits and the remaining grant circuits associated 25 with the same output wir~s in that section. The grant-in terminals of the bottom grant circuits associated with the remaining wire in the channels are coupled to a logical l signal (i.e., grant-ln 0 in Table I) to enable those grant circuits and the remaining grant circuits associated with 30 that wire in that section.
Thus, in the single channel explicitly shown in Fig. lO, only the grant circuits in section D associated with output wire D and output terminal 410 will be enabl~d for switching to an input terminal connected to section D
35 in the crossbar personality router chip. Similarly, in section C, only grant circuits associated with wire C and output terminal 412 will be enabled. In section B, only . ' ' , . "' ""' ~: ; ' ~' . ' . ,, " ' ' " :','"' ' ' ':''""' ' ". ' ;" .'.. ' ' ' . ' ", : . . ' ~73~ 9 - 30 - ~
grant circuits associated with wire B and output terminal 414 will be enabled. And, in section A only grant circuits associated with wire A and output terminal 416 will be enabled.
In each of the other 15 channels in sections A, B, C, and D, only grant circuits associated with one wire are enabled in the same way as the channel illustrated in ' ~ig~ 10.
` In the router chip's crossbar personality, each lO of the sections A, B, C, and D can only access 16 of the 64 output terminals of the crossbar chip, where each of ;the ou~put terminals is associaked with a particular one of the four sections A, B, C and D. Thus, by using the crossbar router chips in the third stage, a specific 15 output terminal of a third stage router chip connected to the input of a PE cluster may be selected for connection to an input terminal of th~ third stage router chip.
: Shown in Figs. lla-lld are circuit diagrams of the switches 400, shown in Fig. 10, connected between the 20 various sections A, B, C, and D.
Fig. lla illustrates the switches coupling the four output disable latches 404A-404D in FigO lO to the grant-in terminals of the four bottom grant circuits in `~
section A. The same circuitry is used for the remaining 25 15 channels in the router chip. N channel transistor 436 in Fig. lla is shown having a control terminal coupled to crossbar enable wire 438 and having terminals connected between grant-in terminal 439 of the bottom grant circuit associated with output wire A in Fig. lO and groundO Also 30 shown in Fig. lla is transmission gate 440, where a control terminal of a P channel transistor is shown coupled to crossbar enable wire 438, and a contxol terminal of an N channel transistor is coupled .

to c~ossbar enable wire 441. Transmission gate 440 is 35 coupled between input wire 442 and output wire 443, which is coupled to grant-in terminal 439. Thus, when crossbar enable wire 438 is at a logical l level, transistor 436 .

will couple output wire 443 (output wire 443 conducts the grant-in signal discussed with respect to Table I) to ground, while transmission gate 440 will be non-conductive and isolates input wire 442 from output wire 443.
5 Consequently, when crossbar enable wire 438 is at a :~ logical 1 level, the grant-ln signal on output line 443 will be a logical 0 and, as seen in Table I, causes the grant circuits in section A associated with output terminal 416 to be enabled.
: 10 It is to be noted that the signals to which switches 400 in Fig. lo are coupled are grant-in signals, which are the inverse o~ the gIant-ln signals on output -- lines 443, 447, 448, and 449 in Fig. lla.
The remainder of the switches 444-446 in 15 Fig. lla are P channel types having control terminals connected to crossbar enable wire 441 and having terminals connected between output wires 447-449, respectively, and supply voltage Vcc. Transmission gates 450-452 connect output wires 447-449 to input wires 453-455, respectively.
Consequently, when cross~ar enable wire 441 is at a logical o lev~l, P channel transistors 444-446 couple output wires 447-449 to a high voltage. This high level (representing the grant-in signal~ is coupled to grant-in terminals 460-462 of the three remaining grant circuits in 25 the bottom row of section A in Fig. lO so as to disable these grant circuits in accordance with the truth table of Table I.
Thus, as shown, a logical l crossbar enable signal on wire 438 will enable only one grant circuit in 30 section A per channel and will disable the remaining three grant circuits in the channel. Conversely, a logical 0 signal on cross~ar enable wire 438 will render non-conductive all transistors 436, 444, 445, and 446 in Fig. lla and render conductive all transmission gates 440, 35 450, 451, and 452 so as to connect input lines 442, 453, 454, and 455 to output lines 443, 447, 448, and 449, WO 91tlO198 PCT/Us91/00094 - 2~73179 - 32 -respectively. In this sti~te, the router chip will be in its hyperbar personality, as discussed wi*h respect to Fig. 6.
The switch array of Fig. llb coupled betwe~n 5 sections A and B in Fig. 10 is constructed so that N
channel transistor ~80 is associated with grant-in terminal 482 in section B. As seen from Fig. 10, this will cause grant circuits in section B associated with output wire B to be enabled in the crossbar personality . 10 mode. P channel transistors 484-486 are associated with grant-in terminals 487-489, respectively, shown in Fig. lo so as to render the grant circuits associated with output wires A, C, and D disabled in the crossbar personality mode .
The switch array of Fig. llc coupled between sections B and ~ in Fig~ lo is constructed so that N
channel transistor 492 is associated with grant-in terminal 494 in section c. As seen from Fig. lo, this will cause grant circuits in section C associated with 20 output wire C to be ena~l2d in the crossbar personality mode. P ~hannel transistors 496-498 are associated with grant-in terminals 499-531, respectively, shown in Fig. lo so as to render the grant circuits associated with output ' wires A, B, and D disabled in the crossbar personality 25 mode.
The switch array of Fig. lld coupled between sections C and D in Fig. 10 is constructed so that N
channel transistor 504 is associated with grant-in terminal 506 in sec~ion D. As seen ~rom Fig. 10, thisi 30 will cause grant circuits in section D associated with output wire D to-be enabled in the crossbar personality mode. P channel transistors 508-510 are associated with grant-in terminals 511-513, respecti~ely, shown in Fig. lo so as to render the grant circuits associated with output 35 wires A, B, and C disabled in the crossbar personality mode, By using the con~iguration of the crossbar ,. . ; . :: . .
,, . , ":, . .. . . .... .. . . . . ... .. .

WO91/10198 PCT~US91/00094 2~3~79 router chip shown in Fig. lO, the entire chip is u~ilized and, consequently, only 16 crossbar router chips are necessary in the third stage to provide connections to 1024 output terminals.
` 5 Not shown in Fig. 10 for simplicity are the various bus transceiYers, pipeline latches, input-disable latches, and drivers previously discussed with re~erence to Figs. 5 and 6.

Description of Operation After Connection is Made After twelve routing address bi~s and three protocol bits have been retired by the three stage routing network, a PE cluster coupled to an input of a first stage router chip is connected to a PE cluster coupled to an output terminal o~ a third stage router chip.
Referring to Fig. 7, bits 19 to 22 are then used to address one of the 16 processor elements within the accessed PE cluster u~ing a simple four bit decode means within each PE cluster. A protocol bit PB is then sent.
~t this time a connection is made between a 20 sending processor element and a receiving processor element, and the connection is frozen by control signals generated by the array control unit (not shown). The message bits are now transmitted through the routing network. A parity bit PY followed by an O bit (to tes~
25 for stuck-at-;1 wires) is first sent ~or testing the accuracy of the succeeding transmitted message and the quality of the connection.
After the ~ull message is sent, the addressed processor element is controlled to send an acknowledgment 30 ACK signal, consisting o~ a O bit followed by a 1 bit, back through the routing network to sig~al the origina~ing processor element that the message has been received~ In ordex to transmit this ACK signal back through the routing network, each stage must enable the appropriate tristate 35 buffers in all the bus transceivers (two such bu~ers shown in Fig. 6 are buffers 314 and 226), and clock - : ., .,: . , :. . : ~ ; . : : ,.. : , , WO91/~0198 PCT/US91/00094 ~3i~9 34 -signals must be applied to the CLK2 inputs of the ~arious pipeline latches (one such pipeline latch is shown in Fig. 6 as pipeline latch 224), in order to pass the ACK
signal through the pipeline latches. At the same time, 5 all buffers used to transmit the routing bits and message bits (e.g., buffers 240 and 326) are disabled.
The routing network arrangement described herein enables two of the routing address bits for the third stage to be redundant with two of the routing address bits 10 retired by the first stage. This is made possible by the third stage crossbar router chips being divided up into four 16 x 16 crossbar switches and the outputs of the second stage chips being applied to one of 16 x 16 crossbar switches instead of being applied to any of 64 15 inputs o~ a routing chip.

Circuitry Used in Router ChiPs for Diaqnostic Testinq Shown in Fig. 12 is the router chip broken wire diagnostic circuit which is used between the first and second stages and between the second and third stages of 20 the router network. Shown in Fig. 12 is backplane int2r-board wire 550, which connects an output terminal 552 from a router chip of a previous stage to an input terminal 554 of a next stage. Between each of the stages there are 1,024 backplane inter-board wires 550. Connected to 25 output terminal 552 is bus transceiver 558, comprising tristate CMOS buffer 560 (shown as buffer 326 in Fig. 6), hysteresis circuit 562 (hysteresis circuit 322 in Fig. 6~, multiplexer 563 ~multiplexer 310 in Fig. 6), and other circuitry described with respect to bus transceiver 300 30 shown in Fig. 6. An output disable latch 570 is connected so as to receive an output signal from hysteresis circuit 562 and output a latch signal through multiplexer 563 to an input terminal o~ tristate buffer 560. An output of disable latch 570 is coupled to a grant-in 35 terminal of an associated grant circuit.
As discussed with respect to Fig. 6, input - ~ . .. , ; , : . , 2~7~79 terminal 554 of a router chip i~ connected to various buffers, designated in Fig. 12 as bus ~ransceiver 574 comprising tristate buffer 580 (shown as buffer 226 in Fig. 6) and hysteresis circuit 578 (hysteresis circui~ 2Z1 5 in Fig. 6) for simplicity. The output of hysterasis circuit 578 is connected to the one i~put o~ pipeline latch 584, and the output of pipeline latch 584 is connected to the input of input disable latch 588. Also connected to the input of disable latch 538 is the input 10 of tristate bu~fer 580. Pull-up resistors 596 and 598, connecting the input and output terminals 554 and 552, respectively, to the supply voltage Vcc, serve to pull the input and output terminals to a high state when these terminals-are not being driven to a low state. These 15 pull-up resistors 596 and 598 are actually transistors forming part of the weak latch circuits 227 and 318 discussed with respect to Fig. 5.
, ~he operation of the diagnostic circuit o~
Fig. 12 is as ~ollows. The diagnostic routine to detec~
20 broken wires in the backplane inter-board bus is implemented each time the parallel processing system containing the router network is turned on. Prior to the diagnostic program being implemented, pipeline latch 584 for each of the input terminals on the router chips 25 comprising the second and third stages is loaded with a logical 0 by a serial scan input applied to a second input terminal o~ pipeline latch 584 and clocXed in with a separate clock signal. During the first diagnose phase, voltage Vcc is applied to the pull-up xesistors 596 and 30 598, and tristate buffer 560 is disabled, while tristate buffer 580 is enabled. (Hysteresis circuits 562 and 578 are always acti~e.) Thus, in this configuration, the low output of pipeline latch 584 is transmitted via tristate bu~fer 580 over backplane wire 550 and through hysteresis 35 circuit 562 and sets output disable latch 570 to have a logical 0 output on line 600.
Durin~ the second diagnose phase, vcc xemains . : :, ;. . ,, . : , ,..... ,, ,, . .: ~ .

WO 91t10198 PCI-/US91/0009q 2~731~9 applied to pull-up resistors 596 and 598, and tristate bu~fer 580 is disabled, while tristate buffPr 560 is enabled. Thus, in this phase, the logical O output of output disable latch 570 is transmitted back through S backplane wir~ 550 to load a loyical O into pipeline latch 584. On the next clock pulse, this logical O is loaded into input disable latch 588 (since pipeline latch 584 delays propagation between its input and its output by one clock pulse). Thus, a logical O state of 10 input disable latch 588 indicates that the backplane wire 550 coupled between terminals 552 and 554 is functional.
As seen with respect to Fig. 6, a logical 1 signal outputted by an input disable latch (such as input disable latch 230 in Fig. 6) prevents any router address 15 bits from entering channel address decoder 202.
I~ backplane wire 550 was broken, then in the second diagnose phase, output disable latch 570 would not have pulled input terminal 554 down to a low state, since there would be no coupling between output disable 20 latch 570 and input terminal 554. In this case, pull-up resistor 596 would have driven input terminal 554 high and loaded a logical 1 into pipeline latch 584 and into input disable latch 588, thus indicating that backplane wire 550 is an open circuit. Further, if ~he output of hysteresis 25 circuit 5~2 was not pulled down to a logi;:al O ~uring the first diagnostic phase, output disable latch 570 would have been loaded with a logical 1, since pull-up resistor 598 would pull up output terminal 552 i~ terminal 552 was not driven low. As shown in Figs. 6 and 10, this 30 logic~ i.e., grant-in = 1) output of output disable latch 570 is coupled.to the grant-in terminal of a bottom grant circuit as50ciated with that output terminal. This logical 1 grant-in signal applied to the grant-in terminals, as described with respect to Table I, disables 35 all grant circuits in that router chip asso~iated with the faulty wire so as to prevent any connection to that faulty wire.

-.. :. : , : :.,, :, ~ . ,. . . . :: :
.. , ,: : : , , , , , ; . . , .

:, . :: - :: : ~ ,: : . .... :

wosl/1ols8 PCT/US91/00094 ~ 37 ~ 2~73179 Following the various diagnose phases, the contents of the pipeline latches 584 associated with each input terminal of the router chips in the s~cond and third stages are serially scanned using the same circuitry which 5 initially loaded the pipeline latc~es 58q with a logical 0. This scan identifies the existence of a broken wire between the input and output terminals of adjacent stages so that corrective maintenance can be conducted.
Note that there is no way to disable (and thereby bypass3 l0 an output shorted to ground since a grounded backplane wire 550 would also result in a final pipeline latch state of a logical 0.

Ac~ual CircuitrY Used to Im~lement Above-Described Router Chi~
Figs. 13-25 illustrate the actual circuitry used in the preferred embodiment router chip. One of ordinary skill in the art upon reviewing ~he circuitry of Figs. 13-25 would understand the interconnections of the circuitry and the control signals necessary to operate the 2G circuitry. The operation and equivalent circuitry o~ the ~arious circuits shown in Figs. 14-~5 have been pre~iously discussed with respect to Figs. 3-12, and further discussion of the detailed operation of the various circuits and components shown in Figs. 14-25 would be 25 largely redundant and unnecessarily detailed.
Figs. 13a-13f show the control circuitry within the router chip used to generate the clocking and control signals for operating the various latches and switches shown in the figures.
Some explanation of the clocking of the circuitry in the router chip and of the pipeline latch 654 shown in Fig. 16 is appropriate.
The router chip uses two-phase non-overlapping clocXs Cl (phase l) and C2 (phase 2), shown in Fig. 17.
35 Cl uses short pulses and is used to sample data at the inputs of latches. C2 uses longer pulses and is used to ., . . ....................... , : ;

- , ~ , ... .... : . ,,.. :.

WO91~10198 PCT/US91/00094 2073~ ~

clock all the various latch outputs at the same time. The changing latch outputs propagate through combinatorial logic through the chip in accordance with C2. Clock Cl always runs but is gated for the various latches to cause 5 the various latches to sample input data only in designated cycles.
In many of the circuits (e.g., Figs. 16, 21, 22, 24-26), complementary clock signals are used. This i5 preferred because o~ the CMOs implementation. Only the lO positive`clock signals are shown in the timing diagram of Fig. 17 however.
In the timing diagram of Fig. 17, Cl is shown as a free-running clock. In a cycle during which a gated Cl ~ clock is ena~led, the enabled Cl clock looks like a copy 15 of ~l for that cycle.
With respect to the operation of the pipeline latch 650 circuitry shown in Fi~. 16, forward-message data is presented to the transparent latch on line DF and is clocked into the latch by line ClF (and, in this CMOS
20 implementation, ClF-bar). Similarly, reverse-message data on line DR is clocked by line ClR, and serial-scan data on line DS is clocked by line ClS. -~
In any cycle, only one of ClF, ClR, nd ClS may - be active, or all three may be inactive. This is 25 controlled by circuitry on the chip which is in turn ; controlled by the external array control unit. During a cycle in which one of ClF, ClR, and ClS is active, if the data sampled (clocked) into the latch is of a polarity opposite to the latch's output, then the output will 30 change on the next rising edge of C2. If none of ClF, ClR, and ClS is active, then the latch's output will not change in that cycle.
Fig. 24 shows the circuitry for the output disable latch 368 and the 2:l multiplexer 310 shown in 35 block diagram form in Fig. 6.

", , ~,~ ", "" , " , " " , , , Wo 91/10198 PCr/US91/00094 SummarY and Concluslon A novel router chip has been des ribed which possesses numerous advantages over prior art router chips.
The specific embodiments herein described are 5 only illustrative of the invention. Other embodiments using the ahove-described concepts are also intended to be encompassed by the invention. Various other changes in structure may additionally occur to those skilled in the art, and all of these changes are to be understood as 10 forming a part of the in~ention insofar as they fall within the true spirit and scope of the f ollowing claims .

... . .. . . .. . ...

- .:

Claims

What is claimed is:

1. A router circuit having address decoder means for processing routing address bits and processing a protocol bit, said router circuit comprising:
first router circuit input terminal;
first routing address bit decoder means coupled to said input terminal, said first routing address bit decoder means having a first decoder latch means coupled to receive a routing address bit of a first or second logic level and a protocol bit of a first or second logic level, said first decoder latch means for disabling a first connecting means within said router circuit upon receipt of a routing address bit or a protocol bit of a first logic level, said first connecting means coupled to said first routing address bit decoder means and controlled by said first routing address bit decoder means to couple a signal applied to said input terminal to an output terminal of said router circuit.

2. The router circuit of Claim 1 wherein said first routing address bit decoder means comprises:
a first routing address bit decoder means input terminal associated with said input terminal of said router circuit;
first inverter means having an input coupled to said first routing address bit decoder means input terminal and having an output terminal;
first switching means coupled between said input and output terminals of said first inverter for bypassing said first inverter means;
second inverter means having an input terminal coupled to said first switching means so as to be either coupled to an output of said first inverter or to said input of said first inverter means, depending on a position of said first switching means;
first logic gate means having a first input terminal coupled to an output of said second inverter means and having a second input terminal and an output terminal;
first decoder latch means having an output terminal coupled to said second input terminal of said first logic means and having an input terminal coupled to said output terminal of said first logic gate means, said output terminal of said first decoder latch being coupled to said first switching means f or controlling said first switching means, said output terminal of said first decoder latch being also coupled to a first grant circuit in said first connecting means for controlling said first grant circuit to connect a data input line, associated with said first router circuit input terminal, to a data output line, associated with one of said router circuit output terminals.

3. The router circuit of Claim 2 further comprising:
input buffer means having an input terminal coupled to said first roster circuit input terminal for passing routing address bit signals applied to said first router circuit input terminal to an output terminal of said input buffer means, said output terminal of said input buffer means being coupled to said first decoder means input terminal, said input buffer means comprising:
a hysteresis circuit having an input coupled to said input buffer means input terminal and having an output terminal;
a pipeline latch having an input terminal connected to said output terminal of said hysteresis circuit, said pipeline latch having an output terminal;
data input signal buffer means having an input terminal coupled to said output terminal of said pipeline latch, said data input signal buffer means having an output terminal coupled to said data input line; and routing address bit buffer means having an input terminal coupled to said output terminal of said pipeline latch, said routing address bit buffer means having an output terminal coupled to said first routing address bit decoder means input terminal.

4. The router circuit of Claim 2 further comprising:
a second routing address bit decoder means coupled to a first output terminal of said first routing address bit decoder means, said second routing address bit decoder means having a second decoder latch means coupled to receive a routing address bit of a first or second logic level and a protocol bit of a first or second logic level, said second decoder latch means for disabling a second connecting means within said router circuit upon receipt of a routing address bit or a protocol bit of a first logic level, said second connecting means being coupled to said second routing address bit decoder means and controlled by said second routing address bit decoder means to couple said signal applied to said first router circuit input terminal to an output terminal of said router circuit.

5. The router circuit of Claim 4 wherein said first decoder latch means has an output terminal coupled to an input of a first grant circuit, said first grant circuit for connecting said first router circuit input terminal to an output wire associated with said first grant circuit, and wherein said second decoder latch has an output terminal coupled to an input of a second grant circuit, said second grant circuit for connecting said first router circuit input terminal to an output wire associated with said second grant circuit.

6. The router circuit of Claim 4 wherein said routing address bit and said protocol bit are generated by a first circuit which generates a specific pattern of routing address bits in order to communicate with a second circuit.

7. The router circuit of Claim 6 wherein said protocol bit is generated by said first circuit after said first circuit selects a single channel in said router circuit, said protocol bit causing said selected single channel to be enabled or disabled depending on the logic level of said protocol bit.

8. The router circuit of Claim 4 wherein said second routing address bit decoder means comprises:
third inverter means having an input coupled to said output of said second inverter means and having an output terminal;
second switching means coupled between said input and output terminals of said third inverter for bypassing said third inverter means;

fourth inverter means having an input terminal coupled to said second switching means so as to be either coupled to an output of said third inverter or to said input of said third inverter, depending on a position of said second switching means;
second logic gate means having a first input terminal coupled to an output of said fourth inverter means and having a second input terminal and an output terminal;
second decoder latch means having an output terminal coupled to said second input terminal of said second logic means and having an input terminal coupled to said output terminal of said second logic gate means, said output terminal of said second decoder latch being coupled to said second switching means for controlling said second switching means, said output terminal of said second decoder latch being also coupled to a second grant circuit in said second connecting means for controlling said second grant circuit to connect said data input line, associated with said first router circuit input terminal, to a data output line, associated with one of said router circuit output terminals.

9. The router circuit of Claim 8 wherein said routing address bit and said protocol bit are applied to said first router circuit input terminal, and wherein the logic level of said routing address bit and said protocol bit is not inverted by said first routing address bit decoder means before being applied to said second routing address bit decoder means if said first connecting means is disabled, and wherein said logic level is inverted by said first routing address bit decoder means before being applied to said second routing address bit decoder means if said first connecting means is enabled.

10. The router circuit of Claim 8 wherein said first and second logic gate means is an AND gate.

11. The router circuit of Claim 8 further comprising a fifth inverter means coupled between said output of said second inverter means and said first input terminal of said first logic means, and also comprising a sixth inverter means coupled between said output of said fourth inverter means and said first input terminal of said second logic means.

12. The router circuit of Claim 2 wherein said first connecting means further comprises a plurality of grant circuits associated with a first channel of said router circuit and said first router circuit input terminal, said plurality of grant circuits being interconnected such that a grant enable signal generated by said first decoder latch means will control only one of said plurality of grant circuits associated with said first channel to connect said first router circuit input terminal to one of a plurality of output wires associated with said first channel.

13. The router circuit of Claim 12 wherein each of said plurality of grant circuits is connected to an associated grant circuit associated with a second router circuit input terminal such that, if one of said plurality of grant circuits associated with said first router circuit input terminal is selected to connect said first router circuit input terminal to an associated output wire within said first channel, an associated grant circuit associated with said second router circuit input channel will be disabled so as not to be selectable to connect said second router circuit input terminal to said associated output wire within said first channel.

14. A router circuit having the ability to either connect a router circuit input terminal to one of a plurality of output channels in a hyperbar mode, where each output channel is associated with a plurality of router circuit output terminals, or connect a router circuit input terminal to a specific one of a plurality of router circuit output terminals in a crossbar mode, said router circuit comprising:
a plurality of sections, each section being associated with a plurality of said router circuit input terminals and comprising routing address hit decoder means and connecting means associated with each of said associated router circuit input terminals, each of said routing address bit decoder means being coupled to an associated one of said router circuit input terminals for decoding routing address bits applied to said router circuit input terminal and outputting enabling/disabling control signals for controlling said connecting means, said connecting means comprising a plurality of grant circuits for connecting a data input line, coupled to an associated one of said router circuit input terminals, to one of aid router circuit output terminals, said grant circuits having grant request terminals coupled to said output terminal of an associated one of said decoder means, said grant circuits being arranged in rows and columns, each row being associated with one of said router circuit input terminals, each column being associated with a single router circuit output terminal, each of said grant circuits having an enable/disable input terminal and an enable/disable output terminal, each of said grant circuits within the same section and associated with the same router circuit output terminal having a respective enable/disable input terminal coupled to an enable/disable output terminal of a grant circuit in an adjacent row, except for grant circuits in a terminal row of a section adjacent to another section; and hyperbar personality/crossbar personality switching means connected between adjacent ones of said sections of said router circuit for coupling enable/disable input terminals of grant circuits in a terminal row of an associated section to either enable/disable output terminals of a terminal row of grant circuits in an adjacent section or to fixed logic levels.

15. The router circuit of Claim 14 wherein said output terminal of said decoder means is directly coupled to a grant request terminal of a first grant circuit in an associated channel, and said output terminal of said decoder means is coupled through logic means to grant request terminals of additional grant circuits associated with said associated channel.

16. The router circuit of Claim 14 wherein said fixed logic levels are of such levels as to enable only one column of grant circuits per channel in each of said sections, each of said sections having different columns of grant circuits enabled in a crossbar personality mode of operation.

17. The router circuit of Claim 14 wherein an enable/disable output terminal of a first grant circuit, associated with a certain output wire, provides a first logic state signal to an enable/disable input terminal of a second grant circuit, also associated with said certain output wire, if said first grant circuit is controlled to connect said router circuit input terminal to said certain output wire, said first logic state signal acting to disable said second grant circuit.

18. The router circuit of Claim 17 wherein said first grant circuit and said second grant circuit are within a single one of said sections.

19. The router circuit of Claim 18 wherein said first grant circuit has its enable/disable input terminal coupled to said hyperbar personality/crossbar personality switching means so that when said hyperbar personality/crossbar personality switching means applies a fixed logic state signal of said first logic state to said enable/disable input terminal of said first grant circuit, said first grant circuit and said second grant circuit will be disabled.

20. The router circuit of Claim 19 wherein when said hyperbar personality/crossbar personality switching means applies a fixed logic state signal of a second logic state to said enable/disable input terminal of said first grant circuit, said first grant circuit will be enabled.

21. The router circuit of Claim 20 wherein said hyperbar personality/crossbar personality switching means applies fixed logic state signals to enable/disable input terminals of grant circuits associated with a single channel in a section so as to enable only one column of grant circuits per channel in each of said sections, each of said sections having different columns of grant circuits enabled.

22. The router circuit of Claim 14 wherein said hyperbar personality/crossbar personality switching means is controlled by crossbar enable signals, and wherein said crossbar enable signals control said hyperbar personality/
crossbar personality switching means so as to disable all except one column of grant circuits per channel for each of said sections, each section having different columns of grant circuits not disabled by said hyperbar personality/
crossbar personality switching means when said router circuit is switched to a crossbar personality mode.

23. The router circuit of Claim 22 wherein each of said grant circuits in a single column are constructed so that signals outputted at said enable/disable output terminals of said grant circuits allow only one grant circuit in said single column to connect a router circuit input terminal to a single data output wire associated with said single column.

24. A diagnostic circuit for testing for and detecting nonfunctional wires, said diagnostic circuit comprising:
an output terminal coupled to one end of a wire to be tested;
an input terminal connected to another end of said wire to be tested;
output terminal pull-up means coupled to said output terminal;
transceiver means coupled to said output terminal, said transceiver means for receiving a signal from said output terminal and transmitting a signal to said output terminal;
first latch means having an input terminal coupled to said transceiver means for receiving a signal from said output terminal and having an output terminal coupled to said transceiver means for transmitting an output signal from said first latch means to said output terminal;
input terminal pull-up means coupled to said input terminal;
second transceiver means coupled to said input terminal, said second transceiver means for receiving a signal from said input terminal and transmitting a signal to said input terminal; and second latch means having an input terminal coupled to said second transceiver means for receiving a signal from said input terminal and having an output terminal coupled to said second transceiver means for transmitting an output signal from said second latch means to said input terminal, an output signal at said output terminal of said second latch means being used to identify the functionality of said wire after testing.

25. The diagnostic circuit of Claim 24 further comprising:
third latch means having an input terminal connected to said output terminal of said second latch means, a signal at an output terminal of said third latch being used to enable or disable additional circuitry.

26. The diagnostic circuit of Claim 24 wherein said output terminal pull-up means is a first transistor means connected between said output terminal and a voltage source, and wherein said input terminal pull-up means is a second transistor means connected between said input terminal and said voltage source.

27. The diagnostic circuit of Claim 26, wherein said input terminal pull-up means also acts as part of a weak latch means coupled to said input terminal to prevent undesired floating of said input terminal.

28. The diagnostic circuit of Claim 24 wherein said second latch means has an additional input terminal for receiving an externally generated signal for initializing an output state of said second latch means prior to detecting said functionality of said wire.

29. An integrated circuit with improved circuitry for preventing undesired floating of input terminals, said improvement comprising:
a weak latch circuit coupled to an input terminal of said integrated circuit, said weak latch circuit comprising:
first inverter means having an input terminal coupled to said input terminal of said integrated circuit, said first inverter means having an output terminal;
second inverter means having an input terminal coupled to said output terminal of said first inverter means, said second inverter having an output terminal coupled to said input terminal of said integrated circuit, an output of said second inverter providing a high output resistance so as to weakly latch said input terminal of said integrated circuit.

30. The integrated circuit of Claim 29 wherein said second inverter means comprises a CMOS device with P-channel and N-channel transistors having channel widths and lengths so as to provide said second inverter means with a relatively high output resistance.

31. The integrated circuit of Claim 29 wherein said weak latch circuit also acts as a weak pull-up means in a diagnostic circuit, said diagnostic circuit for detecting broken wires coupled between said input terminal and an output terminal of a separate circuit.

32. A router circuit comprising:
a router circuit terminal; and pipeline latch means having a first input terminal coupled to said router circuit terminal and having an output terminal coupled to additional circuitry, said pipeline latch having a second input terminal to receive a clock signal, said pipeline latch for retaining a signal applied to said router circuit terminal for one clock cycle for reducing equivalent capacitance and inductance between said router circuit terminal and a circuit providing a signal to said router circuit terminal.

33. A router circuit having the ability to either connect a router circuit input terminal to one of a plurality of output channels in a hyperbar mode, where each output channel is associated with a plurality of router circuit output terminals, or connect a router circuit input terminal to a specific one of a plurality of router circuit output terminals in a crossbar mode, said router circuit comprising:
a plurality of sections, each section being associated with a plurality of said router circuit input terminals and comprising connecting means associated with each of said associated router circuit input terminals, said connecting means comprising a plurality of grant circuits for connecting a data input line, coupled to an associated one of said router circuit input terminals, to one of said router circuit output terminals, said grant circuits having grant request terminals coupled to grant request signal generation means, said grant circuits being arranged in rows and columns, each row being associated with one of said router circuit input terminals, each column being associated with a single router circuit output terminal, each of said grant circuits having an enable/disable input terminal and an enable/disable output terminal, each of said grant circuits within the same section and associated with the same router circuit output terminal having a respective enable/disable input terminal coupled to an enable/disable output terminal of a grant circuit in an adjacent row, except for grant circuits in a terminal row of a section adjacent to another section; and hyperbar personality/crossbar personality switching means connected between adjacent ones of said sections of said router chip for coupling enable/disable input terminals of grant circuits in a terminal row of an associated section to either enable/disable output terminals of a terminal row of grant circuits in an adjacent section or to fixed logic levels.

34. The router circuit of Claim 33 wherein said fixed logic levels are of such levels as to enable only one column of grant circuits per channel in each of said sections, each of said sections having different columns of grant circuits enabled in a crossbar personality mode of operation.

35. A method for performing diagnostic testing for nonfunctional wires, said method comprising the steps of:
loading a first latch means with a first logic level;
applying an output of said first latch means to an input terminal of a device, said input terminal being connected via a wire to be tested to an output terminal of another device;
providing a weak pull-up of said input terminal while said first latch means is applying said first logic state to said input terminal;
detecting a level of said output terminal of said another device by a second latch means coupled to said output terminal while providing a weak pull-up of said output terminal;
providing an output of said second latch means to said output terminal;
detecting a logic state of said input terminal by said first latch means; and reading an output of said first latch means to determine whether said wire coupling said input terminal to said output terminal is functional or nonfunctional.

36. The method of Claim 35 wherein a first transceiver is connected between said input terminal and said first latch means, and a second transceiver is connected between said output terminal and said second latch means.

37. A method for processing routing address bits and protocol bits applied to an input terminal of a router chip, said method comprising the steps of:
processing routing address bits applied to an input terminal of a router chip by the use of decoding circuitry within said router chip to enable or disable connecting means coupled to said decoding circuitry, said connecting means for coupling said input terminal of said router chip to an output terminal of said router chip; and processing one or more protocol bits applied to said input terminal, after said step of processing said routing address bits, by also using said decoding circuitry so as to process said one or more protocol bits in a manner identical to that used to process said routing address bits.

38. A method of controlling a router circuit to either connect a router circuit input terminal to one of a plurality of output channels in a hyperbar mode, where each output channel is associated with a plurality of router circuit output terminals, or connect a router circuit input terminal to a specific one of a plurality of router circuit output terminals in a crossbar mode, said method of controlling a router circuit comprising the step of:
controlling a hyperbar personality/crossbar personality switching means connected between adjacent sections of said router circuit to couple enable/disable input terminals of grant circuits in a terminal row of an associated section to either enable/disable output terminals of a terminal row of grant circuits in an adjacent section or to fixed logic levels.