CA2215598C - Fpga-based processor - Google Patents
Fpga-based processor Download PDFInfo
- Publication number
- CA2215598C CA2215598C CA002215598A CA2215598A CA2215598C CA 2215598 C CA2215598 C CA 2215598C CA 002215598 A CA002215598 A CA 002215598A CA 2215598 A CA2215598 A CA 2215598A CA 2215598 C CA2215598 C CA 2215598C
- Authority
- CA
- Canada
- Prior art keywords
- fpgas
- array
- process controller
- controller
- fpga
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
- 238000000034 method Methods 0.000 claims abstract description 87
- 230000008569 process Effects 0.000 claims abstract description 87
- 230000015654 memory Effects 0.000 claims abstract description 64
- 238000012545 processing Methods 0.000 claims abstract description 47
- 239000011159 matrix material Substances 0.000 claims abstract description 28
- 238000013528 artificial neural network Methods 0.000 claims abstract description 22
- 230000006870 function Effects 0.000 claims abstract description 19
- 238000003491 array Methods 0.000 claims abstract description 9
- 238000012546 transfer Methods 0.000 claims description 27
- 230000003190 augmentative effect Effects 0.000 claims description 5
- 230000003287 optical effect Effects 0.000 claims description 5
- 238000012015 optical character recognition Methods 0.000 claims description 4
- 238000003909 pattern recognition Methods 0.000 abstract description 5
- 238000013461 design Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 5
- 238000013459 approach Methods 0.000 description 3
- 239000013078 crystal Substances 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 238000004377 microelectronic Methods 0.000 description 3
- MXRPNYMMDLFYDL-YFKPBYRVSA-N AMCC Chemical compound CNC(=O)SC[C@@H](C(O)=O)NC(C)=O MXRPNYMMDLFYDL-YFKPBYRVSA-N 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001537 neural effect Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/76—Architectures of general purpose stored program computers
- G06F15/78—Architectures of general purpose stored program computers comprising a single central processing unit
- G06F15/7867—Architectures of general purpose stored program computers comprising a single central processing unit with reconfigurable architecture
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/76—Architectures of general purpose stored program computers
- G06F15/80—Architectures of general purpose stored program computers comprising an array of processing units with common control, e.g. single instruction multiple data processors
- G06F15/8007—Architectures of general purpose stored program computers comprising an array of processing units with common control, e.g. single instruction multiple data processors single instruction multiple data [SIMD] multiprocessors
- G06F15/8023—Two dimensional arrays, e.g. mesh, torus
Abstract
A multiprocessor having an input/output controller, a process controller, and a multidimensional arrays of field programmable gate arrays (FPGAs), each FPGA having its own local memory. The multiprocessor may be programmed to function as a single-instruction, multiple-data (SIMD) parallel processor having a matrix of processing elements (PEs), where each FPGA may be programmed to operate as a submatrix array of PEs. The multiprocessor is especially useful for image processing, pattern recognition, and neural network applications.
Description
FPGA-BASED PROCESSOR
Field of the Invention The present invention relates to computer architecture for parallel processors.
Description of the Related Art Even with the last decade's exponential growth in performance of integrated circuits, many high performance applications, such as image processing and neural network applications, still demand increased hardware speed. One approach to increase performance is to build massively parallel computers. Their high price and difficulty to program have resulted in a very low acceptance rate. Furthermore, the design cycle of such computers is usually long, and, as a result, their technology is often obsolete before they are commercially available. Consequently, users often prefer to use the latest high-performance general workstation that is much less expensive and more easy to program. A second approach to solving the performance problem is to design dedicated parallel hardware for one task (or a set of similar tasks). Their programming is usually simple (or even nonexistent) while their performance/cost ratio is high. However, they are not flexible and their design cycle is long.
Over the last few years, advances in programmable logic devices have resulted in the commercialization of field programmable gate arrays (FPGAs) which are able to support relatively large numbers of programmable logic elements on a single chip. The size and speed of those circuits improve at the same rate as microprocessors' size and speed, since they rely on the same technology.
It is an aspect of the present invention to provide improved apparatus for data processing applications requiring a large number of operations, such as image processing, pattern recognition, and neural network algorithms.
It is another aspect to provide apparatus for data processing which provides greater flexibility and shorter design cycle than existing parallel processors.
A further aspect is to provide a more economical apparatus and process for virtual image processing (VIP) and neural network processing.
Further aspects and advantages of this invention will become apparent from the detailed description which follows.
Field of the Invention The present invention relates to computer architecture for parallel processors.
Description of the Related Art Even with the last decade's exponential growth in performance of integrated circuits, many high performance applications, such as image processing and neural network applications, still demand increased hardware speed. One approach to increase performance is to build massively parallel computers. Their high price and difficulty to program have resulted in a very low acceptance rate. Furthermore, the design cycle of such computers is usually long, and, as a result, their technology is often obsolete before they are commercially available. Consequently, users often prefer to use the latest high-performance general workstation that is much less expensive and more easy to program. A second approach to solving the performance problem is to design dedicated parallel hardware for one task (or a set of similar tasks). Their programming is usually simple (or even nonexistent) while their performance/cost ratio is high. However, they are not flexible and their design cycle is long.
Over the last few years, advances in programmable logic devices have resulted in the commercialization of field programmable gate arrays (FPGAs) which are able to support relatively large numbers of programmable logic elements on a single chip. The size and speed of those circuits improve at the same rate as microprocessors' size and speed, since they rely on the same technology.
It is an aspect of the present invention to provide improved apparatus for data processing applications requiring a large number of operations, such as image processing, pattern recognition, and neural network algorithms.
It is another aspect to provide apparatus for data processing which provides greater flexibility and shorter design cycle than existing parallel processors.
A further aspect is to provide a more economical apparatus and process for virtual image processing (VIP) and neural network processing.
Further aspects and advantages of this invention will become apparent from the detailed description which follows.
Summary of the Invention In one embodiment, the present invention is a multiprocessor comprising a multidimensional array of FPGAs, each FPGA having local memory; a process controller, connected to the array of FPGAs; and an input/output (I/O) controller, connected to the array of FPGAs and to the process controller. The array of FPGAs is adapted to be programmed as one or more processing elements (PEs), the process controller is adapted to control processing implemented by the PEs, and the I/O controller is adapted to control data flow into and out of the PEs and the process controller.
In another embodiment, the invention is a multiprocessor comprising a first FPGA
and one or more other FPGAs, connected to the first FPGA. The first FPGA is adapted to be programmed to function exclusively as a process controller for the one or more other FPGAs and the one or more other FPGAs are adapted to be programmed to function exclusively as processing elements.
In accordance with one aspect of the present invention there is provided a multiprocessor comprising: (a) a multidimensional array of field programmable gate arrays (FPGAs), each FPGA having local memory; (b) a process controller, connected to the array of FPGAs; and (c) an input/output (I/O) controller, connected to the array of FPGAs and to the process controller, wherein: the array of FPGAs is adapted to be programmed as a matrix of processing elements (PEs), wherein each FPGA in the array corresponds to two or more PEs; the process controller is adapted to control processing implemented by the PEs; and the I/O controller is adapted to control data flow into and out of the PEs and the process controller.
Brief Description of the Drawings Fig. 1 is a schematic block diagram of a multiprocessor, according to one embodiment of the invention;
Fig. 2 is a schematic block diagram of an FPGA of the multiprocessor of Fig.
1; and Fig. 3 is a schematic block diagram of a 2D configuration of FPGAs on which 2D
configurations of PEs are programmed, according to one embodiment of the present invention.
2a Detailed Description In one embodiment, the multiprocessor of the present invention is based on a multidimensional array of field programmable gate arrays (FPGAs) adapted to be programmed to function as one or more processing elements (PEs). Each FPGA in the array preferably has its own local memory and can be programmed to use the same local-memory addresses as the other FPGAs in the array.
In a one-dimensional array of FPGAs, each FPGA is connected to either one or two neighboring FPGAs. In a one-dimensional torroidal array, each FPGA is connected to exactly two neighboring FPGAs. If, in an array of FPGAs, at least one FPGA is connected to at least three neighboring FPGAs, then that array is said to be a multidimensional array (that is, an array of dimension greater than one). In a two-dimensional torroidal array, for example, each FPGA is connected to four neighboring FPGAs. A multidimensional array may have more than two dimensions.
Referring now to Fig. l, there is shown a schematic block diagram of multiprocessor 100, according to one embodiment of the present invention. Multiprocessor 100 comprises two-dimensional (2D) array 102 of FPGAs 104 (each with its own local memory 120), input/output (I/O) controller 106, and process controller 108 having data memory 110 and instruction memory 112. Process controller 108 is preferably an FPGA that is not part of array 102. Memories 110 and 112 are independently addressable by process controller 108.
I/O controller 106, process controller 108, and array 102 are interconnected by global bus 114. Similarly, I/O controller 106 and process controller 108 are interconnected by I/O
control bus 116, and process controller 108 and array 102 are interconnected by program control bus 118. I/O controller 106 may also be connected to a computer host by host system bus 122, which may be, for example, a peripheral control interface (PCI) bus.
Referring now to Fig. 2, there is shown a schematic block diagram of each FPGA
in array 102 of Fig. 1. As shown in Figs. l and 2, each FPGA 104 is connected in the north (N), south (S), east (E), and west (W) directions to its four neighboring FPGAs in a torroidal fashion. A torroidal connection scheme means that each FPGA.in the first column is connected to the corresponding FPGA in the last column (in the E-W direction) and each FPGA in the first row is connected to the corresponding FPGA in the last row (in the N-S
direction). Each FPGA 104 has local memory 120, which is preferably static random access memory (SRAM) based. Each FPGA 104 is either one-time programmable or reprogrammable. Each FPGA 104 in array 102 is capable of receiving program control data from process controller 108 via program control bus 118 and other data from I/O controller 106 via global bus 114. Similarly, process controller 108 is able to access each local memory 120 directly via either program control bus 118 or global bus 114.
Since it is based on FPGAs which are programmable devices, multiprocessor 100 provides a flexible computer architecture that can be programmed in a wide variety of ways to perform a wide variety of applications. In particular, multiprocessor 100 is especially suited to be programmed to function as a parallel processor. For example, array 102 of FPGAs 104 can be programmed to function as a matrix of processing elements (PEs), where the various PEs perform the same operations on different data in parallel. In that case, multiprocessor 100 will operate as a single-instruction, multiple-data (SIMD) processor.
Depending on the application, each PE may correspond to one or more programmed FPGAs, or, alternatively, each FPGA may be programmed to operate as one or more PEs.
Similarly, multiprocessor 100 could alternatively be programmed to operate as a multiple-instruction, multiple-data (MIMD) or multiple-instruction, single-data (MISD) processor. It is even possible to program multiprocessor 100 to operate as a single-instruction, single-data (SISD) processor, also known as a serial processor.
Referring now to Fig. 3, there is shown an example of a (2x2) 2D array 102 of FPGAs 104 programmed to operate as a (4x4) matrix of PEs 302. Each FPGA 104 is programmed to function as a (2x2) submatrix of PEs 302, where the four PEs 302 corresponding to each FPGA 104 access the same local memory 120. In one embodiment, each FPGA 104 is programmed to use the same local-memory addresses for accessing each local memory 120.
Referring again to Fig. 1, when multiprocessor 100 is programmed to operate as a SIMD processor, process controller 108 is programmed to function as a SIMD
controller that controls the parallel processing performed by the PEs. Process controller 108 is able to store global data in data memory 110 and programming instructions in instruction memory 112.
Multiprocessor 100 can be programmed such that the PEs are used exclusively for computing operations, while process controller 108 is programmed exclusively for program control operations.
Process controller 108 can be programmed to provide addresses and control to every memory during program execution and input/output (I/O) memory transfers.
Process controller 108 is preferably either a one-time programmable or reprogrammable FPGA.
Process controller 108 decodes and executes the program stored in instruction memory 112, and reads or writes to data memory 110. Process controller 108 is the control unit. It reads a program from instruction memory 112 and uses data memory 110 for storing global information. Once an instruction is decoded, process controller 108 sends data and control signals to the PEs through global bus 114 and dedicated program control bus 118. Global bus 114 may be used to send both control and data signals. Process controller 108 also provides addresses and control to every memory during both program execution and I/O
memory transfers. If configured accordingly, it exchanges status information with I/O
controller 106.
In one embodiment, input/output controller 106 is adapted to control data flow into and out of the PEs via global bus 114 and process controller 108 via I/O
control bus 116. I/O
controller 106 is also adapted to exchange status information with process controller 108 and to manage data transfer from and to a host computer (not shown). In one embodiment, I/O
controller 106 has an electrically programmable read only memory (EPROM). In its role of managing off board communication and initiating memory transfers, I/O
controller 106 is responsible for communicating with the host computer, exchanging status information with process controller 108; and managing data transfer between the host computer and the board (i.e., multiprocessor 100).
5 Global bus 114 is adapted to support data transfer between the array of FPGAs 104, process controller 108, and I/O controller 106. In most cases, the global bus is adapted to carry both control and data signals. Data transfers between the host computer and the board use global bus 114 to send address and data to the PEs and process controller 108. I/O
control bus 116 is adapted to support data transfer between I/O controller 106 and process controller 108. Program control bus 118 is adapted to support data transfer between process controller 108 and the array of FPGAs 104.
In addition, multiprocessor 100 includes a programmable clock signal based on two different clock signals, one of which clock signals is from host system bus 122 and the second ' of which clock signals is from a crystal clock (not shown).
Multiprocessor 100 can be configured with one or more other such multiprocessors 100 to form a large, more powerful parallel processing architecture. For example, four multiprocessors 100 can be configured in a (2x2) array to form an augmented multidimensional array of FPGAs. In one embodiment, the different multiprocessors 100 would have matching PCI bus clocks provided by matching PCI bus controllers.
The SIMD architecture of the present invention, together with the multidimensional torus connection topology, is well suited for image processing, pattern recognition, and neural network algorithms. Multiprocessors of the present invention can be programmed on-line at the logic level, allowing optimal hardware dedication to any given algorithm.
For example, multiprocessor 100 can be programmed with an optical character recognition algorithm functioning to receive data from an optical scanner in the form of pixels and to output character recognition data. Those and other algorithms generally include at least one of matrix multiplication, matrix addition, vector multiplication, vector addition, convolution, neural network forward pass, and neural network learning.
One of the most efficient and cost-effective architectures for parallel vector and matrix processing is the 2D systolic architecture. See, e.g., J. Beichter, U.
Ramacher, and H.
Klar, "VLSI design of a neural signal processor," Silicon Architectures for Neural Nets, pp 245-260, Elsevier Science Publishers B.V. (North-Holland), 1991; D.
Mammerstrom, "Image processing and pattern recognition hardware," Tutorial at Neural Information Processing Systems conference, November 1994; P. Ienne, "Architectures for neuro-computers: Review and performance evaluation," Technical Report 93/91, Ecole polytechnique federate de Lausanne, January 1993; and H. Kung and C. Leiserson, "Systolic arrays (for VLSI)," Sparse Matrix Proceedings (I. S. Duff and G. Stewart, editors), Knoxville: Academic Press, 1979.
However, this architecture is somewhat restrictive for more general applications. According to one embodiment of the present invention, a SIMD architecture together with a 2D torus connection topology, which includes the 2D systolic architecture, is used.
Drawbacks in such an architecture due to rigid data flow are overcome by the configurability of the FPGAs.
Multiprocessors of the present invention may be programmed for efficient computation of basic matrix and vector operations such as 1 D and 2D convolutions, matrix multiplication, matrix addition, matrix-vector multiplication, scalar multiplication, vector addition, and the like.
Multiprocessor 100 of the present invention can be programmed, for example, as a virtual image processor (VIP). An example is a VIP board that uses 5 large FPGAs, has 1.5 MB of static RAM, and communicates through a fast PCI bus. VIP, neural networks, and the like are particularly well suited for FPGA implementation since a relatively simple processing element (PE) may perform their most basic operations. Consequently, many instances of this PE may be fitted on one FPGA. The VIP architecture is composed of three basic components: the SIMD controller, the matrix of processing elements, and the I/O
controller. Those components preferably are connected by a shared global bus and two control buses. The matrix is a set of PEs interconnected in a 2D grid topology.
The same address and control signals are preferably used by every PE. The simple PE
interconnection topology is cheap and very efficient for processing 2D data structures such as images. The complexity of the address buses is reduced since the same address is used by every PE. Many useful vector processing algorithms will perform optimally with such a constraint. Alternatively, the addressing of local memory can vary from PE to PE.
The following example illustrates one possible embodiment of multiprocessor 100 of Fig. 1 as a platform for virtual image processing. Multiprocessor 100 has a (2x2) array 102 of FPGAs 104 and processor controller 108 is also an FPGA, where each of the five FPGAs is an SRAM-based Altera EPF81500FPGA (trade mark). The connection from multiprocessor 100 to the host computer (not shown in Fig. 1 ) is through a PCI bus interface. Multiprocessor 100 has six memory devices (data memory 110, instruction memory 112, and four local memories 120 for the four FPGAs 104), where each memory device is a 20 ns static-RAM
module of 64Kx32 bits. Each FPGA has approximately 16,000 usable gates. I/O controller implementation comprises an EPROM-based EPM7192 EPLD (3,750 usable gates) from Altera Corporation of San Jose, California and an AMCC 55933 PCI controller from Applied Micro Circuits Corporation of San Diego, California, and its configuration EPROM.
Since each PE may be configured on-board, the invention may perform any dedicated function by using exactly the logic needed for its implementation. Increasing the number of PEs increases performance. As will be demonstrated later, such an architecture may be an order of magnitude faster than general-purpose processors having the same cost.
The 128-bit vertical torus connections (two 32-bit north connections and two 32-bit south connections) are routed to a 128-pin connector header. Similarly, 128-bit horizontal torus connections (two 32-bit east connections and two 32-bit west connections) are routed to a different 128-pin connector header. Each N-S and E-W torus connection may be established using jumpers. Such connectors may also be used to carry signals off board for ' multiple-board processing.
As described earlier, process controller 108 is preferably implemented on an FPGA.
This implies that decoding and executing instructions may be different from one application to the other. For example, in a convolution application, both data memory 110 and instruction memory 112 may be used to store large instruction words, while, for a character recognizer, global information may be stored in data memory 110. Once an instruction is decoded, data and control signals are sent to the FPGAs through global bus 114 and program control bus 118. Table I shows the widths of the different buses.
TABLEI
BUS NUMBER OF BITS
Global bus 114 32 bits Three address buses18 bits each I/O control bus 23 bits Program control 10 bits bus 118 2D grid connections32 bits Configuration signals8 bits When multiple PEs are implemented on a single FPGA, the data and control signals for the FPGA are shared among every PE of that sub-matrix of PEs. For example, in a 2D
SIMD implementation, each FPGA may be programmed to function as a 2D sub-matrix of PEs. The FPGA matrix then becomes a single large PE matrix. As shown in Fig.
In another embodiment, the invention is a multiprocessor comprising a first FPGA
and one or more other FPGAs, connected to the first FPGA. The first FPGA is adapted to be programmed to function exclusively as a process controller for the one or more other FPGAs and the one or more other FPGAs are adapted to be programmed to function exclusively as processing elements.
In accordance with one aspect of the present invention there is provided a multiprocessor comprising: (a) a multidimensional array of field programmable gate arrays (FPGAs), each FPGA having local memory; (b) a process controller, connected to the array of FPGAs; and (c) an input/output (I/O) controller, connected to the array of FPGAs and to the process controller, wherein: the array of FPGAs is adapted to be programmed as a matrix of processing elements (PEs), wherein each FPGA in the array corresponds to two or more PEs; the process controller is adapted to control processing implemented by the PEs; and the I/O controller is adapted to control data flow into and out of the PEs and the process controller.
Brief Description of the Drawings Fig. 1 is a schematic block diagram of a multiprocessor, according to one embodiment of the invention;
Fig. 2 is a schematic block diagram of an FPGA of the multiprocessor of Fig.
1; and Fig. 3 is a schematic block diagram of a 2D configuration of FPGAs on which 2D
configurations of PEs are programmed, according to one embodiment of the present invention.
2a Detailed Description In one embodiment, the multiprocessor of the present invention is based on a multidimensional array of field programmable gate arrays (FPGAs) adapted to be programmed to function as one or more processing elements (PEs). Each FPGA in the array preferably has its own local memory and can be programmed to use the same local-memory addresses as the other FPGAs in the array.
In a one-dimensional array of FPGAs, each FPGA is connected to either one or two neighboring FPGAs. In a one-dimensional torroidal array, each FPGA is connected to exactly two neighboring FPGAs. If, in an array of FPGAs, at least one FPGA is connected to at least three neighboring FPGAs, then that array is said to be a multidimensional array (that is, an array of dimension greater than one). In a two-dimensional torroidal array, for example, each FPGA is connected to four neighboring FPGAs. A multidimensional array may have more than two dimensions.
Referring now to Fig. l, there is shown a schematic block diagram of multiprocessor 100, according to one embodiment of the present invention. Multiprocessor 100 comprises two-dimensional (2D) array 102 of FPGAs 104 (each with its own local memory 120), input/output (I/O) controller 106, and process controller 108 having data memory 110 and instruction memory 112. Process controller 108 is preferably an FPGA that is not part of array 102. Memories 110 and 112 are independently addressable by process controller 108.
I/O controller 106, process controller 108, and array 102 are interconnected by global bus 114. Similarly, I/O controller 106 and process controller 108 are interconnected by I/O
control bus 116, and process controller 108 and array 102 are interconnected by program control bus 118. I/O controller 106 may also be connected to a computer host by host system bus 122, which may be, for example, a peripheral control interface (PCI) bus.
Referring now to Fig. 2, there is shown a schematic block diagram of each FPGA
in array 102 of Fig. 1. As shown in Figs. l and 2, each FPGA 104 is connected in the north (N), south (S), east (E), and west (W) directions to its four neighboring FPGAs in a torroidal fashion. A torroidal connection scheme means that each FPGA.in the first column is connected to the corresponding FPGA in the last column (in the E-W direction) and each FPGA in the first row is connected to the corresponding FPGA in the last row (in the N-S
direction). Each FPGA 104 has local memory 120, which is preferably static random access memory (SRAM) based. Each FPGA 104 is either one-time programmable or reprogrammable. Each FPGA 104 in array 102 is capable of receiving program control data from process controller 108 via program control bus 118 and other data from I/O controller 106 via global bus 114. Similarly, process controller 108 is able to access each local memory 120 directly via either program control bus 118 or global bus 114.
Since it is based on FPGAs which are programmable devices, multiprocessor 100 provides a flexible computer architecture that can be programmed in a wide variety of ways to perform a wide variety of applications. In particular, multiprocessor 100 is especially suited to be programmed to function as a parallel processor. For example, array 102 of FPGAs 104 can be programmed to function as a matrix of processing elements (PEs), where the various PEs perform the same operations on different data in parallel. In that case, multiprocessor 100 will operate as a single-instruction, multiple-data (SIMD) processor.
Depending on the application, each PE may correspond to one or more programmed FPGAs, or, alternatively, each FPGA may be programmed to operate as one or more PEs.
Similarly, multiprocessor 100 could alternatively be programmed to operate as a multiple-instruction, multiple-data (MIMD) or multiple-instruction, single-data (MISD) processor. It is even possible to program multiprocessor 100 to operate as a single-instruction, single-data (SISD) processor, also known as a serial processor.
Referring now to Fig. 3, there is shown an example of a (2x2) 2D array 102 of FPGAs 104 programmed to operate as a (4x4) matrix of PEs 302. Each FPGA 104 is programmed to function as a (2x2) submatrix of PEs 302, where the four PEs 302 corresponding to each FPGA 104 access the same local memory 120. In one embodiment, each FPGA 104 is programmed to use the same local-memory addresses for accessing each local memory 120.
Referring again to Fig. 1, when multiprocessor 100 is programmed to operate as a SIMD processor, process controller 108 is programmed to function as a SIMD
controller that controls the parallel processing performed by the PEs. Process controller 108 is able to store global data in data memory 110 and programming instructions in instruction memory 112.
Multiprocessor 100 can be programmed such that the PEs are used exclusively for computing operations, while process controller 108 is programmed exclusively for program control operations.
Process controller 108 can be programmed to provide addresses and control to every memory during program execution and input/output (I/O) memory transfers.
Process controller 108 is preferably either a one-time programmable or reprogrammable FPGA.
Process controller 108 decodes and executes the program stored in instruction memory 112, and reads or writes to data memory 110. Process controller 108 is the control unit. It reads a program from instruction memory 112 and uses data memory 110 for storing global information. Once an instruction is decoded, process controller 108 sends data and control signals to the PEs through global bus 114 and dedicated program control bus 118. Global bus 114 may be used to send both control and data signals. Process controller 108 also provides addresses and control to every memory during both program execution and I/O
memory transfers. If configured accordingly, it exchanges status information with I/O
controller 106.
In one embodiment, input/output controller 106 is adapted to control data flow into and out of the PEs via global bus 114 and process controller 108 via I/O
control bus 116. I/O
controller 106 is also adapted to exchange status information with process controller 108 and to manage data transfer from and to a host computer (not shown). In one embodiment, I/O
controller 106 has an electrically programmable read only memory (EPROM). In its role of managing off board communication and initiating memory transfers, I/O
controller 106 is responsible for communicating with the host computer, exchanging status information with process controller 108; and managing data transfer between the host computer and the board (i.e., multiprocessor 100).
5 Global bus 114 is adapted to support data transfer between the array of FPGAs 104, process controller 108, and I/O controller 106. In most cases, the global bus is adapted to carry both control and data signals. Data transfers between the host computer and the board use global bus 114 to send address and data to the PEs and process controller 108. I/O
control bus 116 is adapted to support data transfer between I/O controller 106 and process controller 108. Program control bus 118 is adapted to support data transfer between process controller 108 and the array of FPGAs 104.
In addition, multiprocessor 100 includes a programmable clock signal based on two different clock signals, one of which clock signals is from host system bus 122 and the second ' of which clock signals is from a crystal clock (not shown).
Multiprocessor 100 can be configured with one or more other such multiprocessors 100 to form a large, more powerful parallel processing architecture. For example, four multiprocessors 100 can be configured in a (2x2) array to form an augmented multidimensional array of FPGAs. In one embodiment, the different multiprocessors 100 would have matching PCI bus clocks provided by matching PCI bus controllers.
The SIMD architecture of the present invention, together with the multidimensional torus connection topology, is well suited for image processing, pattern recognition, and neural network algorithms. Multiprocessors of the present invention can be programmed on-line at the logic level, allowing optimal hardware dedication to any given algorithm.
For example, multiprocessor 100 can be programmed with an optical character recognition algorithm functioning to receive data from an optical scanner in the form of pixels and to output character recognition data. Those and other algorithms generally include at least one of matrix multiplication, matrix addition, vector multiplication, vector addition, convolution, neural network forward pass, and neural network learning.
One of the most efficient and cost-effective architectures for parallel vector and matrix processing is the 2D systolic architecture. See, e.g., J. Beichter, U.
Ramacher, and H.
Klar, "VLSI design of a neural signal processor," Silicon Architectures for Neural Nets, pp 245-260, Elsevier Science Publishers B.V. (North-Holland), 1991; D.
Mammerstrom, "Image processing and pattern recognition hardware," Tutorial at Neural Information Processing Systems conference, November 1994; P. Ienne, "Architectures for neuro-computers: Review and performance evaluation," Technical Report 93/91, Ecole polytechnique federate de Lausanne, January 1993; and H. Kung and C. Leiserson, "Systolic arrays (for VLSI)," Sparse Matrix Proceedings (I. S. Duff and G. Stewart, editors), Knoxville: Academic Press, 1979.
However, this architecture is somewhat restrictive for more general applications. According to one embodiment of the present invention, a SIMD architecture together with a 2D torus connection topology, which includes the 2D systolic architecture, is used.
Drawbacks in such an architecture due to rigid data flow are overcome by the configurability of the FPGAs.
Multiprocessors of the present invention may be programmed for efficient computation of basic matrix and vector operations such as 1 D and 2D convolutions, matrix multiplication, matrix addition, matrix-vector multiplication, scalar multiplication, vector addition, and the like.
Multiprocessor 100 of the present invention can be programmed, for example, as a virtual image processor (VIP). An example is a VIP board that uses 5 large FPGAs, has 1.5 MB of static RAM, and communicates through a fast PCI bus. VIP, neural networks, and the like are particularly well suited for FPGA implementation since a relatively simple processing element (PE) may perform their most basic operations. Consequently, many instances of this PE may be fitted on one FPGA. The VIP architecture is composed of three basic components: the SIMD controller, the matrix of processing elements, and the I/O
controller. Those components preferably are connected by a shared global bus and two control buses. The matrix is a set of PEs interconnected in a 2D grid topology.
The same address and control signals are preferably used by every PE. The simple PE
interconnection topology is cheap and very efficient for processing 2D data structures such as images. The complexity of the address buses is reduced since the same address is used by every PE. Many useful vector processing algorithms will perform optimally with such a constraint. Alternatively, the addressing of local memory can vary from PE to PE.
The following example illustrates one possible embodiment of multiprocessor 100 of Fig. 1 as a platform for virtual image processing. Multiprocessor 100 has a (2x2) array 102 of FPGAs 104 and processor controller 108 is also an FPGA, where each of the five FPGAs is an SRAM-based Altera EPF81500FPGA (trade mark). The connection from multiprocessor 100 to the host computer (not shown in Fig. 1 ) is through a PCI bus interface. Multiprocessor 100 has six memory devices (data memory 110, instruction memory 112, and four local memories 120 for the four FPGAs 104), where each memory device is a 20 ns static-RAM
module of 64Kx32 bits. Each FPGA has approximately 16,000 usable gates. I/O controller implementation comprises an EPROM-based EPM7192 EPLD (3,750 usable gates) from Altera Corporation of San Jose, California and an AMCC 55933 PCI controller from Applied Micro Circuits Corporation of San Diego, California, and its configuration EPROM.
Since each PE may be configured on-board, the invention may perform any dedicated function by using exactly the logic needed for its implementation. Increasing the number of PEs increases performance. As will be demonstrated later, such an architecture may be an order of magnitude faster than general-purpose processors having the same cost.
The 128-bit vertical torus connections (two 32-bit north connections and two 32-bit south connections) are routed to a 128-pin connector header. Similarly, 128-bit horizontal torus connections (two 32-bit east connections and two 32-bit west connections) are routed to a different 128-pin connector header. Each N-S and E-W torus connection may be established using jumpers. Such connectors may also be used to carry signals off board for ' multiple-board processing.
As described earlier, process controller 108 is preferably implemented on an FPGA.
This implies that decoding and executing instructions may be different from one application to the other. For example, in a convolution application, both data memory 110 and instruction memory 112 may be used to store large instruction words, while, for a character recognizer, global information may be stored in data memory 110. Once an instruction is decoded, data and control signals are sent to the FPGAs through global bus 114 and program control bus 118. Table I shows the widths of the different buses.
TABLEI
BUS NUMBER OF BITS
Global bus 114 32 bits Three address buses18 bits each I/O control bus 23 bits Program control 10 bits bus 118 2D grid connections32 bits Configuration signals8 bits When multiple PEs are implemented on a single FPGA, the data and control signals for the FPGA are shared among every PE of that sub-matrix of PEs. For example, in a 2D
SIMD implementation, each FPGA may be programmed to function as a 2D sub-matrix of PEs. The FPGA matrix then becomes a single large PE matrix. As shown in Fig.
3, the connections between FPGAs are used for connecting adjacent PEs located on different FPGAs. Architectures that fit into this framework may be implemented by configuring the FPGAs accordingly. For example, since a 1D topology may be mapped on a 2D
topology, the VIP board may be configured as a 1 D SIMD processor.
The PCI local bus is used for the I/O interface. The PCI local bus standard is widely accepted by the personal computer industry and for its high transfer rate ( 132 MBytes/sec).
The design of the bus interface is greatly simplified by the AMCC S5933 PCI
controller. It is a powerful and flexible controller supporting several levels of interface sophistication. The control of the PCI interface chip is provided by an Altera EPM7192 EPROM-based programmable logic device. A 32-bit bus (multiplexed address and data) interconnects those two chips. This bus is connected to the board's global bus through a 32-bit transceiver. This transceiver has three modes: write-to-global bus, read-from-global bus, and disconnect-from-global bus.
Each Altera EPF81500 FPGA is on-line configurable. The reserved configuration pins for all five FPGAs are connected to the EPM7192 device. It is thus possible to configure each FPGA at any time by providing the configuration data from the host computer. All FPGAs may be configured in parallel in less than 40 ms. The on-board local memories 120 are accessible by the SIMD controller and the FPGAs. Therefore, they may be configured before any memory transfer is initiated.
Two clock signals are available on-board. The first one is the 33-MHz PCI bus clock that is provided by the PCI controller, while the second one is an on-board crystal clock. A
programmable clock is generated from the EPM7192 device, based on those two clocks. For added flexibility, the crystal clock is preferably mounted on a socket.
It is possible to connect together two or more VIP boards by using their 128-pin connectors. In that case, the N-S connections and/or E-W connections of the FPGAs may be routed to other boards. Some of those connections may be used as control signals to synchronize the execution on a multi-board system. This is greatly simplified by the use of the same PCI clock on each board.
To perform computation on the VIP board initially, the designer determines a program for an application and a dedicated logic design for the SIMD
controller and each FPGA. This design is translated into a form that may be used to configure the corresponding FPGAs. This configuration is send to the I/O controller which supervises the configuration of each FPGA. At this point, the SIMD controller and FPGAs have the capability to access each memory bank. Data transfers may be initiated by the host computer. The I/O
controller signals the SIMD controller to start program execution. During program execution, status information is transmitted to the I/O controller. SIMD processing is done in parallel to those transmissions. The SIMD controller indicates to the I/O controller that the processing is done. The result of the computation may be read from registers or memories as initiated by the host computer.
The following example demonstrates the implementation of a convolution process over a binary image on a prototype of the VIP board of the present invention.
Each pixel z;.J
in the resulting feature map is expressed as follows:
1 if yf~ > t 0 otherwise where N M
y;~=~ ~ f(x;~~,wk~
and,() is any 3-input-bit Boolean function, x;~ is a pixel in the original image (precision: 1 bit), wk" is a template pixel (precision: 2 bits), Nand Mare respectively the height and width of the template image. Finally, y;,~ is the cross correlation between image field and the template and t is threshold level for a match.
For a 2D systolic algorithm, a matrix of 8x4 PEs is implemented on each FPGA, for a total of 16X8 PEs for the (2x2) array of four FPGAs. The systolic algorithm computes 16x8 convolutions in parallel in NxM steps. This is done by partitioning the image in (15+N)X(7+M) pixel tiles. Each tile has 16X8 or 128 convolution results. A
tile image is shifted over the PE matrix by using the North, South, and East connections.
The "border" of the sliding window is fed from each PEs' local memory. The template values are broadcasted from the global bus to each PE. At any time, every PE computes a partial sum for its convolution. Reading pixel columns and rows from memory is pipelined with the processing, and an initial latency of 3 cycles is used.
The processing steps for each tile convolution are:
( 1 ) Load a tile ( 1-cycle/memory read, 3-cycle latency);
5 (2) NxM computations (1 cycle each);
(3) Threshold function evaluation to generate z() (1 cycle); and (4) Write result (2 cycles/memory write).
Thus, there are Nx(M+7) cycles per tile.
The prototype of the VIP board of the present invention had a processing rate of 16.6 10 MHz for this application. This implementation speed performance was compared to that of the NET32K board. The NET32K board is described in E. Cosatto and H. Graf, "NET32K
high speed image understanding system," Proceedings of the Fourth International Conference on Microelectronics for Neural Networks and Fuzzy Systems, pp. 413-421, September 1994. The same algorithm was also optimized for a personal computer (PC) having an Intel 90-MHz Pentium~ processor, 32 MB of main memory, and a 256-KB
level-2 cache. Table II shows a performance comparison between these three implementations for convolution over a 512x512 pixel image. The NET32K board was configured to process only templates of size 16x16, which explains why no performance improvement was achieved for smaller templates.
TABLE II
Template size VIP board NET 32K board Pentium-based PC
3 x 4 1.5 ms 6.25 ms 114 ms 8 x 8 6 ms 6.25 ms 238 ms 16 x 16 24 ms 6.25 ms 961 ms The reported performance shows that the prototype VIP board was between 40 and times faster than a 90-MHz Pentium-based PC, depending on template size. For small templates, the VIP board was faster than the NET32K board; however, for larger template, it was 4 times slower. Considering that slow FPGAs and SRAM were used in the VIP
prototype of the present invention, those results are very impressive. The VIP
board outperformed, in some cases, a dedicated processor. This is a surprising achievement for programmable hardware which is much slower than custom chip implementation.
Furthermore, the added flexibility of the VIP board over the NET32K board permits implementation of algorithms that could not be processed by the NET32K board.
For example, the VIP board could process images with many levels of gray which is not possible for the NET32K board.
As another application example, the prototype VIP board was programmed as an optical handwritten character recognizer. The program is based on the evaluation of a feed-forward neural network, using forward propagation without multiplication.
See J.
Cloutier and P. Simard, "Hardware implementation of the backpropagation without multiplication," Proceedings of the Fourth International Conference on Microelectronics for Neural Networks and Fuzzy Systems, pp. 46-55, September 1994. The forward propagation for each unit i, is given by the following equation:
x~ _ .f~ (~ ~'~~ xi ) r where f is the unit function, w~; is the weight from unit i to unit j, and x;
is the activation of unit i. This implementation considers the following resolution and format of neural network variables. The weights are 8-bit fixed-point numbers with 6-bit fractional value. The states are 3-bit floating-point values with 1-bit mantissas (m) and 2-bit exponents (e). The corresponding real numbers are-1 "' x 2-a . Each neuron has the same unit function which is a discretized version of a sigmoid function. The output function is a state value as defined above.
Defining the shift-operation (a « b) , where a is a fixed-point number with the same resolution as the weights and b=(m,e) is a floating-point number, yields the following:
(a x b) _ (a « b) _ -1"'a2-a which may be implemented by a barrel shifter. The sign inversion is performed by the adder used for the multiply-accumulate operation. This implementation permits reduction of the hardware by about 40% over one using an 8-bit multiplier. This is done without any degradation of recognizer performance.
The character recognition algorithm has five layers. The first layer performs convolutions of four different 5x5 kernels over a 32x32 pixel input image having 3 bits per pixel to generate four feature maps. The second layer subsamples the four feature maps to reduce the resolution of each feature map by four. The third layer performs a set of convolutions (similar to the first layer) by combining different feature maps to generate 12 new maps. The fourth layer performs subsampling similar to the second layer, but on smaller feature maps. The fifth layer is a fully connected neural network connected with 73 outputs corresponding to letters, digits, and punctuation. All weights in the neural network were learned from examples using a back-propagation algorithm.
The forward pass algorithm was implemented with a 2D systolic network of 6x6 PEs per FPGA. A square sub-matrix of 8 times 8 performed a shift-accumulate operation and a discretized sigmoid function. The other PEs were used to shift information around. In this implementation, the prototype VIP board performed at a frequency of 16.6 MHz.
The processing was time-multiplexed over the layers of the network. Each feature map was diced into tiles that fit into the 12x12 PEs of the 2D systolic network. For example, the 32x32 input was diced into 16 tiles (nine of size 12x12, six of size 12x8, and one of size 8x8). The resulting feature map for a tile was not larger than 8 times 8, since only 8 times 8 convolutions were computed per tile.
Only the first three layers were executed on the VIP board (86% of the connections).
The remaining layers, which have higher precision, were computed on the host computer.
The total execution time for the three layers is presented in Table III which compares the performance of implementations of the present invention to those of the ANNA
board and a dual-processor SPARC 10-41X2 workstation from Sun Microsystems of Mountain View, California. The ANNA board is described in E. Sackinger, B. E. Boser, J.
Bromley, E.
LeCun, and L. D. Jackel, "Application of the ANNA neural network chip to high-speed character recognition," IEEE Trans. on Neural Networks, 3(3), May 1992 and E.
Sackinger and H. Graf, "A system for high-speed pattern recognition and image analysis,"
Proceedings of the Fourth International Conference on Microelectronics for Neural Networks and Fuzzy Systems, pp. 364-371, September 1994.
TABLE III
VIP board ANNA board SPARC workstation 0.75 ms 1.02 ms 12.3 ms Loading and storing transfer operations were included in the execution time of the VIP board. The VIP board was respectively about 1.4 and 16.4 times faster than the ANNA
and SPARC implementations. This is due mainly to the improved parallelism and 2D systolic network that is well suited for processing 2D data structure. For example, at each step of the convolution, 144 state values of 3 bits were exchanged between PEs, giving an impressive bandwidth of 900 MBytes/sec.
The architecture and implementation of the VIP board described above uses large FPGAs as the main building blocks. This provides a high degree of flexibility for the processor logic design. The VIP board has a SIMD "virtual" architecture with a interconnection network that is well suited for implementing 2D systolic networks.
The two implementation examples described above compare the speed performance I 5 of the VIP board with other dedicated and general hardware designs. In some situations, the VIP board is at least an order of magnitude faster than general processor implementations.
Furthermore, it has speed performance similar to those obtained by dedicated hardware. Such results are excellent, considering that the approach of the present invention is much more general and much cheaper than dedicated hardware. In fact, the initial prototype used the slowest version of the EPF81500 FPGA available at that time from Alters. Using faster chips and faster memory in other implementations of the VIP board may increase performance at least another 2 to 3 times.
Those skilled in the art will understand that the prototype VIP board described in the above examples is only one possible embodiment of the present invention and that many other embodiments fall within the scope of the present invention.
It will be further understood that various changes in the details, materials, and arrangements of the parts which have been described and illustrated in order to explain the nature of this invention may be made by those skilled in the art without departing from the principle and scope of the invention as expressed in the following claims.
topology, the VIP board may be configured as a 1 D SIMD processor.
The PCI local bus is used for the I/O interface. The PCI local bus standard is widely accepted by the personal computer industry and for its high transfer rate ( 132 MBytes/sec).
The design of the bus interface is greatly simplified by the AMCC S5933 PCI
controller. It is a powerful and flexible controller supporting several levels of interface sophistication. The control of the PCI interface chip is provided by an Altera EPM7192 EPROM-based programmable logic device. A 32-bit bus (multiplexed address and data) interconnects those two chips. This bus is connected to the board's global bus through a 32-bit transceiver. This transceiver has three modes: write-to-global bus, read-from-global bus, and disconnect-from-global bus.
Each Altera EPF81500 FPGA is on-line configurable. The reserved configuration pins for all five FPGAs are connected to the EPM7192 device. It is thus possible to configure each FPGA at any time by providing the configuration data from the host computer. All FPGAs may be configured in parallel in less than 40 ms. The on-board local memories 120 are accessible by the SIMD controller and the FPGAs. Therefore, they may be configured before any memory transfer is initiated.
Two clock signals are available on-board. The first one is the 33-MHz PCI bus clock that is provided by the PCI controller, while the second one is an on-board crystal clock. A
programmable clock is generated from the EPM7192 device, based on those two clocks. For added flexibility, the crystal clock is preferably mounted on a socket.
It is possible to connect together two or more VIP boards by using their 128-pin connectors. In that case, the N-S connections and/or E-W connections of the FPGAs may be routed to other boards. Some of those connections may be used as control signals to synchronize the execution on a multi-board system. This is greatly simplified by the use of the same PCI clock on each board.
To perform computation on the VIP board initially, the designer determines a program for an application and a dedicated logic design for the SIMD
controller and each FPGA. This design is translated into a form that may be used to configure the corresponding FPGAs. This configuration is send to the I/O controller which supervises the configuration of each FPGA. At this point, the SIMD controller and FPGAs have the capability to access each memory bank. Data transfers may be initiated by the host computer. The I/O
controller signals the SIMD controller to start program execution. During program execution, status information is transmitted to the I/O controller. SIMD processing is done in parallel to those transmissions. The SIMD controller indicates to the I/O controller that the processing is done. The result of the computation may be read from registers or memories as initiated by the host computer.
The following example demonstrates the implementation of a convolution process over a binary image on a prototype of the VIP board of the present invention.
Each pixel z;.J
in the resulting feature map is expressed as follows:
1 if yf~ > t 0 otherwise where N M
y;~=~ ~ f(x;~~,wk~
and,() is any 3-input-bit Boolean function, x;~ is a pixel in the original image (precision: 1 bit), wk" is a template pixel (precision: 2 bits), Nand Mare respectively the height and width of the template image. Finally, y;,~ is the cross correlation between image field and the template and t is threshold level for a match.
For a 2D systolic algorithm, a matrix of 8x4 PEs is implemented on each FPGA, for a total of 16X8 PEs for the (2x2) array of four FPGAs. The systolic algorithm computes 16x8 convolutions in parallel in NxM steps. This is done by partitioning the image in (15+N)X(7+M) pixel tiles. Each tile has 16X8 or 128 convolution results. A
tile image is shifted over the PE matrix by using the North, South, and East connections.
The "border" of the sliding window is fed from each PEs' local memory. The template values are broadcasted from the global bus to each PE. At any time, every PE computes a partial sum for its convolution. Reading pixel columns and rows from memory is pipelined with the processing, and an initial latency of 3 cycles is used.
The processing steps for each tile convolution are:
( 1 ) Load a tile ( 1-cycle/memory read, 3-cycle latency);
5 (2) NxM computations (1 cycle each);
(3) Threshold function evaluation to generate z() (1 cycle); and (4) Write result (2 cycles/memory write).
Thus, there are Nx(M+7) cycles per tile.
The prototype of the VIP board of the present invention had a processing rate of 16.6 10 MHz for this application. This implementation speed performance was compared to that of the NET32K board. The NET32K board is described in E. Cosatto and H. Graf, "NET32K
high speed image understanding system," Proceedings of the Fourth International Conference on Microelectronics for Neural Networks and Fuzzy Systems, pp. 413-421, September 1994. The same algorithm was also optimized for a personal computer (PC) having an Intel 90-MHz Pentium~ processor, 32 MB of main memory, and a 256-KB
level-2 cache. Table II shows a performance comparison between these three implementations for convolution over a 512x512 pixel image. The NET32K board was configured to process only templates of size 16x16, which explains why no performance improvement was achieved for smaller templates.
TABLE II
Template size VIP board NET 32K board Pentium-based PC
3 x 4 1.5 ms 6.25 ms 114 ms 8 x 8 6 ms 6.25 ms 238 ms 16 x 16 24 ms 6.25 ms 961 ms The reported performance shows that the prototype VIP board was between 40 and times faster than a 90-MHz Pentium-based PC, depending on template size. For small templates, the VIP board was faster than the NET32K board; however, for larger template, it was 4 times slower. Considering that slow FPGAs and SRAM were used in the VIP
prototype of the present invention, those results are very impressive. The VIP
board outperformed, in some cases, a dedicated processor. This is a surprising achievement for programmable hardware which is much slower than custom chip implementation.
Furthermore, the added flexibility of the VIP board over the NET32K board permits implementation of algorithms that could not be processed by the NET32K board.
For example, the VIP board could process images with many levels of gray which is not possible for the NET32K board.
As another application example, the prototype VIP board was programmed as an optical handwritten character recognizer. The program is based on the evaluation of a feed-forward neural network, using forward propagation without multiplication.
See J.
Cloutier and P. Simard, "Hardware implementation of the backpropagation without multiplication," Proceedings of the Fourth International Conference on Microelectronics for Neural Networks and Fuzzy Systems, pp. 46-55, September 1994. The forward propagation for each unit i, is given by the following equation:
x~ _ .f~ (~ ~'~~ xi ) r where f is the unit function, w~; is the weight from unit i to unit j, and x;
is the activation of unit i. This implementation considers the following resolution and format of neural network variables. The weights are 8-bit fixed-point numbers with 6-bit fractional value. The states are 3-bit floating-point values with 1-bit mantissas (m) and 2-bit exponents (e). The corresponding real numbers are-1 "' x 2-a . Each neuron has the same unit function which is a discretized version of a sigmoid function. The output function is a state value as defined above.
Defining the shift-operation (a « b) , where a is a fixed-point number with the same resolution as the weights and b=(m,e) is a floating-point number, yields the following:
(a x b) _ (a « b) _ -1"'a2-a which may be implemented by a barrel shifter. The sign inversion is performed by the adder used for the multiply-accumulate operation. This implementation permits reduction of the hardware by about 40% over one using an 8-bit multiplier. This is done without any degradation of recognizer performance.
The character recognition algorithm has five layers. The first layer performs convolutions of four different 5x5 kernels over a 32x32 pixel input image having 3 bits per pixel to generate four feature maps. The second layer subsamples the four feature maps to reduce the resolution of each feature map by four. The third layer performs a set of convolutions (similar to the first layer) by combining different feature maps to generate 12 new maps. The fourth layer performs subsampling similar to the second layer, but on smaller feature maps. The fifth layer is a fully connected neural network connected with 73 outputs corresponding to letters, digits, and punctuation. All weights in the neural network were learned from examples using a back-propagation algorithm.
The forward pass algorithm was implemented with a 2D systolic network of 6x6 PEs per FPGA. A square sub-matrix of 8 times 8 performed a shift-accumulate operation and a discretized sigmoid function. The other PEs were used to shift information around. In this implementation, the prototype VIP board performed at a frequency of 16.6 MHz.
The processing was time-multiplexed over the layers of the network. Each feature map was diced into tiles that fit into the 12x12 PEs of the 2D systolic network. For example, the 32x32 input was diced into 16 tiles (nine of size 12x12, six of size 12x8, and one of size 8x8). The resulting feature map for a tile was not larger than 8 times 8, since only 8 times 8 convolutions were computed per tile.
Only the first three layers were executed on the VIP board (86% of the connections).
The remaining layers, which have higher precision, were computed on the host computer.
The total execution time for the three layers is presented in Table III which compares the performance of implementations of the present invention to those of the ANNA
board and a dual-processor SPARC 10-41X2 workstation from Sun Microsystems of Mountain View, California. The ANNA board is described in E. Sackinger, B. E. Boser, J.
Bromley, E.
LeCun, and L. D. Jackel, "Application of the ANNA neural network chip to high-speed character recognition," IEEE Trans. on Neural Networks, 3(3), May 1992 and E.
Sackinger and H. Graf, "A system for high-speed pattern recognition and image analysis,"
Proceedings of the Fourth International Conference on Microelectronics for Neural Networks and Fuzzy Systems, pp. 364-371, September 1994.
TABLE III
VIP board ANNA board SPARC workstation 0.75 ms 1.02 ms 12.3 ms Loading and storing transfer operations were included in the execution time of the VIP board. The VIP board was respectively about 1.4 and 16.4 times faster than the ANNA
and SPARC implementations. This is due mainly to the improved parallelism and 2D systolic network that is well suited for processing 2D data structure. For example, at each step of the convolution, 144 state values of 3 bits were exchanged between PEs, giving an impressive bandwidth of 900 MBytes/sec.
The architecture and implementation of the VIP board described above uses large FPGAs as the main building blocks. This provides a high degree of flexibility for the processor logic design. The VIP board has a SIMD "virtual" architecture with a interconnection network that is well suited for implementing 2D systolic networks.
The two implementation examples described above compare the speed performance I 5 of the VIP board with other dedicated and general hardware designs. In some situations, the VIP board is at least an order of magnitude faster than general processor implementations.
Furthermore, it has speed performance similar to those obtained by dedicated hardware. Such results are excellent, considering that the approach of the present invention is much more general and much cheaper than dedicated hardware. In fact, the initial prototype used the slowest version of the EPF81500 FPGA available at that time from Alters. Using faster chips and faster memory in other implementations of the VIP board may increase performance at least another 2 to 3 times.
Those skilled in the art will understand that the prototype VIP board described in the above examples is only one possible embodiment of the present invention and that many other embodiments fall within the scope of the present invention.
It will be further understood that various changes in the details, materials, and arrangements of the parts which have been described and illustrated in order to explain the nature of this invention may be made by those skilled in the art without departing from the principle and scope of the invention as expressed in the following claims.
Claims (35)
1. A multiprocessor comprising:
(a) a multidimensional array of field programmable gate arrays (FPGAs), each FPGA
having local memory;
(b) a process controller, connected to the array of FPGAs; and (c) an input/output (I/O) controller, connected to the array of FPGAs and to the process controller, wherein:
the array of FPGAs is adapted to be programmed as a matrix of processing elements (PEs), wherein each FPGA in the array corresponds to two or more PEs;
the process controller is adapted to control processing implemented by the PEs; and the I/O controller is adapted to control data flow into and out of the PEs and the process controller.
(a) a multidimensional array of field programmable gate arrays (FPGAs), each FPGA
having local memory;
(b) a process controller, connected to the array of FPGAs; and (c) an input/output (I/O) controller, connected to the array of FPGAs and to the process controller, wherein:
the array of FPGAs is adapted to be programmed as a matrix of processing elements (PEs), wherein each FPGA in the array corresponds to two or more PEs;
the process controller is adapted to control processing implemented by the PEs; and the I/O controller is adapted to control data flow into and out of the PEs and the process controller.
2. The invention of claim 1, wherein the process controller is an FPGA.
3. The invention of claim 1, wherein the process controller is a single-instruction, multiple-data (SIMD) controller adapted to control parallel processing by the PEs.
4. The invention of claim 1, wherein the multiprocessor is a first multiprocessor adapted to be connected to a second multiprocessor having a multidimensional array of FPGAs, wherein the array of FPGAs of the first multiprocessor is adapted to be connected to the array of FPGAs of the second multiprocessor to form an augmented multidimensional array of FPGAs.
5. The invention of claim 1, further comprising two memories that are independently addressable by the process controller.
6. The invention of claim 5, wherein the two memories comprise an instruction memory adapted to store programming instructions from the process controller and a data memory adapted to receive to store global data from the process controller.
7. The invention of claim I, further comprising:
(d) a global bus interconnecting and adapted to support data transfer between the array of FPGAs, the process controller, and the I/O controller;
(e) an I/O control bus interconnecting and adapted to support data transfer between the I/O controller and the process controller;
(f) a program control bus interconnecting and adapted to support data transfer between the process controller and the array of FPGAs, and (g) a host system bus adapted to be interconnected to a computer host to support data transfer between the computer host and the I/O controller.
(d) a global bus interconnecting and adapted to support data transfer between the array of FPGAs, the process controller, and the I/O controller;
(e) an I/O control bus interconnecting and adapted to support data transfer between the I/O controller and the process controller;
(f) a program control bus interconnecting and adapted to support data transfer between the process controller and the array of FPGAs, and (g) a host system bus adapted to be interconnected to a computer host to support data transfer between the computer host and the I/O controller.
8. The invention of claim 1, wherein each FPGA is either one-time programmable or reprogrammable.
9. The invention of claim 1, wherein each FPGA is adapted to be programmed to use the same local-memory addresses.
10. The invention of claim 1, wherein at least one FPGA is connected to at least three neighboring FPGAs.
11. The invention of claim 1, wherein the multidimensional array is a two-dimensional (2D) array, wherein each FPGA is connected in north, east, south, and west directions to its four neighboring FPGAs in a torroidal fashion.
12. The invention of claim 1, further comprising:
(d) a global bus interconnecting and adapted to support data transfer between the array of FPGAs, the process controller, and the I/O controller;
(e) an I/O control bus interconnecting and adapted to support data transfer between the I/O controller and the process controller;
(f) a program control bus interconnecting and adapted to support data transfer between the process controller and the array of FPGAs;
(g) instruction memory, connected to the process controller, wherein the process controller is adapted to store programming instructions in the instruction memory; and (h) data memory, connected to the process controller, wherein the process controller is adapted to store global data in the data memory, wherein:
the instruction memory and the data memory are independently addressable by the process controller;
the process controller is an FPGA adapted to be programmed as a SIMD
controller to control parallel processing by the PEs;
each FPGA is either one-time programmable or reprogrammable;
each FPGA is adapted to be programmed to use the same local-memory addresses;
the multidimensional array is a two-dimensional (2D) array, wherein each FPGA
is connected in north, east, south, and west directions to its four neighboring FPGAs in a torroidal fashion; and the 2D array of FPGAs is adapted to be programmed as a 2D matrix of PEs.
(d) a global bus interconnecting and adapted to support data transfer between the array of FPGAs, the process controller, and the I/O controller;
(e) an I/O control bus interconnecting and adapted to support data transfer between the I/O controller and the process controller;
(f) a program control bus interconnecting and adapted to support data transfer between the process controller and the array of FPGAs;
(g) instruction memory, connected to the process controller, wherein the process controller is adapted to store programming instructions in the instruction memory; and (h) data memory, connected to the process controller, wherein the process controller is adapted to store global data in the data memory, wherein:
the instruction memory and the data memory are independently addressable by the process controller;
the process controller is an FPGA adapted to be programmed as a SIMD
controller to control parallel processing by the PEs;
each FPGA is either one-time programmable or reprogrammable;
each FPGA is adapted to be programmed to use the same local-memory addresses;
the multidimensional array is a two-dimensional (2D) array, wherein each FPGA
is connected in north, east, south, and west directions to its four neighboring FPGAs in a torroidal fashion; and the 2D array of FPGAs is adapted to be programmed as a 2D matrix of PEs.
13. The invention of claim 12, wherein the multiprocessor is a first multiprocessor adapted to be connected to a second multiprocessor having a 2D array of FPGAs, wherein the array of FPGAs of the first multiprocessor is adapted to be connected to the array of FPGAs of the second multiprocessor to form an augmented 2D array of FPGAs.
14. The invention of claim 1, wherein the PEs are programmed with an optical character recognition algorithm functioning to receive data from an optical scanner in the form of pixels and to output character recognition data.
15. The invention of claim 1, wherein the PEs are programmed with an algorithm that comprises at least one of matrix multiplication, matrix addition, vector multiplication, vector addition, convolution, neural network forward pass, and neural network learning.
16. A multiprocessor comprising:
(a) a first FPGA; and (b) one or more other FPGAs, connected to the first FPGA, wherein:
the first FPGA is adapted to be programmed to function exclusively as a process controller for the one or more other FPGAs; and
(a) a first FPGA; and (b) one or more other FPGAs, connected to the first FPGA, wherein:
the first FPGA is adapted to be programmed to function exclusively as a process controller for the one or more other FPGAs; and
17 the one or more other FPGAs are adapted to be programmed to function exclusively as processing elements.
17. The invention of claim 16, wherein the one or more other FPGAs comprise a multidimensional array of FPGAs adapted to be programmed as one or more processing elements and the first FPGA is adapted to be programmed as a parallel process controller to control parallel processing implemented by the processing elements.
17. The invention of claim 16, wherein the one or more other FPGAs comprise a multidimensional array of FPGAs adapted to be programmed as one or more processing elements and the first FPGA is adapted to be programmed as a parallel process controller to control parallel processing implemented by the processing elements.
18. A multiprocessor comprising:
(a) a multidimensional array of field programmable gate arrays (FPGAs), each FPGA
having local memory;
(b) a process controller, connected to the array of FPGAs; and (c) an input/output (I/O) controller, connected to the array of FPGAs and to the process controller, wherein:
the array of FPGAs is adapted to be programmed as one or more processing elements (PEs);
the process controller is adapted to control processing implemented by the PEs;
the I/O controller is adapted to control data flow into and out of the PEs and the process controller; and the PEs are programmed with an optical character recognition algorithm functioning to receive data from an optical scanner in the form of pixels and to output character recognition data.
(a) a multidimensional array of field programmable gate arrays (FPGAs), each FPGA
having local memory;
(b) a process controller, connected to the array of FPGAs; and (c) an input/output (I/O) controller, connected to the array of FPGAs and to the process controller, wherein:
the array of FPGAs is adapted to be programmed as one or more processing elements (PEs);
the process controller is adapted to control processing implemented by the PEs;
the I/O controller is adapted to control data flow into and out of the PEs and the process controller; and the PEs are programmed with an optical character recognition algorithm functioning to receive data from an optical scanner in the form of pixels and to output character recognition data.
19. A multiprocessor comprising:
(a) a multidimensional array of field programmable gate arrays (FPGAs), each FPGA
having local memory;
(b) a process controller, connected to the array of FPGAs; and (c) an input/output (I/O) controller, connected to the array of FPGAs and to the process controller, wherein:
the multidimensional array is a two-dimensional (2D) array, wherein each FPGA
in the array is directly connected in north, east, south, and west directions to its four neighboring FPGAs in a torroidal fashion;
the array of FPGAs is adapted to be programmed as one or more processing elements (PEs);
the process controller is adapted to control processing implemented by the PEs; and the I/O controller is adapted to control data flow into and out of the PEs and the process controller.
(a) a multidimensional array of field programmable gate arrays (FPGAs), each FPGA
having local memory;
(b) a process controller, connected to the array of FPGAs; and (c) an input/output (I/O) controller, connected to the array of FPGAs and to the process controller, wherein:
the multidimensional array is a two-dimensional (2D) array, wherein each FPGA
in the array is directly connected in north, east, south, and west directions to its four neighboring FPGAs in a torroidal fashion;
the array of FPGAs is adapted to be programmed as one or more processing elements (PEs);
the process controller is adapted to control processing implemented by the PEs; and the I/O controller is adapted to control data flow into and out of the PEs and the process controller.
20. The invention of claim 19, wherein the process controller is an FPGA.
21. The invention of claim 19, wherein the process controller is a single-instruction, multiple-data (SIMD) controller adapted to control parallel processing by the PEs.
22. The invention of claim 19, wherein the multiprocessor is a first multiprocessor adapted to be connected to a second multiprocessor having a multidimensional array of FPGAs, wherein the array of FPGAs of the first multiprocessor is adapted to be connected to the array of FPGAs of the second multiprocessor to form an augmented multidimensional array of FPGAs.
23. The invention of claim 19, further comprising two memories that are independently addressable by the process controller.
24. The invention of claim 23, wherein the two memories comprise an instruction memory adapted to store programming instructions from the process controller and a data memory adapted to receive to store global data from the process controller.
25. The invention of claim 19, further comprising:
(d) a global bus interconnecting and adapted to support data transfer between the array of FPGAs, the process controller, and the I/O controller;
(e) an I/O control bus interconnecting and adapted to support data transfer between the I/O controller and the process controller;
(f) a program control bus interconnecting and adapted to support data transfer between the process controller and the array of FPGAs; and (g) a host system bus adapted to be interconnected to a computer host to support data transfer between the computer host and the I/O controller.
(d) a global bus interconnecting and adapted to support data transfer between the array of FPGAs, the process controller, and the I/O controller;
(e) an I/O control bus interconnecting and adapted to support data transfer between the I/O controller and the process controller;
(f) a program control bus interconnecting and adapted to support data transfer between the process controller and the array of FPGAs; and (g) a host system bus adapted to be interconnected to a computer host to support data transfer between the computer host and the I/O controller.
26. The invention of claim 19, wherein each FPGA is either one-time programmable or reprogrammable.
27. The invention of claim 19, wherein each FPGA is adapted to be programmed to use the same local-memory addresses.
28. The invention of claim 19, wherein at least one FPGA is connected to at least three neighboring FPGAs.
29. The invention of claim 19, wherein the array of FPGAs is adapted to be programmed as a matrix of PEs.
30. The invention of claim 29, wherein each PE corresponds to one or more FPGAs.
31. The invention of claim 29, wherein each FPGA corresponds to one or more PEs.
32. The invention of claim 19, further comprising:
(d) a global bus interconnecting and adapted to support data transfer between the array of FPGAs, the process controller, and the I/O controller;
(e) an I/0 control bus interconnecting and adapted to support data transfer between the I/0 controller and the process controller;
(f) a program control bus interconnecting and adapted to support data transfer between the process controller and the array of FPGAs;
(g) instruction memory, connected to the process controller, wherein the process controller is adapted to store programming instructions in the instruction memory; and (h) data memory, connected to the process controller, wherein the process controller is adapted to store global data in the data memory, wherein:
the instruction memory and the data memory are independently addressable by the process controller;
the process controller is an FPGA adapted to be programmed as a SIMD
controller to control parallel processing by the PEs;
each FPGA is either one-time programmable or reprogrammable;
each FPGA is adapted to be programmed to use the same local-memory addresses;
and the 2D array of FPGAs is adapted to be programmed as a 2D matrix of PEs.
(d) a global bus interconnecting and adapted to support data transfer between the array of FPGAs, the process controller, and the I/O controller;
(e) an I/0 control bus interconnecting and adapted to support data transfer between the I/0 controller and the process controller;
(f) a program control bus interconnecting and adapted to support data transfer between the process controller and the array of FPGAs;
(g) instruction memory, connected to the process controller, wherein the process controller is adapted to store programming instructions in the instruction memory; and (h) data memory, connected to the process controller, wherein the process controller is adapted to store global data in the data memory, wherein:
the instruction memory and the data memory are independently addressable by the process controller;
the process controller is an FPGA adapted to be programmed as a SIMD
controller to control parallel processing by the PEs;
each FPGA is either one-time programmable or reprogrammable;
each FPGA is adapted to be programmed to use the same local-memory addresses;
and the 2D array of FPGAs is adapted to be programmed as a 2D matrix of PEs.
33. The invention of claim 32, wherein the multiprocessor is a first multiprocessor adapted to be connected to a second multiprocessor having a 2D array of FPGAs, wherein the array of FPGAs of the first multiprocessor is adapted to be connected to the array of FPGAs of the second multiprocessor to form an augmented 2D array of FPGAs.
34. The invention of claim 19, wherein the PEs are programmed with an optical character recognition algorithm functioning to receive data from an optical scanner in the form of pixels and to output character recognition data.
35. The invention of claim 19, wherein the PEs are programmed with an algorithm that comprises at least one of matrix multiplication, matrix addition, vector multiplication, vector addition, convolution, neural network forward pass, and neural network learning.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US748,041 | 1996-11-12 | ||
| US08/748,041 US5892962A (en) | 1996-11-12 | 1996-11-12 | FPGA-based processor |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2215598A1 CA2215598A1 (en) | 1998-05-12 |
| CA2215598C true CA2215598C (en) | 2001-08-14 |
Family
ID=25007741
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA002215598A Expired - Lifetime CA2215598C (en) | 1996-11-12 | 1997-09-16 | Fpga-based processor |
Country Status (2)
| Country | Link |
|---|---|
| US (1) | US5892962A (en) |
| CA (1) | CA2215598C (en) |
Families Citing this family (322)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7266725B2 (en) | 2001-09-03 | 2007-09-04 | Pact Xpp Technologies Ag | Method for debugging reconfigurable architectures |
| US5819027A (en) | 1996-02-28 | 1998-10-06 | Intel Corporation | Bus patcher |
| DE19651075A1 (en) | 1996-12-09 | 1998-06-10 | Pact Inf Tech Gmbh | Unit for processing numerical and logical operations, for use in processors (CPU's), multi-computer systems, data flow processors (DFP's), digital signal processors (DSP's) or the like |
| DE19654595A1 (en) | 1996-12-20 | 1998-07-02 | Pact Inf Tech Gmbh | I0 and memory bus system for DFPs as well as building blocks with two- or multi-dimensional programmable cell structures |
| EP1329816B1 (en) | 1996-12-27 | 2011-06-22 | Richter, Thomas | Method for automatic dynamic unloading of data flow processors (dfp) as well as modules with bidimensional or multidimensional programmable cell structures (fpgas, dpgas or the like) |
| US6542998B1 (en) | 1997-02-08 | 2003-04-01 | Pact Gmbh | Method of self-synchronization of configurable elements of a programmable module |
| US6421817B1 (en) * | 1997-05-29 | 2002-07-16 | Xilinx, Inc. | System and method of computation in a programmable logic device using virtual instructions |
| US8686549B2 (en) | 2001-09-03 | 2014-04-01 | Martin Vorbach | Reconfigurable elements |
| US6904480B1 (en) * | 1997-12-17 | 2005-06-07 | Intel Corporation | Testing a bus using bus specific instructions |
| US6339819B1 (en) * | 1997-12-17 | 2002-01-15 | Src Computers, Inc. | Multiprocessor with each processor element accessing operands in loaded input buffer and forwarding results to FIFO output buffer |
| US6996656B2 (en) * | 2002-10-31 | 2006-02-07 | Src Computers, Inc. | System and method for providing an arbitrated memory bus in a hybrid computing system |
| US7565461B2 (en) * | 1997-12-17 | 2009-07-21 | Src Computers, Inc. | Switch/network adapter port coupling a reconfigurable processing element to one or more microprocessors for use with interleaved memory controllers |
| US20040236877A1 (en) * | 1997-12-17 | 2004-11-25 | Lee A. Burton | Switch/network adapter port incorporating shared memory resources selectively accessible by a direct execution logic element and one or more dense logic devices in a fully buffered dual in-line memory module format (FB-DIMM) |
| US7003593B2 (en) * | 1997-12-17 | 2006-02-21 | Src Computers, Inc. | Computer system architecture and memory controller for close-coupling within a hybrid processing system utilizing an adaptive processor interface port |
| US7197575B2 (en) * | 1997-12-17 | 2007-03-27 | Src Computers, Inc. | Switch/network adapter port coupling a reconfigurable processing element to one or more microprocessors for use with interleaved memory controllers |
| US7373440B2 (en) * | 1997-12-17 | 2008-05-13 | Src Computers, Inc. | Switch/network adapter port for clustered computers employing a chain of multi-adaptive processors in a dual in-line memory module format |
| US7424552B2 (en) * | 1997-12-17 | 2008-09-09 | Src Computers, Inc. | Switch/network adapter port incorporating shared memory resources selectively accessible by a direct execution logic element and one or more dense logic devices |
| US6076152A (en) * | 1997-12-17 | 2000-06-13 | Src Computers, Inc. | Multiprocessor computer architecture incorporating a plurality of memory algorithm processors in the memory subsystem |
| JP3283040B2 (en) * | 1997-12-19 | 2002-05-20 | ビーエイイー システムズ パブリック リミテッド カンパニー | Digital signal filter using unweighted neural technology |
| EP1040582B1 (en) * | 1997-12-19 | 2003-10-08 | BAE SYSTEMS plc | Binary code converters and comparators |
| DE19861088A1 (en) | 1997-12-22 | 2000-02-10 | Pact Inf Tech Gmbh | Repairing integrated circuits by replacing subassemblies with substitutes |
| US6092174A (en) * | 1998-06-01 | 2000-07-18 | Context, Inc. | Dynamically reconfigurable distributed integrated circuit processor and method |
| EP1164544B1 (en) * | 1999-03-16 | 2011-11-02 | Hamamatsu Photonics K.K. | High-speed vision sensor |
| WO2000077652A2 (en) | 1999-06-10 | 2000-12-21 | Pact Informationstechnologie Gmbh | Sequence partitioning in cell structures |
| WO2001009717A1 (en) * | 1999-08-02 | 2001-02-08 | Morton Steven G | Video digital signal processor chip |
| US6728863B1 (en) | 1999-10-26 | 2004-04-27 | Assabet Ventures | Wide connections for transferring data between PE's of an N-dimensional mesh-connected SIMD array while transferring operands from memory |
| AU2471001A (en) * | 1999-10-26 | 2001-05-08 | Arthur D. Little, Inc. | Multiplexing n-dimensional mesh connections onto (n + 1) data paths |
| US6847365B1 (en) * | 2000-01-03 | 2005-01-25 | Genesis Microchip Inc. | Systems and methods for efficient processing of multimedia data |
| US6438737B1 (en) | 2000-02-15 | 2002-08-20 | Intel Corporation | Reconfigurable logic for a computer |
| JP3674515B2 (en) | 2000-02-25 | 2005-07-20 | 日本電気株式会社 | Array type processor |
| EP2226732A3 (en) | 2000-06-13 | 2016-04-06 | PACT XPP Technologies AG | Cache hierarchy for a multicore processor |
| US6628140B2 (en) * | 2000-09-18 | 2003-09-30 | Altera Corporation | Programmable logic devices with function-specific blocks |
| US8058899B2 (en) | 2000-10-06 | 2011-11-15 | Martin Vorbach | Logic cell array and bus system |
| US20020087829A1 (en) * | 2000-12-29 | 2002-07-04 | Snyder Walter L. | Re-targetable communication system |
| JP4688316B2 (en) * | 2001-02-28 | 2011-05-25 | 独立行政法人産業技術総合研究所 | Simulation apparatus and simulation method, and video generation apparatus and video generation method |
| US7844796B2 (en) | 2001-03-05 | 2010-11-30 | Martin Vorbach | Data processing device and method |
| US9552047B2 (en) | 2001-03-05 | 2017-01-24 | Pact Xpp Technologies Ag | Multiprocessor having runtime adjustable clock and clock dependent power supply |
| US9141390B2 (en) | 2001-03-05 | 2015-09-22 | Pact Xpp Technologies Ag | Method of processing data with an array of data processors according to application ID |
| US9436631B2 (en) | 2001-03-05 | 2016-09-06 | Pact Xpp Technologies Ag | Chip including memory element storing higher level memory data on a page by page basis |
| WO2005045692A2 (en) | 2003-08-28 | 2005-05-19 | Pact Xpp Technologies Ag | Data processing device and method |
| US9037807B2 (en) | 2001-03-05 | 2015-05-19 | Pact Xpp Technologies Ag | Processor arrangement on a chip including data processing, memory, and interface elements |
| US9411532B2 (en) | 2001-09-07 | 2016-08-09 | Pact Xpp Technologies Ag | Methods and systems for transferring data between a processing device and external devices |
| US9250908B2 (en) | 2001-03-05 | 2016-02-02 | Pact Xpp Technologies Ag | Multi-processor bus and cache interconnection system |
| US20070299993A1 (en) * | 2001-03-05 | 2007-12-27 | Pact Xpp Technologies Ag | Method and Device for Treating and Processing Data |
| US7444531B2 (en) | 2001-03-05 | 2008-10-28 | Pact Xpp Technologies Ag | Methods and devices for treating and processing data |
| US6901422B1 (en) * | 2001-03-21 | 2005-05-31 | Apple Computer, Inc. | Matrix multiplication in a vector processing system |
| US7752419B1 (en) | 2001-03-22 | 2010-07-06 | Qst Holdings, Llc | Method and system for managing hardware resources to implement system functions using an adaptive computing architecture |
| US7962716B2 (en) | 2001-03-22 | 2011-06-14 | Qst Holdings, Inc. | Adaptive integrated circuitry with heterogeneous and reconfigurable matrices of diverse and adaptive computational units having fixed, application specific computational elements |
| US6836839B2 (en) | 2001-03-22 | 2004-12-28 | Quicksilver Technology, Inc. | Adaptive integrated circuitry with heterogeneous and reconfigurable matrices of diverse and adaptive computational units having fixed, application specific computational elements |
| US6577678B2 (en) | 2001-05-08 | 2003-06-10 | Quicksilver Technology | Method and system for reconfigurable channel coding |
| JP4846924B2 (en) * | 2001-05-31 | 2011-12-28 | キヤノン株式会社 | Pattern recognition device |
| US6633856B2 (en) * | 2001-06-15 | 2003-10-14 | Flarion Technologies, Inc. | Methods and apparatus for decoding LDPC codes |
| US7673223B2 (en) * | 2001-06-15 | 2010-03-02 | Qualcomm Incorporated | Node processors for use in parity check decoders |
| US6938196B2 (en) * | 2001-06-15 | 2005-08-30 | Flarion Technologies, Inc. | Node processors for use in parity check decoders |
| US10031733B2 (en) | 2001-06-20 | 2018-07-24 | Scientia Sol Mentis Ag | Method for processing data |
| US7657877B2 (en) | 2001-06-20 | 2010-02-02 | Pact Xpp Technologies Ag | Method for processing data |
| US7996827B2 (en) * | 2001-08-16 | 2011-08-09 | Martin Vorbach | Method for the translation of programs for reconfigurable architectures |
| US7434191B2 (en) | 2001-09-03 | 2008-10-07 | Pact Xpp Technologies Ag | Router |
| US8686475B2 (en) | 2001-09-19 | 2014-04-01 | Pact Xpp Technologies Ag | Reconfigurable elements |
| EP1450916A4 (en) * | 2001-11-01 | 2006-01-04 | Mattel Inc | Tile-based board game |
| US8412915B2 (en) | 2001-11-30 | 2013-04-02 | Altera Corporation | Apparatus, system and method for configuration of adaptive integrated circuitry having heterogeneous computational elements |
| US6986021B2 (en) | 2001-11-30 | 2006-01-10 | Quick Silver Technology, Inc. | Apparatus, method, system and executable module for configuration and operation of adaptive integrated circuitry having fixed, application specific computational elements |
| CA2468800A1 (en) * | 2001-12-06 | 2003-06-19 | University Of Georgia Research Foundation, Inc. | Floating point intensive reconfigurable computing system for iterative applications |
| US20070067380A2 (en) * | 2001-12-06 | 2007-03-22 | The University Of Georgia Research Foundation | Floating Point Intensive Reconfigurable Computing System for Iterative Applications |
| US7215701B2 (en) | 2001-12-12 | 2007-05-08 | Sharad Sambhwani | Low I/O bandwidth method and system for implementing detection and identification of scrambling codes |
| US20030108012A1 (en) * | 2001-12-12 | 2003-06-12 | Quicksilver Technology, Inc. | Method and system for detecting and identifying scrambling codes |
| US6938177B1 (en) * | 2001-12-19 | 2005-08-30 | Sentient Sensors Llc | Multi-chip module smart controller |
| US7403981B2 (en) * | 2002-01-04 | 2008-07-22 | Quicksilver Technology, Inc. | Apparatus and method for adaptive multimedia reception and transmission in communication environments |
| WO2003062770A1 (en) * | 2002-01-18 | 2003-07-31 | Spm Instrument Ab | An analysis system for analysing the condition of a machine |
| AU2003208266A1 (en) | 2002-01-19 | 2003-07-30 | Pact Xpp Technologies Ag | Reconfigurable processor |
| AU2003214003A1 (en) | 2002-02-18 | 2003-09-09 | Pact Xpp Technologies Ag | Bus systems and method for reconfiguration |
| US8914590B2 (en) | 2002-08-07 | 2014-12-16 | Pact Xpp Technologies Ag | Data processing method and device |
| US9170812B2 (en) | 2002-03-21 | 2015-10-27 | Pact Xpp Technologies Ag | Data processing system having integrated pipelined array data processor |
| US6992925B2 (en) * | 2002-04-26 | 2006-01-31 | Kilopass Technologies, Inc. | High density semiconductor memory cell and memory array using a single transistor and having counter-doped poly and buried diffusion wordline |
| US20030212853A1 (en) * | 2002-05-09 | 2003-11-13 | Huppenthal Jon M. | Adaptive processor architecture incorporating a field programmable gate array control element having at least one embedded microprocessor core |
| US7406573B2 (en) * | 2002-05-09 | 2008-07-29 | Src Computers, Inc. | Reconfigurable processor element utilizing both coarse and fine grained reconfigurable elements |
| US7328414B1 (en) | 2003-05-13 | 2008-02-05 | Qst Holdings, Llc | Method and system for creating and programming an adaptive computing engine |
| US7660984B1 (en) | 2003-05-13 | 2010-02-09 | Quicksilver Technology | Method and system for achieving individualized protected space in an operating system |
| US20110238948A1 (en) * | 2002-08-07 | 2011-09-29 | Martin Vorbach | Method and device for coupling a data processing unit and a data processing array |
| WO2004021176A2 (en) | 2002-08-07 | 2004-03-11 | Pact Xpp Technologies Ag | Method and device for processing data |
| US7657861B2 (en) | 2002-08-07 | 2010-02-02 | Pact Xpp Technologies Ag | Method and device for processing data |
| US6961888B2 (en) | 2002-08-20 | 2005-11-01 | Flarion Technologies, Inc. | Methods and apparatus for encoding LDPC codes |
| US8108656B2 (en) | 2002-08-29 | 2012-01-31 | Qst Holdings, Llc | Task definition for specifying resource requirements |
| US7394284B2 (en) * | 2002-09-06 | 2008-07-01 | Pact Xpp Technologies Ag | Reconfigurable sequencer structure |
| US7042772B2 (en) * | 2002-09-26 | 2006-05-09 | Kilopass Technology, Inc. | Methods and circuits for programming of a semiconductor memory cell and memory array using a breakdown phenomenon in an ultra-thin dielectric |
| US7031209B2 (en) * | 2002-09-26 | 2006-04-18 | Kilopass Technology, Inc. | Methods and circuits for testing programmability of a semiconductor memory cell and memory array using a breakdown phenomenon in an ultra-thin dielectric |
| US7124211B2 (en) | 2002-10-23 | 2006-10-17 | Src Computers, Inc. | System and method for explicit communication of messages between processes running on different nodes in a clustered multiprocessor system |
| US7937591B1 (en) | 2002-10-25 | 2011-05-03 | Qst Holdings, Llc | Method and system for providing a device which can be adapted on an ongoing basis |
| WO2004042562A2 (en) * | 2002-10-31 | 2004-05-21 | Lockheed Martin Corporation | Pipeline accelerator and related system and method |
| US7418574B2 (en) * | 2002-10-31 | 2008-08-26 | Lockheed Martin Corporation | Configuring a portion of a pipeline accelerator to generate pipeline date without a program instruction |
| US6983456B2 (en) * | 2002-10-31 | 2006-01-03 | Src Computers, Inc. | Process for converting programs in high-level programming languages to a unified executable for hybrid computing platforms |
| US6964029B2 (en) * | 2002-10-31 | 2005-11-08 | Src Computers, Inc. | System and method for partitioning control-dataflow graph representations |
| EP1573515A2 (en) * | 2002-10-31 | 2005-09-14 | Lockheed Martin Corporation | Pipeline accelerator and related system and method |
| US6941539B2 (en) * | 2002-10-31 | 2005-09-06 | Src Computers, Inc. | Efficiency of reconfigurable hardware |
| US7299458B2 (en) | 2002-10-31 | 2007-11-20 | Src Computers, Inc. | System and method for converting control flow graph representations to control-dataflow graph representations |
| US7225324B2 (en) * | 2002-10-31 | 2007-05-29 | Src Computers, Inc. | Multi-adaptive processing systems and techniques for enhancing parallelism and performance of computational functions |
| US7155708B2 (en) * | 2002-10-31 | 2006-12-26 | Src Computers, Inc. | Debugging and performance profiling using control-dataflow graph representations with reconfigurable hardware emulation |
| US8949576B2 (en) * | 2002-11-01 | 2015-02-03 | Nvidia Corporation | Arithmetic node including general digital signal processing functions for an adaptive computing machine |
| US8276135B2 (en) * | 2002-11-07 | 2012-09-25 | Qst Holdings Llc | Profiling of software and circuit designs utilizing data operation analyses |
| US7225301B2 (en) | 2002-11-22 | 2007-05-29 | Quicksilver Technologies | External memory controller node |
| WO2004053716A2 (en) * | 2002-12-12 | 2004-06-24 | Koninklijke Philips Electronics N.V. | Dataflow-synchronized embedded field programmable processor array |
| KR20050085545A (en) * | 2002-12-12 | 2005-08-29 | 코닌클리즈케 필립스 일렉트로닉스 엔.브이. | Modular integration of an array processor within a system on chip |
| US20040157626A1 (en) * | 2003-02-10 | 2004-08-12 | Vincent Park | Paging methods and apparatus |
| US6957375B2 (en) * | 2003-02-26 | 2005-10-18 | Flarion Technologies, Inc. | Method and apparatus for performing low-density parity-check (LDPC) code operations using a multi-level permutation |
| JP4373340B2 (en) * | 2003-02-26 | 2009-11-25 | クゥアルコム・インコーポレイテッド | Soft information scaling for iterative decoding |
| US20070234178A1 (en) * | 2003-02-26 | 2007-10-04 | Qualcomm Incorporated | Soft information scaling for interactive decoding |
| US8196000B2 (en) * | 2003-04-02 | 2012-06-05 | Qualcomm Incorporated | Methods and apparatus for interleaving in a block-coherent communication system |
| US7434145B2 (en) * | 2003-04-02 | 2008-10-07 | Qualcomm Incorporated | Extracting soft information in a block-coherent communication system |
| US7231557B2 (en) * | 2003-04-02 | 2007-06-12 | Qualcomm Incorporated | Methods and apparatus for interleaving in a block-coherent communication system |
| US7398359B1 (en) * | 2003-04-30 | 2008-07-08 | Silicon Graphics, Inc. | System and method for performing memory operations in a computing system |
| EP2528000B1 (en) | 2003-05-23 | 2017-07-26 | IP Reservoir, LLC | Intelligent data storage and processing using FPGA devices |
| US6924664B2 (en) * | 2003-08-15 | 2005-08-02 | Kilopass Technologies, Inc. | Field programmable gate array |
| US7266255B1 (en) * | 2003-09-26 | 2007-09-04 | Sun Microsystems, Inc. | Distributed multi-sample convolution |
| FR2860313B1 (en) * | 2003-09-30 | 2005-11-04 | Commissariat Energie Atomique | COMPONENT WITH RECONFIGURABLE ARCHITECTURE DYNAMICALLY |
| JP4104538B2 (en) * | 2003-12-22 | 2008-06-18 | 三洋電機株式会社 | Reconfigurable circuit, processing device provided with reconfigurable circuit, function determination method of logic circuit in reconfigurable circuit, circuit generation method, and circuit |
| US7237181B2 (en) * | 2003-12-22 | 2007-06-26 | Qualcomm Incorporated | Methods and apparatus for reducing error floors in message passing decoders |
| US6972986B2 (en) * | 2004-02-03 | 2005-12-06 | Kilopass Technologies, Inc. | Combination field programmable gate array allowing dynamic reprogrammability and non-votatile programmability based upon transistor gate oxide breakdown |
| US7064973B2 (en) * | 2004-02-03 | 2006-06-20 | Klp International, Ltd. | Combination field programmable gate array allowing dynamic reprogrammability |
| US20050218929A1 (en) * | 2004-04-02 | 2005-10-06 | Man Wang | Field programmable gate array logic cell and its derivatives |
| US8735297B2 (en) | 2004-05-06 | 2014-05-27 | Sidense Corporation | Reverse optical proximity correction method |
| US7755162B2 (en) | 2004-05-06 | 2010-07-13 | Sidense Corp. | Anti-fuse memory cell |
| EP1743380B1 (en) * | 2004-05-06 | 2016-12-28 | Sidense Corp. | Split-channel antifuse array architecture |
| US9123572B2 (en) | 2004-05-06 | 2015-09-01 | Sidense Corporation | Anti-fuse memory cell |
| US7164290B2 (en) * | 2004-06-10 | 2007-01-16 | Klp International, Ltd. | Field programmable gate array logic unit and its cluster |
| US20050275427A1 (en) * | 2004-06-10 | 2005-12-15 | Man Wang | Field programmable gate array logic unit and its cluster |
| US7395490B2 (en) * | 2004-07-21 | 2008-07-01 | Qualcomm Incorporated | LDPC decoding methods and apparatus |
| US7346832B2 (en) * | 2004-07-21 | 2008-03-18 | Qualcomm Incorporated | LDPC encoding methods and apparatus |
| US7127659B2 (en) * | 2004-08-02 | 2006-10-24 | Qualcomm Incorporated | Memory efficient LDPC decoding methods and apparatus |
| US7299339B2 (en) * | 2004-08-30 | 2007-11-20 | The Boeing Company | Super-reconfigurable fabric architecture (SURFA): a multi-FPGA parallel processing architecture for COTS hybrid computing framework |
| US7135886B2 (en) * | 2004-09-20 | 2006-11-14 | Klp International, Ltd. | Field programmable gate arrays using both volatile and nonvolatile memory cell properties and their control |
| US7619541B2 (en) * | 2004-10-01 | 2009-11-17 | Lockheed Martin Corporation | Remote sensor processing system and method |
| JP2006236106A (en) * | 2005-02-25 | 2006-09-07 | Canon Inc | Data processor and data processing method |
| GB2423840A (en) * | 2005-03-03 | 2006-09-06 | Clearspeed Technology Plc | Reconfigurable logic in processors |
| DE102005014323A1 (en) * | 2005-03-24 | 2006-09-28 | Silicann Technologies Gmbh | Field Programmable synapse Array and method for its programming |
| US7461236B1 (en) * | 2005-03-25 | 2008-12-02 | Tilera Corporation | Transferring data in a parallel processing environment |
| US7193436B2 (en) * | 2005-04-18 | 2007-03-20 | Klp International Ltd. | Fast processing path using field programmable gate array logic units |
| WO2007011203A1 (en) * | 2005-07-22 | 2007-01-25 | Stichting Astron | Scalable control interface for large-scale signal processing systems. |
| US8620980B1 (en) | 2005-09-27 | 2013-12-31 | Altera Corporation | Programmable device with specialized multiplier blocks |
| US7890686B2 (en) * | 2005-10-17 | 2011-02-15 | Src Computers, Inc. | Dynamic priority conflict resolution in a multi-processor computer system having shared resources |
| US7716100B2 (en) * | 2005-12-02 | 2010-05-11 | Kuberre Systems, Inc. | Methods and systems for computing platform |
| WO2007082730A1 (en) | 2006-01-18 | 2007-07-26 | Pact Xpp Technologies Ag | Hardware definition method |
| US8301681B1 (en) | 2006-02-09 | 2012-10-30 | Altera Corporation | Specialized processing block for programmable logic device |
| US8266198B2 (en) * | 2006-02-09 | 2012-09-11 | Altera Corporation | Specialized processing block for programmable logic device |
| US8041759B1 (en) | 2006-02-09 | 2011-10-18 | Altera Corporation | Specialized processing block for programmable logic device |
| US8266199B2 (en) * | 2006-02-09 | 2012-09-11 | Altera Corporation | Specialized processing block for programmable logic device |
| US7688721B2 (en) * | 2006-03-13 | 2010-03-30 | Alcatel Lucent | Distributed communication traffic control systems and methods |
| US7836117B1 (en) | 2006-04-07 | 2010-11-16 | Altera Corporation | Specialized processing block for programmable logic device |
| US7577820B1 (en) * | 2006-04-14 | 2009-08-18 | Tilera Corporation | Managing data in a parallel processing environment |
| US7636835B1 (en) * | 2006-04-14 | 2009-12-22 | Tilera Corporation | Coupling data in a parallel processing environment |
| US7774579B1 (en) | 2006-04-14 | 2010-08-10 | Tilera Corporation | Protection in a parallel processing environment using access information associated with each switch to prevent data from being forwarded outside a plurality of tiles |
| US7539845B1 (en) * | 2006-04-14 | 2009-05-26 | Tilera Corporation | Coupling integrated circuits in a parallel processing environment |
| US7337272B2 (en) * | 2006-05-01 | 2008-02-26 | Qualcomm Incorporated | Method and apparatus for caching variable length instructions |
| US7792895B1 (en) * | 2006-06-16 | 2010-09-07 | Nvidia Corporation | Efficient matrix multiplication on a parallel processing device |
| US7836118B1 (en) | 2006-06-16 | 2010-11-16 | Nvidia Corporation | Hardware/software-based mapping of CTAs to matrix tiles for efficient matrix multiplication |
| US7912889B1 (en) | 2006-06-16 | 2011-03-22 | Nvidia Corporation | Mapping the threads of a CTA to the elements of a tile for efficient matrix multiplication |
| US7822799B1 (en) | 2006-06-26 | 2010-10-26 | Altera Corporation | Adder-rounder circuitry for specialized processing block in programmable logic device |
| US7693257B2 (en) * | 2006-06-29 | 2010-04-06 | Accuray Incorporated | Treatment delivery optimization |
| KR100772547B1 (en) * | 2006-08-31 | 2007-11-02 | 주식회사 하이닉스반도체 | Semiconductor device and test method thereof |
| US8386550B1 (en) | 2006-09-20 | 2013-02-26 | Altera Corporation | Method for configuring a finite impulse response filter in a programmable logic device |
| US7660793B2 (en) | 2006-11-13 | 2010-02-09 | Exegy Incorporated | Method and system for high performance integration, processing and searching of structured and unstructured data using coprocessors |
| WO2008061162A1 (en) * | 2006-11-14 | 2008-05-22 | Star Bridge Systems, Inc. | Hybrid computing platform having fpga components with embedded processors |
| US7555469B2 (en) * | 2006-11-16 | 2009-06-30 | L-3 Communications Integrated Systems L.P. | Reconfigurable neural network systems and methods utilizing FPGAs having packet routers |
| US7930336B2 (en) * | 2006-12-05 | 2011-04-19 | Altera Corporation | Large multiplier for programmable logic device |
| US8386553B1 (en) | 2006-12-05 | 2013-02-26 | Altera Corporation | Large multiplier for programmable logic device |
| EP2122542B1 (en) | 2006-12-08 | 2017-11-01 | Medhat Moussa | Architecture, system and method for artificial neural network implementation |
| US7814137B1 (en) | 2007-01-09 | 2010-10-12 | Altera Corporation | Combined interpolation and decimation filter for programmable logic device |
| US8650231B1 (en) | 2007-01-22 | 2014-02-11 | Altera Corporation | Configuring floating point operations in a programmable device |
| US7865541B1 (en) | 2007-01-22 | 2011-01-04 | Altera Corporation | Configuring floating point operations in a programmable logic device |
| US20080182021A1 (en) * | 2007-01-31 | 2008-07-31 | Simka Harsono S | Continuous ultra-thin copper film formed using a low thermal budget |
| US8645450B1 (en) | 2007-03-02 | 2014-02-04 | Altera Corporation | Multiplier-accumulator circuitry and methods |
| US7865695B2 (en) * | 2007-04-19 | 2011-01-04 | L3 Communications Integrated Systems, L.P. | Reading and writing a memory element within a programmable processing element in a plurality of modes |
| US8117137B2 (en) | 2007-04-19 | 2012-02-14 | Microsoft Corporation | Field-programmable gate array based accelerator system |
| US7949699B1 (en) | 2007-08-30 | 2011-05-24 | Altera Corporation | Implementation of decimation filter in integrated circuit device using ram-based data storage |
| US8156313B2 (en) * | 2007-09-22 | 2012-04-10 | Navosha Corporation | Chained operation of functional units in integrated circuit by writing DONE/complete value and by reading as GO/start value from same memory location |
| US8959137B1 (en) | 2008-02-20 | 2015-02-17 | Altera Corporation | Implementing large multipliers in a programmable integrated circuit device |
| US8244789B1 (en) | 2008-03-14 | 2012-08-14 | Altera Corporation | Normalization of floating point operations in a programmable integrated circuit device |
| US8626815B1 (en) | 2008-07-14 | 2014-01-07 | Altera Corporation | Configuring a programmable integrated circuit device to perform matrix multiplication |
| US8301638B2 (en) * | 2008-09-25 | 2012-10-30 | Microsoft Corporation | Automated feature selection based on rankboost for ranking |
| US8131659B2 (en) * | 2008-09-25 | 2012-03-06 | Microsoft Corporation | Field-programmable gate array based accelerator system |
| US8255448B1 (en) | 2008-10-02 | 2012-08-28 | Altera Corporation | Implementing division in a programmable integrated circuit device |
| US8307023B1 (en) | 2008-10-10 | 2012-11-06 | Altera Corporation | DSP block for implementing large multiplier on a programmable integrated circuit device |
| US8645449B1 (en) | 2009-03-03 | 2014-02-04 | Altera Corporation | Combined floating point adder and subtractor |
| US8805916B2 (en) * | 2009-03-03 | 2014-08-12 | Altera Corporation | Digital signal processing circuitry with redundancy and bidirectional data paths |
| US8886696B1 (en) | 2009-03-03 | 2014-11-11 | Altera Corporation | Digital signal processing circuitry with redundancy and ability to support larger multipliers |
| US8549055B2 (en) * | 2009-03-03 | 2013-10-01 | Altera Corporation | Modular digital signal processing circuitry with optionally usable, dedicated connections between modules of the circuitry |
| US8468192B1 (en) | 2009-03-03 | 2013-06-18 | Altera Corporation | Implementing multipliers in a programmable integrated circuit device |
| US8706790B1 (en) | 2009-03-03 | 2014-04-22 | Altera Corporation | Implementing mixed-precision floating-point operations in a programmable integrated circuit device |
| US8150902B2 (en) | 2009-06-19 | 2012-04-03 | Singular Computing Llc | Processing with compact arithmetic processing element |
| US8650236B1 (en) | 2009-08-04 | 2014-02-11 | Altera Corporation | High-rate interpolation or decimation filter in integrated circuit device |
| US8396914B1 (en) | 2009-09-11 | 2013-03-12 | Altera Corporation | Matrix decomposition in an integrated circuit device |
| US8412756B1 (en) | 2009-09-11 | 2013-04-02 | Altera Corporation | Multi-operand floating point operations in a programmable integrated circuit device |
| US8539016B1 (en) | 2010-02-09 | 2013-09-17 | Altera Corporation | QR decomposition in an integrated circuit device |
| US7948267B1 (en) | 2010-02-09 | 2011-05-24 | Altera Corporation | Efficient rounding circuits and methods in configurable integrated circuit devices |
| US8601044B2 (en) * | 2010-03-02 | 2013-12-03 | Altera Corporation | Discrete Fourier Transform in an integrated circuit device |
| US8458243B1 (en) | 2010-03-03 | 2013-06-04 | Altera Corporation | Digital signal processing circuit blocks with support for systolic finite-impulse-response digital filtering |
| US8484265B1 (en) | 2010-03-04 | 2013-07-09 | Altera Corporation | Angular range reduction in an integrated circuit device |
| US8510354B1 (en) | 2010-03-12 | 2013-08-13 | Altera Corporation | Calculation of trigonometric functions in an integrated circuit device |
| US8539014B2 (en) * | 2010-03-25 | 2013-09-17 | Altera Corporation | Solving linear matrices in an integrated circuit device |
| US8862650B2 (en) | 2010-06-25 | 2014-10-14 | Altera Corporation | Calculation of trigonometric functions in an integrated circuit device |
| US8589463B2 (en) | 2010-06-25 | 2013-11-19 | Altera Corporation | Calculation of trigonometric functions in an integrated circuit device |
| US8577951B1 (en) | 2010-08-19 | 2013-11-05 | Altera Corporation | Matrix operations in an integrated circuit device |
| US8645451B2 (en) | 2011-03-10 | 2014-02-04 | Altera Corporation | Double-clocked specialized processing block in an integrated circuit device |
| US9600278B1 (en) | 2011-05-09 | 2017-03-21 | Altera Corporation | Programmable device using fixed and configurable logic to implement recursive trees |
| US10078620B2 (en) * | 2011-05-27 | 2018-09-18 | New York University | Runtime reconfigurable dataflow processor with multi-port memory access module |
| FR2979444A1 (en) * | 2011-08-23 | 2013-03-01 | Kalray | EXTENSIBLE CHIP NETWORK |
| US8812576B1 (en) | 2011-09-12 | 2014-08-19 | Altera Corporation | QR decomposition in an integrated circuit device |
| US8949298B1 (en) | 2011-09-16 | 2015-02-03 | Altera Corporation | Computing floating-point polynomials in an integrated circuit device |
| US9053045B1 (en) | 2011-09-16 | 2015-06-09 | Altera Corporation | Computing floating-point polynomials in an integrated circuit device |
| US8762443B1 (en) | 2011-11-15 | 2014-06-24 | Altera Corporation | Matrix operations in an integrated circuit device |
| US9576046B2 (en) | 2011-11-16 | 2017-02-21 | Ptc Inc. | Methods for integrating semantic search, query, and analysis across heterogeneous data types and devices thereof |
| US9098312B2 (en) | 2011-11-16 | 2015-08-04 | Ptc Inc. | Methods for dynamically generating an application interface for a modeled entity and devices thereof |
| US8909641B2 (en) | 2011-11-16 | 2014-12-09 | Ptc Inc. | Method for analyzing time series activity streams and devices thereof |
| US9443269B2 (en) * | 2012-02-16 | 2016-09-13 | Novasparks, Inc. | FPGA matrix architecture |
| US11436672B2 (en) | 2012-03-27 | 2022-09-06 | Exegy Incorporated | Intelligent switch for processing financial market data |
| US8543634B1 (en) | 2012-03-30 | 2013-09-24 | Altera Corporation | Specialized processing block for programmable integrated circuit device |
| US9098332B1 (en) | 2012-06-01 | 2015-08-04 | Altera Corporation | Specialized processing block with fixed- and floating-point structures |
| US8996600B1 (en) | 2012-08-03 | 2015-03-31 | Altera Corporation | Specialized processing block for implementing floating-point multiplier with subnormal operation support |
| US9153230B2 (en) * | 2012-10-23 | 2015-10-06 | Google Inc. | Mobile speech recognition hardware accelerator |
| US9207909B1 (en) | 2012-11-26 | 2015-12-08 | Altera Corporation | Polynomial calculations optimized for programmable integrated circuit device structures |
| US9383901B1 (en) * | 2012-12-28 | 2016-07-05 | Juniper Networks, Inc. | Methods and apparatus for navagating data center using advanced visualization |
| US9189200B1 (en) | 2013-03-14 | 2015-11-17 | Altera Corporation | Multiple-precision processing block in a programmable integrated circuit device |
| US9374344B1 (en) | 2013-03-29 | 2016-06-21 | Secturion Systems, Inc. | Secure end-to-end communication system |
| US9317718B1 (en) | 2013-03-29 | 2016-04-19 | Secturion Systems, Inc. | Security device with programmable systolic-matrix cryptographic module and programmable input/output interface |
| US9798899B1 (en) * | 2013-03-29 | 2017-10-24 | Secturion Systems, Inc. | Replaceable or removable physical interface input/output module |
| US9355279B1 (en) | 2013-03-29 | 2016-05-31 | Secturion Systems, Inc. | Multi-tenancy architecture |
| US9524399B1 (en) | 2013-04-01 | 2016-12-20 | Secturion Systems, Inc. | Multi-level independent security architecture |
| US9348795B1 (en) | 2013-07-03 | 2016-05-24 | Altera Corporation | Programmable device using fixed and configurable logic to implement floating-point rounding |
| EP2858025A1 (en) * | 2013-10-01 | 2015-04-08 | Enyx SA | An order book management device in a hardware platform |
| US9698791B2 (en) | 2013-11-15 | 2017-07-04 | Scientific Concepts International Corporation | Programmable forwarding plane |
| US9294097B1 (en) * | 2013-11-15 | 2016-03-22 | Scientific Concepts International Corporation | Device array topology configuration and source code partitioning for device arrays |
| US10326448B2 (en) | 2013-11-15 | 2019-06-18 | Scientific Concepts International Corporation | Code partitioning for the array of devices |
| US9379687B1 (en) | 2014-01-14 | 2016-06-28 | Altera Corporation | Pipelined systolic finite impulse response filter |
| US9462085B2 (en) | 2014-03-21 | 2016-10-04 | Ptc Inc. | Chunk-based communication of binary dynamic rest messages |
| US9762637B2 (en) | 2014-03-21 | 2017-09-12 | Ptc Inc. | System and method of using binary dynamic rest messages |
| US9961058B2 (en) | 2014-03-21 | 2018-05-01 | Ptc Inc. | System and method of message routing via connection servers in a distributed computing environment |
| US9560170B2 (en) | 2014-03-21 | 2017-01-31 | Ptc Inc. | System and method of abstracting communication protocol using self-describing messages |
| US10025942B2 (en) | 2014-03-21 | 2018-07-17 | Ptc Inc. | System and method of establishing permission for multi-tenancy storage using organization matrices |
| US9350812B2 (en) | 2014-03-21 | 2016-05-24 | Ptc Inc. | System and method of message routing using name-based identifier in a distributed computing environment |
| US10313410B2 (en) | 2014-03-21 | 2019-06-04 | Ptc Inc. | Systems and methods using binary dynamic rest messages |
| WO2015143416A1 (en) | 2014-03-21 | 2015-09-24 | Ptc Inc. | Systems and methods for developing and using real-time data applications |
| US9350791B2 (en) | 2014-03-21 | 2016-05-24 | Ptc Inc. | System and method of injecting states into message routing in a distributed computing environment |
| US9467533B2 (en) | 2014-03-21 | 2016-10-11 | Ptc Inc. | System and method for developing real-time web-service objects |
| US9749548B2 (en) | 2015-01-22 | 2017-08-29 | Google Inc. | Virtual linebuffers for image signal processors |
| US9684488B2 (en) | 2015-03-26 | 2017-06-20 | Altera Corporation | Combined adder and pre-adder for high-radix multiplier circuit |
| US10095479B2 (en) | 2015-04-23 | 2018-10-09 | Google Llc | Virtual image processor instruction set architecture (ISA) and memory model and exemplary target hardware having a two-dimensional shift array structure |
| US9769356B2 (en) | 2015-04-23 | 2017-09-19 | Google Inc. | Two dimensional shift array for image processor |
| US10291813B2 (en) | 2015-04-23 | 2019-05-14 | Google Llc | Sheet generator for image processor |
| US9756268B2 (en) | 2015-04-23 | 2017-09-05 | Google Inc. | Line buffer unit for image processor |
| US9772852B2 (en) | 2015-04-23 | 2017-09-26 | Google Inc. | Energy efficient processor core architecture for image processor |
| US9785423B2 (en) | 2015-04-23 | 2017-10-10 | Google Inc. | Compiler for translating between a virtual image processor instruction set architecture (ISA) and target hardware having a two-dimensional shift array structure |
| US9965824B2 (en) | 2015-04-23 | 2018-05-08 | Google Llc | Architecture for high performance, power efficient, programmable image processing |
| CN104915195B (en) * | 2015-05-20 | 2017-11-28 | 清华大学 | A kind of method that neural computing is realized based on field programmable gate array |
| US9747546B2 (en) | 2015-05-21 | 2017-08-29 | Google Inc. | Neural network processor |
| US10452971B2 (en) | 2015-06-29 | 2019-10-22 | Microsoft Technology Licensing, Llc | Deep neural network partitioning on servers |
| US11283774B2 (en) | 2015-09-17 | 2022-03-22 | Secturion Systems, Inc. | Cloud storage using encryption gateway with certificate authority identification |
| US9794064B2 (en) | 2015-09-17 | 2017-10-17 | Secturion Systems, Inc. | Client(s) to cloud or remote server secure data or file object encryption gateway |
| US10708236B2 (en) | 2015-10-26 | 2020-07-07 | Secturion Systems, Inc. | Multi-independent level secure (MILS) storage encryption |
| US9830150B2 (en) | 2015-12-04 | 2017-11-28 | Google Llc | Multi-functional execution lane for image processor |
| US10313641B2 (en) | 2015-12-04 | 2019-06-04 | Google Llc | Shift register with reduced wiring complexity |
| US10204396B2 (en) | 2016-02-26 | 2019-02-12 | Google Llc | Compiler managed memory for image processor |
| US10387988B2 (en) | 2016-02-26 | 2019-08-20 | Google Llc | Compiler techniques for mapping program code to a high performance, power efficient, programmable image processing hardware platform |
| US10380969B2 (en) | 2016-02-28 | 2019-08-13 | Google Llc | Macro I/O unit for image processor |
| US10185699B2 (en) | 2016-03-14 | 2019-01-22 | Futurewei Technologies, Inc. | Reconfigurable data interface unit for compute systems |
| US20180005346A1 (en) | 2016-07-01 | 2018-01-04 | Google Inc. | Core Processes For Block Operations On An Image Processor Having A Two-Dimensional Execution Lane Array and A Two-Dimensional Shift Register |
| US10546211B2 (en) | 2016-07-01 | 2020-01-28 | Google Llc | Convolutional neural network on programmable two dimensional image processor |
| US20180005059A1 (en) | 2016-07-01 | 2018-01-04 | Google Inc. | Statistics Operations On Two Dimensional Image Processor |
| US20180007302A1 (en) | 2016-07-01 | 2018-01-04 | Google Inc. | Block Operations For An Image Processor Having A Two-Dimensional Execution Lane Array and A Two-Dimensional Shift Register |
| US10275243B2 (en) | 2016-07-02 | 2019-04-30 | Intel Corporation | Interruptible and restartable matrix multiplication instructions, processors, methods, and systems |
| US10089577B2 (en) | 2016-08-05 | 2018-10-02 | Xilinx, Inc. | Binary neural networks on progammable integrated circuits |
| WO2018119035A1 (en) | 2016-12-22 | 2018-06-28 | Ip Reservoir, Llc | Pipelines for hardware-accelerated machine learning |
| WO2018174925A1 (en) | 2017-03-20 | 2018-09-27 | Intel Corporation | Systems, methods, and apparatuses for dot production operations |
| US10942706B2 (en) | 2017-05-05 | 2021-03-09 | Intel Corporation | Implementation of floating-point trigonometric functions in an integrated circuit device |
| US11275588B2 (en) | 2017-07-01 | 2022-03-15 | Intel Corporation | Context save with variable save state size |
| US10790828B1 (en) * | 2017-07-21 | 2020-09-29 | X Development Llc | Application specific integrated circuit accelerators |
| ES2697693B2 (en) * | 2017-07-24 | 2019-11-13 | Univ Del Pais Vasco / Euskal Herriko Unibertsitatea | IP NUCLEO, ARCHITECTURE UNDERSTANDING AN IP NUCLEUS AND DESIGN PROCEDURE OF AN IP NUCLEUS |
| US10839286B2 (en) | 2017-09-14 | 2020-11-17 | Xilinx, Inc. | System and method for implementing neural networks in integrated circuits |
| WO2019089816A2 (en) | 2017-10-31 | 2019-05-09 | Micron Technology, Inc. | System having a hybrid threading processor, a hybrid threading fabric having configurable computing elements, and a hybrid interconnection network |
| US20190164037A1 (en) * | 2017-11-29 | 2019-05-30 | Electronics And Telecommunications Research Institute | Apparatus for processing convolutional neural network using systolic array and method thereof |
| US11023235B2 (en) | 2017-12-29 | 2021-06-01 | Intel Corporation | Systems and methods to zero a tile register pair |
| US11093247B2 (en) | 2017-12-29 | 2021-08-17 | Intel Corporation | Systems and methods to load a tile register pair |
| US11789729B2 (en) | 2017-12-29 | 2023-10-17 | Intel Corporation | Systems and methods for computing dot products of nibbles in two tile operands |
| US11809869B2 (en) | 2017-12-29 | 2023-11-07 | Intel Corporation | Systems and methods to store a tile register pair to memory |
| US11816483B2 (en) * | 2017-12-29 | 2023-11-14 | Intel Corporation | Systems, methods, and apparatuses for matrix operations |
| US11669326B2 (en) | 2017-12-29 | 2023-06-06 | Intel Corporation | Systems, methods, and apparatuses for dot product operations |
| US20230229450A1 (en) * | 2018-02-23 | 2023-07-20 | Untether Ai Corporation | Computational memory |
| US10664287B2 (en) | 2018-03-30 | 2020-05-26 | Intel Corporation | Systems and methods for implementing chained tile operations |
| EP3776238A1 (en) | 2018-03-31 | 2021-02-17 | Micron Technology, Inc. | Execution control of a multi-threaded, self-scheduling reconfigurable computing fabric |
| US11275710B2 (en) | 2018-03-31 | 2022-03-15 | Micron Technology, Inc. | Loop thread order execution control of a multi-threaded, self-scheduling reconfigurable computing fabric |
| CN112055853B (en) | 2018-03-31 | 2024-04-09 | 美光科技公司 | Circuit and system for multi-thread self-scheduling reconfigurable computing architecture |
| EP3776242A1 (en) | 2018-03-31 | 2021-02-17 | Micron Technology, Inc. | Multi-threaded self-scheduling reconfigurable computing fabric |
| EP3776239A1 (en) | 2018-03-31 | 2021-02-17 | Micron Technology, Inc. | Loop execution control for a multi-threaded, self-scheduling reconfigurable computing fabric using a reenter queue |
| WO2019191743A1 (en) | 2018-03-31 | 2019-10-03 | Micron Technology, Inc. | Conditional branching control for a multi-threaded, self- scheduling reconfigurable computing fabric |
| CN111919204A (en) | 2018-03-31 | 2020-11-10 | 美光科技公司 | Efficient loop execution for multi-threaded self-scheduling reconfigurable computing architectures |
| CN111919206A (en) | 2018-03-31 | 2020-11-10 | 美光科技公司 | Backpressure control using stall signals for multithreaded self-scheduling reconfigurable computing architectures |
| US11615300B1 (en) | 2018-06-13 | 2023-03-28 | Xilinx, Inc. | System and method for implementing neural networks in integrated circuits |
| US11093579B2 (en) | 2018-09-05 | 2021-08-17 | Intel Corporation | FP16-S7E8 mixed precision for deep learning and other algorithms |
| US11579883B2 (en) | 2018-09-14 | 2023-02-14 | Intel Corporation | Systems and methods for performing horizontal tile operations |
| US10970076B2 (en) | 2018-09-14 | 2021-04-06 | Intel Corporation | Systems and methods for performing instructions specifying ternary tile logic operations |
| US10719323B2 (en) | 2018-09-27 | 2020-07-21 | Intel Corporation | Systems and methods for performing matrix compress and decompress instructions |
| US10990396B2 (en) | 2018-09-27 | 2021-04-27 | Intel Corporation | Systems for performing instructions to quickly convert and use tiles as 1D vectors |
| US10866786B2 (en) | 2018-09-27 | 2020-12-15 | Intel Corporation | Systems and methods for performing instructions to transpose rectangular tiles |
| US10929143B2 (en) | 2018-09-28 | 2021-02-23 | Intel Corporation | Method and apparatus for efficient matrix alignment in a systolic array |
| US10963256B2 (en) | 2018-09-28 | 2021-03-30 | Intel Corporation | Systems and methods for performing instructions to transform matrices into row-interleaved format |
| US10896043B2 (en) | 2018-09-28 | 2021-01-19 | Intel Corporation | Systems for performing instructions for fast element unpacking into 2-dimensional registers |
| US10963246B2 (en) | 2018-11-09 | 2021-03-30 | Intel Corporation | Systems and methods for performing 16-bit floating-point matrix dot product instructions |
| US10929503B2 (en) | 2018-12-21 | 2021-02-23 | Intel Corporation | Apparatus and method for a masked multiply instruction to support neural network pruning operations |
| US11886875B2 (en) | 2018-12-26 | 2024-01-30 | Intel Corporation | Systems and methods for performing nibble-sized operations on matrix elements |
| US11294671B2 (en) | 2018-12-26 | 2022-04-05 | Intel Corporation | Systems and methods for performing duplicate detection instructions on 2D data |
| US20200210517A1 (en) | 2018-12-27 | 2020-07-02 | Intel Corporation | Systems and methods to accelerate multiplication of sparse matrices |
| US10942985B2 (en) | 2018-12-29 | 2021-03-09 | Intel Corporation | Apparatuses, methods, and systems for fast fourier transform configuration and computation instructions |
| US10922077B2 (en) | 2018-12-29 | 2021-02-16 | Intel Corporation | Apparatuses, methods, and systems for stencil configuration and computation instructions |
| US11269630B2 (en) | 2019-03-29 | 2022-03-08 | Intel Corporation | Interleaved pipeline of floating-point adders |
| US11016731B2 (en) | 2019-03-29 | 2021-05-25 | Intel Corporation | Using Fuzzy-Jbit location of floating-point multiply-accumulate results |
| US10990397B2 (en) | 2019-03-30 | 2021-04-27 | Intel Corporation | Apparatuses, methods, and systems for transpose instructions of a matrix operations accelerator |
| US11175891B2 (en) | 2019-03-30 | 2021-11-16 | Intel Corporation | Systems and methods to perform floating-point addition with selected rounding |
| US11403097B2 (en) | 2019-06-26 | 2022-08-02 | Intel Corporation | Systems and methods to skip inconsequential matrix operations |
| US11334647B2 (en) | 2019-06-29 | 2022-05-17 | Intel Corporation | Apparatuses, methods, and systems for enhanced matrix multiplier architecture |
| US11573834B2 (en) | 2019-08-22 | 2023-02-07 | Micron Technology, Inc. | Computational partition for a multi-threaded, self-scheduling reconfigurable computing fabric |
| US11150900B2 (en) | 2019-08-28 | 2021-10-19 | Micron Technology, Inc. | Execution or write mask generation for data selection in a multi-threaded, self-scheduling reconfigurable computing fabric |
| US11886377B2 (en) | 2019-09-10 | 2024-01-30 | Cornami, Inc. | Reconfigurable arithmetic engine circuit |
| US11714875B2 (en) | 2019-12-28 | 2023-08-01 | Intel Corporation | Apparatuses, methods, and systems for instructions of a matrix operations accelerator |
| US11941395B2 (en) | 2020-09-26 | 2024-03-26 | Intel Corporation | Apparatuses, methods, and systems for instructions for 16-bit floating-point matrix dot product instructions |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5611038A (en) * | 1991-04-17 | 1997-03-11 | Shaw; Venson M. | Audio/video transceiver provided with a device for reconfiguration of incompatibly received or transmitted video and audio information |
| US5535406A (en) * | 1993-12-29 | 1996-07-09 | Kolchinsky; Alexander | Virtual processor module including a reconfigurable programmable matrix |
| US5512765A (en) * | 1994-02-03 | 1996-04-30 | National Semiconductor Corporation | Extendable circuit architecture |
| US5640106A (en) * | 1995-05-26 | 1997-06-17 | Xilinx, Inc. | Method and structure for loading data into several IC devices |
| US5692147A (en) * | 1995-06-07 | 1997-11-25 | International Business Machines Corporation | Memory mapping method and apparatus to fold sparsely populated structures into densely populated memory columns or rows by selectively transposing X and Y address portions, and programmable gate array applications thereof |
| US5640107A (en) * | 1995-10-24 | 1997-06-17 | Northrop Grumman Corporation | Method for in-circuit programming of a field-programmable gate array configuration memory |
-
1996
- 1996-11-12 US US08/748,041 patent/US5892962A/en not_active Expired - Lifetime
-
1997
- 1997-09-16 CA CA002215598A patent/CA2215598C/en not_active Expired - Lifetime
Also Published As
| Publication number | Publication date |
|---|---|
| CA2215598A1 (en) | 1998-05-12 |
| US5892962A (en) | 1999-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2215598C (en) | Fpga-based processor | |
| US5129092A (en) | Linear chain of parallel processors and method of using same | |
| Cloutier et al. | Vip: An fpga-based processor for image processing and neural networks | |
| Batlle et al. | A new FPGA/DSP-based parallel architecture for real-time image processing | |
| JP7268996B2 (en) | Systems and methods for computation | |
| EP0085520B1 (en) | An array processor architecture utilizing modular elemental processors | |
| Duff | Review of the CLIP image processing system | |
| Downton et al. | Parallel architectures for image processing | |
| EP0390907B1 (en) | Parallel data processor | |
| US6356993B1 (en) | Dual aspect ratio PE array with no connection switching | |
| EP0422348A2 (en) | Two-dimensional systolic array for neural networks, and method | |
| Kung | Computational models for parallel computers | |
| EP1070295A1 (en) | Pattern generation and shift plane operations for a mesh connected computer | |
| US4745546A (en) | Column shorted and full array shorted functional plane for use in a modular array processor and method for using same | |
| WO1999053412A1 (en) | Global input/output support for a mesh connected computer | |
| US4524428A (en) | Modular input-programmable logic circuits for use in a modular array processor | |
| JPH05242065A (en) | Information processor and its system | |
| Sunwoo et al. | A sliding memory plane array processor | |
| Wills et al. | Processing architectures for smart pixel systems | |
| Pan et al. | Computation of singular value decomposition on arrays with pipelined optical buses | |
| Maresca et al. | Parallel computer vision on polymorphic torus architecture | |
| Cloutier et al. | VIP: An FPGA-Based Processor for Image Processing | |
| Borah et al. | The MGAP: a high performance, user programmable, multifunctional architecture for DSP | |
| US5596660A (en) | Compression/reduction method for extracting iconic/symboic information from an image array | |
| Bajwa | Reconfigurable micro-grain VLSI arrays: Architectures and algorithms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKEX | Expiry |
Effective date: 20170918 |
|
| MKEX | Expiry |
Effective date: 20170918 |