WO1998024065A1

WO1998024065A1 - 3d imaging from 2d scans

Info

Publication number: WO1998024065A1
Application number: PCT/GB1997/003250
Authority: WO
Inventors: Christopher Paul Allott; Christopher David Barry; Philip Anthony Arundel; Nigel William John; Paul Maxwell Mellor
Original assignee: Zeneca Limited
Priority date: 1996-11-28
Filing date: 1997-11-27
Publication date: 1998-06-04
Also published as: GB9624753D0; JP2001504378A; CA2269323A1; AU5127998A; IL130056A0; EP1008111A1

Abstract

There is disclosed a method and apparatus for reconstructing in 3D an image of an object scanned in 2D a plurality of times to produce a plurality of 2D image data slices at different angles of inclination. The slices are recorded and stored on a recording medium e.g. video tape, whereon the slices are recorded in succession together with at least one datum which identifies the image data slices as corresponding to at least one changing physical parameter such as positioning, ECG, and/or respiration cycle. The 2D image data slices are processed to create a 3D grid of points containing data values based on at least part of an object being scanned, with the image data values being inserted at the grid points as a result of processing the 2D image data slices in dependence upon said at least one datum. Preferably the scanning is carried out by feehand ultrasound scanning, and can be employed for the non-invasive determination of a condition inside a mammalian body.

Description

TITLE

3D Imaging from 2D Scans

TECHNICAL FIELD

The present invention relates to methods of and apparatus for 3D imaging or diagnosing based on 2D scans .

BACKGROUND ART The value of three-dimensional imaging has been well recognised for many years. Accurate sequential monitoring of pathology and other variables related to volume become possible and meaningful structural information may be more readily communicated. The implementation of three dimensional techniques to ultrasound has however proved to be a difficult and lengthy process. As early as 1956 three-dimensional and stereoscopic observations were made on body structures by ultrasound (Howry et al . 1956) . Reconstruction of ultrasound data in 3D, allowing volumes to be measured independently of the data acquisition views and angles was reported in 1980, (Moritz et al . 1980).

Since then, a considerable number of reports have appeared on the development of methods for 3D reconstruction of ultrasound data (reviews see Rankin et al.1993; Vogel et al . 1995; Levme et al . 1992).

Several studies describe methods using conventional 2D transducers to acquire parallel or near-parallel image sets. The transducer may be advanced with a stepping motor (Franceschi et al . 1992; Hell et al . 1995; Moskalik et al . 1995; Vogel et al . 1995), by a freehand sweep (Geiser et al . 1982; Gardener et al . 1991; Kelly et al . 1994; King et al . 1990; Moritz et al.1980; Nelson et al . 1996; Riccabona, et al 1995 ), or m intravascular studies, by timed pull-back of the catheter ( von Birgelen et al . 1995; Mmtz et al .

1992, Rosenfield et al . 1991), or by rotation about a central axis (Kok-Hwee et al . 1994) .

Where freehand techniques are used, the image positioning can be obtained from simultaneous recording of the position and orientation of the transducer using mechanical arm, acoustic spark gap or electromagnetic sensor techniques (Detmer et al . 1994; Geiser et al . 1982; Gardener et al . 1991; Hernandez et al . 1996; Kelly et al . 1994; King et al . 1990; Moritz et al . 1980; Moskalik et al . 1995; Nelson et al . 1996; Riccabona et al.1995).

Dedicated 3D ultrasound systems have also been described which utilise tomographic transducers to interrogate a volume by either internal mechanical sweeping techniques or by use of a 2 dimensional array. ( Hamper et al . 1994; von Ramm et al.1996; Zosmer et al. 1996) .

To date, all the methods that have been described for 3D ultrasonography have limitations. Described systems based on stepping motors or freehand sweeps have been limited by the relatively low number of data slices contributing to the 3D reconstruction, or the large slice interval. Data interpolation techniques have been used to "fill in" the missing data for reconstruction. Coincident image frames have been actively avoided to overcome the problem of dealing with multiple values being generated for a single point in the data block. As has already been pointed out in the literature (Nelson et al.1996) this prohibits any possibility of compounding data to improve the quality of the 3D images. Dedicated 3D transducers are often bulky and can only be used to acquire a 3D volume equal to the insonated volume from a stationary probe. Such systems may limit the ability to obtain preferred views, particularly where shadowing may occur from overlying structures. Insonation angle for data acquisition is often fixed in the 3D methods, while 2D ultrasonography routinely allows interrogation from a variety of angles to optimise structure boundary definition. The angle dependency of ultrasound reflection and backscatter intensities when investigating tissue composition was reported in 1985 (Picano et al.1985). An ability to "compound" these multiple angles of insonation into a single data set would significantly improve signal to noise and thus speckle contrast and produce the most coherent object for segmentation and reconstruction (Hernandez et al . 1996; Hughes et al . 1996; Moskalik et al . 1995; Nelson et al.1996; Shattuck and von Ramra 1982) .

DISCLOSURE OF THE INVENTION

An object of the present invention is to provide a system based preferably, but not essentially, on freehand 2D ultrasound scanning, capable of delivering precise and rapid 3D reconstruction and leading to successive grey-style segmentation and volumetric analysis .

Embodiments of the invention may be capable of fitting into the biomedical and clinical research environments to allow 3D Ultrasound to take its place alongside the other mainstream imaging modalities of Magnetic Resonance Imaging (MRI) and Computerised Tomography (CT) that can routinely exploit the advantages of 3D imaging and analysis.

According to one aspect of the invention there is provided a method for reconstructing in 3D an image of an object scanned in 2D a plurality of times to produce a plurality of 2D image data slices at different angles of inclination, said plurality of 2D image data slices being recorded and stored on a recording medium whereon said 2D image data slices are recorded in succession together with at least one datum which identifies said 2D image data slices as corresponding to at least one changing physical parameter which varies in time as the 2D scanning takes place, said 3D image of the object being reconstructed from the recorded 2D image data slices in dependence upon said recorded at least one changing physical parameter.

According to a second aspect of the invention there is provided an apparatus for use in reconstructing in 3D an image of an object scanned in 2D a plurality of times to produce a plurality of 2D image data slices, said apparatus comprising scanning means operable to scan an object to produce said 2D image data slices at different angles of inclination, recording means coupled to said scanning means and operable to record the output thereof onto a recording medium whereon said 2D image data slices will be recorded in succession, said recording means also being operable to record onto said recording medium, together with said 2D image data slices, at least one datum which identifies said 2D image data slices as corresponding to at least one changing physical parameter which varies in time as the 2D scanning takes place, and processing means coupled to said recording means and operable to reconstruct said 3D image of the object from said recording medium, in dependence upon said recorded at least one changing physical parameter. According to a third aspect of the invention there is provided a method for reconstructing in 3D an image of an at least part of an object scanned in 2D a plurality of times, wherein a plurality of 2D image data slices produced as a result of said scanning at different angles of inclination are processed to create a 3D grid of points containing data values, the said plurality of 2D image data slices being associated with at least one datum which identifies the various positionings of said 2D image data slices relative to said object, said 3D grid being constructed based on said at least part of an object being scanned, and image data values being inserted at said grid points as a result of processing of said 2D image data slices in dependence upon said least one datum.

According to a fourth aspect of the invention there is provided a method of calibrating a scanning and position detecting device having a position detecting transmitter defining a registration frame, a position detecting receiver cooperable with said position detecting transmitter and having its own coordinate frame, and a scanning transducer mechanically connected to said position detecting receiver and having a coordinate frame associated with the image it produces, wherein the transformation from said image coordinate frame to said position detecting receiver coordinate frame is determined by scanning a point or volume in space from different transducer angles and positions, and employing an iterative mathematical process on the resultant data, thereby to calculate said transformation.

It is to be appreciated that this fourth aspect of the invention may be employed in combination with the other stated aspects of the invention. According to a fifth aspect of the invention there is provided a method for the non-invasive determination of a condition inside a mammalian body, comprising reconstructing in 3D an image of at least part of said body scanned in 2D a plurality of times to produce a plurality of 2D image data slices at different angles of inclination, said plurality of 2D image data slices being recorded and stored on a recording medium whereon said 2D image data slices are recorded in succession together with at least one datum which identifies said 2D image data slices as corresponding to at least one changing physical parameter which varies in time as the 2D scanning takes place, said 3D image of said body being reconstructed from the recorded 2D image data slices in dependence upon said at least one changing parameter. According to a sixth aspect of the invention there is provided a method for the non-invasive determination of a condition inside a mammalian body, comprising reconstructing in 3D an image of an at least part of an object scanned in 2D a plurality of times, wherein a plurality of 2D image data slices produced as a result of said scanning at different angles of inclination are processed to create a 3D grid of points containing data values, the said plurality of 2D image data slices being associated with at least one datum which identifies the various positionings of said 2D image data slices relative to said object, said 3D grid being constructed based on said at least part of an object being scanned, and image data values being inserted at said grid points as a result of processing of said 2D image data slices in dependence upon said least one datum.

Preferred additional features are set out in the subsidiary claims .

There is disclosed as a preferred embodiment a system that rapidly produces a regular 3D data block suitable for processing by conventional 3D analysis and volume measurement software . The system uses electromagnetic spatial location of freehand-scanned Ultrasound B-Mode image frames or slices, signal conditioning hardware and UNIX based computer processing. An efficient algorithm has been developed that populates a Cartesian grid with data extracted from the 2D image frames, acquired with a variety of interrogation angles. The utilisation of data from multiple angles of insonation reduces the angle- dependency of reflection intensity from each interface. Such "compounding" was found to significantly reduce speckle contrast, improve structure coherence within the 3D greyscale image and enhance the ability to detect, segment and measure volumes on the basis of structure boundaries .

Volume measurement based on automated greyscale segmentation of a series of water filled latex and cylindrical foam rubber phantoms with volumes in the range 0.9 to 8.0 ml . show that a high degree of accuracy, precision and reproducibility can be obtained. The 3D reconstruction and automatic greyscale segmentation of water filled latex phantoms gave volumes for the enclosed water with rms accuracy of 1.1%, while the volumes of the foam rubber phantoms showed rms coefficients of variation of 1.4% (test- retest) and 1.3 (inter-observer).

The disclosure here shows that two-dimensional (2D) images acquired with conventional, freehand, scanning techniques can be reconstructed to provide a three-dimensional (3D) map of echo-intensities that allows reliable and accurate volume measurement of structures of interest following grey-scale segmentation. Extension of the technique to handle in vivo data sets by allowing physiological criteria to be taken into account in selecting the images used for reconstruction is also illustrated.

Satisfactory 3D reconstruction from freehand 2D ultrasound images requires precise spatial registration of the ultrasound image in a common reference frame. Compounding of the data places a particular emphasis on the precision of this registration and requires compensation for systematic errors associated with any position sensing device (Detmer et al . 1994; Moskalik et al . 1995) . A 3D data block can then be generated and the intensity data from each image extracted into its appropriate position. A precise method of compensating for, or gating to, physiological motion such as respiration or the cardiac pressure cycles in blood vessels using electro-cardiographic (ECG) recording, is often preferred to ensure that such motion does not disrupt or deform the integrity of the structure to be reconstructed. Also, if the technique is to have any practical application, the output from the process must be available within a short time of completing the scan and be in a form that can exploit the highly efficient image processing and analysis tools developed for other medical imaging modalities. Although computer model fitting to generate volume measurement could be used, this may not deal adequately with pathology. A sufficiently high degree of confidence and reliability in the results of automated segmentation on the basis of reconstructed, grey-scale, echo intensities and any subsequent volume measurements and analysis require the generated 3D ultrasound image quality to be improved beyond that represented by a typical 2D ultrasound frame.

Thus, a preferred embodiment of our optimal 3D ultrasound system may have the features that:

1. The ultrasound scanning equipment not be irretrievably modified.

2. The freehand, interactive nature of ultrasound scanning be maintained as near normal as possible with the option for 2D analysis retained.

3. All data be recorded continuously rather than restrict acquisition to particular periods within the cardiac or respiratory cycles. Post processing techniques can then be used to select appropriate image frames. This allows re-interrogation of data at different physiological states without the need to repeat the ultrasound scan.

4. The intensity values in the 2D ultrasound images be preserved to allow reconstruction, volume measurement and greyscale analysis of the acquired data.

5. Image quality be improved beyond that represented by typical 2D ultrasound.

6. A 3D data block be generated, directly compatible with 3D image analysis products.

Hereinafter we describe a system which is capable of fulfilling these criteria, delivering significant benefits, at no risk to the equipment or the ability to revert to conventional 2D analysis of the ultrasound images .

Accordingly, preferred embodiments and methods are described with reference to the accompanying drawings, and stressing the importance of particular features of those preferred embodiments and methods. For example only, an electromagnetic position and orientation sensor (EPOS) is described wherein particular emphasis is given to the particular method of calculation of a Δ transform between coordinates of data points in a scan slice, and coordinates of the EPOS receiver. It is to be understood that none of the particular features of the preferred embodiments and methods beyond the general features of the claims are essential features of the invention per se, and can be replaced by alternative features in different particular systems.

Although the invention is described as applied to a biological sample or living tissue, it could also be applied to an inanimate object for the detecting of flaws or cracks, for example in a manufactured metallic obj ect .

It is further to be understood that the invention could be applicable to scanning employing radiation other than ultrasound, and scanning other than freehand scanning.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention and to show how it may be put into effect, reference will now be made by way of example to the accompanying drawings attached to the rear of this specification, wherein: Figure 1 is a diagram of system components of a preferred embodiment of the present invention;

Figure la shows circuitry details of Figure 1; Figure 2a shows an ultrasound transducer combined with a Polhemus receiver;

Figure 2b shows transforms relating to the apparatus of Figure 2a;

Figure 2c shows processed signals used for frame selection;

Figure 3 is volumetric reconstruction of a foam phantom; Figure 4 shows images of a phantom produced by the system of Figure 1; and the remaining tables and figures relate to the description herein of the diagnostic scans carried out in the Watanabe rabbit, and the scans of human carotid artery bifurcations, as discussed hereinafter.

DETAILED DESCRIPTION OF THE DRAWINGS

In Figure 1, a Polhemus transmitter 1 is fixed adjacent to an object to be scanned, and emits electromagnetic waves. The transmitter 1 is coupled to an EPOS Polhemus 2 which, in return, receives signals from a Polhemus receiver 2a rigidly coupled to a Toshiba scanner 3 and a probe 4.

Synchronisation and processing hardware 5 coupled to the EPOS Polhemus 2, scanner 3 and probe 4 is itself coupled to a video recorder 6 to provide image data to a video channel thereof, with positioning (EPOS) and physiological (ECG) information to two respective audio channels.

The output of video recorder 6 can be fed to a processor 7 with storage, which is itself coupled to a frame select circuit 8.

The output of the processor storage is fed to a further processor 9 for the reconstruction of a 3D grid map, whilst a PC 10 is arranged for reconstructing 3D images and measurements from the 3D grid map. The function of the system is described hereinafter in three phases . Figure la shows details of the synchronisation and processing hardware 5 and will be described later.

Figure 2a shows the physical arrangement of the Polhemus receiver 2a and scanner 3 and probe 4 of Figure 1, with a perspex strip connecting the receiver 2a to the other parts.

Figure 2b indicates the transformations required in order to transform between a point in a scanned image data slice having coordinates qrs within the slice, and the coordinates XYZ of the reference frame of the transmitter 1. The significance of these transforms will become apparent later.

The system implementation falls into three separate phases (figure 1) .

• Data Acquisition to S-VHS Video tape • Data Extraction and 3D Reconstruction

• Analysis and Presentation of 3D Reconstructions Fig. 1 System organisation II.1 DATA ACQUISITION TO S-VHS VIDEO TAPE

Phase 1 comprises the complete and continuous recording of the ultrasound images, their encoded positional information and any ECG wave form. The three inputs were captured on the video and two associated stereo audio channels of an S-VHS video recorder 6 (Panasonic AG 7350, Matsushita Electric Industrial Co., Ltd. Osaka , Japan). The ultrasound image frames were generated on a Toshiba SSH 140a ultrasound scanner 3 (Toshiba Medical Systems U.K. Crawley. England) fitted with a 7.5MHz linear array transducer 4. The positional information for registration was obtained from an electromagnetic position and orientation sensor [EPOS] Polhemus '3

Space Isotrak II' system (Polhemus Inc., Colchester, Vermont, USA) , controlled and synchronised by custom hardware 5 shown in Fig, la. This hardware also conditioned any ECG signal to a format suitable for recording on the video audio channel.

ACQUISITION OF ULTRASOUND IMAGES TO TAPE

All ultrasound scans were performed using a Toshiba SSH 140a ultrasound scanner 3 fitted with a 7.5MHz linear array transducer 4. Image generation parameters were pre-set to optimise the visualisation of the object of interest. B-Mode ultrasound images were acquired at the standard rate of 31 frames per second (fps) and during interrogation were recorded continuously using an S-VHS PAL video recorder

(Panasonic) at the lower rate of 25 fps, the remaining 6 fps being dropped systematically during video readout from the internal frame buffer of the scanner: a single interrogation generating around 10,000 2D images on videotape. SIGNAL PROCESSING HARDWARE

Modular signal processing hardware 5 allows the recording of positional and any ECG information on to the two hi-fi audio channels of the video recorder.

RECORDING THE POSITIONAL INFORMATION

Ultrasound transducer positional information was obtained from an electromagnetic position and orientation sensor (EPOS), Polhemus '3 Space Isotrak II' 2, 2a attached to the ultrasound scanning transducer 3 as shown in fig 2a.

In Fig la, the first process module generated the required command line for the EPOS by a programmable controller 11 (PIC16C54) , following a single manual contact closure. The controller was clocked by a 8MHz crystal oscillator, communicating with the EPOS via an RS-232 line driver (MAX233) which operates at 9600 baud. The command line consisted of thirteen characters defining the output list as the x,y,z linear co- ordinates, plus angles for roll, pitch and yaw, interspersed with "carriage return" and "line-feed" commands. On receipt of a synchronising pulse (see below) , the EPOS transmitted the output list corresponding to the sensor position at that instant, and therefore corresponding to the current video frame. This output binary data stream was returned to the module and scaled, using a line driver 12 (MAX483) and potential divider 13, to provide a 0.4V amplitude unbalanced signal suitable for the audio input channel of the S-VHS video tape recorder.

For each video frame to be recorded together with the corresponding positional information, synchronisation was necessary between the scanner and EPOS. The latter was run in "non-continuous" mode, awaiting a pulse to start the data stream. To achieve this, the composite video output from the scanner was taken to a purpose-designed "synchronisation " module. The composite signal was fed to a sync separator circuit 14 (TDA8128) which provided an output pulse for each video field. As two interlaced fields form each frame, a bi-stable circuit 15 (4013) was used as a divide-by-two to give a single output pulse per frame. This monophasic pulse activated a differential line driver 16 (MAX483) to provide the external synchronisation signal for the EPOS. A simple diode- pump circuit 17 also integrated field pulses to light an indicator as a visual confirmation of synchronisation pulse presence.

The above processing was designed to ensure that the positional information was sent to the tape concurrently with the corresponding video frame. In practice, there was a slight latency of around 18ms between the instant of frame starting and the positional data stream. This was accurately measured and compensated for during subsequent editing in the digital frame store medium.

ECG RECORDING

In choosing to record ultrasound interrogations continuously and not utilise instrument gating, a facility has been provided to enable frames to be selected for reconstruction on the basis of their temporal position in the cardiac cycle in the case of in vivo interrogations. When necessary, a conditioned ECG signal is recorded on the second hi-fi audio channel of the video recorder.

The ECG conditioning module can be considered in two sections. The electrically-isolated front end consisted of a conventional differential instrumentation pre-amplifier (Burr-Brown Ltd. Livingston, West Lothian.) with a voltage gain of 50, configured together with an external operational amplifier to provide active drive of the indifferent electrode. This front end was interfaced with the second section of the circuit via an isolation amplifier (Burr-Brown ISO 107) which also provided isolated power supplies for the pre-amplifier and driver. There was no ground connection to the subject, the resultant isolation (2500 volts a.c. rms., 3500 volts d.c.) allowing safe clinical use. The isolator output was connected, via a variable attenuator, to the second, non-isolated, section of the circuit. This consisted of a capacitively coupled inverting amplifier, with a fixed d.c. gain of x 10, and low pass filtering to remove the high frequency ripple artefact induced by the oscillator within the isolation amplifier. An output socket at this point provided amplified ECG for display on the ultrasound scanner screen by connection to the high level d.c, non- isolated, patient input socket. A second output of this ECG stage was routed within the module to a differentiator, for enhancement of the QRS wave and partial suppression of the P and T waves. A fast time constant removed baseline shifts. This heavily filtered version of the ECG was scaled for recording through a hi-fi audio channel (20Hz to 20kHz) on the S-VHS video tape recorder, and was stored concurrently with the video images .

Fig. 2c shows ECG and Polhemus signals recorded on tape by the video tape recorder.

Instead of or in addition to an ECG signal, a respiratory signal detected by conventional means could be employed.

II.2 DATA EXTRACTION AND 3D RECONSTRUCTION

Phase 2 starts with the transfer of a continuous sequence of image frames, each with its associated EPOS and ECG data, from the video tape to the computer. This is followed by the selection of a subset of frames on the basis of their image content and/or timing within the cardiac cycle, and finally the reconstruction process .

ULTRASOUND IMAGE EXTRACTION

Data from the videotape were digitised on a 100MHz Silicon Graphics Indy R4600PC workstation 7 configured with 96Mbytes of memory, a Video option card, and a Cosmo Compress motion JPEG (Joint Photographic Experts Group) video compression card. Four external SCSI disks were connected to the Indy and striped to provide a fast 8 Gbyte disk store. To achieve real-time video acquisition with ultrasound images (using PAL timing) , the JPEG quality factor on the Cosmo card was set to

75%, giving approximately 15:1 compression ratio. Data were stored in Silicon Graphics' proprietary movie file format. Typically four minutes of video were acquired, providing movie files containing 6000 image frames and occupying 600 Mbytes.

EPOS AND ECG DATA EXTRACTION

The audio hi-fi channels containing the ECG and EPOS data were sampled using the Indy analogue audio input at 48 kHz. The audio files produced in Silicon

Graphics format were linked to the Moviefile in such a way that the audio associated with any individual image frame could be uniquely extracted. The positional information and ECG could be then be carried and processed with any selected image frame.

Silicon Graphics' IRIS Digital Media software tools Audio Panel , Video Panel , and Capture, supplemented with appropriate software, provided a user interface for all data capture. Review and selection of image and audio frames was provided by Movie Player, Movie Maker, and Sound Edi tor again supplemented with appropriate software written in-house.

IMAGE FRAME SELECTION

For data sets where there was no respiratory or cardiac cycle motion associated with the structure under investigation, intact sequences of frames, or appropriate sub-sets, were selected and extracted using custom command line utilities indicating start, end and frame interval . Where ECG gating of images is required to eliminate motion associated with the cardiac cycle, this can be done retrospectively using the digitised QRS waveform. For this purpose a utility has been developed with an easy to use graphical interface which allows frames to be selected based on their position within cardiac cycles whose duration falls within a specified range. Interrogation of the digitised audio channel containing the processed ECG waveform displays a histogram of cycle lengths from which the user selects the most relevant samples. The user specifies a time or percentage based window in the cardiac cycle for which frames will be selected, avoiding the use of excessively long or short duration cycles. For each valid ECG cycle, the exact time for the end points of this window is calculated and mapped to the corresponding frames on the image track of the movie file to generate a "valid frames list" file.

Typically, around 500 valid frames will be selected from each data set for 3D reconstruction and recorded in the "valid frames list" file.

Similar processing and gating of images to eliminate motion associated with the respiratory cycle can be achieved similarly, replacing the ECG signal by a conventionally generated signal indicating the respiratory cycle. IMAGE FRAME PROCESSING

Some preliminary processing of the ultrasound video frames was undertaken. The data captured in 32 bit integer format by the Cosmo Compress card was reduced to 8-bit grey scale values. De-interlacing of the PAL video was provided, and redundant video information such as text, grey scale bar, blank spaces and grid lines were removed. Finally, to reduce further the sheer quantity of data to be processed by the 3D reconstruction software, images were scaled to half their original size using an anamorphic image scaling algorithm (Schumacher 1991) . This preliminary processing achieved an overall reduction of around 98% in data size without compromising significantly the quality of the final reconstructions. Extraction from the movie file and conversion to the required format was performed by a background batch process controlled by customised command line utilities and the previously generated "valid frames list" file. Using our 100 MHz, R4600 based SGI Indy, this task took approximately 3 seconds per image; however, this reduces very significantly using R5000 and R10000 based systems now available at a similar cost.

3D REGISTRATION OF ULTRASOUND IMAGE DATA

FIGURE 2a - Physical configuration of EPOS - ultrasound transducer mounting

FIGURE 2b - Co-ordinate systems and transformations

In our system, the EPOS transmitter 1 was sited at a convenient location within a 30 cm radius of the object being scanned and remained fixed throughout the data acquisition. The origin and axes of the transmitter established the fixed "registration frame" (designated XYZ in figure 2b) . The EPOS readings tracked the position and orientation of the receiver 2a, and thus the receiver co-ordinate frame (designated xyz) , relative to the fixed transmitter frame. The transformation relating the co-ordinates of a point in frame xyz to those of the same point in frame XYZ was designated M and obtained directly from the EPOS. The EPOS receiver 2a was attached to the ultrasound transducer 3,4 by mounting on a short (15cm) Plastic strip fixed securely to the transducer to minimise electromagnetic influences (figure 2a) . The co-ordinate frame associated with the ultrasound image itself (designated qrs, with q always zero), while fixed relative to the receiver frame xyz, has an offset in both position and orientation. A transformation, including both translation and rotation, had to be applied to correct for the position and orientation of the EPOS receiver in relation to the 2D ultrasound image. We refer to this as the Delta (Δ) transformation which has to be determined only once for each specific mounting of the EPOS on the ultrasound transducer before the ultrasound images can be properly registered.

Specifically, the transformations M and Δ together relate the co-ordinates of a point in space measured using the ultrasound image axis system qrs to the coordinates of the same point measured in the registration axis system XYZ.

If the former co-ordinates are represented by p(qrs) and the latter by P(XYZ), then

P(XYZ) = M * Δ* p(qrs) } or p(qrs) = ( M * Δ)^"1 * P(XYZ) } Equation 1

Δ^"1 * M^"1 * P(XYZ) } DETERMINATION OF THE DELTA TRANSFORMATION

The Delta (Δ) transformation was determined by scanning a calibration phantom consisting of two crossed threads suspended in a bath of 20%w/v galactose solution using a wide range of transducer angles and positions.

The galactose solution provided a transmission medium for the ultrasound that more closely corresponded to the speed of sound transmission in normal tissue than does water. The crossover provided a point in space whose co- ordinates were unknown but fixed in relation to the EPOS transmitter. After digitisation on the Silicon Graphics Indy the frames containing images where the cross was visible were extracted from the moviefile. These 2D frames were displayed and the image location of the centre of the cross (r,s) determined and noted. A minimum of two observations were made for each attitude, totalling some 50 observations of this unique point in space. The vastly over-determined series of equations relating the unknown fixed position, P(XYZ) and transformation Δ to the known value pairings of p(qrs) and M (equation 1) was then solved for P(XYZ) and Δ. Here an iterative process was used based on a starting trial Δ followed by a steepest descent refinement of the 3 angles and 3 distances defining the Δ transformation. The criteria function minimised was the root-mean-square deviation of the individual estimates of P(XYZ) from their group mean. The resulting Δ transformation matrix was subsequently used for all reconstruction with the specific ultrasound transducer/EPOS receiver configuration for fixed settings of the "field of view and scaling" parameters for the scanner. Any change in these two parameter settings, in general, will result in the introduction of a new "qrs" co-ordinate system and require experimental determination of the new Δ transformation matrix; a mathematical relationship will exist between any two such Δ's that will be unique for the specifc transducer.

3D RECONSTRUCTION USING GRID-MAPPING

Determination of the Δ transformation calibrates the ultrasound transducer/EPOS configuration necessary for the accurate registration of the 2D ultrasound images in the 3D co-ordinate system established by the fixed transmitter. The second critical step is Grid- mapping. This is the process which uses the data values observed on a number of oblique, non-parallel ultrasound planes to compute the echo intensities at points on a regular 3D grid in a format compatible with software for presentation, segmentation and analysis. It is essential that the algorithm used for this application is very efficient as a typical reconstruction will involve the calculation of around 2 million new positional intensity values from an input data set of some 12 million values contained on 500 registered frames.

There are two possible implementations of this algorithm. The first involves sequencing through the regular array of grid positions and at each point identifying and using the sub-set of the US data that is "relevant" to the calculation of the grid value. The second involves sequencing through the input data points and accumulating the contributions to the "relevant" subset of grid positions. In both cases the "relevant" subsets can be specified by introducing a limiting radius "R" within which the relative weights of contributions are made a function of the distance "r" between the data point and the grid position. While the first option appears the more direct, the second is much more efficient. In addition to allowing full advantage to be taken of the systematic organisation of the US data, the second approach enables effective use to be made of the limiting radius to reduce very significantly the number of grid-point, data-point pairs used in the calculation of the 3D reconstruction. In typical cases where the limiting radius "R" is of the same magnitude as the data spacing on the Ultrasound frame and of the grid spacing (0.2mm), the saving in total computation time can approach the ratio of the volume of the grid-mapped box to the volume of a sphere of radius "R" . Noting the intrinsic parallel nature of the algorithm, this second approach has potential to support close to real-time implementations, utilising state of the art computing techniques. Our implementation follows the second approach and uses an "inverse distance l/r" weighting scheme with two nested cycles, the outer cycle indexing through the individual 2D ultrasound frames, the inner cycle indexing through the data values and positions associated with each of the US frames. As the 2D ultrasound frames are processed two sums are accumulated for each voxel in the reconstruction. The terms of the first sum are the measured echo intensities of the ultrasound data points which, when registered in 3D, fall within the limiting radius "R" of the voxel centre, scaled by a factor equal to the inverse distance from the voxel centre to the data point . The terms of the second sum are the inverse distance scaling factors themselves. The ratio of the final values of these two sums provides the normalised, distance weighted average assigned to the voxel .

The location and orientation of the 3D reconstruction grid is defined in relation to a user selected "KEY" ultrasound frame, typically one that is centrally located and depicts a complete cross-section of the object of interest. The orientation and position of the KEY ultrasound frame relative to the EPOS transmitter co-ordinate system is determined by the EPOS values and knowledge of the Δ transformation. The origin of the 3D grid will be at the centre of the KEY frame and the grid axes will be parallel to those of this ultrasound frame. The transformation relating the EPOS transmitter co-ordinate system and that of the 3D reconstruction grid is obtained from the co-ordinates of the centre of the KEY frame and the normalised axis- vectors of the oriented KEY US frame.

II.3 ANALYSIS AND PRESENTATION OF RECONSTRUCTIONS

Phase 3 of the system covers the segmentation, presentation and analysis of the information inherent in the reconstructed 3D array of echo intensities.

Image analysis and measurement was provided by commercially available software from IBM : TOSCA (TOols for Segmentation, Correlation and Analysis) and DX (Data eXplorer) mounted on Power PC UNIX workstations configured with 128 Mbytes or more of memory, 3Gbytes of disc, 80 MHz processor and GT4E graphics adapter. TOSCA implements a three-dimensional region-growing algorithm for automatic grey-scale segmentation (Elliot et al . 1996; Sivewright et al . 1994). Using the statistics for the voxel values in the immediate neighbourhood of a user-selected seed-point which are characterised by window and level parameters, the algorithm tests adjacent voxels for inclusion in the same region-of -interest (ROI) , iterating this process until there are no more contiguous voxels consistent with the seed-point statistics. For display purposes only, a smooth contour or surface is generated bounding the ROI; the volume is determined by counting the statistically acceptable voxels rather than by estimating the volume bounded by the smoothed surface. Such volume estimates compare well with results obtained using other methods based on contouring techniques or on edge detection algorithms provided the boundary of the ROI is reasonably continuous and uniform in intensity. After processing, all data sets were reconstructed and displayed, processing usually taking less than 30 minutes . The regular shapes of the phantoms (see later) allowed visual assessment of the reconstructions regarding distortions or disruptions of shape,- no obvious distortions or disruptions of shape were discerned and even the 0.5mm wire used to suspend the foam rubber phantoms was successfully resolved (fig. 3).

The three critical elements in our system were : -

1) Attainment of a high degree of precision in positioning the individual frames of 2D ultrasound data in space so that the extensive Compounded data sets can be accurately "registered" in 3D and used effectively in the reconstruction process.

2) Implementation of a highly efficient "grid mapping" algorithm that allows 3D reconstructions to be generated in a timely and routine manner with a typical UNIX workstation.

3) Development of objective "image quality measures" that correlate well with the ability to segment on the basis of grey-scale echo intensity values.

Accurate determination of the Δ transformation is the first critical step in image reconstruction. It provides the calibration of the ultrasound transducer - EPOS transmitter configuration necessary for the accurate registration of the 2D ultrasound images in the 3D coordinate system established by the fixed transmitter. The next critical step is "grid-mapping" and involves mapping of the ultrasound intensity values onto a regular 3D grid suitable for input to commercial software for presentation, segmentation and analysis including volume assessment .

Grid-mapping involves computing ultrasound echo intensities at points on a regular 3D grid from the data values observed on a number of oblique, non-parallel 2D ultrasound planes. It is very important that the algorithm used for this application is efficient as a typical reconstruction will involve calculating 1 to 2 million values from an input data set containing some 500 frames each with 25,000 echo intensity values. The number and distribution of echo intensities will depend on the scanning pattern, frame selection and the radius examined around each 3D grid point. The single value assigned to each grid point must represent the ultrasound echo intensities observed within this radius. This was achieved through a 1/r weighting scheme with the cut-off radius ( R ) normally set to be 0.25 mm. , slightly larger than the separation between data points in the ultrasound image planes which is typically 0.2 mm. With a sufficient density of data points the result will be insensitive to the exact functional form of the weighting scheme.

The specifics of our implementation are as follows. Within the outer cycle, the position and orientation of the 2D ultrasound frame is computed with reference to the grid co-ordinate system The box bounding the grid is then mapped onto the ultrasound image plane and the indices (x and y) representing the limits of an ex- scribed rectangle containing all data points relevant to the reconstruction and with sides parallel to those of the ultrasound frame are computed. This is done by transforming the co-ordinates of the eight corners of the grid into the co-ordinate system of the data frame using its origin and orientation cosines. The intersections of the ultrasound data plane (z=0) with the twelve edges of the grid are determined and the limits in x and y expressed in terms of the discrete data indices. If the data plane pass through the grid box a minimum of three and a maximum of six edges will be intersected. Data points with x,y indices lying outside these limits can be ignored, reducing the size of the inner cycle computation. In some cases , for example frames captured as the probe is repositioned between sweeps, this test allows entire data planes to be set aside with a minimum of computation.

The inner cycle sequences through the data points lying within the limits of the x,y index range. As in the 3D grid co-ordinate system these data points lie on a regular, square lattice, the position of each point express in terms of fractional 3D grid indices can be computed incrementally using the two vectors which specify the unit displacements along the rows and columns of the ultrasound data. For a selected limiting radius ( R) and using the 1/r weighting function, each final grid value can be calculated from the number of individual contributions and the sums of their individual weights and weighted echo intensities.

For this purpose the grid is represented by a 3 x N array with an appropriate mapping of the linear positional indices. The three sums for each grid position are accumulating as each US data point is processed and the contributions calculated for each of the indices corresponding to the "relevant" sub-grid positions. Our program code makes provision for handling the special case where the Ultrasound data point coincides exactly with a grid-map point . Upon completion of the cycles over data planes and data points, the intensity value for each and every grid point was computed by dividing the weighted intensity sum by the sum of the weights.

Three additional facilities were implemented to support specific requirements.

1) The dimensions of the reconstruction box can be extended by an amount equal to a "fringe" parameter. This has been used to combine two adjacent reconstructions into a single block. In this case the "fringe" would be set to the same value as the limiting radius ( R ) , avoiding potential edge effects where the two boxes touch. This allowed data points outside the actual box but which were within the limiting radius of the grid points on, or close to, the box surface to be included in the calculations.

2) In order to handle a range of Ultrasound frame densities, a parameter specifying the minimum number of contributions that must be present for any grid-point to be assigned a value, was implemented. The settings for this parameter will also be influenced by the resolution of the grid and the limiting radius ( R ) .

3) Data planes which deviated significantly from the orientation of the "KEY" frame can be rejected. The angle of the vector normal to the key plane and to all other contributing data planes was calculated in turn and tested with the option to reject the entire candidate plane if the angle was greater than a specified value (typically 45 degrees) .

THE ZENECA 3D ULTRASOUND ENVIRONMENT

The system developed within Zeneca has many advantages over existing 3D systems for ultrasound imaging. Scanning is achieved freehand in real time rather than using a step acquisition of frames and gives access to normal 2D scanning procedures and measurements. The storage of registered data onto videotape provides a cost effective data storage system that can be reviewed in normal 2D. Sets or portions of data sets can then be loaded into the 3D system at any time after acquisition. Throughout the 3D processing the original ultrasound intensity data is retained. This gives opportunity for interrogation of data within structures in addition to rendering and visualisation of the surface.

A unique system of image frame sub-set selection, from the digitised data set, allows interrogation of the data at precise points within the cardiac cycle. Reconstructions of 3D objects may be obtained at specified intervals within the cardiac cycle to identify any motion or structural changes resulting from the pressure changes during the cardiac cycle. The system also features an output of a 3D datablock suitable for volume measurement utilising state of the art automatic segmentation packages designed for other imaging modalities such as MRI and CT.

Briefly, the system we have in place to generate a 3D data block has the following features.

* Freehand scanning - no significant change to equipment or normal 2D scanning allows the powerful interaction between the sonographer and interrogation to be preserved. Features of interest are intuitively scanned more thoroughly and this results in increased data density in such areas. Poorly defined boundaries or shadowed areas can be scanned from alternative angles and the data contribute to the dataset .

* The generated data sets may be analysed in commercially available 3D analysis software. The compounded data is independent of any individual insonation angle and provides coherent data suitable for automated greyscale segmentation techniques. The original data is preserved throughout the process of generating a data block and may be separately analysed to determine data acquisition density, angles of insonation explored, contributing data values and statistics.

* The storage of data to S-VHS videotape provides a cost effective and efficient data storage medium. Video can be reviewed for 2D data acquisition or for 3D data output. Portions of the video can be captured for interesting features requiring 3D analysis and retrospectively captured during different segments of the cardiac cycle without requiring patient re-examination. All the 3D data from individual interrogations are spatially registered so that relationships between structures are apparent even when captured from differing portions of video.

* All the processes of data block generation utilise linear algorithms. Processing time is therefore proportional to the number of frames used in generation of that block. As processing algorithms are linear, considerable increase in performance could be available with parallel processing or silicon chip technology. We can confidently predict that the output of a 3D datablock is achievable in near to real time.

* The generation of a 3D data block is independent of the analysis package we have in use (IBM-DX and TOSCA) and is transportable to any UNIX based workstation.

Further System details; -

An electromagnetic orientation and position sensor (Polhemus, Colchester, Vermont) is attached to the scanning transducer, providing accurate positional information in 6° of freedom for each image frame generated. A transformation matrix to allow accurate spatial positioning of the image frame has to be established for each sensor/transducer combination.

Freehand, free running (25fps) scanning of the object of interest is recorded to videotape (S-VHS) with concurrent ECG, respiration and Polhemus registration data being stored on the audio channels. (Tape bandwidth 2-20KHz) .

These signals are processed and synchronised with the image data by a Zeneca constructed hardware module connected to video, Polhemus and the ultrasound scanner.

The module uses the synchronisation for video from the scanner to provide a synchronisation pulse to Polhemus, which then outputs the positional information for this image frame. The binary RS232 Polhemus information is then buffered to allow this to be captured on the audio Hi-Fi channel 1 of the Video recorder. Input of ECG is similarly buffered, multiplexed with the respiration signal from a Pneumotrace II (UFI California USA) and frequency modulated to preserve the slow phases of respiration to allow capture onto the Hi-Fi channel 2. The video (both image and audio data) is then digitised using a real-time frame grabber (SG-COSMO utilising JPEG compression (15:1).) onto a Silicon Graphics Indy workstation, (approx 7,500 frames for a 5 minute video) . The frequency modulated audio channel 2 signal being demodulated before capture. The audio signals being oversampled at 48KHz on each channel.

A custom written Graphical user interface utilising the IRIX media enables image frames occurring at precise portions of the cardiac cycle and respiration phase to be extracted. This uses the audio file of the

ECG/respiration trace to recognise the intervals of the cardiac cycle and identifies the associated image frames and Polhemus data. Typically a data set of 500 frames is extracted. Further custom written software decodes the positional information, applies the transformation matrix to spatially align the image frames and extracts the greyscale information to a regular 3D data block. This software provides for the multiple observations, from a variety of angles, of any single data point within the prescribed data block by scavenging data from a prescribed radius to the predetermined data point . This "compounded" greyscale data provides a significantly enhanced image in that the process reduces "speckle" and provides coherent data for segmentation algorithms.

III. METHODS - SYSTEM EVALUATION

The system was evaluated against three criteria, the ability to provide accurate estimates of volumes in the range encountered with atherosclerotic plaques and small tumours, the ability to improve overall image quality through carefully registered spatial compounding and finally, the ability to carry out in vivo reconstructions of clinical relevance. The first two studies employed phantoms scanned at room temperature using galactose solutions to match the 1540 m/s sound velocity for tissue inherent in the scanner's internal calibration of image depth.

III.l PHANTOM STUDIES The accuracy of reconstruction and volume measurement of the system was evaluated using a series of 3 water filled balloon phantoms of volumes between 0.9 and 8.0 ml. The phantoms were mounted in a bath of 20%w/v galactose solution and each scanned using the system described, employing several free-hand sweeps to explore a wide variety of insonation angles. After 3D reconstruction the volumes were measured three times using the automatic greyscale segmentation and analysis tools provided by TOSCA. After measurement the latex balloons were dried and weighed. The balloons were then punctured, dried and reweighed. The volume of distilled water contained within the balloon was obtained from the weight difference assuming a density value at room temperature (22°C) of 0.9978 gm./ml. A second series of phantoms was used to introduce internal structure and texture as well as provide volumes of different shape. These phantoms were cylinders cut from a block of foam rubber by cork borers of known diameters then trimmed to length providing volumes of approximately 0.75, 1.00 and 2.50 ml. The phantoms were again scanned in a bath of 20%w/v galactose solution. Measurements of the "wet" cross sectional diameters and cylinder length were made from the 2D Ultrasound images for volume calculation. A more direct estimation of volume by displacement was not possible here because the integrity of cells in the foam could not be guaranteed. As a consequence, each cylinder length and diameter was measured in three positions on each scan; the mean values from these observations being used to calculate "best estimated volume" . This was compared to the 3D volume again measured in TOSCA by volume growing algorithms based on segmentation of the 3D data block at pre-set level and window. The level and window were chosen by displaying a profile of the intensity values across the object within the data block, then selecting a mean value for level' and an appropriate 'window' to delineate the edges of the object. Two observers measured each data set with their own preferred level and window. The two data sets on each phantom were processed independently in an attempt to evaluate the reproducibility and objectivity of volume measurement by this method.

III.2 IMAGE QUALITY STUDIES

The effect of grid-mapping high density sampled data acquired with multiple angles of insonation, was investigated by 2D image comparison. One image was the "key frame" taken directly from the ultrasound acquisition video and the other was the central frame or plane of the grid-mapped 3D data set. This reconstructed plane was identical in position and orientation with the input "keyframe" and representative of the quality of all reconstructed planes at this data density. For the results to be meaningful, it was important in these studies that the boundary of the ROI was well defined. The foam-rubber phantom described above was chosen to provide a high contrast interface with the galactose bath and a high degree of uniformity in the echogenicity of boundary.

Statistical analysis of these images and the contributing data provided ; the mean pixel value for the whole image; the number of 2D ultrasound data values contributing to each grid point; and the percent coefficient of variation (% CoV) of the weighted contributions to the computed mean pixel value. These characterise the quality of individual pixel values. The %CoV serves as the ultrasound analogue of the "noise to signal" ratio encountered with other imaging modalities and is thus a direct measure of speckle contrast (Hernandez et al . 1996) . There is also a need to characterise the ability of statistical segmentation tools to generate a volume of interest (VOI) on the basis of grey-scale intensities. This involves considering the " coherence" of the intensities of the pixels forming the boundary of the VOI , as defined below.

A simple measure of coherence was obtained by examining the number of pixels identified in the interface boundary at a fixed level (40) and window (2) and the variation of intensity of the pixels along the boundary in the IBM DX environment. The "segmentation window width" required to produce a continuous boundary at a fixed level (40) provided a similar but independent measure of the inverse of this coherence.

A more automatic and tool independent measure has been implemented by applying a 3x3 inverse distance filter (F) at each pixel location in the 2D image plane.

The filter result for each pixel in the image was then normalised by the original pixel value and the absolute value of the result expressed as a percentage. These percentage values can be used to colour-code a display to reveal areas of high or low coherence within a region of interest, greater coherence again represented by the lower percentage values. The average of these percentage values taken over a region offers an over-all coherence measure. Specifically, we define a "Coherence Number" for the reconstructed map or an individual plane which is equal to 100 divided by the mean of the percentage values for the map or plane.

IV. RESULTS RECONSTRUCTIONS

After processing, all data sets were reconstructed and displayed, processing usually taking less than 30 minutes including batch processing time. The regular shapes of the phantoms allowed visual assessment of the reconstructions regarding distortions or disruptions of shape: no obvious distortions or disruptions of shape were discerned and even the 0.5mm wire used to suspend the foam rubber phantoms was successfully resolved, (fig. 3) .

The spatial resolution of the system as implemented has a theoretical isotropic limit of 0.2 mm. from the anamorphic scaling to half size of 2D ultrasound images with an in-plane resolution of 0.1 mm. The out-of-plane resolution of the 2D images is controlled by the beam- thickness profile of the transducer and is of order 1 mm even at the focal depth. Without the use of the multiple insonation angle, spatial compounding technique, this much larger value would dominate the spatial resolution characteristics and render them very significantly anisotropic. The degree to which the system's spatial resolution approaches the theoreticaly limit will instead depend on the extent of the compounding (number of ultrasound planes and diversity of insonation angle) and on the quality of the Δ matrix used in image registration. On the basis of the statistics related to the Δ matrix determination using the cross-wire phantom, the spatial resolution of the system was determined to be 0.5 mm isotropic. This figure had an associated standard deviation of 0.18 mm, and was consistent with the quality of the results and images presented throughout the paper.

ACCURACY OF VOLUME DETERMINATION Volume measurement from the balloon phantoms was straightforward in that the greyscale segmentation tools within TOSCA were able to detect the internal surface of the balloon. Grey-scale segmentation within this surface provided reproducible volumes with rms accuracy of 1.1% ( Table 1) .

Table 1 (located hereinafter in this text)

Best estimated volume of the foam cylinders was calculated from the mean diameter and length measurement by

voi = π(^d/₂)². L

where d = mean diameter of the cylinder

L = mean length.

Overall mean values for volumes were obtained for precision determination, and individual operator values for volumes from both independent data sets on each phantom for reproducibility (Table 2) .

Table 2 (located hereinafter in this text)

Foam Phantom Volume measurements : 2 observers, 3 measurements each.

Three measurements of the two data sets from each of the foam phantoms, by two independent observers, resulted in good overall reproducibility. Observer 1 demonstrated a CoV of between 5% and 2% for the model based calculation of volumes compared to an overall 1% CoV for 3D analysis. Observer 2, demonstrated a 2% CoV for the model based volume measurement against a 1% CoV for 3D analysis. Overall precision of 3D measurement is illustrated by an rms coefficient of variation of 1.4% (test-retest) and 1.3% (inter-observer).

COMPARATIVE MEASURES OF IMAGE QUALITY

Table 3 (located hereinafter in this text) - Statistical properties of raw and reconstructed image planes.

In all cases the "compounded" grid-mapped reconstructions offer very significant improvements in interface coherence and hence the ability to segment volumes of interest based on intensity values. By using: a) the foam-rubber phantom to provide high contrast interfaces with a relative uniform boundary density and b) positioning the 3D grid using the "key" 2D plane so that direct comparison of a "raw" versus a reconstructed data plane can be made, it has been possible to quantify the improvements in image quality that can be achieved with reconstructions based on compounded, free-hand scanned, high-density sampled data sets, (table 3, figures 4a and 4b) .

The extent to which compounding was used in the reconstruction is shown by the range of angles and the average number of ultrasound data points used in calculating each value in the reconstructed central plane. This average increased with both the number of ultrasound frames in the input data set and with the grid-mapped cut-off radius ( R ) . Using the statistics for the whole of the grid-mapped volume, a very strong linear correlation was shown to exist between the average number of data points used per grid point and the product of the number of frames with the cube of the cut-off radius (linear regression coefficient with 15 observations = 0.999998 ) . On average, 60 values (Table 3) from contributing ultrasound image frames were used to determine the value of a single voxel in the reconstructed data block, compared to just one for a conventional 2D image. The percent coefficient of variation (CoV) of the 1/r weighted values used in the reconstructions indicated the consistency of the distribution of values averaged during the grid-mapping procedure for every pixel in the reconstruction. In the example presented in Table 3, the average CoV taken over the whole plane was 0.23%. Assessment of boundary continuity for a fixed segmentation level involved determining the width of the "Window" needed to establish a continuous perimeter for the volume of interest (VOI) . These attributes related directly to the ability to segment structures based on the greyscale intensity values. The tabulated results (Table 3) corresponded to a "Level" setting of 40, the mean value of pixels in the reconstructed plane. The "Segmentation window width" then had to be set to +_, 3.1 (15% of level) for the VOI in the reconstructed plane to obtain a qualitatively continuous and unbroken boundary. This compares well with a window width of 8.5 (42% of level) required for the same effect in the 2D ultrasound image . The more quantitative continuity measure of counting the number of pixels in the perimeter, detected by a standard setting of the Window parameter, demonstrated similar improvements.

A maximum pixel count of 345 for the perimeter of a VOI was obtained from a 1000 frame reconstruction, gridmapped using a limiting radius (R) of 0.5mm, at a window of ±2 (10% of level) . In the typical reconstruction example (500 frames and R = 0.25mm), using the same ±2 (10% of level) window, 300 boundary pixels were detected (87% of the maximum) in the 3D reconstructed plane compared to 121 (35% of maximum) in the original 2D ultrasound image (Table 3) . This 2 to 3 fold increase in boundary pixel count contributed directly to improvements seen both in (a) the performance of statistical segmentation tools carrying out volumetric segmentation based on greyscale intensity and structure boundaries and in (b) the 3D depictions of reconstructed phantoms (figure 3) .

The derived "Coherence Number" (see methods section) provided an abstract measure of ability to segment on greyscale intensity that is independent of the tools to be used. The coherence number generated from the statistical analysis of the plane was found to correlate well with :

1) Qualitative assessment of image quality based on ease and consistency of TOSCA segmentation ;

2) Qualitative assessment of image quality based on the appearance of interfaces when revealed by displaying only pixels within a narrow range of values corresponding to an appropriate "level and window" selection;

3) Quantitative assessment of image quality based on edge continuity of constituent interfaces using the "segmentation window width" measure. (Linear regression coefficient with 17 observations 0.9737) .

In all cases the grid-mapped 3D reconstructions of "compounded" ultrasound data sets offered very significant improvement in interface coherence and the ability to segment structures of interest from greyscale intensity.

V. DISCUSSION

The results presented support the assertion that data acquired using conventional 2D ultrasound scanning techniques can be reconstructed to provide high quality 3D echo intensity images. These are capable of supporting reliable and consistent greyscale structure segmentation and the accurate measurement (and progressive monitoring) of volume. The key points are as follows;

• The surface rendered 3D reconstruction of the foam rubber phantom (figure 3) convincingly depicts not only details of the cylindrical body of the phantom but also the supporting wire ( 0.5 mm. diameter) . • Tables 1 and 2 demonstrate the high degree of accuracy and reproducibility attained for volume measurements using water filled balloon and cylindrical foam rubber phantoms.

• The percent coefficients of variation show the distribution of l/r weighted pixel values used in the 3D reconstructions of the reference plane to be tightly grouped, with average values for the whole plane (13,000 pixels) of 0.25% for the 500 frame reconstruction (Table 3) . The effective "noise to signal" figures are 1:400.

• The side-by-side comparison of a single 2D ultrasound image frame and the echo intensities for the same plane, extracted from the 3D reconstructed image of the foam rubber phantom (figure 4) , shows the differences in continuity of the interface obtained when a narrow range of pixel intensities is selected. Of the 345 pixels that might possibly be identified, the conventional 2D image has only 35% of these while the plane from the 500 frame reconstruction has 87% (Table 3) .

• The coherence numbers which characterise the ease and reliability of segmentation using the TOSCA level and window type algorithm improve from 51 for the average of the "raw" 2D image plane (13,000 pixels) to 190 for the 500 frame reconstruction (Table 3) . This

370% improvement is an order of magnitude larger than that found when comparing coherence numbers for images before and after "smoothing " using a 2x2 box filter.

The power of the approach described originates in the freehand acquisition of extensive, compounded, data sets. Firstly, the freehand aspect allows advantage to be taken of instinctive scanning behaviours and ensures a high density of data in the areas of greatest interest. Secondly, in association with the precision that can be achieved with extensive data sets and the Δ matrix formulation for the 3D registration of the 2D ultrasound images , the grid-mapping procedure described ensures that the echo intensity for each of the points in the 3D reconstruction is calculated from a very significant number of observed data values. Thirdly, compounding ensures an extensive positional sampling of echo intensities in the immediate neighbourhood of the reconstruction point and a range of insonation angles; this results in speckle reduction. The overall improvement in image quality allows reliable and accurate structure segmentation and volume assessment on the basis of greyscale, 3D reconstructed echo intensities. Finally, the format of the reconstruction allows routine handling of ultrasound image data in the advanced 3D analysis environments that once were exclusive to MRI and CT.

The ability to segment ultrasound data on the basis of grey-scale intensities has to date been controversial. This has arisen from attempts to segment 2D or 3D ultrasound images that remain contaminated by speckle. The fact also remains that the grey-scale attributed to the same object might differ significantly between scanning instruments. In the present study reliable automated grey- scale segmentation was only achieved after the high density sampling of compounded data was introduced and used in association with grid-mapping procedures . Reconstructions with the necessary level of coherence were then obtained that allowed standard statistical segmentation methods to succeed. IBM's TOSCA product provides for a profile of intensities to be obtained across a representative slice of the data block. This allows an appropriate level and window to be selected for the region of interest and used for segmentation throughout the block. Reference features can then be used to provide cross correlation of structures and their intensities between instruments and subsequent data sets. This has been checked using three ultrasound scanners and two calibrated 7.5 MHz linear array transducer/EPOS sensor configurations in various combinations . An additional advantage of the methodology described is common to all 3D imaging techniques and derives from the intrinsic limitations of 2D data in volumetric analysis. While any 3D image can be considered as a series of parallel 2D plane images, volume assessments using 2D data employ a relatively small number of independent planes. These are used to establish a set of ID or 2D parameters to define a 3D "model" and compute its volume. The quality of the predicted volume will depend on the accuracy of the measurement of the parameters, the precision with which the planes selected meet the geometric requirements of the model, and the degree to which the structure "conforms" to the model. The major limitation lies with the "model" itself. Simple models make assumptions about the symmetry and regularity of structure. Even the more sophisticated, adaptive, modelling techniques which use training sets to select the most appropriate parametric model, become progressively less valid with increasing degrees of pathology ( Cootes et al . 1994, Syn et al . 1995. The emphasis placed in our approach on acquiring over-determined data sets to support the critical steps associated with 3D registration and reconstruction is in marked contrast to what is described elsewhere. In the literature regular parallel scanning techniques using a motor driven linear transducer or specialised 2D crystal arrays restricts the insonation angle to a single value. Fixed slice intervals and physiological gating on acquisition limit the density of data points available for reconstruction. Where freehand scanning has been used, the subsequent 3D reconstructions do not use a high density of compounded images in association with the grid-mapping approach. Instead, a relatively small number of approximately parallel, non-compounded, 2D ultrasound image planes have been registered in 3D and structure boundaries identified on these planes. These boundaries have then been used to produce a 3D surface; the additional data points necessary to generate the surface triangles or polygons associated with standard surface rendering techniques being provided by interpolation. Alternatively, 3D shape modelling has been used with constraints provided by the structure boundaries established on the various, registered 2D image planes. While, in general, the appearance of such surface rendered objects will be good, the reliance on relatively low data densities and single insonation angle must restrict the precision associated with such segmentation and volume assessments.

In contrast these problems (associated with both reliance on limited data or volume measurement from potentially inaccurate model fitting) are avoided by direct assessment of volume based on segmentation of 3D reconstructed images produced in the manner described in this paper.

The 3D reconstruction techniques described are not restricted to use with linear array transducers and can, in principle, be applied wherever 2D ultrasound is used. Reconstruction of data acquired with other probe types has been achieved in our laboratory. Extension of the system to address in vivo studies in a Research or Clinical environment requires the physiological monitoring of cardiac and respiratory functions that can produce relative motion or distortion of the structures of interest. The facility to select those 2D ultrasound images captured under identical cardiac cycle conditions has been described. Extension to handle respiratory motion has also been developed. The second stereo channel of the S-VHS tape is used to record such information. Provided that the selection criteria are sufficient to effectively "freeze" the associated motion, a consistent 3D reconstruction can then be obtained. This has already been demonstrated in this laboratory through 3D reconstruction of in vivo vascular structures using only the 2D ultrasound images recorded close to the midpoint of diastole (Allott, et al . 1995).

Using the equipment described, a 3D reconstruction is available for segmentation and analysis within 30 minutes from acquisition of the 2D ultrasound images. This is believed to be an acceptable processing time even in a Clinical Diagnostic environment and would allow routine use in Clinical and Pre-Clinical studies. ( ref Halliwell, Rees , Berman , Woodcock, Seif - personal communications. Permission to be sought) . In this laboratory in excess of 2,000 3D reconstructions of ultrasound data sets, from both in vi tro and in vivo sources, have been made with this system in the past 12 months, providing 3D images of potential clinical relevance.

Uses

The ability to interrogate a compound data block generated from freely acquired ultrasound images has many possible applications. Medical Applications

The readily derived volume measurement may be used to follow progression of disease process or regression during treatment where the dimensions of a structure are affected by the disease process - e.g. Tumour growth, Atherosclerotic plaque, cardiac hypertrophy and renal disease. Other diagnostic functions may include the maturation of ovarian follicles or endometria in fertility monitoring or foetal development applications. The ability to differentiate structure on the basis of tissue characteristics allows further discrimination in disease progression monitoring. Tissue characteristics rather than dimensions may change during some disease processes. These changes may be monitored by analysis of the greyscale attribution within the 3D data set, which is independent of view and acquisition angle. These characteristic changes may be found in a wide variety of disease processes where tissue damage and subsequent scarring occurs as a result of vascular insufficiency, toxic or fibrotic response e.g. renal disease, liver disease and infarction.

The ability to interrogate all the data contributing to the 3D volume, from a variety of angles, allows analysis of the contributing greyscale attribution and position with the angle of insonation being known. Using this data the surface characteristics (i.e. roughness) may also be determined. This may have application in monitoring cartilage or ulceration of atherosclerotic plaque which is known to promote thrombus formation and lead to vascular occlusion.

The use of compounded data in generation of a regular 3D data block allows the data to be presented as a slice at any preferred angle or orientation, generating a novel 2D ultrasound image that is independent of the orientations used in data acquisition. The isotropic nature of the data voxels within the reconstructed compound data block make such images meaningful and useful in presenting views that are unobtainable in conventional ultrasound scanning. For example an orientation of scan that illustrates the carotid bifurcation in plan may be obtained, as described and illustrated later herein. This would equate to a view taken with the transducer positioned inside the head or chest of the subject.

Reconstruction of interrogations may also provide an aid to surgical planning. In many instances the spatial relationships between structures such as blood vessels is of great importance in determination of best approach for a surgical procedure. Reconstruction in 3D with the ability to differentiate structures and tissue character and display and rotate these on screen provides a comprehensive overview so that such planning can be made with increased confidence.

Non Medical applications Ultrasound has many uses outside the medical fields already indicated.

Crack testing of pressure vessels is a widespread application that is dependent on angle of interrogation. When the ultrasound beam is aligned with the direction of the crack it will not be detected as there is no interface within the material . Compounding data from a variety of insonation angles could allow more accurate detection of the crack and precise spatial location. Similar compounding techniques to those described could also be used to interrogate pipelines for sedimentation and blockage. The compounded data would improve the location and differentiation of the sediment from the fluid phase within the pipe. This may be useful in locating a blockage, determination of extent and consistency of the blockage and decision making on means of repair. The compounding techniques could also be applied to manufacturing processes where voids in materials are important in material performance such as for example in explosives . Our study has presented results that support the following assertions :

1) Accurate 3D reconstruction from compound, freehand, ultrasound images has been achieved. 2) Compounding of 2D images and generation of a regular 3D data block leads to better image structure coherence so that the 3D reconstructions are comparable with CT and MRI images and formatted appropriately for use with state-of-the-art 3D Medical Image Analysis products.

3) Provided that EPOS data and system scaling factors can be obtained with the required accuracy, any ultrasound image data set, from any 2D transducer type or anatomical location, could be reconstructed in 3D by these methods.

4) The accuracy and precision of 3D reconstruction and volume measurement from these compounded images is sufficient for clinical utility in patient monitoring. 5) The computation processes involved in 3D spatial orientation and generation of a regular 3D data block from ultrasound images no longer present a bottleneck in data processing. Reconstruction and segmentation of a data set can be achieved in under 30 minutes, and so is appropriate for clinical utility.

We have described the process of 3D reconstruction and accurate volume measurement, starting with data acquired using 2D ultrasound scanning techniques. These procedures, together with readily available commercial software and equipment, produce a system which is capable of delivering sufficiently significant benefits to propose its use in the clinical environment.

REFERENCES Allott CP, Arundel PA, Barry CD, John NW, Mellor PM, et al . Freehand, quantitative 3-D ultrasound imaging and its application. Eur. J. Ultrasound 1996; 3 (Suppl. ) : S9. von Birgelen C, Slager CJ, Di Mario C, de Feyter PJ, Serruys PW. Volumetric intracoronary ultrasound; A new maximum confidence approach for the quantitative assessment of progression-regression of atherosclerosis? Atherosclerosis 1995; 118 (Suppl.): S103-S113.

Cootes TF, Hill A, Taylor CJ, Haslem J. The Use of Active Shape Models for Locating Structures in Medical Images. Image and vision Computing 1994; 12:276-285.

Detmer PR, Bashein G, Hodges T, Beach KW, Strandness DE Jr. 3D ultrasonic image feature location based on magnetic scan head tracking: In vitro calibration and validation. Ultrasound Med Biol 1994; 20: 923-936.

Elliot PJ, Diedrichsen J, Goodson K, Riste-Smith R, Sivewright GJ. An object oriented system for 3D medical image analysis. IBM Systems Journal 1996 35:1-18.

Fine D. Three-dimensional ultrasound imaging of the gallbladder and dilated biliary tree: reconstruction from real - time B-scans. Br J Radiol 1991; 64: 1056-????.

Franceschi D, Bondi J, Rubin JR. A new approach for three-dimensional reconstruction of arterial ultrasonography. J Vase Surg 1992; 15: 800-805. Gardener JE, Lees WR, Gillams A. Volume imaging with ultrasound. Radiology 1991; 181 (P) : 133-??? .

Geiser EA, Ariet M, Conetta DA, Lupkiewcz SM, Christie LG. Dynamic three-dimensional echocardiographic reconstruction of the intact human left ventricle; Technique and initial observations in patients. Am H J 1982; 103: 1056-10?? Gilja OH. In Vitro evaluation of three-dimensional ultrasonography in volume estimation of abdominal organs. Ultrasound Med Biol 1994: 20:157- ???.

Hamper UM, Trapanotto V, Sheth S, Dejong MR, Caskey CI . Three-dimensional U.S. preliminary clinical experience. Radiology 1994; 191:397-401.

Hell B. 3D Sonography. Int J Oral Maxillofac Surg 1995; 24: 84 -89.

Hernandez A, Basset O, Chirossel P, Gimenez G. Spatial compounding in ultrasound imaging using an articulated scan arm. Ultrasound Med Biol 1996; 22: 229- 238.

Howdry DH, Posakony G, Cushman R, Holmes JH. Three dimensional and Stereoscopic observation of body structures by ultrasound. J Appl Physiol 1956; 9:304-306.

Hughes SW, D'Arcy TJ, Maxwell DJ, Chiu W, Milner A, et al . Volume estimation from multiplanar 2D ultrasound images using a remote electromagnetic position and orientation sensor. Ultrasound Med Biol 1996; 22: 561- 572.

Kelly IM, Gardener JE, Brett AD, Richards R, Lees WR. Three-dimensional US of the fetus- work in progress. Radiology 1994; 192: 253-259.

King DL, King DL Jr, Yi-Ci Shao M. Three-dimensional spatial registration and interactive display of position and orientation of real-time ultrasound images. J Ultrasound Med 1990; 9:525-532.

Kok-Hwee Ng, Evans JL, Vonesh MJ, Meyers SN, Mills TA. Atrerial imaging with a new forward-viewing intravascular ultrasound catheter, II. Three-dimensional reconstruction and display of data. Circulation 1994; 89: 718-723.

Levine RA, Weyman AE, Handschumaker MD. Three- dimensional echocardiography : Techniques and applications. Am J Cardiol 1992; 69 : 121H-130H.

Mintz GS, Keller MB, Fay FG . Motorised IVUS transducer pull-back permits accurate quantitative axial measurements (abstr.) Circulation 1992; 86-1 :323s.

Moritz WE, Medema DK, Ainsworth M, McCabe D, Pearlman AS. Three-dimensional reconstruction and volume calculation from a series of nonparallel, real-time, ultrasonic images. Circulation 1980; 62 (Suppl .); 111- 539. ?????

Moskalik A, Carson PL, Meyer CR, Fowlkes JB, Rubin JM, et al . Registration of three-dimensional compound ultrasound scans of the breast for refraction and motion corrections. Ultrasound Med Biol 1995; 21: 769-778.

Nelson TR, Pretorius DH, Slansky M, Hagen-Ansert S. Three-dimensional echocardiographic evaluation of fetal heart anatomy and function. J Ultrasound Med 1996; 15:1- 9.

Picano E, Landini L, Distante A, Salvadori M, Lattanzi F. Angle dependence of ultrasonic backscatter in arterial tissues; a study in vitro . Circulation 1985; 72: 573-576. von Ramm OT, Durham NC, Smith SW, Carroll BA. Real time volumetric US imaging. Radiology 1994; 193 (P) : 308 - ??? .

Rankin et al . 1993 - Review cited on page 3

Riccabona M, Nelson TR, Pretorius DH, Davidson TE. Distance and volume measurement using three-dimensional ultrasonography. J Ultrasound Med 1995; 14:881-886.

Rosenfield K, Losordo DW, Ramaswamy K, Isner JM. Three-dimensional reconstruction of human coronary and peripheral arteries from images recorded during two- dimensional intravascular ultrasound examination. Circulation 1991; 84: 1938-1956.

Schumacher D A, Fast Anamorphic Image Scaling. In Aruo J ed. Graphics Gems II. Academic Press 1991: 78-79.

Shattuck DP, von Ramm OT. Compound scanning with a phased array. Ultrason Imag 1982; 4:93-107.

Sivewright GJ, Elliot PJ. Interactive region and volume growing for segmenting volumes in MR and CT images. Medical Informatics 1994; 19:71-80.

Steen E, Olstad B. Volume rendering of 3D medical ultrasound data using direct feature mapping. IEEE Trans Med Imag 1994; 13:517-525.

Syn M, Prager RA. Model based approach to three- dimensional ultrasound imaging. Proc XlVth Information Processing in Medical Imaging 1995; 95-???

Vogel M, Ho SY, Buhlmeyer K, Anderson RH. Assessment of congenital heart defects by dynamic three-dimensional echocardiography; methods of data acquisition and clinical potential. Acta Paediatr 1995; 410 (Suppl.): 34-39.

Zosmer N, Jurkovic D, Jauniaux E, Gruboeck K, Lees C. Selection and Identification of standard cardiac views from three-dimensional volume scans of the fetal thorax. J Ultrasound Med 1996; 15: 25-32.

Table and figure headings

Table 1; Accuracy of Balloon phantom volume measurements .

Table 2 ; Precision of Foam phantom volume measurements Table 3; Statistical properties of 2D ultrasound image and reconstructed image plane.

Figure 1; Diagram of 3D Freehand ultrasound system components

Figure 2 ; a) Photograph of the physical arrangement of ultrasound transducer and Polhemus EPOS receiver b) Co-ordinate systems and transformations for 3D spatial location of 2D ultrasound images.

Figure 3; Surface rendered 3D reconstruction of a 700 mm³

Foam rubber cylinder and 0.5mm diameter supporting wire used as a phantom for the reported studies.

Figure 4 Segmentation of pixels in the boundary of a foam phantom using a level of 40 and a window of 2. Panel a represents the segmentable pixels in the original 2D image. Panel b represents the segmentable pixels in a compounded reconstructed plane identical in location.

Table 1

Tabl e 2

Volume estimated Volume measured from 3D from 2D slices ultrasound

(cylindrical (model -free) model) μl ± S.D. μl ± S.D. μl ± S.D.

First scan Repeat scan

Observer 1 699 ± 19 709±10 703±4

Phantom 1

Observer 2 717 ± 34 719±0 694+20

Observer 1 956 ± 8 967±9 971±12

Phantom 2

Observer 2 975 ± 10 985±5 986+1

Observer 1 2405 ± 13 2356±10 2325+23

Phantom 3

Observer 2 2398 ± 42 2339±0 2401±33

Table 3

Our overall goal has been to develop a system to produce precise and rapid 3D reconstructions capable of supporting reliable grey-scale segmentation and successful volume assessment based on free-hand ultrasound scanning.

Two-dimensional (2D) , percutaneous, Ultrasound B- Mode imaging is well established in a wide variety of clinical and biomedical applications including the investigation and diagnosis of cardiac, vascular and oncological disease. However significant difficulties have been encountered in using 2D ultrasound to quantify disease progression or the response to therapies and to communicate the findings to third parties that have not been involved in the scanning procedure. It is almost impossible to ensure successive identical registrations of 2D image views and insonation angles to allow accurate temporal comparisons to be made. These problems are exacibated when the structures involved have irregular shape or changing echogenicity. This has prevented the modality from reaching its full potential in either clinical diagnosis or studies to evaluate novel disease modifying drugs.

In contrast, three-dimensional (3D) reconstruction of ultrasound images allows volumes, and other variables related to volume, to be measured independently of the data acquisition views and angles (Picano 1985) . Accurate sequential monitoring of pathology and communication of the information then becomes available as the whole structure is included at each interrogation. During the past decade, a considerable number of reports have appeared on the development of methods for 3D reconstruction of ultrasound data ( reviews see Rankin, 1993; Vogel, 1995; Levine 1992). Where freehand techniques are used, the image positioning can be obtained from simultaneous recording of the position and orientation of the transducer using mechanical arm, acoustic spark gap or electromagnetic sensor techniques

(Detmer, 1994; Geiser, 1982; Gardener, 1991; Kelly, 1994; King, 1990; Moritz, 1980; Nelson, 1996; Riccabona, 1995) . We have already described hereinbefore (Barry 1996) our implementation of a system that produces a regular 3D data block suitable for analysis by conventional 3D analysis and volume measurement software. This system incorporated electromagnetic spatial location of freehand-scanned Ultrasound B-Mode image frames, custom built signal conditioning hardware and UNIX based computer processing. It utilised a high speed algorithm that populates a Cartesian grid with data extracted from the 2D image frames and positioned accurately in 3D. The inclusion of reflectivity data from multiple 2D ultrasound frames with a wide range of insonation angles reduced the angle-dependency of reflection intensity values from each interface. Such "compounding" significantly improved structure coherence within the 3D greyscale image and made possible segmention and volume assessment on the basis of structure boundaries.

Satisfactory reconstruction of 3D data from freehand acquired 2D ultrasound images requires accurate spatial registration of the ultrasound image in a fixed reference frame. This allows the data from each image to be extracted and positioned appropriately within a common 3D data block. For the technique to have practical application, the 3D reconstruction must be available within a short time of completing the scan and be in a format that allows use of the highly efficient image processing and analysis tools that have been developed for other medical imaging modalities. A successful system also requires that the 3D image quality is improved beyond that of a typical 2D ultrasound frame in order that segmentation on the basis of grey-scale, reconstructed echo intensities, can be used and lead to a sufficiently high degree of confidence and reliability in the results of automated segmentation and subsequent volume measurement and analysis.

The three critical elements in our system were : -

3) Development of objective "image quali ty measures " that correlate well with the ability to segment on the basis of grey-scale echo intensity values.

This section describes the mathematical and computational principles behind 3D registration, grid- mapping and the assessment of image quality and grey- scale segmentability. We analyse the performance of the system and key procedures over a range of conditions and parameter settings and draw comparisons with the work of other groups .

II 3D REGISTRATION OF ULTRASOUND IMAGES

11.1 THE DELTA TRANSFORMATION

FIGURE 2 - Co-ordinate systems and transformations

The position and orientation of the 2D ultrasound image plane in 3D space was established using an the electromagnetic position and orientation system (EPOS) . The sensor was attached to the ultrasound probe while the origin and axes of the transmitter established the fixed "registration frame" ( designated XYZ in figure 2b) . The EPOS tracked the receiver and provided read-outs of the vector and the three eulerian angles which defined the position and orientation of the receiver and its co- ordinate frame (designated xyz ) , relative to the fixed transmitter frame. The transformation relating the co- ordinates of a point in frame xyz to those of the same point in frame XYZ was designated M .The co-ordinate frame associated with the ultrasound image itself (designated qrs ) , while fixed relative to the ultrasound probe and thus the receiver frame xyz, has an offset in both position and orientation. A transformation, including both translation and rotation, had to be applied to correct for the position and orientation of the EPOS receiver in relation to the frame, qrs, of the 2D ultrasound image. We define this as the Delta (A) transformation.

The transformations M and Δ together relate the coordinates of a point in space in the ultrasound image axis system, designated p(qrs), to the co-ordinates of the same point measured in the registration axis system, designated P(XYZ), through the following equations :

P(XYZ) = M * Δ* p(qrs) Equation la and p (qrs) = ( M * Δ) ^"1 * P(XYZ) Equation lb

Δ^"1 * M^"1 * P(XYZ)

The transformations M and Δ have translation and rotation components and can either be represented by a vector and a 3x3 rotation matrix, or by a single 4x4 matrix using the formalism of homogeniuos co-ordinates (Newman 1973 ) .

11.2 DETERMINATION OF THE DELTA TRANSFORMATION

This has to be determined for the ultrasound images to be registered.

The Δ transformation was determined by scanning a calibration phantom consisting of two crossed threads suspended in a 20% w/v galactose solution using a wide range of transducer angles and positions. The crossover provides a point in space whose co-ordinates are unknown but fixed in relation to the EPOS transmitter. The galactose solution provides a medium matched to the sound transmission velocity of normal tissue and thus to the internal depth scaling of the ultrasound instrument. After digitisation on the Silicon Graphics Indy the frames containing images where the cross was visible were extracted from the moviefile. These 2D frames were then displayed and the image location of the centre of the cross (r,s) determined and noted. A minimum of two observations were made for each attitude, totalling some 50 observations of this unique point in space. The vastly over-determined series of equations relating the unknown fixed position, P(XYZ) and transformation Δ to the known value pairings of p(qrs) and M (equation la) was then solved for P(XYZ) and Δ. Here an iterative process was used based on a starting trial Δ followed by a steepest descent refinement of the 3 angles and 3 distances defining the Δ transformation. The criteria function minimised was the root-mean-square deviation of the individual estimates of P(XYZ) from their group mean. The resulting Δ transformation matrix was subsequently used for all reconstruction with the specific ultrasound transducer / EPOS receiver configuration.

Because the Δ necessary for reconstructions remains constant for a given transducer- EPOS receiver configuration, independent sets of data corresponding to scans of either different point phantoms or the same phantom with different transmitter locations can be combined into a single, improved, estimate. This is acheived by computing the rms measure of the criteria function for each set of data and using the sum of these as the refinement criteria. Direct extension of this procedure to use a weighted sum allows incorporation an assessment of the quality of the individual scans. Equation lb has also been used as the basis of an alternative estimation of Δ using the same point phantom approach. The group mean P(XYZ) was back transformed and the co-ordinates of this point in the individual Ultrasound image co-ordinate systems, p(qrs) calculated. The difference between these back-transformed co- ordinates and their original, observed positions were used to compute the criteria function, with the option now of reducing the contribution (weighting) of the component perpendicular to the ultrasound image plane, q, relative to the better resolved in-plane components (r and s) . A relative weighting of 1.0 reproduced the results of the original refinement procedures while a weight of zero typically produced a Δ differing by several degrees in orientation and millimetres in translation, being a highly significant change.

Volume Refinement

The next development involved using volume rather than point phantoms. Initially the aim was to establish a more useful and objective measure of the Δ estimates than the criteria function residual. Subsequently, volume phantom measurements and their analysis became the basis for a refinement process in its own right.

A single cylinder of foam rubber was mounted in a 20% galactose bath and scanned with the equipment described. Several data sets were recorded using a variety of scan directions and angles, but each set containing only scans from approximately the same alignment. These data sets were then processed individually to provide multiple observations of the phantom volume, acquired from different positions and orientations .

Reconstructed phantoms, from a series of scanning passes, using less than perfect Δ show an occurrence of multiple discrete images which converge into one when the same data is processed using an optimised Δ matrix. Two hypotheses can be drawn up immediately. - First, the degree to which reconstructed images exhibit convergence of components stemming from different sub- sets of scanned Ultrasound images can provide a measure of the quality of Δ or degree to which it has converged and is optimal .

- Second, quantification of this 3D-image convergence is likely to provide a new, and better, criteria for driving an iterative refinement of Δ.

Quantification can be achieved by using any one of several BOOLEAN operations between volume reconstructions from two or more scanning subsets, as long as each is sufficient to reconstruct the whole object. The relevant BOOLEAN operators are INTERSECTION, SUBTRACTION and UNION. The convergence of the separate reconstructed volumes to a common value can also be used.

11. 3 PROPERTIES OF THE DELTA TRANSFORMATION

The analysis starts with equation la but with P and p redefined to be the position of a generalised data point in the two co-ordinate systems so that the equation now represents registration of the data for a whole 2D ultrasound image plane.

P(XYZ) = M * Δ * p(qrs) Equation la (repeat)

For an object defined by a subset of 2D Ultrasound images having a general index i, and a known EPOS transmitter-receiver transformation M_j , we have

P_i(XYZ) = M_j * Δ * P_j(qrs) Defining W_j = M_j * Δ * M_j ^'1

Pi(XYZ) = W; * M_j * p_j(qrs) Equation 2

T

Representing the true Δ by Δ and the corresponding generalised data point in the accurate reconstruction by -,T

P_j ^T(XYZ) = W_j ^T * M_j * p_j(qrs)

Where W_j ^T = M_j * Δ^T * M_j ^"1

So that M_j * p_j(qrs) = (W_j ¹)^"1 * P_j ^T(XYZ) Equation 2 can now be re-written as

P_j(XYZ) = W_j * (W_j ⁷)^{"1 *} P_j ^T(XYZ)

And then simplified, giving

P_j(XYZ) = U_j ^* P_j ^T(XYZ) Equation 3

Where U_j = W_j * (W_j ⁷)^"1 = M_t * Δ*(Δ^T)^{"1 *} M_j ^"1 Equation 4 Equations 3 and 4 define the effect of Δ and the

M_j transformations in determining how close the 3D registration of individual 2D Ultrasound images is to the true result. The U_s transformation is a product of rotations and translations and so will itself correspond to a translation and a rotation about an axis.

If a reconstruction is to be accurate in shape and volume equation 3 must reduce to the equation for a rigid body transformation with U_j the same for all 2D images, independent of i. If, in addition the position of the reconstruction is to be correct, for all the 2D images used in the reconstruction every U will have to be the identity transformation (no rotation, no translation) .

In the general case, unrestricted, free-hand scanning, there will be no fixed relationship between the M_j for one 2D Ultrasound image and the M_k for another. Only if Δ equals Δ T so that U_j = U_k = the identity transformation ( I ) for every i and k, will the reconstructed object be accurate in shape, volume and position. Next consider the case where the reconstruction is performed omitting Δ. This corresponds to setting Δ = I in equations 3 and 4, so that

P_j' (XYZ ) = U_j ¹ * P_j ^T (XYZ )

T * .1 -1

. ( M_j * Δ M_j )

Again the reconstruction will only be accurate if U_j = I for all i and thus not be true in the general case. If, however, we relax the requirement for positional accuracy and analyse the conditions under which just the shape and volume will be correct, it will be sufficient for U_; to correspond to the same rigid body transformation for all the 2D Ultrasound images used in the reconstruction, that is for every index i. If we let K stand for this common rigid body transformation and C for its inverse, and consider any two specific values of i, namely j and k . Then

U_j ¹ = ( M_j * Δ^{T *} M^"1 )^_1 = K C^"1 and U_k ^! = ( M_k* Δ^{T *} M_k ^_1 )^_1 = K C^"1

These equations must be satisfied simultaeously for any j or k if shapes and volumes are to be reconstructed accurately. This requires that

(Mj * Δ^{T *} Mj^"1 ) = (M_k * Δ^{T *} M_k ^_1 ) = C Equation 5a

M and Δ each have rotational and translational components defined in three dimensions by 3 angles and a vector. Representing the rotational and translational components of M , Δ T and C for the 2D images with general index j and k by Rot_Mj , Tran_Mj , Rot_Mk , Tran_Mk , Rot_DT, Tran_DT , Rot_c and Tran_c respectively, the rotation components must satisfy the equation

Rot_Mj * Rθt_DT * Rot_Mj ^" =Rot_Mk * Rot_DT * Rot_Mk ^" = Rot_c Equation 5b

This required that Rot_Mj = Rot_Mk = Rot_M and corresponds to the case where the orientation is fixed and the image planes parallel .

Simultaneously, the equation for the translational component must also be satisfied for shapes and volumes to be preserved. This requires that

Rot_M * Tran_DT + ( I - Rot_M * Rot_DT * Rot_M ^"! ) * Tran_Mi = Tran_c

and Rot_M * Tran_DT + ( I - Rot_M * Rot_DT * Rot_M ^_1 ) * Tran_Mk = Tran_c

As Tran is arbitrary and the term ^Rot _M * Tran_DT is both common and constant for parallel scanning, the necessary and sufficient condition for the reconstruction to give accurate shapes and volumes is that

(I - Rot_M* Rot_DT* Rot_M ^_1) * Tran_Mj = (I - Rot_M * Rot_DT * Rot_M ^_1) * Tran_Mk

The solution requires that

Tran_Mj = A_j * V + B and Tran_Mk = A_k * V + B where A: and A_k are arbitrary scalars, B is a common, constant origin or offset vector and V is the Eigen vector of Rot_M*Rot_DT*Rot_M ^" with Eigen value 1. These solutions represent an arbitrary straight line parallel to the eigen vector V which, as noted in Appendix 3?? ,

V will be the axis of the rotation matrix defined by tthhee pprroodduucctt RRoott_MM**RRoott_DDTT**lRot_M ^" . This transformation product can be expanded Rot_M*Rot_DT*Rot_M ^"' = Rot_M*Rot_DT * (Rot_DT * Rot_DT ^_1) *Rot_M ^_1

= (Rot_M*Rot_DT) * Rot_DT * (Rot_M*Rot_DT )^"* revealing the form of the standard expression relating a rotation (Rot_DT) defined in in one frame to the same rotation as defined in a new co-ordinate reference system when the transformation between reference systems is Rot_M*Rot_DT . This shows that when V is defined in the parallel Ultrasound image co-ordinate frames ( qrs in figure 2) it is co-incident with the axis of Rot_DT . We have designated this unique direction as the "Zeta direction" . Parallel scans taken along this direction reproduce shape and volume accuarately whatever value is assigned to Δ.

To summarise, the practical implications of this analysis are : -

1) An accurate determination of Δ is essential for reconstructions to be accurate in position, shape and volume. This holds even for reconstruction with a highly COMPOUNDED (non-parallel ) data sets.

2) While the equations show that Δ can be ignored and reconstructions can still accuratlely reproduce shape and volume if the scans are parallel and along the "Zeta direction", Δ must still be known to determine this unique direction.

3) If Δ is close to the true Δ and scans are reasonably parallel, errors in reconstruction will be relatively small. If Δ is ignored or estimated without a high degree of precision, estimates of shape, volume and position will be poor unless the scans are all approximately parallel.

IV ASSESSMENT OF IMAGE QUALITY

For many imaging modalities the quality of an image can be improved by taking an average over multiple, identically obtained, samples. The basis for this is that the signal is systematic while the noise is random. By averaging over ' n' samples the discrimination between the signal and the noise is increased over that for a single sample by the square root of n' . This is done in MRI to improve the image, but requires that corresponding points in succeeding samples are identically placed. Under these condition where the samples being combined differ only in their noise content, the signal to noise ratio itself provides an important characterisation of quality.

In using "compounding" and "grid mapping" techniques in the reconstruction of ultrasound images it is true that the data from multiple ultrasound images is combined. However, in this case the samples are not identically obtained, the angle of insonation will differ from frame to frame. Also they are not identically placed: they are simply nearest neighbours to the relevant grid-point and thus distributed within a sphere whose radius is equal to the cut-off radius. Under these conditions an alternative measure to signal to noise is needed which can take account of these facts and be expressed in terms of the statistical properties of the distribution of 1/ r weighted values within the cut-off sphere. The analogy of the "signal" will be the mean value while the analogy of "noise" will be the standard deviation of this mean. This leads directly to the % coefficient of variation for the mean of the distribution providing the appropriate analogue to a "noise to signal" ratio.

However, there is also need not to just characterise the statistics of each picture element (pixel or voxel) in the reconstructed object as in independent entity, but to provide a measure that relates directly to the ability to segment a volume of interest

(VOI) on the basis of intensity. This requires a measure that will reflect the consistency or "coherence" of intensities within the sub-set of pixels that define the boundary (or interface) of the VOI.

Using level and window parameters common to statistical segmentation methods (e.g. IBM's TOSCA -

Elliot 199??) where pixels or voxels with intensities within the range level ± window are included in the VOI, simple measures of coherence can be obtained by quantifying the variation of intensity of the pixels along the boundary. One such measure would be a count of the number of boundary pixels that lie within a narrow window of the level equal to the mean intensity of the VOI boundary. Alternatively, the width of the window required to produce a continuous boundary at the mean level provides a similar but independent measure of the inverse of this coherence .

The filter result for each pixel in the image was then normalised by the original pixel value and the absolute value of the result expressed as a percentage. These percentage values can be used to colour-code a display to reveal areas of high or low coherence within a region of interest, greater coherence again represented by the lower percentage values. The average of these percentage values taken over a region offers an over-all coherence measure. Specifically, we define a "Coherence Number" for the reconstructed map or an individual plane which is equal to 100 divided by the mean of the percentage values for the map or plane. The Coherence Number quantifies the degree to which the value of each pixel is correlated to those of its immediate neighbours, i.e. the pixels that will form the interface or boundary of the VOI. It also provides a measures of the continuity of an edge.

V. RESULTS

V.l DELTA MATRIX

2) While the equations show that Δ can be ignored and reconstructions can still accurately reproduce shape and volume if the scans are parallel and along the "Zeta direction" , Δ must still be known to determine this unique direction.

V.2 GRID MAPPING PERFORMANCE & DEPENDANCIES

Timings moved from Results & maps renamed cdb 23/4/96 Times for Grid-mapping using a 80 Mhz IBM Power PC with 256 Mbytes of memory are presented in Tables X2 and X3. Table X2 Variations in time in minutes with Limiting radius R and number of 2D US frames for a fixed map size (26x20x22 = 11,440 mm ³ ) .

Table X3 Variations in time in minutes with Limiting radius R and Grid-mapped Volume

The structure of the grid mapping algorithm itself indicates that the execution time will be the sum of three terms .

The first relates to the input of the ultrasound data frames and the selection of the relevant sub-set of data for each. This term will be linear in the number of frames in the input set .

The second relates to the grid map computation itself. This will be proportional to the average number of ultrasound data used per grid point and to the total number of grid points. This term will dominate for all but the lowest R values .

The third term relates to the output of the computed grid-map and will be linear with the grid-mapped volume.

T = A * Number of ultrasound frames

+ B * Average number of data/grid point * Volume of the Grid-map Box + C * Volume of the Grid-map Box

The grid spacing used in the reconstructions was 0.2 mm giving a total number of grid points per map equal to 125 times the map volume in mm. The 2D ultrasound frames were cropped to 31 by 32 mm and have a data spacing of 0.2mm. each frame yielding a maximum of 24,800 input data values to the reconstruction.

In an idealised model where the volume of interest is scanned in a 100% effective way, the data would be distributed uniformly and all values would fall within the volume being reconstructed. If the reconstructed volume is V cubic millimetres and there are N contributing ultrasound frames the density of input 2D ultrasound data values per millimetre will be : p = 24800 * N / V As grid-mapping process uses all ultrasound data points within the sphere of limiting radius R, the typical number of data values used in the calculation per grid point will this be : p * 4 π R³ /3 = 4 π R³ * 24800 * N / 3V = 1.04 * N * R³ / V * 10⁵ The free-hand scanning pattern will not be completely uniform and the volume that bounds all of the ultrasound data will only partially match the grid mapped volume. Thus the actual number of ultrasound data values contributing to the value assigned to a specific grid point will depend on the relative position and orientation of the ultrasound frame within the reconstruction grid and also to the specifics of the free-hand scanning pattern.

However, using the statistics relating to the reconstruction of the same volume of interest using different numbers of frames (maps A, B and C) , the plot of the average number of data used per grid point versus N*R³ confirms that the linear relationship does hold in practice. The r value for the regression for the whole map with 15 observations is 0.999998. The same linear correlation also holds for individual planes, with an r value of 0.9987 for the reconstruction of the central "KEY frame" plane.

Moreover, the value of 9.09 predicted by the model for the constant of proportionality for the three maps (A,B and C) with a volume of 11,440 mm³ is consistent with the values determined for the whole map and for the reconstructed central, key plane. These values are respectively 4.53 and 9.45, indicating that some 50 % of the total ultrasound data points fell within the grid- mapped volume while in the central plane, which was the focus for the scans, the density of ultrasound data was highest .

This result provides an alternative functional form for the second term in the timing expression allowing the term

B * Average number of data / grid point * Volume of the Grid-map Box to be replaced by

B^/ * N * R³ The residual dependence on V seen in table X3 arises because the proportion of the ultrasound data frame involved in the calculation will change with the shape and volume of the grid mapped box.

V.3 IMAGE QUALITY ESTIMATIONS Table XI lists the observed values for parameters that relate to image quality, as the number of data planes and the limiting radius R vary for a fixed grid mapped volume. The edge continuity data for R = 0.1 mm. for map B and R = 0.1 and 0.2 mm for map C need to be treated as suspect as the low density of data points / grid point in these cases has led to there being a number of grid points at which there are no data values available within the cut-off radius. In these cases the resultant "tears" in the reconstructed plane make segmentation difficult.

The extent to which compounding is used in the reconstruction is represented by the entries showing the average number of data used in calculating the values for each grid point in the reconstructed central KEY plane. This average increases with both the number of ultrasound frames in the input data set and with the grid-mapped cut-off radius ( R ) . Using the statistics for the whole of the grid-mapped volume , a very strong linear correlation can be shown to exists be the average number of data used per grid point and the product of the number of frames with the cube of the cut-off radius R.

The percent coefficient of variation of the 1/r weighted values indicates the "tightness" of the distribution of values being averaged during the grid- mapping procedure. The average pixel value for the reconstructed KEY frame lies within the range 39.9 to 40.6 for the 15 cases presented. The relationship between this % CoV and the degree of compounding as represented by the statistics on the average number of data / grid point and thus the product of the number of frames with the cube of the cut-off radius R is apparent. The log:log plot indicated a correlation with power law of - 1.68 and a regression coefficient of 0.9960 once the points for map A with R = 0.5 ( % CoV too small to estimate reliably ) and map C with R = 0.1 (suspect in general) are ignored.

The Coherence number and inversely correlated Filter percent mean and maximum have been defined earlier in section III.l. Using the values for the smoothed data, the correlation with the average number of data / grid point and thus the product of the number of frames with the cube of the cut-off radius R is apparent. The log: log plot confirms an approximate cube root dependency and thus a linear relationship in R.

The Coherence number correlates well with another Edge Continuity measure based directly on the Level and Window parameters of a TOSCA segmentation. As with the Signal to Noise measure, the Coherence Number improves as the sample size increases, but as the results show the relationship involves the cube root of the product of ultrasound frames and the cube of the limiting radius, or, in fact to the average number of data contributing values to a grid point (all distinct, i.e. with different angles of insonation) rather than the square root of ' n' (the number of repeated samples with potentially the same value) .

The Edge continuity measures for a fixed segmentation level involved determining the width of the "Window" needed to establish a continuous perimeter for a VOI and the count of the number of pixels in that perimeter for a standard setting of the Window. These attributes relate directly to TOSCA segmentations. The tabulated results correspond to the "Level" being set to the mean value of pixels in the reconstructed plane, that is 40, and the standard Window for the pixel count being set to 2. This gives a maximum, "target", pixel count of 345, obtained using map A with 1009 frames and an R of 0 . 50 mm .

The Window entries provide an independent confirmation that the Coherence number represents an appropriate, objective and quantitative way to characterise the quality of a reconstruction in regard grey scale TOSCA segmentation. The correlation between the width of the Window to ensure edge continuity and the inverse of the Coherence number shows that the regression coefficient for the linear fit with 17 observations is 0.9734. Only the data for map C when R = 0.10 mm. was discarded.

Setting aside the suspect values for map B when R = 0.1 mm and for map C when R = 0.1 or 0.2 mm. the pixel count shows the anticipated gradual fall-off from the maximum value of 345 as the number of ultrasound frames used for reconstruction or the cut-off radius R is reduced. The minimum value (268) occurring with map C for R = 0.3 mm. is still very significantly larger than the count of 121 recorded for the raw 2D ultrasound frame.

Finally, two points concerning image smoothing. First, the Coherence Number, Filter values and Edge Continuity measures presented in Table XI are based on images that have been explicitly "smoothed" , the mean of the intensity values at the four corners of the 2D grid cell being used for the intensity at the centre. In the case of the Coherence number an assessment can also be made of this parameter without the smoothing (figures in parenthesis in Table XI) . There is a linear correlation between these two options with a gradient of 1.33 and regression coefficient r of 0.9977. The fact that conventional "smoothing" produces an improvement of order 33% provides a useful metric for assessing the scale of the improvements resulting from "compounding", these being of order several 100 %. Second, if the Grid Mapping cut-off radius R equals or exceeds the data spacing on the ultrasound frame (here 0.2 mm.) , each data value in an ultrasound frame has potential to contribute to the grid map calculation for two neighbouring grid points. This potential becomes a certainty when R exceeds the length of the grid diagonal (0.346 mm) . For larger values of R the number of grid points involved will be more than two. This represents a form of image smoothing and is implicit in grid mapping process. The magnitude of the effect can be assessed by looking for discontinuities in the various quality measures at R = 0.19 and .20 mm. and again between R = 0.3 and 0.4 mm.

The results presented in Table XI indicate no significant discontinuities.

To summarise, the "Coherence Number" was found to correlate well with:

1) Qualitative assessment of image quality based on ease and consistency of TOSCA segmentation ; 2) Qualitative assessment of image quality based on the appearance of interfaces when revealed by displaying only pixels within a narrow range of values corresponding to an appropriate "level and window" selection; 3) Quantitative assessment of image quality from the coherence of constituent interfaces based on edge continuity using the "level and window" measure. In all cases the "compounded" grid-mapped reconstructions offered very significant improvements in interface coherence, the improvements being 6 to 10 times as large as that produced by simple data smoothing. (table XI ) . Using the procedure detailed here of positioning the 3D grid using a "key" plane of 2D data has enabled, possibly for the first time, the true evaluation of the improvements in image quality which are achieved following reconstruction based on compounded free-hand scanned, high-density sampled data. This will have important implications for routine clinical use.

Table XI Quality parameters for Grid mapped central plane which is co-incident with the location of the KEY 2D ultrasound frame. As in tables XI & 2 the maps A, B and C are of a fixed size (26x20x22 mm. ) but represent reconstructions based on data sets with respectively 1009, 505 and 253 ultrasound frames.

NOTE - data for R 0.1 for map B and R 0.1 and 0.2 for map C are suspect due to tears in the reconstructed plane - insufficient data density at these lower R values.

Coherence Number and Signal to Noise

It is important to distinguish between these two measures and to use one or the other as appropriate to the specific requirements to characterise "Image Quality" . For many imaging modalities the quality of an image can be improved by taking an average over multiple, identically obtained, samples. The basis for this is that the signal is systematic while the noise is random. By averaging over ^λn' samples the discrimination between the signal and the noise is increased over that for a single sample by the square root of ^Λn' . This is done in MRI to improve the image, but requires that corresponding points in succeeding samples are identically placed. Under these condition where the samples being combined differ only in their noise content, the signal to noise ratio itself provides an important characterisation of quality.

In using "compounding" and "grid mapping" techniques in the reconstruction of ultrasound images it is true that the data from multiple ultrasound images is combined. However, in this case the samples are not identically obtained, the angle of insonation will differ from frame to frame. Also they are not identically placed: they are simply nearest neighbours to the relevant grid-point and thus distributed within a sphere whose radius is equal to the cut-off radius.

Under these conditions an alternative measure to signal to noise is needed which can take account of these facts and be expressed in terms of the statistical properties of the distribution of 1/R weighted values within the cut-off sphere. The analogy of the "signal" will be the mean value while the analogy of "noise" will be the standard deviation of this mean. This leads directly to the % coefficient of variation for the mean of the distribution providing the appropriate analogue to a "noise to signal" ratio. Average values for this measure for the entire reconstructed central key plane are recorded in Table XI for the various map parameters. Typical values are in the range 0.1% to 0.75 % corresponding to "signal to noise" values of better than 100 : 1. The relationship between the % CoV of the mean and the product of the number of ultrasound frames and the cube of the limiting radius approximates to an inverse 5/3 power law.

However, the need here with ultrasound is not to just characterise the statistics of each picture element (pixel or voxel) in the reconstructed object as in independent entity, but to provide a measure that relates to the ability to segment a volume of interest (VOI) on the basis of intensity. This involves using a measure that will reflect the consistency of intensities within the sub-set of pixels that define the boundary (or interface) of the VOI. The "Coherence Number" defined in the paper has the appropriate properties as it quantifies the degree to which the value of each pixel is correlated to those of its immediate neighbours, i.e. the pixels that will form the interface or boundary of the VOI. It also provides a measures of the continuity of an edge. As shown in figure X6, the Coherence number correlates well with another Edge Continuity measure based directly on the Level and Window parameters of a TOSCA segmentation. As with the Signal to Noise measure, the Coherence Number improves as the sample size increases, but as the results presented in Figures X3 and X5 show the relationship involves the cube root of the product of the number of ultrasound frames and the cube of the limiting radius. or, in fact to the average number of data contributing values to a grid point (all distinct, i.e. with different angles of insonation) rather than the square root of 'n' (the number of repeated samples with potentially the same value) .

GRID MAPPING & SEGMENTATION PARAMETERS

The choice of 0.25 mm. for the cut-off radius in Grid Mapping provides a good compromise when the ultrasound data spacing and grid size are 0.2 mm. Lower values of R would :

1) ensure that any data point would be used in the calculation of only one grid point, avoiding any undesired data smoothing ;

2) reduce the total grid-mapping calculation time; 3) reduce the broadening of the edges of features in the reconstruction; 4) leave reconstructed frames looking more like "normal" ultrasound images .

However lower values of R would reduce the number of data values and thus angles of insonation used to calculate the grid mapped echo intensities and thus :

1) increase the chances that there would be "tears/gaps" in the reconstruction which would interfere with segmentation;

2) reduce the degree to which ultrasound artefacts such as "speckle" will be removed by grid mapping;

3) reduce the overall "quality" of the image as measured by the Coherence number and thus the ability to segment out volumes of interest.

The results obtained when the reconstructed image is segmented using TOSCA to establish the boundaries of the volume of interest (VOI) and to allow its characterisation in terms of distances, areas and volumes will depend on the setting of a grey-scale "level" . Choice of an appropriate value will in turn depend on settings of the ultrasound machine and to a degree on choice of Grid mapping parameters such as the cut-off radius ( R ) . In order to make measurements consistent and objective it is necessary to identify the optimum setting of the level for each specific VOI . This is easily done using the facilities in TOSCA (or in a DX net) to "profile" the edge/interface in question and to select a level that is the mean of the intensity difference across the interface.

V. DISCUSSION

The results presented form a "chain of confidence" supporting the assertion that data acquired using conventional 2D ultrasound scanning techniques can be reconstructed to provide high quality 3D greyscale echo intensity images which are capable of supporting reliable and consistent segmentation of structures and the accurate measurement and progressively monitoring of their volume. The links in this chain are :

3) The percent coefficients of variation show the distribution of l/r weighted pixel values used in the 3D reconstructions of the reference plane to be very well grouped, with average values for the whole plane (13,000 pixels) ranging from 0.75% for the 250 frame reconstruction, dropping to 0.1 % with 1000 frames (Table XX) . The effective "signal to noise" figures are 133:1 rising to 1000:1.

4) The side-by-side comparisons of a single 2D ultrasound image frame and the echo intensities for the identical

2D plane extracted from the 3D reconstructed image of the foam rubber phantom (figure 4) . This shows the differences in continuity of the interface obtained when a narrow range of pixel intensities are selected. Of the 345 pixels that might possibly be identified, the conventional 2D image has 35% while the reconstructed planes have 70% reconstructing with 250 frames rising to 92% with 1000 frames (Table XX) . The coherence numbers which serve to characterise the ease and reliability of segmentation using the TOSCA level and window type algorithm again improve from the average for the plane (13,000 pixels) for the "raw" 2D image of 51 to 140 for the 250 frame reconstruction and to 220 with 1000 frames used (Table XX) . The percentage improvements of 275% and 430% are respectively 8 and 13 times as large an effect as the 33% improvement typically found when comparing coherence numbers for images before and after "smoothing " using a 2x2 box filter.

Table XX - A summary of the "typical values" for various statistical properties of raw and reconstructed image planes. (Cut-off radius R between 0.2 and 0.3 mm. - Maps A,B & C) .

The results presented in this paper and others demonstrate that 3D reconstruction may be achieved and that a high degree of precision can be obtained in reconstructing and analysing volumes even in the sub- milli-litre range. This conflicts with Detmer's view, that the magnitude of the positional uncertainties is sufficient to prevent meaningful reconstruction and analysis of anything other than "larger anatomical features". In taking this position, Detmer appears to have assumed that the errors involved would be random in nature, would apply in an uncorrelated way to each point in a reconstruction and thus, because of the milli-meter magnitude, limit meaningful reconstructions to only those larger anatomical features .

On the basis of our own observations and results, notably those relating to the appearance of multiple, sharp, discrete images in compound reconstruction where the Δ matrix has been perturbed, we find that the 2D frame registration errors are systematic rather than random and that the one milli-meter residual represents a measure of accuracy rather than precision, being associated with the positioning of the reconstructed object as a whole rather than the individual data points making up the reconstruction.

Nelson et al . have taken the precise registration of frames (system calibration and scaling factors) further in that they have achieved volumeric reconstruction and measurement of larger objects (ml) .

Previous Reconstruction algorithms Pitt, Humphreys BIR 1994 & BMUS ; Detmer; Nelson;

McPherson (Sci. American). Individual US planes (usually approximately parallel) are registered in a common 3D reference frame and then surface rendered using interpolation methods to fill in the gaps. Prager, Gee (Cambridge) use similar method but also acquire data on transverse planes and are working on ways to "adjust" intersecting planes so as to bring boundaries into alignment .

These methods of reconstruction still rely on data from a very small number of independent ultrasound frames. There is no intrinsic use of compounding to limit or remove speckle.

These reconstructions are no less vulnerable than grid mapping on registration errors.

>>>>>>>>>>>>>>

Previous workers have taken sequences of 2D images and assessed the position of each one in space. They have then calculated the position of every data point in each 2D image and assigned its data value to a nearest grid position on a 3D Cartesian grid. At best they have had one single data value (subject to speckle and shadow artifacts) for each grid position and where there was no value they have drawn information from adjoining positions using a nearest neighbour procedure. The positional information used has relied on either all the 2D planes being parallel or, for freehand scanning, that the image plane has been parallel to the axes of the electromagnetic positioning device attached to the ultrasound wand. Neither of these two assumptions is desirable or even valid in many cases.

Because of the 'single' nature of the data on this 3D grid, the reliability of the data values was no better than that from each 2D image. It is known that grey level segmentation of a typical interface within a 2D image is inherently unreliable and to be avoided. The Edge continuity measurements (Table XI) , for a fixed segmentation level, involved determining the width of the "Window" needed to establish a continuous perimeter for a VOI and the count of the number of pixels in that perimeter for a standard setting of the Window. These attributes relate directly to TOSCA segmentations. The tabulated results correspond to the "Level" being set to the mean value of pixels in the reconstructed plane, that is 40, and the standard Window for the pixel count being set to 2. This gives a maximum, "target" , pixel count of 345, obtained using map A with 1009 frames and an R of 0.50 mm.

In an example to be discussed in detail here segmentation of the 'raw' 2D image, using these parameters, would result in a 'path' (the perimeter of the VOI), containing only 121 pixels, the rest being gaps. This could not be used for grey-level segmentation directly and would have to be handled by 'active shape models' or some other model based procedures. [Models imply some a priori knowledge of likely shape, a situation of less validity with increasing pathology] .

In that example the 'path' had a mean pixel value of 40, which in order to achieve continuity based on the 2-D image would have required the window width to be set at 8 or 9, in other words grey level limits of 31.5 - 48.5.

Such a wide window would then have allowed about 50% of the whole image plane to be "included" as the interface, and for segmentation this would be a lost cause.

The technique described below can be divided into 5 sections :-

1. Greatly increased data density enabling grey-level segmentation

2. A high speed algorithm for reconstructions of these large data sets, along with a computing environment to match.

3. Much greater precision of the positional data achieved by using regression techniques on the large data volume to calibrate the ultrasound wand initially.

4. Exploiting the advantages of free-hand compounded data to increase the reliability of the 'raw' grey-level values, suppressing artifacts. 5. Direct comparison of an original 2D data frame with its exactly located equivalent following reconstruction. This enables a full statistical analysis of the improvements produced by the method to be carried out, thus increasing confidence in the validity of the approach.

These will be enlarged on in the discussion below. >>>>>>>>>>>>>>>>>>>>>> 1. Data density: grey-scale segmentation

In developing a three-dimensional capability for ultrasound we have utilised a freehand scanning technique which capitalises on instinctive scanning behaviours. This thereby enables high density scanning of the areas of greatest interest . Typically the number of ultrasound echo intensity values on selected frames exceeds the number of 3D grid-points by a factor of 5 to 10. It was only after the features of high density, over-sampling of data and multiple angles of insonation (compounding) were implemented, producing data with significantly greater coherence, that the commercial software for 3D data analysis, developed for modalities such as MRI and CT, was able to provide automated grey-scale segmentation. The ability to segment ultrasound data on the basis of grey-scale intensities has to date been controversial because of speckle and because the grey-scale attributed to the same object may be significantly different between machines .

However with the full technique described in this paper it has been possible to segment successfully on grey level . Firstly sufficient numbers of data planes have been recorded and registered (typically 250 - 500) that each grid point value is now the average of between 17 and 60 data points for the phantom examined. In Table XI, setting aside the suspect values for map B when R = 0.1 mm and for map C when R = 0.1 or 0.2 mm. the pixel count shows the anticipated gradual fall-off from the maximum value of 345 as the number of ultrasound frames used for reconstruction or the cut-off radius R is reduced. The minimum value (268) occurring with map C for R = 0.3 mm. is still very significantly larger than the count of 121 recorded for the raw 2D ultrasound frame. This value (268) was achieved within the grey-level range 38.0 - 40.0 and a continuous path within grey-level limits of 36.5 - 43.5 . This radical improvement in discrimination allowed precise grey-level segmentation to become a reality.

2. High-speed; environment

This enormous increase in data density has been achieved by the use of; an S-VHS video recorder to record all the frames in a free-hand interrogation along with an audio-channel recording of the EPOS position; a sophisticated frame selection interface allowing for timings to match an e.c.g. cycle if required (recorded on the second channel of S-VHS) ; and a high-speed data acquisition and positioning algorithm to allow reconstruction in less than 30 minutes from typically 500 data frames .

The S-VHS videotape provided a cost effective and efficient archive as the equivalent of around 300 Gbytes of information could be stored on each tape. Data acquired, free- running, can be subsequently extracted as data sub-sets, without the need to re-interrogate the subject. This ability to post process video tape archives and extract data sub-sets from specified periods of the cardiac cycle will allow comparison of co-registered structure shape, volume and texture patterns during these different phases. The system described here provides a means to monitor volume accurately from conventional 2D ultrasound scanning techniques, the basic equipment and software used being for the most part commercially available. However it is the customised hardware and computer software described in this paper that have developed this prototype laboratory tool into a system sufficiently flexible and efficient to propose starting its use in the clinical environment. In our laboratory some 2,000 3D reconstructions of ultrasound data sets have been made with this system in the past 12 months, each providing 3D images of potential clinical relevance.

3. Precision, delta: reconstructions

The increased data density has had a second benefit in that the precision of the location of individual data points in each 2-D image has been greatly improved: this is because the DELTA matrix obtained using a point phantom has been optimised using a linear least squares minimisation approach with this vastly over-determined data set, giving precision to the level of 0.01 degrees and 10 microns for each of the three beam axes and probe coordinates respectively. Previous workers, using only individual data points in their optimisations have had to stop at an rms deviation of 1.0 mm or more.

The key to successful 3D reconstruction is accurate registration of the 2D US data. In the case of free-hand scanning, this registration depends on the accurate determination of the set of system scaling parameters embodied in our Δ matrix. Reconstruction from parallel plane data, with or without use of an EPOS device, represents a special case but still must be based on the properties and role of the Δ matrix that have been derived from the analysis of the mathematical transformations (Appendix 1) and confirmed by the detailed appearance of reconstructed volume and cross- thread phantoms . Reconstructions of objects such as the cylindrical foam-rubber phantoms using multiple scans all consist of a number of discrete images, each associated with a subset of the 2D ultrasound images. If the Δ used is ideal, the 2D Ultrasound images will be correctly registered in the 3D transmitter frame XYZ and the discrete images will converge into a single object. During refinement of Δ, the registration of the 2D ultrasound images in 3D will not be perfect, so that the individual discrete objects will differ in shape and, in general, volume and will not converge exactly. The degree to which the 2D ultrasound images are mis-registered and the individual discrete objects are distorted and mis- positioned depends partly on the estimate of Δ but also on the translations and orientations that define the EPOS values (M' s) for the images making up the particular scan subset.

Analysis of the mathematics for free-hand scanning (APPENDIX 1) verifies that :

1) Only the true Δ will result in reconstructions which accurately represent shape, volume and position using unrestricted free-hand patterns of scanning .

2) Even an imperfect Δ can provide reconstructions which accurately represent shape and volume, but only for a highly restricted range of scanning patterns.

3) For every transducer- EPOS receiver configuration there is a unique scanning direction - the "Zeta direction" , which will result in a reconstruction with accurate shape and volume, independent of the Δ used. However, even in this special case, the true Δ matrix is required to correctly position the reconstruction and to determine the unique scanning direction relative to the ultrasound reference frame, qrs. Papers reporting reconstructions from evenly spaced parallel 2D US frames (Refs. Lee & Gardner etc.) have in general ignored the Δ matrix. This is equivalent to assuming that the rotation component of Δ is a Unit matrix or that the vector defining the incremental displacements is exactly perpendicular to the 2D US frame and thus corresponds to the unique "Zeta scanning direction" for this arrangement. As with free-hand scanning, the degree of divergence between the effective (Unit) matrix used for reconstructions and the matrix that represents the true situation, will control the magnitude of any errors in the shape, volume or position of the reconstructed object.

Even small deviations in the Δ matrix can have a profound effect on the quality of the resulting reconstructions and, in general, the US beam plane defined by the crystals in the US probe can not be relied upon to be exactly aligned with respect to the faces of the probe itself. However, in the case of single parallel scan reconstructions, even very significant Δ matrix discrepancies may produce good shape reconstructions but this would result in poor volume measurements. In the free-hand case this combination of events would alert the reconstructer to the presence of a problem as identified by multiple discrete images. Using the point phantom approach to determine a translation matrix for registration of ultrasound images, the value of the criteria function at convergence has typically an rms. deviation of order 1.0 millimetre or more. Detmer et al (1994 paper) using a point phantom but a somewhat different method of determining Δ and more sophisticated positional equipment found the same magnitude of residual. Similar results have been found by other workers (Pitt at BMUS 95) .

Again in agreement with Detmer, our general finding is that the principal source of the deviation comes from the positional uncertainty inherent in ultrasound because of the finite thickness of the ultrasound beam at the focal distance. We have examined this issue in more detail by using equation lb to back transform the group mean P(XYZ) and to calculate the co-ordinates of this point in the individual ultrasound image co-ordinate systems, p(qrs). The results show that the co-ordinate (q) corresponding to a height above or below the ultrasound image plane has a distribution consistent with this model .

Deviations of the refined Δ from the true optimum Δ will also contribute to positional uncertainty. The Δ transformation has a profound effect on the quality and accuracy of the 3D reconstruction and a considerable degree of precision is required. Fortunately, the over-determined system of equations has been found extremely well behaved, yielding a single minimum and a long range, almost quadratic dependence of the criteria function on each of the six parameters of Δ. Refinements from widely differing starting trial Δ parameters converge to the same solutions for Δ and P(XYZ) . These estimates are better than can be measured directly, (angles to within 0.01 degrees, positions to within 10 microns) .

4. Compounding Multiple angles of data acquisition are provided that reduce the angle dependency of intensity data used in construction of the 3D data block.

In our system, compounding reduces the effect of speckle while the TOSCA product allows a profile of intensities to be obtained across a representative slice of the data block. This allows an appropriate level and window to be selected for the region of interest for segmentation throughout the block and reference features to be used to provide cross correlation of structures and their intensities between machines and subsequent data sets. Without this approach grey-level segmentation f ai l s .

Further the reliability of the data values at each grid point has been enhanced by compounding, where faults in grey-level caused by shadowing or poor angles of view are on average outweighed by the other 'good' planes.

Remembering that 17 to 60 data values/grid point means 17 - 60 separate data planes contributing (no plane providing more than one point) it is easy to see that the adverse effect of one or two "faulty" planes is rapidly diminished. This has not been available to previous workers .

In all cases the "compounded" grid-mapped reconstructions offered very significant improvements in interface coherence, the improvements being 6 to 10 times as large as that produced by simple data smoothing. (table XI ) . Using the procedure detailed here of positioning the 3D grid using a "key" plane of 2D data has enabled, possibly for the first time, the true evaluation of the improvements in image quality which are achieved following reconstruction based on compounded free-hand scanned, high-density sampled data. This will have important implications for routine clinical use.

5. Statistics: errors The level of confidence with which such work can be carried out is assessed by the statistical analysis of %CoV now made possible by the power of the DX system from IBM. For the first time, using the 'key frame concept' described here it has now been possible to compare a 'raw' 2-D frame and its reconstructed, "exactly located", equivalent from 3-D. Such statistics could not be obtained from a 2-D image, but for the 3-D reconstruction the 0.75 %CoV for the worst case is most reassuring. With regard to the coherence number as defined in the text, it is important to remember that data smoothing was the only tool previously available for making improvements. In Table XI the Coherence Number, Filter values and Edge Continuity measures are based on images that have been explicitly "smoothed" , the mean of the intensity values at the four corners of the 2D grid cell being used for the intensity at the centre. In the case of the Coherence number an assessment can also be made of this parameter without the smoothing (figures in parenthesis in Table XI) . Figure X7 (Sheet 2 chart 4) shows that there is a linear correlation between these two options with a gradient of 1.33 and regression coefficient r of 0.998. The fact that conventional "smoothing" produces an improvement of order 33% provides a useful metric for assessing the scale of the improvements resulting from "compounding", these being of order of 150-300%. The results presented in this paper and others demonstrate that 3D reconstruction may be achieved and that a high degree of precision can be obtained in reconstructing and analysing volumes even in the sub- milli-litre range. This conflicts with Detmer' s view, that the magnitude of the positional uncertainties is sufficient to prevent meaningful reconstruction and analysis of anything other than "larger anatomical features" . In taking this position, Detmer appears to have assumed that the errors involved would be random in nature, would apply in an un-correlated way to each point in a reconstruction and thus, because of the milli-meter magnitude, limit meaningful reconstructions to only those larger anatomical features.

On the basis of our own observations and results, notably those relating to the appearance of multiple, sharp, discrete images in compound reconstructions where the Δ matrix has been perturbed, we find that the 2D frame registration errors are systematic rather than random and that the one milli-meter residual represents a measure of accuracy rather than precision, being associated with the positioning of the reconstructed object as a whole rather than the relationship of the individual data points making up the reconstruction.

Nelson et al . have taken the precise registration of frames ( system calibration and scaling factors) further in that they have achieved volumetric reconstruction and measurement of larger objects ( ml) .

To summarise, the "Coherence Number" was found to correlate well with: 1) Qualitative assessment of image quality based on ease and consistency of TOSCA segmentation ;

3) Quantitative assessment of image quality from the coherence of constituent interfaces based on edge continuity using the "level and window" measure. Final

Once the grid-mapped data block has been generated, measurement of volume is only one example of many features and utilities that may be exploited in 3D analysis. Segmented objects may be displayed and rotated to provide views that are unobtainable in normal scanning. Filters and colour maps may be applied to highlight features or textures. Any plane required through the data block may be displayed, and positioned relative to segmented 3D objects. This will have relevance when comparing ultrasound with histological sections or co-registered images from other modalities such as MRI or CT. Data may be mapped to a regular surface such as a model of an organ or structure. This may provide an ability to locate abnormalities in shape, texture or provide a map of distribution of a feature.

All this information is summarised in Table ( ) , along with a Chain of Confidence outlining the relevant stages.

SUMMARY - CHAIN OF CONFIDENCE

Phantom picture -> Phantom Volumes -> % CoV for reconstructed plane -> figure of raw and reconstructed central plane -> Coherence Numbers

Can summarise in a table using "typical values" R20 - R30 Maps A, B and C.

REFERENCES

Detmer P.R.; Bashein G. ; Hodges T.; Beach K.W. Strandness D.E.Jr. 3D ultrasonic image feature location based on magnetic scan head tracking: In vitro calibration and validation. Ultrasound Med. Biol. 20: 923-936 1994.

Fine D . Three-dimensional ul trasound imaging of the gallbladder and dilated biliary tree : reconstruction from real - time B-scans . Br. J. Radiol 64 : 1056 1991 .

Franceschi D. ; Bondi J, ; Rubin J.R. A new approach for three-dimensional reconstruction of arterial ultrasonography. J.Vasc.Surg. 15: 800-805, 1992.

Gardner J.E.;Lees W.R. : Gillams A. Volume imaging with ultrasound. Radiology 181(P):133, 1991.

Geiser E.A.; Ariet M. ; Conetta D.A. ; Lupkiewcz S.M.; Christie L.G. Dynamic three-dimensional echocardiographic reconstruction of the intact human left ventricle; Technique and initial observations in patients. Am.H.J. 103: 1056-10 . 1982.

Gilja O . H. In Vi tro evaluation of three-dimensional ul trasonography in volume estimation of abdominal organs .

Ul trasound Med . Biol 20 : 157 1994 .

Hamper U.M. ; Trapanotto V.; Sheth S . ; Dejong M.R.; Caskey C.I. Three-dimensional U.S preliminary clinical experience. Radiology 191:397-401.1994. Hell B. 3D Sonography. Int.J. Oral Maxillofac. Surg. 24: 84 -89 1995. Kelly I.M; Gardener J.E.; Brett A.D.; Richards R.; Lees W.R. Three-dimensional US of the fetus- work in progress. Radiology 192: 253-259, 1994.

King D.L.;King D.L. jr . ; Yi-Ci Shao M. Three- dimensional spatial registration and interactive display of position and orientation of real-time ultrasound images. J Ultrasound Med 9:525-532,1990.

Levine R.A. ; Weyman A.E.; Handschumaker M.D. Three- dimensional echocardiography : Techniques and applications. Am.J. Cardiol . 69:121H-130H 1992.

Mintz G.S.; Keller M.B.; Fay F.G.; Motorised IVUS transducer pull-back permits accurate quantitative axial measurements (abstr.) Circulation 86-1 :323s. 1992.

Moritz W.E.; Medema D.K.; Ainsworth M. ; McCabe D.; Pearlman A.S. Three-dimensional reconstruction and volume calculation from a series of nonparallel, realtime, ultrasonic images. Circulation 62: 111-143 1980.

Nelson T.R. ; Pretorius D.H. ; Slansky M. ;Hagen-Ansert S. Three-dimensional echocardiographic evaluation of fetal heart anatomy and function. J. Ultrasound Med. 15:1-9, 1996.

Picano E.; Landini L.; Distante A.; Salvadori M. ; Lattanzi F. Angle dependence of ultrasonic backscatter in arterial tissues; a study in vitro . Circulation 72: 573-6 1985.

Riccabona M. ; Nelson T.R. ; Pretorius D.H.; Davidson T.E. Distance and volume measurement using three- dimensional ultrasonography. J. Ultrasound. Med 14:881-886, 1995. Rosenfield K. ; Losordo D.W.; Ramaswamy K. ; Isner

J.M. Three-dimensional reconstruction of human coronary and peripheral arteries from images recorded during two- dimensional intravascular ultrasound examination. Circulation 84: 1938-1956 1991. Steen E . ; Olstad B. Volume rendering of 3D medical ultrasound data using direct feature mapping. IEEE Trans.Med.Imag. 13:517-525. 1994.

Vogel M. ; Ho S.Y.; Buhlmeyer K. ; Anderson R.H. Assessment of congenital heart defects by dynamic three- dimensional echocardiography; methods of data acquisition and clinical potential. Acta Paediatr. suppl. 410: 34-9 1995. von Birgelen C; Slager C.J.; Di Mario C; de Feyter P.J.; Serruys P.W Volumetric intracoronary ultrasound; A new maximum confidence approach for the quantitative assessment of progression-regression of atherosclerosis? Atherosclerosis 118 (Suppl.) S103-S113. 1995. von Ramm O.T.; Durham N.C. Smith S.W.; Carroll B.A. Real time volumetric US imaging. Radiology 193 (P) : 308, 1994. Zosmer N. ; Jurkovic D. ; Jauniaux E.; Gruboeck K. ; Lees C. Selection and Identification of standard cardiac views from three-dimensional volume scans of the fetal thorax. J.Ultrasound Med. 15: 25-32, 1996.

scaling algori thm for images ref . Pi tt ( BMUS ' 95)

APPENDIX 2 - Rotation Matrices & Homogeneous Co-ordinates

M and Δ each have rotational and translational components defined in three dimensions by 3 angles and a vector. It is convenient to use an abbreviated form of homogeneous co-ordinate notation (MAXW46 & 51 / 176 & 175 Roberts 233 and Newman & Sproull) . Such transformations can then be represented symbolically by a 2 x 2 partitioned matrix of the form

(Rot Tranλ

0 1 where Rot is itself a 3 x 3 rotation matrix and Tran is a 3 element column vector. Rotation matrices are represented by matrices that are Unitary and have several important properties that are made use of in the main text. Specifically :

1) The transpose of a rotation matrix its also its inverse. The inverse of

is thus (Rot ^_1 j - Rot^"1 * Trans)

2) If the co-ordinates for a fixed point in space given in terms of a reference frame XYZ are P^ and relative to a new frame QRS are P^._g and the origin of QRS is at the point 0_xyz and the axes of QRS are defined by the unit vectors A^, B_xyz and C_xyz, then the transformation to get P^ given P^ has a translation component Trans = O_xyz and a rotation component Rot whose three columns are the vectors A_xyz, B_xyz and C_xyz .

3) If the matrix representing a rotation by an angle T about a vector whose orientation in space is specified by the three direction cosines a, b and c has elements by row of R_llr R₁₂, R₁₃ , R_21/ R₂₂, R₂₃ , R₃₁, R₃₂ and R₃₃ then

T = Cos ^_1 ( ( Rn + R₂₂ + R₃₃ -D/2) and a = (R₃₂ - R₂₃ )/2*Sin ( T) ; b = (R₁₃ - R₃₁)/2*Sin ( T) ; c = (R₂₁ - R₁₂)/2*Sin ( r)

4) The generalised rotation matrix has three mutually orthogonal eigen vectors, one real and coincident with the rotation axis, the other two complex. Using the notation above and with i standing for the square root of minus one, the eigen values and vectors are : Eigen value : 1.0 Eigen vector : (a,b,c) Eigen value : Cos (T) + i Sin(τ) Eigen vector : 0.707 (VA - i VB)

Eigen value : Cos (T) - i Sin (T) Eigen vector : 0.707 * (VB - i VA) VA is the normalised vector formed from the cross product of (a,b,c) with the vector (1,0,0) and VB is the normalised vector formed from the cross product of (a,b,c) with VB . 5) The POLHEMOUS transformation, M must be defined in terms of the parameters available from the device, three positional and three directional. Following the conventions set out in the POLHEMUS manual ( reference ??) the rotation component Rot is defined by the 3 Eulerian angles designated as Azimuth, Elevation and Roll (here shortened to a, e and r) representing Z, Y and X axis rotations. Noting that rotation matrices do not, in general, commute so that the order is critical

Rot (a,e,r) = Z(a) * Y(e) * X(r) or, writing this product out in full as a 3 x 3 matrix

ca *ce ca ^*se *sr-εa *cr ca *se *cr+sa *sr)

Rot (a , e, r) =1 sa *ce sa *se *sr+ca ^*cr sa *se *cr-ca "sr -se ce *sr ce *cr where sa, ca, se, ce, sr and cr are the sine and cosine of the angles a, e and r respectively.

Note that Rot (a,e,r) = Rot (a', e' , r' ) where a' =π + a, e' = n - e and r' = π + r

There now follows a description of a non-invasive technique which may be employed as a diagnostic tool.

Abstract; -

The progression of atherosclerotic lesions in the Watanabe rabbit and the modification of lesion components by Probucol treatment were sequentially monitored over a 30 week period using 3D reconstructed ultrasound image data. The 3D ultrasound data, generated by spatial orientation and gridmapping of a large number (>500) of image frames from conventional 2D hand held ultrasound interrogation, was heavily compounded over the angles of insonation used, improving both signal to noise and structure coherence. This compounding of high data density was found to allow accurate and reproducible greyscale analysis and segmentation of the data, providing sequential volume measurements of structure components identified and selected on the basis of differing greyscale intensity.

Two major components of the lesions developed in these animals were identified and sequentially monitored using 3D reconstruction and segmentation of the ultrasound images. At the end of the experiment histological techniques were used to establish the lesion composition. One identified component correlated with the development of a smooth mucle cell/collagen cap on the lesions during Probucol treatment of the animals. A second component, at a low echo level, indicated the lipid content of the lesions and together with the first component correlated with total lesion volume as measured from histological evaluation. The compounding of heavily sampled ultrasound data into a 3D reconstruction can provide differential greyscale identification and volume measurement of plaque components, including lipid which is normally difficult to differentiate from the blood filled vessel lumen in a conventional ultrasound image. The data presented here shows that 3D reconstruction of ultrasound images may be used to provide accurate and objective sequential volume measurements on atherosclerotic disease progression.

Introduction.;

With the development of high resolution transducers and scanning equipment offering a wide variety of imaging and Doppler facilities, percutaneous ultrasound has become the modality of choice for the interrogation of vascular disease and the assessment of peripheral atherosclerosis (Strandness,

1986) . The non -invasive nature of ultrasound interrogations and the flexibility of Duplex or Triplex scanning provides a diagnostic tool that can provide information not only on the degree of stenosis of the vessel and impairment of bloodflow but also on the textural appearance of the atherosclerotic plaque on the vessel wall. Investigation of plaque image texture may provide an ability to recognise those plaques presenting the greatest risk to the patient in terms of potential for rupture and resultant thrombotic events. Although interrogations of the vessel can be readily obtained, the quantification of such images, the extent of disease and the accurate sequential monitoring of disease progression remains a difficult task.

The suitability of ultrasound imaging to reproducibly monitor progression of carotid artery atherosclerosis has been under investigation for some time. Available evidence would suggest that linear measurements of the changes in intimal - medial thickness (IMT) from 2D images of the carotid artery have quantitative utility in monitoring vascular disease provided great care is taken in producing the image (Salonen, 1991; Poli, 1988; Pignoli, 1989; Riley, 1992) . The dependence that these measurements place on the reproducibility of interrogation angle and B-mode image quality gives the sonographer a critical role in the measurement process (Salonen, 1993) . Volume, rather than linear dimension, would allow the entire structure to be measured at each interrogation and be independent of the acquisition angle and view obtained. Several techniques including step motor advancement of the probe position in a single plane, tomographic probes or spatially registered single manual sweeps of a probe have been developed to generate multi -slice 3D ultrasound data that allow volume to be used for sequential measurement (Levine, 1992; Hell, 1995; Gardener, 1991; Kelly, 1994; King ,1990; Moritz 1980; Nelson, 1996; Riccabona, 1995) . These techniques have been successfully applied to carotid artery interrogations but are limited by relatively low data density as only a single observation is made through any plane. Parts of the structure under examination may be shadowed from calcium deposits within the atherosclerotic lesions. Derivation of volume measurements from such images normally require manual segmentation or model fitting algorithms as the data within the single images is often contaminated by speckle generated throughout the system and is not of sufficient integrity to allow automated greyscale segmentation techniques (Nelson , 1995 , 1996 ; Pretorius, 1995) . Multiple observations from a variety of interrogation angles of any single point within a volume, and compounding of these observations, allows the speckle in ultrasound images to be reduced, with significant improvements in image quality (Nelson, 1996) . Shadowing may also be reduced or eliminated if a sufficiently wide range of angle is utilised during interrogation.

The various components of atherosclerotic plaque possess differing arrangements and densities of cellular and particulate matter that act as reflectors and scatterers of ultrasound to differing degrees. The textural appearance within the ultrasound image will therefore depend on the proportions of reflectors and scatterers, their size, arrangement and density, also the frequency of the ultrasound beam and the angle at which it is presented. This textural information is well recognised and often applied in the subjective assessments of plaque structure, allowing differentiation of plaque into four or five classes indicating the homogeneity of the plaque depending on the degree and distribution of calcification, fibrosis and lipid content. Following the realisation that it is the unstable, lipid laden, plaque which is most closely associated with thrombotic events, considerable effort is being made to provide quantitative identification of these components by analysis of the textural information within the ultrasound image. The lipid component of such plaques, having few reflector or scatterer particles, may be identified by its echolucent nature and appearance as low level greyscale attribution in the ultrasound image (Geroulakos et al . ; 1993, 1994) . Differentiation of such low level greyscale attribution from the blood filled lumen of a blood vessel is however often difficult. The angle dependence of ultrasound image generation makes the sequential comparison of textural information extremely difficult. Obtaining identical views, using precisely the same acquisition angles, from different interrogations in vivo, is virtually impossible. As yet, derivation of information on plaque component change from such analysis has met with little success and the alternative technique of interrogation of integrated backscatter data has been investigated using excised tissues and high frequency transducers ( Wickline, 1993 ; Picano, 1985). Assessment of plaque components from single 2D images or interrogation beam by backscatter analysis may however be of very limited value as the components would almost certainly not be uniformly distributed throughout the volume.

The problems associated with quantification of atherosclerotic plaque progression, including both sequential dimension measurement and beam angulation in textural characterisation, may be overcome by the use of three- dimensional reconstruction and volumetric analysis of compound data generated from freehand two-dimensional ultrasound imaging as recently described by this group ( Allott,1996) . Volume rather than linear dimensions allows structures, whether they be regular or irregular in shape, to be accurately measured at each interrogation. The various insonation angles used to interrogate the structure during freehand scanning may be compounded to provide more reproducible reflection intensity data. The segmentation of such data throughout the volume of interest more accurately represents the characteristics of the plaque .

The Watanabe heritable hyperlipidaemic (WHHL) - rabbit (Watanabe, 1980) has a mutation in the LDL-receptor gene (Kita, 1983) which is similar to that seen in human familial hypercholesterolaemia . These animals develop a persistent increase in plasma cholesterol levels, resulting in an early and accelerated atherosclerosis, including plaque formation in the aortic arch. The WHHL-rabbit has been used by a number of workers to study various aspects of atherosclerotic plaque progression in this vessel ( Carew,1987; Daugherty, 1991 ; Braesen, 1995 ;0' Brien, 1991 ;Nagano , 1992) .

The present study was undertaken to develop and evaluate a post -processing system for 2D ultrasound images of the WHHL- rabbit aortic arch to permit 3D reconstruction, volumetric segmentation and analysis of the developing atherosclerotic lesions. These techniques also being used to identify and quantify changes in these advanced atherosclerotic lesions during treatment with the antioxidant and hypolipidaemic agent Probucol . This agent has been described as having effects on both volume and the cellular composition of atherosclerotic plaques in the Watanabe rabbit (Nagano, 1992 ; O'Brien 1991; Carew, 1987 ;Daugherty, 1991; Braesen, 1995; )

Methods Female Watanabe Homozygous Hyperlipidaemic Rabbits (WHHL) 12-14 weeks of age (Charles River, UK Ltd.) were housed under standard conditions with free access to water. Lipid content of the diet was controlled by providing 150 g of normal rabbit chow per day, supplemented with 500 g of assorted fresh vegetables and free access to sweet grass hay. The animals were weighed at weekly intervals to ensure no loss of bodyweight . At 5 week intervals the animals were monitored for serum cholesterol and the aortic arch examined by conventional 2D ultrasound. The animals were first sedated with 0.3 ml fentanyl (Hypnorm, Janssen) intramuscularly (im) and 2.5 ml blood withdrawn from an ear vein for serum cholesterol determinations using a commercial colourimetric kit ( Boehringer Mannheim) . The animals were then anaesthetised by intravenous (iv) injection of 0.25 ml midazolam (Hypnovel, Roche) and the aortic arch of each animal interrogated by ultrasound. At 30 weeks the animals were allocated to two groups. Group 1 received standard diet containing 1% olive oil while group 2 received 0.5% w/w Probucol (Sigma) dissolved in olive oil added to the diet. Animals were allocated on the basis of bodyweight, serum cholesterol levels and appearance of aortic arch as determined by ultrasound, so that all selection criteria were evenly distributed between the two groups. The animals were maintained on the diets for a further 30 weeks. Blood samples for serum cholesterol and ultrasound examinations from which 3D reconstructions were made were continued at 5 week intervals for the remainder of the experiment .

After the final examination the animals were terminated by iv injection of an overdose of pentobarbitone sodium (Euthatal, May & Baker Ltd.), the aortic arches were excised, washed in saline and stored in formal saline.

Ultrasound Examination

A Toshiba SSH 140a ultrasound scanner fitted with a 7.5 MHz linear array transducer was used throughout. Interrogation parameters were pre-set on the scanner by optimisation of images at the start of the experiment. The pre-set was used for all subsequent examinations, only gain and focus settings being optimised for each animal to provide a clear view of the aortic arch. To achieve 3D reconstruction the scanning transducer was fitted with an electromagnetic position and orientation sensor (EPOS -Polhemus - Isotrak II ) . This provided the transducer position and orientation information to enable reconstruction of the freehand 2D ultrasound images into a compounded 3D data block by off line computer processing.

Following anaesthesia as described ( vide supra) the chest and neck area of each animal were shaved with fine cut electric clippers and transmission gel applied liberally. Views of the aortic arch and origin of the carotid artery were obtained from a right lateral window over the clavicle. The tissue volume around the aortic arch and carotid origin was thoroughly and continuously interrogated at 30 frames per second (fps) for approximately 3-5 minutes, utilising as many angles of insonation as possible, the images being stored onto S-VHS video tape. Simultaneously, an ECG signal was obtained from recording electrodes applied to the animals chest. This signal was passed through a module in custom built processing hardware and recorded to one S-VHS video recorder audio channel. Positional information from the EPOS, synchronised with image generation, was processed by a second hardware module and recorded on the second S-VHS recorder audio channel. These signals were later used for post processing selection of ultrasound image frames and spatial orientation. The videotapes containing the image data, positional information and ECG record, were stored until 3D reconstruction and analysis .

At the end of the experiment the videotapes were digitised in real time onto a Silicon Graphics Indy workstation utilising a COSMO video compression card and IRIS media tools. This produced a moviefile for each interrogation of approximately 6,000 digitised image frames in which the ultrasound images were uniquely identified and locked to the appropriate section of the audio record, containing the positional information for that frame and the position within the ECG cycle.

The image frames from the data set were then further selected utilising a customised motif application on the basis of their temporal location within the ECG cycle. Each selected data set contained approximately 500 image frames from the same portion of the cardiac cycle, but from a variety of interrogation angles. The greyscale intensity data from the selected frames were then extracted, spatially oriented and gridmapped into a regular 3D grid to produce a compound 3D data block. These data blocks were transferred onto an IBM RS 6000 workstation and analysed using IBM 3D automated segmentation and analysis software TOSCA. (TOols for Segmentation, Correlation and Analysis) .

Each data set was visualised in TOSCA. A region of interest that avoided the bone shadows cast by the ribs and clavicle was outlined to include the lumen of the aortic arch. ( Fig 1) Semi-automatic statistical greyscale segmentation using the TOSCA volume growing algorithm (Cootes, 1994) , at fixed window and level settings, selected by using the greyscale profile facility, was used to identify different structures within the images. Segmentations were performed through the outlined region of the 3D data set with the parameters : - a) level 32 + 2 to delineate the lumen of the vessel. b) level 42 + 4 to delineate hypoechoic structures adjacent to the vessel wall protruding into the vessel lumen c) level 60 ± 10 to delineate structures adjacent to and within the vessel wall and surrounding tissue. d) level 90 ± 20 to delineate hyperechoic areas.

Volumes for all segmented structures were obtained and expressed as segmented volume /cm³ total tissue, to normalise animal to animal anatomical variation.

Histology

The excised aortic arches from four animals from each treatment group were chosen at random for histological processing. The arches were placed in O.C.T Tissue-Tek embedding compound (Miles Inc.), frozen at -28°C, trimmed and prepared for sectioning using an Anglia Scientific Cryotome. The arches were orientated so that sequential cross sections were taken proceeding from the heart end of the vessel . Three successive lOμ sections were taken at 170μ intervals along the entire length of the aortic arch finishing near the carotid sinus. The sections were mounted on 3-aminopropyl- triethoxysilane (Merck-Schuchardt) coated slides. The second slide of each triplicate was immediately stained with Oil Red 0 (Sigma) to identify lipid within the plaque and counterstained haematoxylin (Shandon) . The remaining slides were snap frozen and stored at -70°C for immunohistochemistry or cytochemical investigation.

The area of each section occupied by unstained normal tissue or Oil Red 0 stained tissue was determined by planimetry. Each section was viewed on a colour video monitor using a Zeiss microscope equipped with a Panasonic colour video camera. Areas were traced onto transparent film and subsequently digitised using a TDS Bit Pad. A calibration grid was used to determine the magnification of the tracings and calculate the area measurements. The area values for normal tissue and lesion for each sequential lOμ section was obtained. Volume measurements encompassing the whole arch were calculated assuming a linear relationship for the 190μ gap between successive sections. The data was expressed as segmented volume (mm³) /cm³ total tissue.

Lesion Collagen Estimation

Slides of the descending portion of the aorta were prepared as above to determine the influence of antioxidant treatment on the lesion collagen concentration. A spectrophotometric method (De Leon et al . ) based on the selective binding of Sirius Red to collagen was used. Slides were treated with a freshly filtered saturated Picric acid solution containing 0.1% w/v of both Fast Green FCF and Sirius Red for 30 min. They were then immediately rinsed with 5 changes of distilled water and allowed to air dry at room temperature. The sections were then destained by extraction with two volumes of 100 μl of 0.1 M NaOH in absolute methanol (1:1 v/v) which were then combined in a well of a flat bottomed 96 well plate and the absorbance at 540 nm determined. A standard calibration curve to sirius red (1-15 μg/ml) was simultaneously constructed and the concentration of sirius red eluted from each section was determined by reference to this curve .

Immunohistochemist y The frozen sections stored at -70°C were used to determine the cellularity of the lesions, in particular the relative numbers of smooth muscle cells and macrophages were assessed by the following immunohistochemical approach. Sequential adjacent pairs of slides, covering the entire length of the aortic arch, were selected. The first slide of each pair was probed with a mouse monoclonal antibody (Mab) against rabbit macrophages, RAM-11. This antibody reacts with a macrophage cytoplasmic antigen and has been used extensively in studies investigating the cellular components of atherosclerotic lesions in rabbits. The slides were incubated at 21°C with Immunopure peroxidase supressor (Pierce and Warriner) for 1 hour , rinsed in TBS for 5 mins and incubated with 1:50 normal rabbit serum in TBS for 40 minutes. This was followed by incubation for 1 hour with 1:40 Mab anti rabbit macrophage RAM-11 (IgGl) . They were then rinsed for 5 minutes in TBS and incubated with 1:40 Rabbit anti-mouse peroxidase conjugate F(ab)² IgG (Serotec) for 1 hour. After a further 5 minute rinse in TBS the slides were developed for 10-15 mins using Metal enhanced 3,3' Diamino Benzidine Tetrahydrochloride (Pierce and Warriner) .

The second slide of each pair was probed with an anti- a smooth muscle cell actin (Anti α-SM-1) which recognises the 42 kD α-smooth muscle isoform of actin but does not react with other isoforms present in endothelial cells. This actin isoform is the major actin component of vascular tissue in the aorta and is an ideal probe for smooth muscle cells in atherosclerotic lesions (1 reference) . Slides were incubated at 21 °C for 1 hour with alkaline phosphatase suppressor (Pierce and Warriner) , rinsed for 5 mins in Tris buffered saline (TBS) and then incubated with 1:50 normal mouse serum in TBS for 40 mins. This was followed by incubation with 1:100 alkali phosphatase conjugated anti-smooth muscle cell actin (Sigma) for 1 hour and a further 5 min TBS rinse. A 10 minute incubation with Immunopure alkaline phosphatase substrate kit (Pierce and Warriner), was used to develop the slides. The slides were washed in tap water and counterstained for 30 seconds with Haeamatoxylin.

Frozen sections of rabbit spleen, bladder and brain were simultaneously probed with the same antibodies to provide positive and negative control slides.

The atherosclerotic lesions from adjacent sections were then assessed for smooth muscle cell and macrophage content using an arbitrary scoring system from 0 - 5. These arbitrary scores were then converted to relative percentage area values. The proportion of smooth muscle cells to macrophages found in each slide was calculated by comparing matched adjacent areas and a mean score for each parameter was calculated by averaging all the scores for all the lesions from individual rabbits.

Results Ultrasound Analysis

Good visualisation of the aortic arches of the Watanabe rabbits was obtained with the 7.5MHz linear transducer.

Identification and measurement of the atherosclerotic lesions was difficult in the 2D images due to a lack of differentiation between the lesion and the blood in the lumen of the vessel. (Fig 2) The ability to post-process these 2D images to reconstruct in 3D greatly aided this identification .

A single slice through the aortic arch of one of the Watanabe rabbits from within a compound 3D data set is shown in Figure 3. The right hand panel shows a pixel by pixel profile of the grey scale intensities between point A above the superior curve of the arch , through the vessel wall (w) , lumen

(1), the posterior wall (pw) and into the pulmonary artery (pa) to point B. The blood filled lumen of this vessel profiled at a consistent grey scale intensity of 32 + 2. A structure having low level echo of greyscale intensity 42 + 4 intruded into the lumen. Other structures on the lumenal side of the vessel wall, the wall itself and surrounding tissues were more echogenic and could be identified by voxel greyscale intensities of 60 + 10.

Finally, randomly distributed areas that were even more echogenic in the region of 90 + 20 were also seen. The volumes segmented at the two grey scale levels of 42 + 4 and 60+ 10 were hypothesised to be those associated with atherosclerotic lesion because of their location around the vessel wall and intrusion into the lumen of the arch.

Ultrasound Volume Changes Greyscale volumetric analysis of the 3D ultrasound data sets pre and post treatment was made in 8 control and 8 Probucol treated animals. The failures to make analysis were due to corruption of data sets by unregistered motion during acquisition. For the same reason, full longitudinal kinetic evaluation was only possible in 7 of the 9 control and 5 of the 9 Probucol treated animals.

The results in Table 1 show that in control animals the volume of the low echo material (42) increased significantly during the 30 week treatment period whereas the echogenic material (60) did not change. In contrast, the Probucol treated animals showed little change in the low echo material but had a significantly greater volume of the echogenic material. The total volume as expressed by a summation of these two grey scale segmented volumes showed a significant increase in both groups with time. The results of the 5 weekly kinetic evaluation are summarised in Figure 4. The data shows a similar trend, the control animals low echo 42 grey scale volume increasing with time and the echogenic 60 grey scale remaining static, whereas in the Probucol treated animals the 60 grey scale volume increased with time and the 42 grey scale remained constant . The Probucol induced increase was however only statistically different from control at the final time point.

Histological Assessment

The excised aortic arches from four Probucol treated animals ( 3,5,8 & 26) and four control animals (16, 17, 19 & 28) were selected randomly for histological evaluation. Lesion volumes were determined by planimetry from serial sections stained with oil red O but no difference between the two groups could be found (control mean lesion volume 490 ± 8.5 Probucol mean lesion volume 514 ± 83.8, p= N.S.) . A statistically significant linear relationship was however found between the lesion volumes as determined by histology and the 60/10 (p< 0.01) ultrasound grey scale segmented volumes and the 42/4 + 60/10 values together (p< 0.02) (Figure 5). The correlation between total lesion volume and the 42/4 greyscale segmentation volumes failed to reach statistical significance (p=0.11).

Immunohistochemical Assessment Representative adjacent frozen sections along the length of the aortic arch were processed to identify macrophages and smooth muscle cells as described in methods. Figure 6 shows the typical appearance and cellular composition of lesions in both the control and Probucol treated groups. A summary of the analysis of cell distribution throughout the lesions are presented in Table 2. Both cell types were found to be fairly evenly distributed throughout the body of the lesions of the two groups. Only along the edge of the lesion in the upper intima were statistically larger numbers of smooth muscle cells found in those lesions from animals treated with Probucol. The Probucol treatment appeared to have stimulated the recruitment of SMCs into the lesion, particularly into the upper portion of the plaque to initiate the formation of a fibrotic cap. Following normalisation to adjust for anatomical variation and physical shrinkage of the tissue during fixation there was a statistically significant linear correlation between the smooth muscle cell content of the lesion cap and the segmentation volume of the 60 ± 10 grey scale intensity calculated from the ultrasound reconstruction (Figure 7) . Sections of aorta were also stained with Sirius Red which was then extracted to give a means of quantifying the collagen content of the section. As shown in Table 2 Probucol treatment tended to increase the sirius red concentration, and thereby the collagen content of the lesions. This increase, however, did not achieve statistical significance. There was however a significant correlation between the sirius red content and smooth muscle cell scores (Figure 6) .

Fig. 6 A) Histological section through lesion in Probucol treated WHHL Rabbit showed increased smooth muscle cell -Ill- population in lesion cap - Anti a SMC actin conjugated with alkali phosphatase anti SMC actin.

B) Control animal - as above.

C) Histological section through aortic arch of a Probucol treated WHHL rabbit showing extent of lesion stained with Oil red 0 and counterstained haematoxylin.

D) Histological section through lesion in Probucol treated animal probed with Mouse monoclonal antibody vs Rabbit macrophage (RAM-11) .

Discussion

The non-invasive measurement of volume, detection and quantification of early textural changes in established plaques, indicative of cellular changes, was achieved. The treatment of Watanabe rabbits with Probucol did not produce any dramatic change in advanced atherosclerotic plaque volume. It did, however, change the cell composition of the plaque from the macrophage laden fatty streak typically found in the Watanabe rabbit model, determined by both histology and the ultrasound image characteristics. With treatment, the ultrasound images reported increasing echogenic structure that correlated with significantly increased numbers of smooth muscle cells. Histologically it was observed that these were in the outer edges of the plaque and had started to form a smooth muscle laden cap. These observations are in agreement with previous workers who have shown both the lack of volume changes in advanced lesions (Daugherty, 1991) and the smooth muscle cell compositional changes (0' Brien, 1991 ; Braesen, 1995) .

The ability to reconstruct data in 3D from freehand ultrasound scans of rabbit aortic arches is illustrated in the figures 1,3 &5. The use of multiple angles of insonation for interrogation and image acquisition enabled data compounding which proved to be an essential step necessary to permit accurate grey scale segmentation of the 3D reconstructed data blocks. Likewise the relocation problems commonly associated with similar attempts at sequential quantification of 2D images was overcome by determination of volumetric data from the sequential 3D reconstructions. The reproducibility of the 3D volumetric assessment is best exemplified by analysis of the control grey scale (60) volumes which month on month showed minimum change and had an average % Coefficient of Variation (%CoV) of 5.2%. This compares favourably with reported intraobserver variations for linear carotid intimal -medial thickness measurements (Salonen, 1993 ; Chambless, 1996) .

There was a considerable degree of consistency in the greyscale attribution to various compartments across the blood vessel . The lumen of the vessel in each animal was used as a reference intensity and in every reconstruction produced a grey scale intensity of 32 + 2. This was, perhaps not unexpected as the pre-sets on the ultrasound scanner were kept constant throughout the experiment.

The hypoechoic region of interest, segmented at the 42 ± 4 greyscale intensity, was found toward the edges of the lumen but distinct from the wall itself. Investigation of the contributing values from the 2D image frames proved that this compartment of greyscale value was not a consequence of signal averaging between the low lumen value and higher wall value (unpublished observations) . This region was considered to relate most closely with the macrophage rich, lipid laden portions of the lesion for several reasons. Firstly, it is well recognised that lipid is poorly echogenic and would be associated with regions of the image having low level greyscale attribution. Secondly, the volume of the segmentable structure at this grey scale intensity was demonstrably smaller in normal rabbit aorta ( Fig 9a) . Finally, the position of this region, protruding into the lumen of the vessel, was identical to that seen in the histological sections. (Fig 9b) . In the control group this region continued to increase in volume and could therefore reflect continued lipid uptake as previously described ( Daugherty, 1991) . This increase was not seen in the Probucol treated group and could relate to the retardation of lipid deposition previously demonstrated for this compound. Although histological evaluation showed there was a slight reduction in macrophage number in the Probucol treated animals, this did not achieve statistical significance. As macrophage number does not necessarily correlate with lipid volume in the lesion the relevance of the finding is not clear.

The remaining region of interest had an intermediate grey scale intensity of 60 ± 10, and while also being associated with the lesion, was also to be found near to and merging with the vessel wall . The volumes of this echogenic region in the two groups produced completely the opposite results to those of the hypoechoic regions. The control animals showed no increase in this region, whereas the Probucol treated animals demonstrated a time dependent increase. These changes in ultrasound image suggested that the Probucol treatment was responsible for the development or recruitment of material which possessed improved echogenic properties within the plaque. Immunohistochemical analysis of the lesions showed an increase in smooth muscle cells in the Probucol treated animals that showed a linear correlation with the echogenic volumes. Although it was not possible to demonstrate a significant increase in collagen, by sirius red content, the concentration of sirius red per unit volume of lesion showed a significant linear correlation with smooth muscle cell numbers. This suggests that either SMCs alone or in combination with stimulated collagen deposition was responsible for the changes in lesion texture identified by the selective grey scale volume segmentation at this intermediate intensity. Although other hyperechoic regions (grey scale intensities >70) were identified and quantified these did not change in either of the groups throughout the treatment period.

The Watanabe rabbit proved useful in developing the technology to reconstruct ultrasound images in three dimensions. We have been able to demonstrate the use of freehand ultrasound scanning and reconstruction as an accurate and reproducible means of non-invasive monitoring atherosclerotic plaque progression and regression, including the cellular changes associated with those events. Rabbit models, however, have lesions of very small dimensions and have a limited potential to represent experimentally the full range of features normally associated with human atherosclerotic plaques .

The animal lesions are predominantly lipid laden, analogous to the human type 1 plaque (Geroulakos, 1993,1994). Heterogeneous and more echogenic plaques are seldom seen. Nevertheless, it is now widely accepted that it is those, lipid laden plaques which, because of their inherent instability, are the culprit lesions associated with cardiovascular events . The poor differentiation between blood and lipid making these the most difficult to identify and measure in conventional ultrasound images. The present study, therefore, demonstrates that even small culprit lesions may be both identified and quantified reproducibly, and that cellular changes can be monitored in such lesions by volumetric segmentation of selected grey scale intensities. The 3D compounding of a large number of sampled images from a pre-calibrated scanner proved extremely reliable and appeared to eliminate the problems normally associated with such segmentation in both the 2D and multi-slice 3D environment.

These animal studies suggest that the 3D reconstruction of freely acquired ultrasound images is practical and that there are considerable advantages to be gained in the quantification of disease processes by the use of such technology.

During the described study it was not always possible to reconstruct the data in 3D because of movement artefacts which could not be rectified by appropriate gating techniques. Unregistered movement during the scanning procedures remains a significant problem in the spatial alignment and compounding of images acquired over a period of minutes . Great care must therefore be taken to minimise any unregistered movement during image sequence acquisition and to optimise the gating parameters to try to eliminate these potential problems.

Respiration has been identified as a potential source of such movement and respiratory gating has subsequently been introduced into the post processing selection of images in addition to the existing ECG selection criteria.

It is also worth noting the reduction in animal use that results from using such non invasive technology in experimental studies as described here. A similar study to investigate temporal changes in plaque components, using histological evaluation techniques alone, would require groups of animals to be culled at each of the various time points. Large numbers of animals would also be required in each group to obtain statistical significance, due to the considerable individual variability in the rate of progression or regression of the disease. To provide the temporal data described here, by histological techniques, would require in excess of 200 animals. Such large scale use of a scarce animal resource such as the Watanabe rabbit, the associated housing, treatment of large numbers of animals and processing of the histology generated from this number of animals makes such studies impractical. In being able to utilise non invasive and benign sequential monitoring of the same individual, each acts as its own control and the total numbers of animals required to obtain statistical significance can be greatly reduced.

The ability to accurately monitor plaque volumes and the textural characteristics of identified culprit lesions by non- invasive interrogation would be of considerable advantage in both clinical diagnosis and clinical trials of new anti atherosclerotic agents. It is anticipated that, using such technology, clinical studies to investigate anti- atherosclerotic therapies could be designed on a much smaller scale than required previously. Studies are now underway to reconstruct human carotid artery and to investigate the relevant criteria needed to identify the various plaque types in relation to their cellular composition and histological appearance associated with atherosclerotic disease in this vessel. With this information it will then be possible to achieve the above objectives and use non- invasive monitoring to quantify disease changes.

Measurement of human carotid bifurcation lumen volume by 3D ultrasound. Summary: -

The 3D ultrasound system developed at Zeneca was used to interrogate the carotid artery bifurcation in four volunteers on two occasions. There was good correspondence between the reconstructions for each individual in both visualised geometry and measured lumen volume. The four volunteers however differed in the visualised geometry of the vessel bifurcation. One volunteer was seen to have a small area of vascular pathology which was recognised in both scans. Method: The right carotid bifurcation was interrogated in four volunteers using the 3D ultrasound system. ECG was recorded via adhesive electrodes attached to both arms and the left leg during the interrogation. The volunteer's head was restrained to eliminate unregistered motion by the use of a framework that provided contact points on the chin, forehead and cheekbones. Each carotid was continuously interrogated for approximately 3 minutes. One week later the interrogations were repeated. The data sets were extracted using ECG cycle gating in the processing and gridmapped at 0.3 mm cut-off radius to a 0.2 mm isotropic grid. The data sets were then segmented over 50 slices of 0.2 mm thickness, covering a total of 1cm length of vessel, with the slice at which the complete internal/external carotid septum was seen at the middle of the dataset . Volumes for this structure were recorded for each segmentation. Surface rendered display of the segmented structure was used to examine for correspondence of geometry between the two scans done 1 week apart on each individual . RESULTS

The surface rendering of the lumen segmentation showed differences in geometry between the four volunteers. Each scan appeared to have identifying characteristics. Subject 1 had a regular bifurcation, subjects 2 and 3 a more acute divergence into the internal carotid with a much larger external carotid artery, subject 2 having some pathology around the ostium to the internal carotid. Subject 4 had a long length of incomplete septum between the internal and external carotid arteries. The repeat scan in each volunteer showed similar geometry to that seen in the first scan. (Fig. 11)

The volumes recorded for the 1cm length lumen segmentation were consistent between scans.

Volume of Carotid Bifurication lumen (cm³) over 1cm length.

DISCUSSION:

Visual examination of the surface rendered lumen segmentation from the two scans on each volunteer provided good geometrical correspondence. There were notable differences in geometry between the volunteers .

The consistent volume measurements over a 1cm length of the carotid artery would allow sequential measurements to be used for detection of disease progression or regression.

REFERENCES ; -

Allott C.P.; Arundel P.A.; Barry CD.,-John N.W. ; Mellor P . M .; Thomson D . S . ;Waterton J.C. Freehand, quantitative three- dimensional ultrasound imaging and its applications. Europ J. Ultrasound.1996; Suppl S9. Atmeh R.F.; Stewart J.M. ; Boag D.E.; Packard C.J.; Lorimer A.R. The hypolipidaemic action of probucol: A study of its effects on high and low density lipoproteins . J. Lipid Res. 1983;24: 588.

Braesen J.H;Beisiegel U;Niendorf A; Probucol inhibits not only the progression of atherosclerotic disease, but causes a different composition of atherosclerotic lesions in WHHL- rabbits. Virchows Archiv 1995; 426: 179-188.

Carew T.E.; Schwenke D.C; Steinberg D. ntiatherogenic effect of probucol unrelated to its hypocholesterolemic effect:

Evidence that antioxidants in vivo can selectively inhibit low density lipoprotein degredation in macrophage-rich fatty streaks and slow the progression of atherosclerosis in the

Watanabe heritable hyperlipidemic rabbit. Proc .Nat1.Acad. Sci

1987;84: 7725-7729.

Chambless L.E.; Zhong M.M. ; Arnett D.; Folsom A.R. Riley W.A. ; Heiss G. Variability in B-Mode ultrasound measurements in the Atherosclerosis Risk in Communities (ARIC) study.

Ultrasound Med.& Biol 1996;22:545-554.

Daugherty A.; Zweifel B.S.; Schonfeld G. Probucol attenuates the development of aortic atherosclerosis in cholesterol-fed rabbits Br.J. Pharmacol .1989; 98 : 612-618.

Daugherty A.; Zweifel B.S.; Schonfeld G. The effects of probucol on the progression of atherosclerosis in mature

Watanabe heritable hyerlipidaemic rabbits. Br.J. Pharmacol

1991;103: 1013-1018. Franceschi D.; Bondi J, ; Rubin J.R. A new approach for three-dimensional reconstruction of arterial ultrasonography.

J.Vasc.Surg. 1992;15: 800-805.

Gardner J.E.;Lees W.R. : Gillams A. Volume imaging with ultrasound. Radiology 1991; 181(P):133. Geiser E.A.; Ariet M. ,- Conetta D.A. ; Lupkiewcz S.M.;

Christie L.G. Dynamic three-dimensional echocardiographic reconstruction of the intact human left ventricle; Technique and initial observations in patients. Am.H.J. 1982; 103: 1056-

10. Geroulakos G. ;Ramaswami G. ;Nicolaides A.; James K. ;

Labropoulos N (et al) . Characterisation of symptomatic and asympotomatic carotid plaques using high-resolution real-time ultrasonography. Br.J.Surg 1993;80: 1274-1277.

Geroulakos G. ;Domjan J. ; Nicolaides A. ; Stevens J. ; Labropoulos N. (et al) . Ultrasonic carotid artery plaque structure and the risk of cerebral infarction on computed tomography. J.Vasc.Surg 1994; 20: 263-6.

Goes E.;Janssens W. ; Mailler B. Tissue characterisation of atheromatous plaques; correlation between ultrasound image and histological findings. J. Clin. Ultrasound 1990 18: 611-7. Hell B. 3D Sonography. Int.J. Oral Maxillofac. Surg. 1995; 24: 84 -89.

Kelly I.M; Gardener J.E.; Brett A.D.; Richards R. ; Lees W.R. Three-dimensional US of the fetus- work in progress. Radiology 1994; 192: 253-259. King D.L.;King D . L. j r . ; Yi-Ci Shao M. Three-dimensional spatial registration and interactive display of position and orientation of real-time ultrasound images. Ultrasound Med 1990;9:525-532.

Kita T. Nagano Y, ; Yokode M. Ishu K. ; Kume N. (et al) Probucol prevents the progression of atherosclerosois in

Watanabe heritable hyperlipidemic rabbit, an animal model for familial hypercholesterolemia. Proc .Nat1.Acad. Sci 1987 84: 5928 - 5931.

Levine R.A. ; Weyman A.E.; Handschumaker M.D. Three- dimensional echocardiography: Techniques and applications. Am.J. Cardiol. 1992 ; 69 : 121H-130H.

Moritz W.E.; Medema D.K.; Ainsworth M. ; McCabe D.; Pearlman A.S. Three-dimensional reconstruction and volume calculation from a series of nonparallel, real-time, ultrasonic images. Circulation 1980; 62: 111-143.

Nagano Y. ; Nakamura T.; Matsuzawa Y. ; Cho M. ; Ueda Y. ; Kita T. Probucol and atherosclerosis in the Watanabe heritable hyperlipidemic rabbit. - long term antiatherogenic effect and effects on established plaques. Atheroscerosis 1992; 92: 131 - 140.

Nelson T.R.; Pretorius D.H.; Slansky M. ;Hagen-Ansert S. Three-dimensional echocardiographic evaluation of fetal heart anatomy and function. J. Ultrasound Med. 1996;15:1-9.

O'Brien K; Nagano Y.;Gown A.; Kita T.; Chait A .Probucol Treatment affects the cellular composition but not anti- oxidized low density lipoprotein immunoreactivity of plaques from Watanabe Heritable Hyperlipidemic rabbits. Arterioscler . Thromb. 1991; 11: 751-759.

Picano E . ; Landini L.; Distante A.; Salvadori M. ; Lattanzi F. Angle dependance of ultrasonic backscatter in arterial tissues; a study in vitro. Circulation 1985;72; 573-6.

Pignoli P.;Tremoli E.; Poli A.,- Oreste P.; Paoletti R. Intimal plus medial thickness of the arterial wall : a direct measurement with ultrasound imaging. Circulation 1986,-74: 1399- 1406. Poli A.; Tremoli E.; Colombo A.; Sirtori M. ; Pignoli P.

Paoletti R. Ultrasonographic measurement of the common carotid artery wall thickness in hypercholesterolemic patients. Atheroscerosis 1988;70: 253-261.

Riccabona M. ; Nelson T.R. ; Pretorius D.H.; Davidson T.E. Distance and volume measurement using three-dimensional ultrasonography. J. Ultrasound. Med 1995;14:881-886.

Riley W.A. ; Barnes R.W; Applegate W.B.; Dempsey R. ; Hartwell T. ; (et al) Reproducibility of noninvasive ultrasonic measurement of carotid atherosclerosis. The Asymptomatic Carotid Artery Plaque Study (ACAPS) . Stroke 1992; 23: 1062- 1068.

Riley W.A ; Craven T.; Romont A.; Furberg CD. Assessment of temporal bias in longitudinal measurements of carotid intimal -medial thickness in the asymptomatic carotid artery progression study (ACAPS) Ultrasound Med.& Biol 1996; 22: 405- 411.

Salonen J.T. Salonen R. Ultrasonographically assed carotid morphology and the risk of coronary heart disease. Arterioscler. Thromb 1991;11: 1245-1249. Salonen J.T. Salonen R. Ultrasound B-Mode imaging in observational studies of atherosclerotic progression. Circulation 1993; 87: {suppl II} II -56 - II- 65.

Steen E . ; Olstad B. Volume rendering of 3D medical ultrasound data using direct feature mapping. IEEE Trans. Med. Imag. 1994;13:517-525.

Strandness D.E Ultrasound in the study of Atherosclerosis . Ultrasound in Med. & Biol 1986; 12: 453-464.

Vogel M. ,- Ho S.Y.; Buhlmeyer K. ; Anderson R.H. Assessment of congenital heart defects by dynamic three-dimensional echocardiography; methods of data acquisition and clinical potential. Acta Paediatr. 1995; suppl. 410: 34-9.

Watanabe Y. Serial inbreeding of rabbits with hereditary hyperlipidemia (WHHL-rabbit) incidence and development of atherosclerosis. Atherosclerosis 1980; 36: 261-268.

Wickline S.A.; Shepard R.K.; Daugherty A. Quantitative ultrasonic characterisation of lesion composition and remodeling in atherosclerotic rabbit aorta. Arterioscler. Thromb. 1993;13: 1543-1550.

REFERENCES Allott CP, Arundel PA, Barry CD, John NW, Mellor PM, et al . Freehand, quantitative 3-D ultrasound imaging and its application. Eur. J. Ultrasound 1996; 3 (Suppl.): S9. von Birgelen C, Slager CJ, Di Mario C, de Feyter PJ, Serruys PW. Volumetric intracoronary ultrasound; A new maximum confidence approach for the quantitative assessment of progression-regression of atherosclerosis? Atherosclerosis 1995; 118 (Suppl.): S103-S113.

Detmer PR, Bashein G, Hodges T, Beach KW, Strandness DE Jr. 3D ultrasonic image feature location based on magnetic scan head tracking: In vitro calibration and validation. Ultrasound Med Biol 1994; 20: 923-936. Elliot PJ, Diedrichsen J, Goodson K, Riste-Smith R,

Sivewright GJ. An object oriented system for 3D medical image analysis. IBM Systems Journal 1996 35:1-18.

Franceschi D, Bondi J, Rubin JR. A new approach for three-dimensional reconstruction of arterial ultrasonography. J Vase Surg 1992; 15: 800-805.

Gardener JE, Lees WR, Gillams A. Volume imaging with ultrasound. Radiology 1991; 181 (P) : 133-??? . Geiser EA, Ariet M, Conetta DA, Lupkiewcz SM, Christie LG. Dynamic three-dimensional echocardiographic reconstruction of the intact human left ventricle; Technique and initial observations in patients. Am H J 1982; 103: 1056-10??

Gilja OH. In Vitro evaluation of three-dimensional ultrasonography in volume estimation of abdominal organs. Ultrasound Med Biol 1994: 20:157- ???.

Hamper UM, Trapanotto V, Sheth S, Dejong MR, Caskey CI . Three-dimensional U.S. preliminary clinical experience. Radiology 1994; 191:397-401. Hell B. 3D Sonography. Int J Oral Maxillofac Surg 1995; 24: 84 -89.

Hernandez A, Basset O, Chirossel P, Gimenez G. Spatial compounding in ultrasound imaging using an articulated scan arm. Ultrasound Med Biol 1996; 22: 229-238. Howdry DH, Posakony G, Cushman R, Holmes JH. Three dimensional and Stereoscopic observation of body structures by ultrasound. J Appl Physiol 1956; 9:304-306.

Hughes SW, D'Arcy TJ, Maxwell DJ, Chiu W, Milner A, et al . Volume estimation from multiplanar 2D ultrasound images using a remote electromagnetic position and orientation sensor. Ultrasound Med Biol 1996; 22: 561-572.

Kelly IM, Gardener JE, Brett AD, Richards R, Lees WR. Three-dimensional US of the fetus- work in progress. Radiology 1994; 192: 253-259. King DL, King DL Jr, Yi-Ci Shao M. Three-dimensional spatial registration and interactive display of position and orientation of real-time ultrasound images. J Ultrasound Med 1990; 9:525-532.

Levine RA, Weyman AE, Handschumaker MD. Three-dimensional echocardiography: Techniques and applications. Am J Cardiol 1992; 69:121H-130H. Mintz GS, Keller MB, Fay FG. Motorised IVUS transducer pull-back permits accurate quantitative axial measurements (abstr.) Circulation 1992; 86-1 :323s.

Moritz WE, Medema DK, Ainsworth M, McCabe D, Pearlman AS. Three-dimensional reconstruction and volume calculation from a series of nonparallel, real-time, ultrasonic images. Circulation 1980; 62 (Suppl. ) ;111- 539. ?????

Nelson TR, Pretorius DH, Slansky M, Hagen-Ansert S. Three-dimensional echocardiographic evaluation of fetal heart anatomy and function. J Ultrasound Med 1996; 15:1-9.

Picano E, Landini L, Distante A, Salvadori M, Lattanzi F. Angle dependence of ultrasonic backscatter in arterial tissues; a study in vitro . Circulation 1985; 72: 573-576. von Ramm OT, Durham NC, Smith SW, Carroll BA. Real time volumetric US imaging. Radiology 1994; 193 (P) : 308 -???.

Rankin et al . 1993 - Review cited on page 3. Riccabona M, Nelson TR, Pretorius DH, Davidson TE. Distance and volume measurement using three-dimensional ultrasonography. J Ultrasound Med 1995; 14:881-886.

Rosenfield K, Losordo DW, Ramaswamy K, Isner JM. Three- dimensional reconstruction of human coronary and peripheral arteries from images recorded during two-dimensional intravascular ultrasound examination. Circulation 1991; 84: 1938-1956.

Shattuck DP, von Ramm OT. Compound scanning with a phased array. Ultrason Imag 1982; 4:93-107. Sivewright GJ, Elliot PJ. Interactive region and volume growing for segmenting volumes in MR and CT images. Medical Informatics 1994; 19:71-80.

Steen E, Olstad B. Volume rendering of 3D medical ultrasound data using direct feature mapping. IEEE Trans Med Imag 1994; 13:517-525. Syn M, Prager RA. Model based approach to three- dimensional ultrasound imaging. Proc XlVth Information Processing in Medical Imaging 1995; 95-???

Zosmer N, Jurkovic D, Jauniaux E, Gruboeck K, Lees C Selection and Identification of standard cardiac views from three-dimensional volume scans of the fetal thorax. J Ultrasound Med 1996; 15: 25-32.

Claims

CLAIMS :

1. A method for reconstructing in 3D an image of an object scanned in 2D a plurality of times to produce a plurality of 2D image data slices at different angles of inclination, said plurality of 2D image data slices being recorded and stored on a recording medium whereon said 2D image data slices are recorded in succession together with at least one datum which identifies said 2D image data slices as corresponding to at least one changing physical parameter which varies in time as the 2D scanning takes place, said 3D image of the object being reconstructed from the recorded 2D image data slices in dependence upon said recorded at least one changing physical parameter, with said 3D image of the object being reconstructed by post processing the data recorded on the recording medium, employing 2D image data slices selected in dependence upon a said changing physical parameter.

2. A method as claimed in claim 1, wherein said recording medium is video tape, and said 2D image data slices are recorded thereon by a video recorder, with said at least one changing physical parameter being recorded on an audio track of said video tape.

3. A method as claimed in claim 1 or 2 , wherein there is recorded as a said changing physical parameter the positionings of said 2D image data slices relative to said object.

4. A method as claimed in claim 1, 2 or 3 , wherein there is recorded as a said changing physical parameter a time- varying effect acting on said object during a period in which said 2D image data slices are recorded.

5. A method as claimed in claim 4, wherein said object comprises biological tissue.

6. A method as claimed in claim 5, wherein said object comprises at least a part of a living body.

7. A method as claimed in claim 6, wherein there is recorded as a said changing physical parameter a signal dependent upon a physiological time-varying effect acting on said object during a period in which said 2D image data slices are recorded.

8. A method as claimed in claim 7, wherein said physiological time-varying effect comprises an ECG.

9. A method as claimed in claim 7 or 8 , wherein said physiological time-varying effect comprises a respiration cycle.

10. A method as claimed in any one of the preceding claims, wherein the scanning is freehand scanning.

11. A method as claimed in any one of the preceding claims, wherein the scanning is ultrasound scanning.

12. An apparatus for use in reconstructing in 3D an image of an object scanned in 2D a plurality of times to produce a plurality of 2D image data slices, said apparatus comprising scanning means operable to scan an object to produce said 2D image data slices at different angles of inclination, recording means coupled to said scanning means and operable to record the output thereof onto a recording medium whereon said 2D image data slices will be recorded in succession, said recording means also being operable to record onto said recording medium, together with said 2D image data slices, at least one datum which identifies said 2D image data slices as corresponding to at least one changing physical parameter which varies in time as the 2D scanning takes place, and processing means coupled to said recording means and operable to reconstruct said 3D image of the object from said recording medium, in dependence upon said recorded at least one changing physical parameter, by post processing the data recorded on the recording medium, employing 2D image data slices selected in dependence upon a said changing physical parameter.

13. An apparatus as claimed in claim 12, wherein said recording medium is video tape, and said recording means is a video recorder provided to record thereon said 2D image data slices, with said at least one changing physical parameter on an audio track of said video tape.

14. An apparatus as claimed in claim 12 or 13, and comprising means to record as a said changing physical parameter the positionings of said 2D image data slices relative to said object.

15. An apparatus as claimed in claim 12, 13 or 14, and comprising means to record as a said changing physical parameter a time-varying effect acting on said object during a period in which said 2D image data slices are recorded.

16. An apparatus as claimed in any one of claims 12 to 15, and comprising means to record as a said changing physical parameter a signal dependent upon a physiological time-varying effect acting on said object during a period in which said 2D image data slices are recorded.

17. An apparatus as claimed in claim 16, wherein said physiological time-varying effect comprises an ECG.

18. An apparatus as claimed in claim 16 or 17, wherein said physiological time-varying effect comprises a respiration cycle .

19. A method for reconstructing in 3D an image of at least part of an object scanned in 2D a plurality of times, wherein a plurality of 2D image data slices produced as a result of said scanning at different angles of inclination are processed to create a 3D grid of points containing data values, the said plurality of 2D image data slices being associated with at least one datum which identifies the various positionings of said 2D image data slices relative to said object, said 3D grid being constructed based on said at least part of an object being scanned, and image data values being inserted at said grid points as a result of processing of said 2D image data slices in dependence upon said least one datum, wherein said image data values are inserted at said grid points by sequencing through the array of grid point positions and at each said grid point position identifying and using a sub-set of the data which is contained in said plurality of 2D image data slices and which is relevant to the calculation of said image data value at said each grid point position.

20. A method for reconstructing in 3D an image of at least part of an object scanned in 2D a plurality of times, wherein a plurality of 2D image data slices produced as a result of said scanning at different angles of inclination are processed to create a 3D grid of points containing data values, the said plurality of 2D image data slices being associated with at least one datum which identifies the various positionings of said 2D image data slices relative to said object, said 3D grid being constructed based on said at least part of an object being scanned, and image data values being inserted at said grid points as a result of processing of said 2D image data slices in dependence upon said least one datum, wherein said image data values are inserted at said grid points by sequencing through said plurality of 2D image data slices and accumulating the contributions to a relevant sub-set of said grid point positions.

21. A method as claimed in claim 19, wherein said relevant sub-set is specified by introducing a limiting radius within which the relative weights of contributions are made as function of the distance between the image data point and said grid point position.

22. A method as claimed in claim 20, wherein said relevant sub-set is specified by introducing a limiting radius within which the relative weights of contributions are made as function of the distance between the image data point and the grid point position.

23. A method as claimed in claim 20 or 22, and comprising two nested sequencing cycles including an outer cycle sequencing through the individual 2D image data slices, and an inner cycle sequencing through the image data and positions associated with each of said 2D image data slices.

24. A method as claimed in any one of claims 19 to 23, wherein the location and orientation of said 3D grid of points is defined in relation to a user-selected one of said plurality of 2D image data slices.

25. A method as claimed in claim 24, wherein the origin of said 3D grid of points is at the centre of said selected one of said plurality of 2D image data slices.

26. A method as claimed in any one of claims 19 to 25, wherein the scanning is freehand scanning.

27. A method as claimed in any one of claims 19 to 26, wherein the scanning is ultrasound scanning.

28. A method as claimed in any one of claims 1 to 18, wherein said plurality of 2D image data slices produced as a result of said scannings are processed to create a 3D grid of points containing data values, the said plurality of 2D image data slices being associated with at least one datum which identifies the various positionings of said 2D image data slices relative to said object, said 3D grid being constructed based on said at least part of an object being scanned, and image data values being inserted at said grid points as a result of processing of said 2D image data slices in dependence upon said least one datum.

29. A method of calibrating a scanning and position detecting device having a position detecting transmitter defining a registration frame, a position detecting receiver cooperable with said position detecting transmitter and having its own co-ordinate frame, and a scanning transducer mechanically connected to said position detecting receiver and having a coordinate frame associated with the image it produces, wherein the transformation from said image coordinate frame to said position detecting receiver coordinate frame is determined by scanning a point or volume in space from different transducer angles and positions, and employing an iterative mathematical process on the resultant data, thereby to calculate said transformation, wherein calculation of the transformation from the image co-ordinate frame to the detecting receiver coordinate frame is determined by scanning a point and adjusting the mathematical transformation to minimise spread of point positions in the receiver coordinate frame for a plurality of 2D image data slices.

30. A method of calibrating a scanning and position detecting device having a position detecting transmitter defining a registration frame, a position detecting receiver cooperable with said position detecting transmitter and having its own co-ordinate frame, and a scanning transducer mechanically connected to said position detecting receiver and having a coordinate frame associated with the image it produces, wherein the transformation from said image coordinate frame to said position detecting receiver coordinate frame is determined by scanning a point or volume in space from different transducer angles and positions, and employing an iterative mathematical process on the resultant data, thereby to calculate said transformation, wherein calculation of the transformation from the image co-ordinate frame to the detecting receiver coordinate frame is determined by scanning a point and adjusting the transformation to maximise correspondence of point position observed within a plurality of individual 2D image data slices to a corresponding back transformation of a calculated mean point position.

31. A method of calibrating a scanning and position detecting device having a position detecting transmitter defining a registration frame, a position detecting receiver cooperable with said position detecting transmitter and having its own co-ordinate frame, and a scanning transducer mechanically connected to said position detecting receiver and having a coordinate frame associated with the image it produces, wherein the transformation from said image coordinate frame to said position detecting receiver coordinate frame is determined by scanning a point or volume in space from different transducer angles and positions, and employing an iterative mathematical process on the resultant data, thereby to calculate said transformation, wherein the calculation of the transformation between the image co-ordinate frame to the detecting receiver coordinate frame is defined by scanning a volume in space using a plurality of sweeps from different angles and employing the iterative mathematical process to optimise 3D correspondence between volumetric reconstructions of the sweeps.

32. A method for the non-invasive determination of a condition inside a mammalian body, comprising reconstructing in 3D an image of at least part of said body scanned in 2D a plurality of times to produce a plurality of 2D image data slices at different angles of inclination, said plurality of 2D image data slices being recorded and stored on a recording medium whereon said 2D image data slices are recorded in succession together with at least one datum which identifies said 2D image data slices as corresponding to at least one changing physical parameter which varies in time as the 2D scanning takes place, said 3D image of said body being reconstructed from the recorded 2D image data slices in dependence upon said at least one changing parameter, with said 3D image of the object being reconstructed by post processing the data recorded on the recording medium, employing 2D image data slices selected in dependence upon a said changing physical parameter.

33. A method as claimed in claim 32, wherein said recording medium is video tape, and said 2D image data slices are recorded thereon by a video recorder, with said at least one changing physical parameter being recorded on an audio track of said video tape.

34. A method as claimed in claim 32 or 33, wherein there is recorded as a said changing physical parameter the positionings of said 2D image data slices relative to said object .

35. A method as claimed in claim 32, 33 or 34, wherein there is recorded as a said changing physical parameter a time- varying effect acting on said body during a period in which said 2D image data slices are recorded.

36. A method as claimed in claim 35, wherein there is recorded as a said changing physical parameter a signal dependent upon a physiological time-varying effect acting on said body during a period in which said 2D image data slices are recorded.

37. A method as claimed in claim 36, wherein said physiological time-varying effect comprises an ECG.

38. A method as claimed in claim 36 or 37, wherein said physiological time-varying effect comprises a respiration cycle.

39. A method as claimed in any one of claims 32 to 38, wherein the scanning is ultrasound scanning.

40. A method as claimed in any one of claims 32 to 39, wherein the scanning is freehand scanning.

41. A method for the non-invasive determination of a condition inside a mammalian body, comprising reconstructing in 3D an image of an at least part of an object scanned in 2D a plurality of times, wherein a plurality of 2D image data slices produced as a result of said scanning at different angles of inclination are processed to create a 3D grid of points containing data values, the said plurality of 2D image data slices being associated with at least one datum which identifies the various positionings of said 2D image data slices relative to said object, said 3D grid being constructed based on said at least part of an object being scanned, and image data values being inserted at said grid points as a result of processing of said 2D image data slices in dependence upon said least one datum, wherein said image data values are inserted at said grid points by sequencing through the array of grid point positions and at each said grid point position identifying and using a sub-set of the data which is contained in said plurality of 2D image data slices and which is relevant to the calculation of said image data value at said each grid point position.

42. A method for the non-invasive determination of a condition inside a mammalian body, comprising reconstructing in 3D an image of an at least part of an object scanned in 2D a plurality of times, wherein a plurality of 2D image data slices produced as a result of said scanning at different angles of inclination are processed to create a 3D grid of points containing data values, the said plurality of 2D image data slices being associated with at least one datum which identifies the various positionings of said 2D image data slices relative to said object, said 3D grid being constructed based on said at least part of an object being scanned, and image data values being inserted at said grid points as a result of processing of said 2D image data slices in dependence upon said least one datum, wherein said image data values are inserted at said grid points by sequencing through said plurality of 2D image data slices and accumulating the contributions to a relevant sub-set of said grid point positions.

43. A method as claimed in claim 41, wherein said relevant sub-set is specified by introducing a limiting radius within which the relative weights of contributions are made as function of the distance between the image data point and the said grid point position.

44. A method as claimed in claim 42, wherein said relevant sub-set is specified by introducing a limiting radius within which the relative weights of contributions are made as function of the distance between the image data point and the grid point position.

45. A method as claimed in claim 42 or 44, and comprising two nested sequencing cycles including an outer cycle sequencing through the individual 2D image data slices, and an inner cycle sequencing through the image data and positions associated with each of said 2D image data slices.

46. A method as claimed in any one of claims 41 to 45, wherein the location and orientation of said 3D grid of points is defined in relation to a user-selected one of said plurality of 2D image data slices.

47. A method as claimed in claim 46, wherein the origin of said 3D grid of points is at the centre of said selected one of said plurality of 2D image data slices.

48. A method as claimed in any one of claims 41 to 47, wherein the scanning is freehand scanning.

49. A method as claimed in any one of claims 41 to 48, wherein the scanning is ultrasound scanning.

50. A method as claimed in any one of claims 32 to 49, wherein the determination of the condition comprises measurement of the dimensions of a structure affected by a disease.

51. A method as claimed in any one of claims 32 to 50, wherein the determination of the condition comprises measurement of the tissue characteristics of a structure affected by a disease.

52. A method as claimed in claim 51, wherein the structure is a lipid laden, unstable atherosclerotic plaque.

53. A method as claimed in any one of claims 32 to 52, wherein the determination of the condition comprises measurement of the surface characteristics of a structure affected by a disease.

54. A method as claimed in any one of claims 32 to 53, wherein the determination of the condition comprises measurement of the spatial relationships of structures within the body as an aid to surgical planning.