Device and method for correlation of an ultrasound image and an X-ray image
The invention relates to a device and a method for the mutual correlation of an ultrasound image with an X-ray image of a body volume.
From US 6 396 940 B 1 is known a method with which an ultrasound image and an X-ray image from the same body volume can be correlated (registered) with each other. Thereby, the images obtained are subjected to a series of tests of a large number of geometric transformations, and the transformed images so obtained are correlated with one another. The transformation, in which a maximum correlation occurs, is considered the correct correlation of ultrasound image and X-ray image. With this method, the high calculating costs, and/or the necessity of special optical correlators is disadvantageous. Further, the correlation only based on the image area may be difficult or even impossible, if no suitable estimation of the initial values for the sought-after correlation exist. Beyond this, it is to be considered that body structures are represented differently on an ultrasound image and an X-ray image, so that a high correlation of two images not necessarily mean that the images optimally match in their geometric position.
Against this background, it is an object of the present invention to provide a method for the mutual correlation of an ultrasound image and an X-ray image of a body volume, which allow a reliable geometric correlation in a simple manner.
This object is achieved by a method with the characteristics of claim 1 as well as a device with the characteristics of claim 3. Advantageous embodiments are contained in the dependent claims. The method according to the invention for mutual correlation of at least an ultrasound image with at least an X-ray image of a body volume, comprises the following steps: a) The spatial position of the ultrasound system relative to the body volume is determined. By definition the ultrasound system comprises all parts of the device generating
the ultrasound image, whose spatial positions have an effect on the image produced. Typically, the ultrasound system comprises an ultrasound transmitter, which transmits ultrasound signals and ultrasound receivers, which pick up the ultrasound signals after passing through the body. Normally, the ultrasound receivers, are identical with the ultrasound transmitter ("ultrasound-transducer") and are integrated into an ultrasound probe, b) The spatial position of the X-ray system relative to the body volume is determined. By definition, the X-ray system comprises all parts of the device producing the X-ray image, whose spatial positions have an effect on the X-ray image produced. Typically, the X-ray system comprises an X-ray source and an X-ray detector. c) A correlation is made of at least one ultrasound image with the at least one X- ray image on the basis of the positions of the ultrasound system and the X-ray system, known from steps a and b), relative to the body volume as well as on the basis of the respective image parameters during the acquisition. Particularly, the image parameters may be field of vision, projection intervals, scaling factors and the like. With the method described above, a very accurate geometric correlation between an ultrasound image and an X-ray image of the same body volume is possible in a simple manner. The method is based on the fact that all data relevant for the geometry of images are determined so that, subsequently, a point on one of the images can be transferred to a point of the other image - that is, assigned thereto - by in essence purely mathematical transformations. The method is then independent of the properties of the imaged body volume and its representation in the various image modalities. In particular, it is also possible with the method to correlate the recordings of different body volumes geometrically with one another, in case this is desired. The spatial position of the ultrasound system or the X-ray system can be determined directly or indirectly relative to the body volume. An indirect determination exists, for example, if the position relative to a patient table is determined quantitatively (i.e. in a coordinate system) and the patient takes up a constant (but, if need be, quantitatively unknown) position relative to the patient table during the acquisition.
The invention furthermore relates to a device for mutual correlation of an ultrasound image and an X-ray image of a body volume. The device comprises: a) An ultrasound system for production of an ultrasound image in which, according to definition, all portions are set to be part of the ultrasound system, the position of which is relevant for the developing image.
b) An X-ray system for production of an X-ray image, in which, according to definition, all portions are set to be part of the X-ray system, the position of which is relevant for the developing image. c) A position determining system for determining the position of the ultrasound system and the X-ray system relative to the body volume. d) A data processing unit, which is coupled to the ultrasound system, the X-ray system and the position determining system and is set up for the purpose of effecting a correlation between ultrasound image and X-ray image from the known relative positions of ultrasound system and X-ray system as well as from the respective image parameters. With the device described, the method explained above can be carried out so that its advantages are achievable. The device can be improved further in such a way that it facilitates also an execution of the variants of the method explained.
Ultrasound imaging may particularly deal with a two-dimensional section through the body volume to be imaged. X-ray imaging preferably deals with a two- dimensional projection of the body volume. Such two dimensional sectional or projection images respectively are the most widespread kinds of images of ultrasound systems or X-ray systems. The method and the device, respectively, are not limited to this, however, and can also directly be used for three-dimensional ultrasound or X-ray images, for X-ray sectional images and the like. In place of the ultrasound or the X-ray image, in principle, also other imaging modalities such as magnetic resonance (MR) in particular or a nuclear spin tomograph could be employed in extending the method or the device.
The correlation between the ultrasound image and the X-ray image involves finding which point in the ultrasound image corresponds to which point of the X-ray image and vice versa. According to a first embodiment of the device and/or method, this correlation is exploited to transform the ultrasound image in the way of representing the X-ray image. According to an alternative embodiment of the device and/or the method, the X-ray image is transformed in the way of representing the ultrasound image. Both variants have the advantage that, according to a calculation of the transformation, both images can be shown in the same way of representation, in which this is familiar to the attending doctor and enables him to quickly recognize the corresponding structures. In particular, the transformed image (for example, the transformed ultrasound image) can also be represented superimposed on the other image (X-ray image).
The ultrasound image and the X-ray image can be produced by the method, in principle, at different times and/or at different places, since the knowledge of the position of
the recording apparatus relative to the body volume is sufficient for the desired correlation. Preferably, the ultrasound image and the X-ray image, however, are to be acquired at the same time. In this case, only the absolute spatial position of the ultrasound system and the X- ray system must be determined, since the body volume in both images must necessarily lie on the same spot of the space. A further advantage of a simultaneous acquisition consists in the fact that the exact agreement of the acquired body structures is guaranteed. This is important particularly in medical applications, since a body volume usually constantly changes its own shape through heart beat, respiration or other body movements, so that a constant average spatial position is only meaningful to a limited extent. The spatial position of the ultrasound system, the X-ray system and/or the body volume can be determined in many ways. Preferably, the position is determined with the help of position marks, which are fixed to the ultrasound system, X-ray system and/or in the body volume. Such position marks may be localized for example, with the help of electromagnetic or optical position measuring systems. Alternatively, a localization by itself can also take place from the X-ray image (cf. EP 1 127 545 A2). A further possibility consists in determining the position of the ultrasound system and/or the X-ray system by a mechanical position measuring system. This can be developed for the ultrasound system, e.g. as an arm with several joints from the measured angular positions, of which joints the position of ultrasound head at the end of the arm can be calculated. A mechanical position measurement for the X-ray system could for example be based on angle-step emitters, which are built into its carrier system.
According to a further embodiment of the device or method, the intended correlation between the ultrasound image and the X-ray image is fine-tuned or improved with the aid of image processing methods. Such methods usually depend on the fact that they determine striking structures such as for example the position of a bone in each of the images. The correlation between the images can then be so finely tuned that the structures each time assume the same position. A suitable method of image processing is described for example in US 6 396940 Bl.
The invention will be explained by way of example in the following Fig. The single image shows schematically the components of a device for acquisition and mutual correlation of an ultrasound image and an X-ray image.
In the figure, a patient is seen whose chest region represents a body volume 1 to be photographed via an X-ray image and an ultrasound image. The X-ray image is produced with the help of an X-ray source 6 and an X-ray detector 2, which are arranged at different ends of a C-arc. The X-ray images X are forwarded to a data processing unit (PC, workstation) 5. The ultrasound image is produced by the ultrasound probe 3. The measured data of the ultrasound probe 3 are processed by an ultrasound control instrument 7 and made available as images US to the data processing unit 5.
Furthermore, a position determining system 4 is represented in the Figure, which determines, with the help of magnetic or electromagnetic signals and/or optical methods, the absolute spatial position of a position mark 8 on the X-ray detector 2, a position mark 10 on the ultrasound probe 3, and a position mark 9 on the patient. The measured positions are likewise transmitted to the data processing unit 5.
Provided that X-ray images and ultrasound images are produced in parallel during medical operations, they are usually represented close together. It is up to the attending doctor to recognize corresponding structures in different images and correlate them with each other.
The system represented in the figure permits, on the other hand, an automatic correlation of ultrasound images US with X-ray images X. The associated method then proceeds in the following steps:
1. Calibration phase before the operation
1.1. In a first calibration phase, the image parameters of the fluoroscopic X-ray system 2, 6 are determined. Such a calibration must be done once, for example, during manufacturing, or when the X-ray system is installed. With the help of the calibration data obtained, it is possible to calculate, on which pixel of the X-ray image a given point in space is projected. In reversing this method it is also possible to determine by which X-ray beam a given pixel of the X-ray image is produced. The calibration of an X-ray system is described for example, in DE 101 56445.7.
1.2. In a second calibration phase, the imaging parameters of the ultrasound system are determined. This calibration can also be carried out during manufacturing or during the installation of the system. If necessary, a lookup-table of the calibration parameters is to be generated for every setting of the ultrasound equipment. With the help of the calibration data, it is possible to calculate on which pixel of an ultrasound image a given point in space is represented, and vice versa.
2. Representation phase during the medical operation
2.1. Data gathering:
With the help of X-ray fluoroscopic system 2, 6, an X-ray image X is produced on which the anatomy 1 under study is to be seen. At the same time, an ultrasound image US of the anatomy 1 under study is produced with the ultrasound probe 3 and the control instrument 7. Finally, further additional information is acquired at the time of imaging, namely, particularly the spatial positions of the X-ray system (i.e. of C-arc) and of the ultrasound system (i.e. of ultrasound probe 3) and relevant imaging parameters (magnifying factors etc.). 2.2. Data Transfer
All images US, X as well as other data are transmitted to the data processing unit 5 where all information necessary for the correlation method is then present. 2.3. Image correlation:
For the mutual correlation of an ultrasound image US with an X-ray image X, there are (inter alia) the following two possibilities:
2.3.1. Transformation of the ultrasound image US into the X-ray image X
Based on the calibration data and the position information of the ultrasound system 3, the spatial position of an anatomy point is calculated, which corresponds to an examined pixel in the ultrasound image US. Furthermore, based on the calibration data and the position information of the X-ray system 2,6, the projection of the existing spatial position referred to in the X-ray image is calculated. That is, there is calculated which pixel of the X-ray image X corresponds to this spatial position. Thereby, the correlation of an examined pixel of the ultrasound image US with a pixel of the X-ray image X is known.
The steps mentioned can be carried out either for all pixels of the ultrasound image US or they are carried out for a limited number of pixels, whereas interpolation methods are used for the correlation of the remaining pixels.
In conclusion, the complete transformation of the ultrasound image into the X- ray image is known, so that the two images are merged now in the method of representation of the X-ray image and can be represented geometrically correctly i.e. identically on a display of the data processing unit 5. The images can then in the simplest case be shown side by side. Preferably, a superimposed representation is done, however, in the way of an "Alpha- blending", in which the X-ray and the ultrasound image can have for example different colors.
2.3.2. Transformation of the X-ray image X into the ultrasound image US
Based on the calibration data and the position information of the X-ray system 2,6, the geometric position of the projection line (i e. of the X-ray beam) can be calculated, which corresponds to a given pixel in the X-ray image. Based on the calibration data and the position information of the ultrasound system 3, the spatial position of the point of intersection of the existing projection line referred to can then be calculated with the body plane represented in the ultrasound image. From this is calculated which pixel of the ultrasound image US corresponds to this point of intersection. Thereby, the correlation of an examined pixel of the X-ray image X with a pixel of the ultrasound image US is known. The above mentioned steps can be carried out either for all pixels of the X-ray image X or for a specific number of pixels, whereas interpolation methods are used for the correlation of the remaining pixels.
In conclusion, the complete transformation of the X-ray image into the ultrasound image is known, so that the two images can now be merged in the method of representation of the ultrasound image and can be represented geometrically correctly i.e. identically (for example, side by side or superimposed) on a display of the data processing unit 5. 2.4. Repetition of the steps 2.1-2.3 is preferably done in real time
The method explained above can also be carried out in analog form, if one of the images - for example, the ultrasound image - is tliree dimensional and the other image (X-ray image) is two dimensional. The operations during the transformation of the three- dimensional image into the two-dimensional image (see above no. 2.3.1) can then be carried out solely with voxels (three-dimensional pixels) instead of pixels.
According to a modification of the method described above, single images instead of real-time images can be used or processed respectively. In particular, the X-ray images can be acquired only at rather large intervals and be used for the correlation to continuously updated ultrasound images, in order to reduce the X-ray burden for the patient in this way. Further, a plurality of images can be correlated simultaneously, for example, several ultrasound images with one X-ray image. The individual images can further be acquired from different directions or positions. This makes it possible to effect an appropriate superpositioning even under unfavorable conditions (for example, if the ultrasound plane is oriented largely parallel to the X-ray beams of an X-ray image). During the registration of images lying apart in time, for example, of an earlier X-ray image and a real time-ultrasound
image, the position is calculated directly relative to the patient or indirectly with an object that is fixed relative to the patient like for example, a patient table.
The device represented in figure 1 can be combined with a navigation system for a catheter, for biopsy needles or for other for example surgical instruments. Preferably, navigation information is in this case inserted into the representation of the recordings. The device can advantageously be used in various medical applications such as e.g. therapy (minimal invasive surgery, cardiology, radiological operations, "high intensity focused ultrasound", extracorporeal shock wave lithotripsy, radiation therapy) or diagnostics (for example, Urology).