WO2013109966A1 - Method and apparatus for quantitative hyperspectral fluorescence and reflectance imaging for surgical guidance - Google Patents

Method and apparatus for quantitative hyperspectral fluorescence and reflectance imaging for surgical guidance Download PDF

Info

Publication number
WO2013109966A1
WO2013109966A1 PCT/US2013/022266 US2013022266W WO2013109966A1 WO 2013109966 A1 WO2013109966 A1 WO 2013109966A1 US 2013022266 W US2013022266 W US 2013022266W WO 2013109966 A1 WO2013109966 A1 WO 2013109966A1
Authority
WO
WIPO (PCT)
Prior art keywords
light
target
images
fluorescence
tissue
Prior art date
Application number
PCT/US2013/022266
Other languages
French (fr)
Inventor
Pablo A. VALDES
David W. Roberts
Keith D. Paulsen
Frederic Leblond
Original Assignee
The Trustees Of Dartmouth College
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US14/373,443 priority Critical patent/US20140378843A1/en
Application filed by The Trustees Of Dartmouth College filed Critical The Trustees Of Dartmouth College
Publication of WO2013109966A1 publication Critical patent/WO2013109966A1/en
Priority to US15/044,097 priority patent/US11510600B2/en
Priority to US15/367,243 priority patent/US10568535B2/en
Priority to US15/402,077 priority patent/US11813100B2/en
Priority to US16/443,760 priority patent/US11656448B2/en
Priority to US16/785,882 priority patent/US11129562B2/en
Priority to US17/486,839 priority patent/US11690558B2/en
Priority to US17/961,480 priority patent/US11857317B2/en
Priority to US18/144,767 priority patent/US20230280577A1/en
Priority to US18/204,091 priority patent/US20230320649A1/en
Priority to US18/389,128 priority patent/US20240081759A1/en

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/06—Means for illuminating specimens
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/063—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements for monochromatic or narrow-band illumination
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00—Microscopes
    • G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements

Definitions

  • the present application relates to the fields of surgery and imaging of surgical sites.
  • the present application relates to hyperspectral fluorescence and reflectance imaging to improve ability of a surgeon to distinguish tissue types during surgical procedures, such as, for example, surgical procedures involving tumor resection in the brain or other organs.
  • lesions treatable with surgical removal or modification include tissues abnormal for any location in the body, such as malignant (or cancerous) tumors, and many slower-growing "benign" tumors. These lesions also include tissues that are abnormal for their location in a particular organ, but resemble normal tissues found in other locations in the body. Other lesions may incorporate material foreign to the body, including bacteria, viruses, or parasites, and associated zones of immune reactions. Still others involve developmental anomalies, such as arteriovenous malformations and berry aneurisms. Other lesions may incorporate scars and adhesions from prior illness or injury. While lesions are of many kinds, it is generally desirable for a surgeon to be able to visualize the lesion being treated and to be able to discriminate between normal and lesion tissues.
  • tumors and other lesions do not have a capsule or other connective tissue that separates them from nearby normal tissues; they may have irregular boundaries.
  • Invasive malignant tumors in particular often have infiltrations and filaments containing malignant cells that penetrate into adjacent normal tissue.
  • Some benign and some malignant tumors have tissue that superficially resembles tissue like that from which the tumor arose in both color and, to a certain extent, in texture.
  • Some tumor types, including gliomas have motile cells that may migrate a short distance away from the tumor into normal tissue; once these cells have found a hospitable location they may grow and form a new spinoff tumor.
  • the new tumor may or may not become attached to the parent tumor, if it becomes attached it may resemble a filament of tumor. Either way, the tumor may develop a somewhat ragged edge with filaments and spots penetrating into adjacent tissue.
  • filaments of tumor, and motile cells may stop extending for a time when they reach an organ capsule, resulting in tumor encapsulated in the organ, it is often undesirable to remove an entire organ or organ lobe—especially when an organ is critical for life and the tumor may not have invaded the entire organ. For example, removal of more brain tissue or spinal cord than necessary can cause life-altering neurological impairment. Similarly, it may be desirable to save as much as possible of a patient's only kidney. There are other organs and body structures where tumors may form but where it may be desirable to retain as much post-surgery organ structure and function as possible.
  • Invasive filaments and clones from formerly motile cell portions of tumors may not be readily visible to a surgeon - even under magnification.
  • Other lesion types may also have portions that have color and structure that resemble nearby healthy tissue, making it difficult for the surgeon to distinguish the lesions from the healthy tissue.
  • a prior method of ensuring complete tumor removal while retaining as much organ as possible involves a pathologist cooperating with the surgeon.
  • the surgeon removes the tumor and some adjacent tissue, while the pathologist immediately examines frozen sections to verify that the removed tissue includes a tumor-free margin. Should tumor portions be found to extend to boundaries of the removed tissue, extension of tumor beyond the removed tissue is assumed and more adjacent tissue is removed before closing the incision.
  • This method tends to be slow, requiring extended anesthesia times and repeated frozen sections, and may require removal of more tissue than necessary because frozen sections can only be performed on tissue after the tissue is removed from the patient. Further, not all abnormal tissue types are readily distinguished in a frozen section.
  • An alternative or supplemental method involves pathological examination of stained sections to verify complete tumor removal with removal of adequate margins of healthy tissue, however stained sections often take so much time to prepare that any further removal requires re-operation.
  • tissue including tissue both abnormal for the organ and malignant tissue, including small invasive branches of tumors in tissue adjacent to tumors.
  • lesions and tumors may lie under the surface of an organ, or under a visible and exposed surface of an operative site, where they may be obscured by overlying tissue and not readily visible, or may have poor contrast relative to surrounding stroma. It is desirable to make these lesions, including portions of malignant tumors, visible to a surgeon so that they can be more readily treated, with less normal overlying tissue damaged during treatment, than with current techniques.
  • the turbid medium of Ntziachristos and Weissleder is about one inch thick, far thinner than many human organs and tissues, because it is used in small animals such as laboratory mice.
  • the device of Ntziachristos and Weissleder applies light to the medium from a pulsed laser, and detects light from the medium, through an arrangement of optical fibers placed about the medium.
  • the device of Ntziachristos and Weissleder uses an intensified charge-coupled device (ICCD) camera to time-resolve the detected fluorescence in a time-domain system. Additional devices for optical imaging of biological tissues have been reported in Hillman, E., Optical brain imaging in vivo:
  • US Patent application 20080218727 to Djeziri, et al., entitled Method And Apparatus For Optical Image Reconstruction Using Contour Determination, 2008, describes the importance of determining tissue contours and the impact of tissue contour in diffuse optical tomography reconstruction algorithms in context of intact breast imaging.
  • Djeziri proposes raster-scanning to determine an intensity profile, and using the intensity profile as a surface contour of the breast. He specifies using an optical fluid to fill space between the breast surface and the optical fibers of his diffuse optical tomography apparatus during diffuse optical imaging.
  • tissues During surgery, as adjacent structures such as skin, muscle, and bone are moved and pressure applied to soft tissues with instruments such as retractors, these tissues will deform and shift. Since these tissues may deform readily both between imaging and surgery, and during surgery, it is common for surgeons to find lesions, including tumors and foreign objects, and other surgical targets are no longer in the exact positions they occupied in preoperative images.
  • MRI and CT imaging are often used to provide high resolution preoperative images of surgical targets.
  • the equipment required to make these images is bulky, expensive, and not easily incorporated into an operating-room environment.
  • the intense magnetic fields required for MRI may be incompatible with other operating room instruments and equipment, and radiation emitted by CT machines may require surgeon and staff wear bulky and heavy lead-lined garments or leave the room during intraoperative imaging.
  • Heme is generally found in both oxygenated and de-oxygenated forms in the body, and it is well known that absorption spectra of heme differs between the oxygenated and de-oxygenated forms; this difference in absorption may be used to identify tissues having different oxygen concentrations.
  • heme concentrations and oxygenation may assist in locating some types of malignant tumor tissue, as well as of imaging tissues such as muscle, bone marrow, liver, spleen, and blood vessels including arteriovenous malformations and aneurysms that naturally have high heme concentrations.
  • Djeziri's diffuse optical imaging system of the breast described above is intended to visualize heme concentrations, such as those that result from rapid blood-vessel proliferation.
  • Muscle, including cardiac muscle, and brain activities are known to consume oxygen.
  • a normal physiological response to this increase of oxygen consumption with activity is to dilate blood vessels to increase blood flow in affected tissue.
  • diseases including peripheral vascular disease, and cardiovascular disease, as well as cerebrovascular disease, ischemic bowel disease, the centers of some types of tumors, and other conditions, this physiological increase of flow is impaired resulting in a local decrease in oxygenation of heme.
  • a significant decrease in oxygenation may produce pain or other signs and symptoms, as in intermittent claudication or angina.
  • mapping increases in blood flow due to brain activity can be of interest in monitoring activity in the brain.
  • Fluorescence-guided neurosurgery marks tumor tissue with a fluorescent contrast agent, and uses fluorescence detection technologies to identify tumor tissue using the fluorescence signals emitted from tumor tissue.
  • Current surgical microscopes enabled for fluorescence imaging typically perform single-band, single-spectral-wavelength, detection. Although useful, significant levels of tumor tissue can be left undetected using this fluorescence imaging approach.
  • Prior fluorescence detection technologies in the operating room display features and functionalities such as: i) surgical microscopes modified for broad-beam fluorescence imaging currently allow for wide-field, for example, up to 50 cm 2 , single- band, 620 - 720 nm, non-spectrally resolved fluorescence detection that assesses fluorescence qualitatively, without accounting for non- linear effects of tissue optical properties on the emitted fluorescence; and ii) surgical point-probe spectroscopy devices for fluorescence detection currently allow single-point, for example a 1 mm 2 , spectrally- resolved fluorescence detection that may or may not measure fluorescence quantitatively.
  • an imaging system includes an illumination device for illuminating a target and a surgical microscope for receiving light from the target.
  • the surgical microscope includes at least one optical output port at which at least a portion of the received light is provided as an output from the surgical microscope.
  • a tunable filter receives the portion of the received light provided as the output from the surgical microscope, the tunable filter being tunable to pass a filtered portion of the received light, the filtered portion of the received light having a plurality of wavelengths selected by the tunable filter and provided as output from the tunable filter.
  • a high-resolution, broad-bandwidth electronic camera receives the light of a plurality of wavelengths selected by the tunable filter, the electronic camera converting the light of a plurality of wavelengths selected by the tunable filter to a plurality of electrical signals.
  • a processor processes the plurality of electrical signals to form an image of the target.
  • a surgical microscope system includes an illumination device for illuminating a target and at least one optical output port at which at least a portion of received light received from the target is provided as an output.
  • a tunable filter receives the portion of the received light provided as the output from the optical output port, the tunable filter being tunable to pass a filtered portion of the received light, the filtered portion of the received light having a plurality of wavelengths selected by the tunable filter and provided as output from the tunable filter.
  • a high-resolution, broad-bandwidth electronic camera receives the light of a plurality of wavelengths selected by the tunable filter, the electronic camera converting the light of a plurality of wavelengths selected by the tunable filter to a plurality of electrical signals.
  • a processor processes the plurality of electrical signals to form an image of the target.
  • a real-time hyperspectral fluorescence and reflectance imaging device is provided that may, in an embodiment, be integrated into a surgical microscope. In other embodiments, the device is used with another operating room instrument or used independently.
  • the device provides:
  • oxygen saturation oxy- and deoxy-hemoglobin, myoglobin, and bilirubin concentrations, and spectral distortions
  • iii) quantitative hyperspectral fluorescence imaging for intraoperative guidance of neurosurgical procedures, including but not limited to brain tumor resection procedures to more accurately and quantitatively detect tumor tissue across the whole field of view using spectrally resolved fluorescence imaging.
  • a method of imaging includes: providing a white light to a target; taking a series of images through a tunable optical filter of the target while tuning the filter to a plurality of wavelengths, the series of images forming a spectral image; providing a stimulus light to the target; taking a fluorescence emission image of the target; processing the spectral image to determine an absorption and a scattering parameter at pixels of the fluorescence emission image; and correcting the fluorescence emission image using the absorption and scattering parameters to produce a corrected fluorescence emission image.
  • a method of imaging includes providing a white light to a target; taking a series of images through a tunable optical filter of the target while tuning the filter to a plurality of wavelengths, the series of images forming a spectral image; processing the spectral image to determine an absorption and a scattering parameter at pixels of the images; and decomposing the spectral image to provide images of components of the target, the components including specific scattering and absorbing substances expected to be in the target.
  • advantages of the present system and method include the ability to provide: i) wide-field and ii) spectrally-resolved fluorescence detection for iii) quantitative fluorescence and reflectance assessments in near real time.
  • the present system combines the advantages of current wide-field fluorescence imaging approaches, which allow for a wide field of view of the surgical field, and the spectrally-resolved and quantitative capabilities of single-point spectroscopy devices, to provide the surgeon with quantitative fluorescence and reflectance images of the whole field of view.
  • FIG. 1 contains a schematic diagram of a through-microscope spectral resolved quantitative fluorescence imaging system and a surgical microscope enabled for fluorescence imaging, according to some embodiments.
  • FIG. 2 contains a schematic functional block diagram of a system, according to some embodiments.
  • FIG. 3 contains a schematic functional block diagram of workflow of the quantitative fluorescence imaging according to some embodiments.
  • FIG. 4 is a schematic functional block diagram of the intraoperative standard probe system according to some embodiments.
  • FIG. 5 illustrates the PpIX fluorescence spectrum (left) and the diffuse reflectance spectrum (right) measured in a high-grade glioma following tissue excitation with blue and white light, respectively.
  • FIG. 6A-6F illustrate raw and corrected emission spectra measured for tissue phantoms, according to the some embodiments. Three different ⁇
  • FIG. 9 illustrates (a) empirical determination of the optimal a value for ⁇ quantification and (b) relationship between the estimated an actual PpIX concentration in phantoms following correction of the raw fluorescence, according to some embodiments.
  • FIG. 10 contains ROC curves quantifying the performance of three different fluorescence measurement schemes for optical datasets acquired in vivo during 14 brain tumor resection procedures.
  • FIG. 11 contains a schematic block diagram illustrating a system for supporting surgery, according to some embodiments.
  • FIG. 12 is a schematic block diagram of a hyperspectral imaging device, according to an alternative embodiment.
  • FIG. 13 is a flow chart of the method of depth resolution of fluorescence.
  • FIG. 14 illustrates spectra of fluorescent light emitted at several depths in tissue and measured above a surface of tissue.
  • FIG. 15 is a flow diagram of a method of imaging.
  • FIG. 16 is a flowchart of correction of a fluorescence image for absorption and scattering in a target, such as tissue.
  • a hyperspectral imaging device is one that covers a broader range of the electromagnetic spectrum than visible light alone, such as visible and infrared light, and which is capable of resolving received light into more than the traditional three colors.
  • a hyperspectral imaging device is capable of acquire full field of view images of the region of interest, such as the surgical field of view, similar to broad beam fluorescence imaging devices used for wide field imaging.
  • a hyperspectral imaging device is one that is capable of selecting wavelengths of interest in the visible and infrared regions of the elelectromagnetic spectrum, and as such capable of acquiring multiple images at user-defined wavelengths of interest
  • a hypespectral imaging device is capable of pixel by pixel full spectra reconstruction using multiple images acquired at user-defined wavelengths of interest
  • the system herein described provides near real-time quantitative hyperspectral fluorescence and reflectance imaging evaluation of tissue during surgical procedures to determine levels of fluorescent contrast agent and additional optical biomarkers in tissue in a field of view. Such information allows the surgeon to make decisions as surgery progresses and in conjunction with the surgical workflow, enabling a safer, more accurate and more effective surgical procedure through improved extent of tumor removal.
  • This system is applicable to other fluorescent contrast agents that may also mark tissue of different types, including tumor and stroma types, with high sensitivity and specificity.
  • the present system uses a hyperspectral fluorescence and reflectance imaging device to determine fluorophore levels, such as PpIX levels, and fluorophore locations, throughout the field of view, giving the surgeon information on the levels of fluorescent agent which has accumulated in tissue and the locations of concentrations in tissue.
  • fluorophore levels such as PpIX levels, and fluorophore locations
  • the additional optical biomarkers resolvable by the system include, but are not limited to, hemoglobin concentration and oxygen saturation. These biomarkers are of use separately or in conjunction with fluorophore assessments to help inform surgical decisions.
  • the system has a probe-based hyperspectral contact point spectroscopy device, which interrogates 1mm 2 of tissue at a time, and which also provides a significant improvement in tumor tissue detection.
  • the combined fluorescence and reflectance probe and method measures optical biomarkers, including fluorophore concentrations, oxygen saturation, hemoglobin concentration, and scattering parameters, to improve tumor detection across a range of glioma histologies.
  • Fluorescence signals are strongly affected by variations in fluorophore depth, tissue absorption and transport scattering properties, i.e., tissue optical properties.
  • a previous in vivo contact point spectroscopy technique quantifies fluorescence in tissue by compensating for the effects of tissue optical property variation and quantifies additional optical biomarkers such as hemoglobin concentration, oxygen saturation and scattering parameters. This technique was described in, for example, Valdes, P.A., Quantitative fluorescence in intracranial tumor: implications for ALA-induced PpIX as an intraoperative biomarker. Journal of Neurosurgery, 2011. 115: p. 11-17., which is incorporated herein in its entirety by reference.
  • the device sequentially acquires white light reflectance and fluorescence emissions over a broad range of wavelengths, in a particular embodiment in the range of 400-720 nm for white light reflectance and 550-720 nm for fluorescence when using, for example, PpIX.
  • the wavelengths imaged by the hyperspectral imaging device may include additional wavelengths from 400 to 1100 nm.
  • a light absorption-scattering correction applied across the whole imaged field of view uses reflectance data to correct the detected fluorescence spectrum.
  • the device uses a spectrally-constrained normalization correction algorithm, in which the ratio of the reflectance near the region of the excitation wavelength and the reflectance near the region of the emission wavelength divides intensities of raw spectrally resolved fluorescence data to provide a corrected fluorescence data.
  • Spectral decomposition is then used to extract individual fluorophore contributions from the corrected fluorescence data, and quantify the concentration of fluorophores in tissue, and the depth of concentrations of fluorophores in tissue.
  • the reflectance data is used to retrieve additional optical parameters, including but not limited to oxygenation fraction, oxy- and deoxy-hemoglobin, myoglobin, and bilirubin concentrations, that may be of interest to a surgeon.
  • reflectance hyperspectral imaging to quantify reflectance-derived optical parameters, including but not limited to, oxygen saturation, oxy- and deoxy-hemoglobin, myoglobin, and bilirubin concentrations, and spectral distortions;
  • the system provides near real-time, more accurate, quantitative, and spectrally resolved fluorescence and reflectance assessments of the whole field of view.
  • the system provides combined use of spectrally resolved reflectance and fluorescence for more accurate tissue identification across the whole field of view.
  • FIG. 1 is a schematic diagram of a through-microscope spectral resolved quantitative fluorescence imaging system and a surgical microscope enabled for fluorescence imaging, according to some embodiments.
  • the surgical microscope includes two optical ports (1 and 2) associated with two respective oculars for two surgeons and a third optical port (3) that is coupled to a RGB CCD camera for video rate and still image acquisition.
  • this CCD camera is a color or black-and-white stereo camera usable for image acquisition or for surface profile extraction using stereo-optic techniques.
  • a fourth free optical port (4) allows for integration of the portable, quantitative fluorescence and reflectance imaging device (QFI) according to the present system.
  • QFI quantitative fluorescence and reflectance imaging device
  • the QFI includes: i) an optical component that fits onto a surgical microscope, or which can be used with another operating room instrument or used independently, ii) a high-speed, tunable, wavelength-selection device, for example, a liquid crystal tunable filter, iii) a camera, and iv) a digital image processor.
  • the tunable wavelength-selection device is a liquid crystal tunable filter such as described in Nourrit, V. et al., "High-resolution hyperspectral imaging of the retina with modified fundus camera". J Fr Opthalmol 33(10), 686-692 (2010).
  • the tunable wavelength- selection device is an acousto-optic tunable filter such as described in Fellers, T., et al., "Acousto-Optic Tunable Filters (AOTFs),” National High Magnetic Field Laboratory, 1800 East Paul Dirac Dr., The Florida State University, Tallahassee, FL 32310 (2004).
  • AOTFs Acoustic-Optic Tunable Filters
  • the hyperspectral imaging device includes a scanning line
  • spectrophotometer which includes an entrance slit and a rotating scanning mirror.
  • the mirror scans the image over the slit to a dispersive element, such as a prism or a diffraction grating.
  • a two-dimensional image sensor receives a plurality of spectra along a first dimension of the image sensor, with each spectrum being spread along the second orthogonal dimension of the two-dimensional image sensor.
  • Such scanning line spectrophotometers are described in Gao, et al., "Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy," Opt.Express 18, 14330- 14344 (2010).
  • the microscope has an illumination device adapted to provide illumination in both a blue or stimulus light mode for fluorescence imaging, and in a white light mode for white light reflectance imaging; wherein the stimulus light mode has a filter, or an emitter that provides specific wavelengths, such that light of fluorescence wavelength is excluded from the tissue.
  • the QFI system connects to the free optical port of the surgical microscope.
  • a hyperspectral fluorescence and reflectance imaging device is adapted to a surgical microscope.
  • Hyperspectral fluorescence data are acquired, processed and displayed in near real-time during a surgical procedure, enabling a surgeon to view images representing quantitative levels of fluorescent agent and additional optical markers across the whole field of view.
  • the quantitative information aids the surgeon in making surgical decisions given the levels of optical markers present, for example, PpIX, oxygen saturation, etc.
  • the QFI provides two sets of 3-dimensional image cubes ( ⁇ , ⁇ , ⁇ ) in memory of the digital image processor, each image cube having a sequence of two dimensional images recorded by the QFI, each image of the sequence recorded at different wavelengths.
  • the image cubes include a white light reflectance image cube under white light mode and a fluorescence image cube under a selected fluorescence excitation or stimulus light mode, such that for each (x,y) coordinate of a 2- dimensional image of the image cubes, either a reflectance or fluorescence spectrum can be reconstructed.
  • FIG. 3 contains a functional block diagram of workflow of the quantitative fluorescence imaging of the present device, according to some embodiments.
  • both reflectance and fluorescence image cubes are acquired for each pixel coordinate (x,y).
  • the reflectance spectra are used in a light attenuation correction algorithm to derive a correction map which accounts for the distorting effects of tissue optical properties on the measured fluorescence at each pixel coordinate (x,y).
  • the correction map is applied to the fluorescence spectra at each pixel coordinate (x,y) to derive fluorescence spectra which have been corrected for the effects of tissue optical properties.
  • Spectral un-mixing techniques are applied to decouple the individual contributions of each fluorophore of interest, for example, autofluorescence, ⁇ , to the corrected fluorescence spectra, and calculate a quantitative image of the absolute concentration of fluorescent biomarker in tissue, for example, Cp p ix.
  • the spectrally-resolved quantitative fluorescence imaging system and technique of the present system described herein were designed for seamless integration with the flow of surgery and compatibility with the state-of-the-art surgical microscopes.
  • the imaging system is portable and affordable.
  • the ability to spatially resolve down in the submillimeter range enables both pre-clinical and clinical studies and ease of translation between each platform.
  • the imaging system provides significant improvements in
  • the imaging system provides an ideal pre-clinical-to-clinical testing platform for testing of novel imaging agents and preclinical models, with subsequent ease of translation from bench-to-bedside.
  • seamless integration into the surgical workflow, portability, reasonable costs, and small footprints, allows for implementation across modern operating rooms and visualization of multiple fluorescent biomarkers.
  • the imaging system includes a set-up with a small footprint, which allows for ease of implementation in the operating room.
  • the main components in a prototype embodiment include an optical adapter, a liquid crystal tunable filter (LCTF) (Cambridge Research Instruments), and a charge-coupled device (CCD) camera (Cooke Corp.) connected to a digital image processor and computer control system.
  • the LCTF is a wavelength selection device to enable acquisition of images at each wavelength of interest and create the 3D spectrally resolved image cube.
  • Other type of electronic camera for example, EMCCD, ICCD, etc., or CMOS sensor cameras, which allow for collection of light and conversion to digital form, can also be used.
  • the prototype embodiment liquid crystal tunable filter performs fast (50 ms) single band selection (7 nm FWHM) for filtration of incoming light in the range 400 - 720 nm.
  • the CCD camera 1396 x 1024 pixels; 2 x 2 binning; 62% quantum efficiency at 580 nm.
  • a fully integrated, functional system requires only standard maintenance provided to conventional surgical microscopes.
  • the system is portable, which allows for ease of transportation between operating rooms, and installation into a commercial surgical microscope is seamless.
  • An optical adapter connects the main components (LCTF and camera) to an optical port of a surgical microscope, and between LCTF and a c-mount CCD camera.
  • the CCD camera 1396 x 1024 pixels; 2 x 2 binning; 62% quantum efficiency at 580 nm, PCO.Pixelfly, Cooke Corporation) senses the incoming light and transmits the digital counts to a computer control system.
  • a computer control system In a particular exemplary illustration, we operated in the visible region of the spectrum, given that two of the three major clinically used fluorescent imaging dyes operate in this region (i.e., fluorescein, PpIX).
  • the system can be extended to the near infrared (650 - 1100 nm), given the advantages of working in this range. With the advent of novel imaging agents in the near infrared (NIR), this set up could readily accommodate either a LCTF operating in the NIR or a dual visible-NIR region LCTF.
  • NIR near infrared
  • Each 3-dimensional image cube (x, y, ⁇ ) includes a series of 2- dimensional images (x, y) (i.e., spatial dimensions) at a plurality N of specific wavelengths ( ⁇ ) (i.e., spectral dimension) in the range of interest acquired under white light mode for reflectance or under blue light mode for fluorescence signals. That is, the image cube is a stack or series of 2-dimensional images taken at a plurality of wavelength settings of the tunable filter. Each individual pixel spatial coordinate (x, y) corresponds to a location on the field of view.
  • the white light reflectance spectra are used to correct the detected fluorescence spectra for the attenuation caused by tissue absorption and scattering.
  • a non-negative least-squares routine is used on the corrected fluorescence spectra to spectrally un-mix the individual contributions of main fluorophores: PpIX, fluorescein, and tissue
  • c rel ( ⁇ ⁇ ⁇ )- 1 ⁇ ⁇ ⁇ C ⁇ 1 o ⁇ r ⁇ rected (Eq.2)
  • B [bib 2 . . .b ] is a matrix of basis spectra for N fluorophore components (for example, PpIX, fluorescein, autofluorescence)
  • C rel is a relative concentration fluorophore vector.
  • McCormick absorber dye
  • Intralipid Intralipid
  • PpIX concentrations covering the range found in brain tumors (0.019, 0.039, 0.078, 0.156, 0.313, 0.625, 1.250, 2.500, and 5.000 ⁇ g/ml) were added, and a final set with non PpIX added for a total of 90 phantoms of varying absorption, scattering and PpIX.
  • a USAF 1951 (Edmund Optics) contrast resolution target was used to determine the spatial resolution of the imaging system.
  • the full fluorescence spectrum is measured single points at the tissue surface, together with the full diffuse reflectance spectrum.
  • a spectrally-constrained diffusion theory model is then used to determine the absorption and transport scattering spectra of the tissue ( ⁇ 3 and ⁇ ' 8 , respectively) and these values are then used to correct the measured fluorescence spectrum yielding the true fluorescence signal.
  • Data has been taken with the Standard Probe, this data has been used to confirm practicability of the hyperspectral imaging device described herein, and to test practicability of the spectrally -constrained normalization approach described herein.
  • the local concentration of PpIX (Cp p ix) is then determined by spectral unmixing to eliminate contributions from PpIX photoproducts and tissue autofluorescence.
  • Cp p ix The local concentration of PpIX (Cp p ix) is then determined by spectral unmixing to eliminate contributions from PpIX photoproducts and tissue autofluorescence.
  • a non model-based spectrally-constrained normalization approach is described herein. The potential advantages are that it simplifies the instrumentation, since only two reflectance detection channels are required. It is expected that the hyperspectral imaging device herein described will be able to provide similar information when used in vivo.
  • FIG. 4 is a schematic functional block diagram of the intraoperative standard probe system according to some embodiments.
  • measurements are made with three adjacent optical fibers making contact with the tissue at the distal end of a 1.1 mm diameter intraoperative probe.
  • Each fiber has a core diameter of 200 ⁇ and the center-to-center separation between adjacent fibers is 260 ⁇ .
  • LED broadband light-emitting diode
  • the effective tissue sampling depth is ⁇ ⁇ 1mm.
  • FIG. 5 shows typical spectra acquired during a surgical resection of a high-grade glioma. That is, FIG.
  • FIG. 5 illustrates the PpIX fluorescence spectrum (left) and the diffuse reflectance spectrum (right) measured in a high-grade glioma following tissue excitation with blue and white light, respectively.
  • the vertical bands in the diffuse reflectance spectrum (right) indicate the detection windows used in the normalization measurements.
  • ⁇ D Huo is the raw measured fluorescence spectrum
  • the wavelength range for ⁇ x was chosen to be as close as possible to the fluorescence excitation band in order to correct for tissue attenuation of the excitation light.
  • (7) are: (1) that most of the spectral distortion effect is due to tissue absorption at the excitation wavelength, and (2) that the impact of light diffusion in the emission band, where scattering dominates over absorption, can be corrected by further dividing the raw fluorescence using an empirical power function of
  • the phantoms comprised Intralipid (Fresenius Kabi, Uppsala, Sweden) which provided the tissue-like background scattering, yellow food coloring (McCormick, London, Ontario) as the absorber and PpIX.
  • Intralipid Fresenius Kabi, Uppsala, Sweden
  • McCormick, London, Ontario yellow food coloring
  • PpIX concentrations of PpIX (0.15625, 0.3125, 0.625, 1.25, 2.5, and 5 ⁇ g/ml) for each phantom were mixed, giving a total of 54 separate phantoms with different absorption coefficient ( ⁇ 3 ), reduced scattering coefficient ( ⁇ 8 ') and fluorophore concentration.
  • FIG. 6A-6F illustrate raw and corrected emission spectra measured for tissue phantoms, according to the some embodiments. Three different ⁇ concentrations and a range of absorption and scattering values were used: (Fig. 6a, Fig. 6b, Fig. 6c) varying absorption with constant scattering; (Fig.
  • Normalization to a function of ⁇ Ref x and ⁇ Ref m as in Eq. (7) can further correct for changes in both absorption (FIG. 7c) as well as scattering (FIG. 7f), and yield quantitative fluorescence spectra that are largely independent of variations in the optical properties.
  • the optimal tissue-specific empirical parameter a is determined experimentally using the tissue- simulating phantoms (or, for example, Monte Carlo modeling) in order to achieve the highest accuracy in CppK .
  • the spectral unmixing algorithm was applied to the corrected fluorescence spectrum, using the known ⁇ emission spectrum as input.
  • FIG. 7 and Fig. 8 shows, as a function of the true ⁇ concentration, both the raw fluorescence intensity and the calculated Cppix following correction only for the strong absorption effects at the excitation wavelength.
  • FIG. 9 illustrates (a) empirical determination of the optimal a value for PpIX quantification and (b) relationship between the estimated an actual ⁇ concentration in phantoms following correction of the raw fluorescence, as described herein.
  • the spectrally constrained normalization method was then evaluated in terms of its efficacy to quantify Cp p ix in biological tissue.
  • lxlO 6 cells from the human glioma VX2 line were orthotopically implanted to a depth of 2 mm in the brains of five mice. Following three weeks of incubation, 100 mg/kg body weight of ALA (Sigma- Aldrich Inc., MO) was injected via the tail vein, and the animals were subsequently sacrificed by cervical dislocation under anesthesia at 0.5, 1, 2 , 3 and 4 h later, respectively.
  • the brains were then removed intact and measurements made with the Standard Probe at three different locations for each sample. The probe measurements (fluorescence and reflectance) were used to compute the ratio in Eq.
  • the Standard Probe has been used in ALA-induced PpIX fluorescence guided resection of human brain tumors.
  • FIG. 10 contains ROC curves quantifying the performance of three different fluorescence measurement schemes for optical datasets acquired in vivo during 14 brain tumor resection procedures.
  • AUC area under the curve
  • Cp p ix in vivo may also be of value in applications such as photodynamic therapy in order to monitor and optimize the dose of photosensitizer and the drug-light time interval for treatment.
  • FIG. 11 is a schematic block diagram illustrating a system 100 for supporting surgery, according to some embodiments.
  • the system of FIG. 11 includes a microscope body 102, which has multiple beam splitters 104 that permit light to be diverted to several optical ports simultaneously. Attached to a first optical port of body 102 is a tube 106 leading to a surgeon's binocular optical eyepieces 108.
  • Attached to a second optical port of body 102 are a first high definition electronic camera 120 and a second high definition electronic camera 122. Cameras 120, 122 are coupled to provide images to image capture interface 124 of a digital image processing system 126. Attached to a third optical port of body 102 is a hyperspectral imaging device 128 that in an embodiment has a tunable filter 130 adapted to receive light from body 102 and a high resolution broad-bandwidth electronic camera 132. The camera 132 of the hyperspectral imaging device 128 is also coupled to provide images to image capture interface 124 of the digital processing system 126.
  • tunable filter 130 is a liquid crystal tunable filter. In an alternative embodiment, tunable filter 130 is an acousto-optic tunable filter.
  • FIG. 12 is a schematic block diagram of a hyperspectral imaging device, according to an alternative embodiment.
  • hyperspectral imaging device 128 of FIG. 11 is replaced with the hyperspectral imaging device of FIG. 12.
  • light received through optical interface port 202 is scanned by a rotating mirror 204 across a spectrometer slit 206.
  • spectrometer slit 206 Light passing through spectrometer slit 206 is dispersed according to wavelength by a dispersive device 208, such as a prism or a diffraction grating, and thence onto a planar image sensor 210 such that each column of planar image sensor 210 provides a spectrum of light received at a corresponding point in slit 206, and light received at slit 206 corresponds to light scanned from a row of a virtual image received through microscope body 102.
  • a dispersive device 208 such as a prism or a diffraction grating
  • a tracker interface 140 of the image processing system 126 is coupled to use sensors 142 rigidly attached to a reference location within an operating room to track locations of microscope location sensors 144 and patient location sensors 146.
  • Microscope location sensors 144 are rigidly attached to the microscope body 102
  • patient location sensors 146 are attached to a frame 148 that may be attached to a patient while the patient is undergoing a surgical procedure.
  • frame 148 is adapted to be attached to a patient's skull 150 by screws (not shown) for the duration of a neurosurgical procedure during which the patient's brain 152 is exposed, and during which patient's brain 152 may be operated on with surgical instruments 154 to remove one or more lesions 156.
  • Microscope body 102 also has zoom optics 160, adapted for operation by a zoom motor/sensor 162, and a focus adjustment (not shown) adapted for operation by a focus motor (not shown).
  • the microscope also has multiple illuminators 166, 168.
  • illuminators 166, 168 include white-light illuminators 166, and fluorescent stimulus illuminators 168, operating under control of an illumination interface 170 of the image processing system 126.
  • the microscope body also has a heads-up display (HUD) projector 172 capable of providing graphical images through a combiner 174 of body 102 such that the graphical images are presented for viewing by a surgeon through surgeon's eyepieces 108
  • HUD heads-up display
  • the surgeon's field of view through the operating microscope and its associated HUD is co-registered with that of the imaging system, allowing display of tissue classifications and hyperspectral imaging results superimposed on visible brain tissue, one-to-one comparisons, and intraoperative surgical decision making.
  • surgical tools 154 fit between objective 160 and tissue 152.
  • Image processing system 126 also has a memory 178 into which image capture interface 124 saves images received from cameras 120, 122, 132; and at least one processor 180.
  • Processor 180 is adapted for executing processing routines such as surface profile extraction routines 182, brain deformation modeling routines 184, fluorescence depth modeling routines 186, and hyperspectral image processing routines 188 stored in memory 178 and operable on images stored in memory 178.
  • Processor 180 is also adapted for preparing images for display through display interface 190 onto monitor 192, and for communicating through network interface 194 to server 196; server 196 has a database containing information derived from preoperative MRI and CAT scans 198.
  • the image processing routines in memory of image processing subsystem 126 also has a classifier for classifying tissue types according to a combination of fluorescence and backscatter information.
  • the classifier is a K-nearest-neighbor classifier.
  • Image processing system 126 is also interfaced to a handheld backscatter fluorescence probe 143 that acts as a single-channel, single-point subset of the fluorescence imaging microscope.
  • Probe 143 has white 145 and fluorescence stimulus 147 illuminators, and a spectrophometric detector 149, the illuminators 143, 145, and detector 149 coupled through an optical fiber and lens subsystem 151 to apply light to, and read light from, a single point on tissue surface.
  • the probe couples into the image processing subsystem 126 through a probe interface 153.
  • Operation of the system 100 has several modes, and each mode may require execution of several phases of processing on processor 180, executing one or more of several routines, as mentioned above.
  • processor 180 can execute the hyperspectral image processing routine to perform the hyperspectral fluorescence and reflectance imaging of the system, as described above in detail.
  • the hyperspectral fluorescence and reflectance imaging can also be performed in connection with stereo- optical extraction, using images captured by stereo cameras 120, 122, to perform tissue surface extraction and modeling, as described in U.S. Provisional Patent Application No. 61/583,092, filed in the United States Patent and Trademark Office on January 4, 2012, and incorporated herein in its entirety by reference.
  • the tracking using tracking sensor 142, patient location sensors 146 and microscope location sensors 144 described above can be used in performing the brain deformation modeling routine.
  • the hyperspectral fluorescence and reflectance imaging can also be performed in connection with fluorescence depth modeling, as described in U.S. Patent Application No. 13/145,505, filed in the United States Patent and Trademark Office on July 2, 2011, and incorporated herein in its entirety by reference, and described below in detail, where fluorescence and reflectance spectral information is derived from hyperspectral imaging device 128.
  • the depth modeling is described as using a ratio of images at two wavelengths. Depth modeling using the hyperspectral fluorescence imaging of the present system can be improved with a modified process in which several ratios are used simultaneously using a fitting routine.
  • 5-ALA may also be usable to locate tumors in other organs.
  • 5-ALA has been shown to be preferentially metabolized into protoporphyrin IX in tumors of the prostate as compared to protoporphyrin IX production in surrounding normal prostate tissue. This may offer the opportunity to remove the prostate tumors with reduced risk of damaging nerves responsible for penile sensation and erection; thereby preserving functions thought important by many patients.
  • dye administered to a patient to locate a tumor contains a fluorescent molecule or prodrug metabolized into fluorescent molecules in- vivo that is attached to a monoclonal antibody having specificity for tumor tissue.
  • dye administered to a patient comprises a fluorescent molecule or prodrug attached to an antibody, such as a monoclonal antibody, having specificity towards a tissue type of the distant organ. Because many metastatic tumors retain some cell-surface markers similar to those of the parent tissue, dye preferentially delivers the fluorescent molecule or prodrug to the tumor tissue, while delivering a substantially lower concentration of the fluorescent molecule or prodrug to the organ.
  • a surgeon visualizes the tumor through the hyperspectral imaging device described herein.
  • tumors and organs are light propagation media where absorption and scattering is particularly important at short wavelengths, making them seem opaque to visible light. Consequently the white light images give good resolution but do not give good visibility of features beneath a surface of the tissue of tumor and organ.
  • light at the blue end of the visible spectrum is often absorbed and scattered within less than a millimeter of the surface, rendering it difficult to see deeper structures.
  • fine invasions of tumor may have color and structure superficially resembling that of surrounding tissues, these fine invasions of tumor at the surface may not be readily visible to the surgeon.
  • the interaction of far-red and near-infrared light with biological tissue is such that propagation is inherently different when compared to that of visible light.
  • the monochromatic illuminator provides red or near infrared light between 600 and 1000 nanometers wavelengths, this light is capable of deeper penetration into organ and tumor than blue or green light and is within the range of wavelengths that can be detected by available electronic cameras.
  • the wavelength used is chosen to be appropriate for the fluorophore used to label the organ or tumor.
  • the hyperspectral imaging device herein described may, in an embodiment, be operated to provide basic fluorescent imaging. This is done by illuminating the tissue with a light at a stimulus wavelength, in a particular embodiment by illuminating with a blue light-emitting diode (LED) having a filter to exclude illumination at a fluorescence emission wavelength.
  • the tunable liquid-crystal filter is then set to pass only light at the fluorescence emission wavelength. This provides the surgeon with non depth-resolved information pertaining to fluorophore distribution beneath the surface; this information may be somewhat hazy due to scattering in the tissue.
  • multiple monochromatic illuminators are provided, and the tunable liquid crystal filter adjusted appropriately, permitting rapid selection of surface and deep imaging wavelengths, and wavelengths suitable for different fluorophores.
  • the surgeon views the fluorescence image information on monitor 192 before, after, or at intervals while using surgical instruments such as surgical tools 154 to remove or destroy part or all of tumor 156.
  • the surgeon When the surgeon wants to determine if any tumor tissue remains in the organ 152, the surgeon triggers the system to obtain new fluorescence tomographic image information, views that information of monitor 192, and may remove additional portions of tumor 156 and organ 152.
  • blue or ultraviolet monochromatic stimulus- wavelength emitters have peak emission between three hundred fifty and four hundred five nanometers to induce visible fluorescence from protoporphyrin IX in tumor.
  • Alternative embodiments may replace blue or ultraviolet flood emitters with, or provide an additional set of monochromatic flood emitters, for providing stimulus specifically appropriate to a fluorophore other than protoporphyrin IX present in tumor.
  • monochromatic emitters may be used with other dyes such as those based on indocyanine green.
  • monochromatic emitters are LEDs; LEDs are available in a wide variety of spectral characteristics, can provide intense light, and are small and easy to use.
  • monochromatic emitters are a tungsten halogen or high intensity discharge light source equipped with a monochromatic filter for transmitting a wavelength appropriate for inducing fluorescence in fluorophores of, or metabolically produced from, the dye in use.
  • Such an alternative light source may also include optical fibers and lenses for directing light from the monochromatic filter to the tissue and tumor.
  • a matching liquid applied to the subject to give the area being interrogated a geometry that is simpler to model.
  • a left and right stereo image pair is captured 402. This image pair is used by the processor 180 to extract a surface profile of the tissue. Image cubes are then captured with the hyperspectral imaging device and placed in memory as previously described. A computer model of diffuse scattering in the tissue is then constructed.
  • the tomography reconstruction algorithm is based on the assumption that the received light from the tissue is diffused, resulting from scattering and absorption at each voxel in a model of the tissue.
  • Each voxel is assigned a scattering parameter, absorption parameters for stimulus and emissions wavelengths, and fluorescence parameters, such as concentration and fluorophore lifetime.
  • fluorescence parameters such as concentration and fluorophore lifetime.
  • this color for each pixel of the image cube may be determined from ratios of received light intensity in each of several of the 2-D images of the image cube captured at different wavelengths near peak fluorescent emissions. For example, should spectral distortions resemble those of FIG. 14, ratios, and thus "colors", may be computed from image data acquired at multiple wavelengths in the 680 to 710 nanometer range.
  • Voxels external to the modeled surface contour of the tissue have absorption, scattering, and fluorescence parameters assumed to be constant, known, and zero in most cases to set boundary conditions on the model of the tissue during the process of solving the differential equations associated with light transport.
  • Voxels within the organ, tumor, or tissue are initially assumed to scatter light and have absorbance as per an average absorption and scattering of biological tissues of the tissue type being observed. In some applications the average values of these properties are chosen based on literature values.
  • the model is constrained to produce image data of "colors,” or ratios between light received at each of several wavelengths, of intensity matching data at selected wavelengths in the fluorescent image cube.
  • optical diffuse tomography images of absorbance may be displayed to the surgeon, showing concentrations of chromophores in the tissue. These images may be of use during surgery in locating hidden structures such as blood vessels, aneurysms, and other concentrations of heme.
  • absorption and scattering parameters are refined, or in embodiments displaying fluorescent parameters only a default absorption and scattering parameters may be used; for voxels within the tissue an illumination at each voxel is computed for each position of the incident beam using a computer-based light propagation model of the tissue and organ.
  • the fluorescence concentration parameters of each voxel are computed by using a least-squares fit of the fluorescence parameters to the captured fluorescence image data.
  • the fluorescence parameters thus computed for each voxel form a three-dimensional model of fluorophore distribution in the tissue.
  • a sequence of tomographic images is constructed and displayed 414 by considering intersections of a plane with fluorescence distribution of the three- dimensional model of fluorescence distribution.
  • these tomographic images are displayed on monitor 192 and/or the HUD to the surgeon as a sequence of increasing depth, the sequence being preceded by a white light photographic image of the organ or cavity for reference.
  • the tomographic images of increasing depth are encoded by coloring each image with a false color ranging from blue for images at the organ surface to red for images more than a centimeter deep into the surface. These images are then summed. The sum image, having fluorescence intensity encoded as intensity and fluorescence depth encoded as color, is then displayed on display device. White light images and images of reflected and scattered laser light may also be displayed on display device to provide the surgeon with a visual reference.
  • protoporphyrin-IX-tagged tumor tissue detected at the surface of an operative site with a surface fluorescence microscope enhance survival; this has also been shown to be true by Stummer for human glioma. It is expected that microscope will result in both improvements in survival beyond those obtained with surface fluorescent instruments and a reduction in post-surgical neurological impairment.
  • the microscope will be of use in treating other cranial tumors such as meningioma, pituitary tumors, acoustic neuroma, and some metastatic tumors. It is expected that the microscope will also be of use in surgical treatment of tumors in other sites, including skin, breast, liver, bowel, thyroid, eyes, pancreas, kidney, bladder, prostate and muscle, as well as some lesions of other types including vascular abnormalities.
  • the microscope is also of use with indocyanine green for visualization of aneurismal vasculature obscured by overlying vessels, aneurysms, or other tissue.
  • Light as the term used herein includes electromagnetic of the visible, near ultraviolet and near infrared portions of the spectrum that may be focused with lenses and imaged with readily available semiconductor imaging devices.
  • An alternative embodiment of the method uses the fluorescent tomographic microscope heretofore described to visualize a fluorescent chromophore such as may occur naturally in some types of tumor tissues. In this embodiment, there is no need to administering a prodrug or a drug.
  • the fluorescent tomographic microscope may also be used to visualize tumors or tissues having concentrations of such a naturally occurring fluorescent chromophore that differ from concentrations in nearby normal organ stroma. Nicotinamide Adenine Dinucleotide Hydride (NADH) is an example of a naturally-occurring fluorescent chromophore that is critical to function of mitochondria.
  • NADH Nicotinamide Adenine Dinucleotide Hydride
  • NADH occurs in all eukaryotic tissues, including human organ stroma and tumor tissues, but may be present at different concentrations in tumor and nearby normal stroma.
  • Metabolically inactive tissues such as fatty tissue of breast, may have very low concentrations of NADH, while adjacent malignant tumor tissue may be metabolically very active and therefore possess large quantities of NADH; this difference in concentration permits using the fluorescent tomographic microscope to visualize the malignant tumor tissue.
  • scattered, reflected, and fluorescent light 1336 from organ 1306 and tumor 1344 passes through a lens of lenses 1308, a collimating optic 1338, a filter system 1340, and into a high resolution CCD camera 1342.
  • Filter system 1340 has transparent modes for use with multiple wavelengths for imaging heme, as well as excitation- wavelength blocking and fluorescent-light passing modes for use when imaging fluorescent chromophores.
  • CCD camera 1342 is actively cooled to provide reduced noise and enhanced sensitivity to infrared light.
  • images are captured by CCD camera 1342 at each of the three wavelengths separately; these images are passed to the image processing system 232 for processing.
  • Image processing at the first and third wavelengths is as heretofore described with reference to FIG. 22; however absorbance and scattering parameters are refined as indicated in step 1006.
  • Image processing at the second wavelength is performed by constructing the voxel-based model of scattering and absorbance 1002 at the second wavelength, but without fluorescence parameter.
  • the absorbance parameters at each voxel are then ratioed to provide a measure of heme oxygenation at each voxel, and summed to provide a measure of heme concentration at each voxel.
  • the fluorescence, heme concentration, and heme oxygenation parameters at each voxel are then mapped into tomographic image planes and displayed as tomographic images for inspection by the surgeon. The surgeon can then use these images to distinguish tumor from adjacent and nearby normal organ stroma.
  • This embodiment is useful during surgery for visualizing below- surface heme concentrations as well as fluorescence during a wide variety of surgical procedures; and is of particular utility for distinguishing malignant breast tissue from adjacent normal breast tissue. In addition to cancer surgery, this embodiment is also useful for intraoperative identification of berry aneurisms, arteriovenous malformations, hematomas, and blood vessels.
  • An embodiment as heretofore described with reference to FIG. 11 is capable of imaging heme and heme oxygenation below organ surfaces in real time, and is capable of generating an additional tomographic image indicating changes in heme concentration and / or heme oxygenation. These images indicating changes in heme concentration and oxygenation are useful for functional neuroimaging to confirm identification of foci of seizure activity because neural activity depletes oxygen and increases blood flow. Precise identification of foci of seizure activity is of importance during surgical procedures intended to alleviate epilepsy, and such surgery is occasionally done under local anesthesia to permit identification of such foci. Since the apparatus can identify changes in heme concentration and oxygenation over a centimeter below a surface, this embodiment may also be used for functional neuroimaging through the intact skull, although resolution is less than that available with the skull open.
  • fluorescence excitation of fluorophores in tissue is performed using a broad-beam source in the NIR at the excitation wavelength, ⁇ ⁇ . It has been found that fluorescent light emitted by concentrations of fluorophores like protoporphyrin IX is more strongly absorbed by tissue in the 650-670 nanometer wavelength range than in longer wavelengths such as near 700-710 nanometers, and that with proper stimulus radiation, protoporphyrin IX will emit radiation having wavelengths from 650 to 720 nanometers. Similar effects may also occur with fluorescent light emitted by other fluorophores at similar or other wavelengths.
  • FIG. 14 illustrates spectra of fluorescent radiation emitted by fluorophores at several depths in tissue as observed above the tissue. These spectra have been normalized to the same peak level. Spectra is that of fluorophores at the surface of the tissue, and is close to the spectra of light as emitted by the fluorophores themselves. Spectra is that of light from fluorophores four millimeters below the surface as seen from above the surface, spectra is that of light from fluorophores one centimeter below the surface as seen from above the surface, and spectra is that of light from fluorophores two centimeters below the surface as observed from above the surface.
  • emitted fluorescence signals are measured at two wavelengths, ⁇ 1>2 , each of which is longer than the excitation wavelength ⁇ ⁇ , using band-pass filters and a cooled CCD camera; in other embodiments emitted fluorescence signals are measured at multiple, meaning three or more, wavelengths using appropriate band-pass filters having center pass-band wavelengths at different wavelengths in the range of 650-720 nanometers. For example, if tissue has absorption and scattering characteristics similar to the tissue assumed in FIG.
  • fluorescent light intensity II is measured through a filter having bandpass 1610 between 680 and 690 nanometers while a fluorescent light intensity 12 is measured through a filter having bandpass 1612 between 700 and 710, a ratio of II to 12 will be considerably smaller when the fluorophore is two centimeters below the surface than when the fluorophore is at the surface of the tissue.
  • Embodiments for use with other fluorophores than protoporphyrin ⁇ will require different calibration tables and may operate at wavelengths other than those specified in the previous paragraph. Further, it is known that optical absorption and scattering characteristics differ from one type of tissue to another, and may also change somewhat with time as a subject's oxygenation levels change during surgery. In order to provide for this variability, an embodiment, has a library with normal optical
  • the library may contain optical characteristics for normal brain grey matter and for normal brain white matter when the microscope is used during removal of brain tumors such as gliomas. An operator then selects the appropriate tissue type for tissue in the field of view, and optical
  • library has optical characteristics of a variety of tissue types with each tissue type measured under different levels of oxygenation.
  • the operator measures tissue oxygenation with direct or optical techniques and optical characteristics of the tissue are determined based on both tissue oxygenation and tissue type.
  • the light signals at each emission wavelength can be modeled using the expression where R ⁇ is the vector corresponding to the detection point on the tissue surface, which is mapped onto a pixel or sub-ensemble of pixels on the CCD chip.
  • Qp is the quantum yield, the emission spectrum and C F (I" ⁇ ) the concentration of fluorescent molecules at location r ⁇ , where r ⁇ is an arbitrary vector corresponding to the location of a point inside the interrogated tissue.
  • light transport is modeled as a diffusive process.
  • the fluence function yf x is the excitation light field
  • G em is the diffusion equation Green's function, which corresponds to the radiant exposure in response to a light impulse at r ⁇ .
  • G (R, r, S ⁇ D*) exp( - l *-I Si) where the diffusion constant is and the penetration depth is is tne absorption coefficient of the tissue while JJ, S ' is the reduced scattering coefficient.
  • a closed form expression is derived from Eq. (8) and Eq. (9), from which a depth value can be estimated for each point imaged on the tissue surface, using an approximation that all fluorescence emitted from the surface is from a point-like distribution of fluorophore with molar concentration Cp location at position r s ⁇ .
  • FIG. 11 An alternative embodiment superficially resembling that of FIG. 11 as heretofore described constructs a fluorescence image cube having images taken at a different center wavelength in the range of wavelengths emitted by a typical fluorescent material such as protoporphyrin IX, such as in the range 650-720 nanometers.
  • the spectra of fluorescent light passing through tissue is altered by absorption and scattering in the tissue, such that spectra of the received fluorescent light encodes depth information.
  • two bandpass elements are used to derive a spectral deformation and depth for the fluorescent light captured in each pixel; in an alternative embodiment three or more bandpass elements are used to more precisely estimate spectral deformation due to passage of emitted light through tissue and thereby estimate depth for each pixel of the image.
  • the depth information determined as described above, using modeling of incident laser light and depth information derived from spectral information in received fluorescent light, are used to provide a refined depth information, and to thereby provide a refined model of the tumor.
  • the scanning laser may be disabled and depth information derived solely from spectral deformation of fluorescent light stimulated by an appropriate flood illuminator.
  • a two dimensional table is used. This table is generated according to the optical properties of the tissue at the wavelengths of the bandpass filters as determined by measurement through imaging of scattering of a beam, or from a library of optical properties as previously discussed herein. This table is indexed by intensity at the higher wavelength bandpass filter in X, and intensity in the lower wavelength bandpass filter in Y, the table having entries of approximate depth of tumor based upon differences between intensity of fluorescent light passing the higher wavelength filter and the lower wavelength filter. The table is precomputed and stored in memory of the image processor to permit fast access.
  • Intensity from each pixel in high wavelength and low wavelength images are used with a table interpolation algorithm as known in the art of computing to derive a depth associated with each pixel, and a three-dimensional model of fluorophore distribution is computed. Information from the model of fluorophore distribution is then displayed to the surgeon; this may be in the form of tomographic images, a topographic map, or by color coding of fluorescence magnitude with determined depth.
  • the optical properties of the tissue as determined above at the wavelengths of the bandpass filter elements is used to compute the 'slope' for the log-linear relationship of the ratio of intensity at two wavelengths as described above with reference to substitution of Eq. (9) into Eq. (11).
  • This 'slope' allows us relate the depth in tissue of the fluorophores to the difference in intensity between wavelengths ⁇ 1 and ⁇ 2 , ⁇ Then, the depth can be estimated based on the measured
  • the hyperspectral imaging system with white-light illumination can be used to estimate the tissue optical properties across the full field of view to create maps of tissue optical properties, such as absorption ( ⁇ ) and reduced scattering ( ⁇ 8 ') coefficients.
  • An embodiment of this method uses a look-up table (LUT), which has of a set of basis functions of spectrally resolved diffuse reflectance including reflectance (Rd( )) at known absorption( ⁇ a( ⁇ )) and scattering ( ⁇ 8'( ⁇ )).
  • the table is indexed by at least reflectance Rd, and wavelength ⁇ , the components of reflectance spectra; and contains multiple spectra, each captured at different values of known absorption and scattering.
  • the table has values recorded therein for absorption ⁇ a, and scattering ⁇ 8' at each table spectra. Captured spectra measured at each pixel, which includes reflectance at a number of different wavelengths, are fit to determine at least the closest table entries, and absorption and scattering parameters are read from the table. In order to conserve table space while providing adequate accuracy, lookups into the table for absorption and scattering parameters are interpolated using an interpolation method selected from the group of linear, cubic, or spline interpolation functions.
  • Rd( ) diffuse reflectance spectra, Rd( ), are measured in phantoms of known and varying optical absorption, ⁇ a, and reduced scattering, ⁇ 8' , across a range of values typically found in human tissues, including both normal tissues and pathological tissues such as tumor.
  • the collected Rd k) at known ⁇ a( ⁇ ) and ⁇ 8'( ⁇ ) are used to create a 4-dimensional set of basis functions [Rd, ⁇ , ⁇ a, ⁇ '].
  • Wide-field spectrally-resolved reflectance images of tissue of unknown tissue optical properties, e.g., acquired during surgery, are then decomposed into Rd at each pixel.
  • regularized minimization is applied to fit the basis functions to observed Rd k) at each pixel such that a best fit to a weighted sum of the basis functions matches Rd.
  • the fitted basis functions yield estimates of ⁇ ( ⁇ ) and ⁇ ' ⁇ ) at every pixel.
  • the estimated absorption and reduced scattering coefficients, ⁇ ( ⁇ ) and ⁇ ' ⁇ ) are also used to determine additional quantitative biomarkers that include but are not limited to hemoglobin concentration and oxygen saturation.
  • This technique enables a hyperspectral imaging system to collect diffuse reflectance data, and apply a minimization algorithm unto a set of known basis functions, (in a particular example using the lookup table described above) to make explicit estimates of tissue optical properties across the full field of view.
  • qFI Hyperspectral quantitative fluorescence imaging
  • the hyperspectral imaging system with white-light illumination and fluorescence excitation light can be used to perform quantitative fluorescence imaging using a LUT and light transport model approach.
  • the estimates of ⁇ ( ⁇ ) and ⁇ ' ⁇ ) determined by fitting spectra to entries of the table, and looking up ⁇ ( ⁇ ) and ⁇ ' ⁇ ) in the lookup table at every pixel m are used to create a 'correction image' .
  • the 'correction image' is then used to correct the measured raw fluorescence based on a light transport model, such as the light transport model of Eq. 13 below, to produce corrected quantitative fluorescence images.
  • This light transport model of Eq. 13 is derived from a model published as Kim, A., et al.,
  • R tx is the total diffuse reflectance. This parameter depends on the internal reflectance parameter ⁇ and the reduced albedo, a which in turn are defined as:
  • Fjcm is the raw (uncorrected) fluorescence emissions
  • R m is the diffuse reflectance at the emission wavelength.
  • Spectral unmixing e.g., least squares approach
  • PpIX and tissue autofluorescence are then used to separate the contributions from the fluorophore(s) being imaged, which may include PpIX and tissue autofluorescence, to construct a full-field of view image of fluorophore(s) concentrations, such as a PpIX concentration map.
  • This method enables quantitative fluorescence imaging qFI by correcting the raw fluorescence using explicit estimates of the tissue optical properties at the fluorescence wavelength and stimulus wavelength to create a 'correction image' to apply a light transport model of quantitative fluorescence.
  • the fluorescence model presented here assumes that the optical absorption at the excitation wavelength hx is high relative to that at the emission wavelength, and as such, most fluorophore absorption occurs close to the excitation light source.
  • the model assumes that fluorescence migration paths are approximate by those of the reflectance photons at the emission wavelength.
  • FIG. 15 is a flow diagram of a method of imaging.
  • FIG. 16 is a flowchart of obtaining and correcting a fluorescence image for absorption and scattering in a target, such as tissue, in order to quantify fluorescence and reflectance spectrally-derived images of the tissue.
  • white light is provided 502 to the target, or tissue, 152 by illuminators 166; and light from the target is imaged 504 by camera through a tunable optical filter of the target while tuning the filter to a plurality of wavelengths, the series of images forming a spectral image; providing a stimulus light to the target 506 using illuminators 168; and taking a fluorescence emission image of the target 508 through filters tuned to block stimulus light.
  • the fluorescence emission images of target 508 are captured as fluorescence spectra by sweeping the tunable optical filter and capturing multiple images of the target, the fluorescence spectra are then decomposed 509 to provide individual fluorescence images related to each of several fluorophores expected to be present in the target.
  • the images are then processed to determine absorption and scattering spectral parameters; in an embodiment determining the absorption and scattering parameters is performed by interpolating in the table as previously described.
  • the fluorescence emission image or images are then corrected 510 by using the absorption and scattering parameters to produce corrected fluorescence emission images giving a quantitative distribution of each fluorophore, such as PPIX 524 and other fluorophores 526 that may be native to the tissue or administered to a patient.
  • the absorption and scattering parameter spectra at each pixel are then decomposed 512 to provide tissue component images mapping concentrations of typical tissue constituents such as oxygenated and deoxygenated hemoglobin; these are then added and ratioed 514 to provide a total hemoglobin image 520 and a percent- oxygenation 522 map or image of hemoglobin oxygenation in the tissue.
  • the images, including corrected, quantitative, fluorescence, hemoglobin, and percent oxygenation, are then displayed 516 for use in diagnosis.
  • optical system may be a surgical microscope, while in other embodiments other optical systems having the ability to illuminate with stimulus or white light, and to image in white or stimulus light and then in fluorescence emitted light may be used.
  • other optical systems having the ability to illuminate with stimulus or white light, and to image in white or stimulus light and then in fluorescence emitted light may be used.
  • the following combinations of features are anticipated:
  • An imaging system designated A having an illumination device for illuminating a target, the illumination device adapted to selectively emit a white light and a fluorescent stimulus light; a device for receiving light from the target, the device for receiving light comprising at least one optical output port at which at least a portion of the received light is provided as an output; a tunable filter for receiving the portion of the received light provided as the output from the device for receiving light, the tunable filter being tunable to pass a filtered portion of the received light, the filtered portion of the received light having a plurality of wavelengths selected by the tunable filter; a high- resolution, broad-bandwidth electronic camera for receiving the light of a plurality of wavelengths selected by the tunable filter and encoding a plurality of electronic images therefrom, the electronic images representing spectra at pixels of the images; and a processor configured to process the plurality of electronic images to form an image of the target; wherein the image includes an image of fluorescent emissions by any fluorophore in the target.
  • An imaging system designated AB including the imaging system designated A wherein the device for receiving light is a surgical microscope.
  • An imaging system designated AC including the imaging system designated A or AB, wherein at least some of the electronic images the processor is configured to process are generated from fluorescence light received by the surgical microscope and wherein at least some of the electronic images processed by the processor are generated from reflectance light received by the surgical microscope.
  • An imaging system designated AD including the imaging system designated A, AB, or AC, wherein, in forming the image of the target, the processor is configured to use the electronic images generated from reflectance light to compensate the electronic images from the fluorescence light for absorption and scattering while generating the image of fluorescent emissions from the target.
  • An imaging system designated AE including the imaging system designated AD wherein the processor is configured to determine absorption and a scattering parameter at each pixel while compensating the electronic images from the fluorescence light for absorption and scattering.
  • An imaging system designated AF including the imaging system designated AE wherein a table interpolation is used to determine an optical absorption and a scattering parameter at each pixel.
  • An imaging system designated AG including the imaging system designated A, AB, AC, AD, AE, or AF, wherein the tunable filter is a liquid crystal tunable filter.
  • An imaging system designated AH including the imaging system designated A, AB, AC, AD, AE, or AF wherein the tunable filter is an acousto-optic tunable filter.
  • a surgical microscope system designated B including: an illumination device for illuminating a target, the illumination device adapted for selectively emitting a fluorescent stimulus light and a white light; at least one optical output port at which at least a portion of received light received from the target is provided; a tunable filter for receiving the portion of the received light, the tunable filter being tunable to pass a filtered portion of the received light, the filtered portion of the received light having a plurality of wavelengths selected by the tunable filter and provided as output from the tunable filter; an electronic camera for receiving the light of a plurality of wavelengths selected by the tunable filter, the electronic camera adapted to convert the light selected by the tunable filter to a plurality of electronic images while the tunable filter is tuned; and a processor configured to process the plurality of electronic images to form images of the target, wherein the images of the target comprise images of fluorophores in the target.
  • a surgical microscope system designated BA including the system designated B, wherein the electronic images the processor is configured to process include images generated from fluorescence light received from the target and images generated from reflectance light received from the target.
  • a surgical microscope system designated BB including the system designated B or BA, wherein, in forming the images of the target, the processor uses the images generated from reflectance light to compensate the images generated from the fluorescence light for absorption and scattering in the target.
  • a surgical microscope system designated BC including the system designated B, BA, or BB, wherein the tunable filter is selected from the group consisting of a liquid crystal tunable filter and an acousto-optic tunable filter.
  • a method of imaging designated C including providing a white light to a target; taking a series of images through a tunable optical filter of the target while tuning the filter to a plurality of wavelengths, the series of images forming a spectral image; providing a stimulus light to the target; taking a fluorescence emission image of the target; processing the spectral image to determine an absorption and a scattering parameter at pixels of the images; and correcting the fluorescence emission image using the absorption and scattering parameters to produce a corrected fluorescence emission image.
  • a method designated CA including the method designated C further including decomposing the spectral image to provide images of components of the target, the components including specific scattering and absorbing substances expected to be in the target.
  • a method designated CB including the method designated CA wherein the target comprises tissue, and wherein the components of the target include oxygenated and deoxygenated hemoglobin.
  • a method of imaging designated D including providing a white light to a target; taking a series of images through a tunable optical filter of the target while tuning the filter to a plurality of wavelengths, the series of images forming a spectral image; processing the spectral image to determine an absorption and a scattering parameter at pixels of the images; and decomposing the spectral image to provide images of components of the target, the components including specific scattering and absorbing substances expected to be in the target.
  • a method designated DA including the method designated D wherein the target comprises tissue, and wherein the components of the target include oxygenated and deoxygenated hemoglobin.

Abstract

An imaging system includes an illumination device for illuminating a target. A surgical microscope receives light from the target, the surgical microscope comprising at least one optical output port at which at least a portion of the received light is provided as an output from the surgical microscope. A tunable filter receives the portion of the received light provided as the output from the surgical microscope, the tunable filter being tunable to pass a filtered portion of the received light, the filtered portion of the received light having a plurality of wavelengths selected by the tunable filter and provided as output from the tunable filter. A high-resolution, broad-bandwidth electronic camera receives the light of a plurality of wavelengths selected by the tunable filter, the electronic camera converting the light of a plurality of wavelengths selected by the tunable filter to a plurality of electrical signals. A processor processes the plurality of electrical signals to form an image of the target.

Description

METHOD AND APPARATUS FOR QUANTITATIVE HYPERSPECTRAL FLUORESCENCE AND REFLECTANCE IMAGING FOR SURGICAL
GUIDANCE RELATED APPLICATIONS
[0001] The present application claims priority to United States Provisional Patent Application 61/588,708 filed 20 January 2012, the contents of which are incorporated herein by reference.
GOVERNMENT RIGHTS
[0002] This invention was made with government support under grant number R01NS052274-01 A2 awarded by the National Institutes of Health. The government has certain rights in the invention.
FIELD
[0003] The present application relates to the fields of surgery and imaging of surgical sites. In particular, the present application relates to hyperspectral fluorescence and reflectance imaging to improve ability of a surgeon to distinguish tissue types during surgical procedures, such as, for example, surgical procedures involving tumor resection in the brain or other organs.
BACKGROUND
[0004] There are many types of lesions treatable with surgical removal or modification. These lesions include tissues abnormal for any location in the body, such as malignant (or cancerous) tumors, and many slower-growing "benign" tumors. These lesions also include tissues that are abnormal for their location in a particular organ, but resemble normal tissues found in other locations in the body. Other lesions may incorporate material foreign to the body, including bacteria, viruses, or parasites, and associated zones of immune reactions. Still others involve developmental anomalies, such as arteriovenous malformations and berry aneurisms. Other lesions may incorporate scars and adhesions from prior illness or injury. While lesions are of many kinds, it is generally desirable for a surgeon to be able to visualize the lesion being treated and to be able to discriminate between normal and lesion tissues.
[0005] Many tumors and other lesions do not have a capsule or other connective tissue that separates them from nearby normal tissues; they may have irregular boundaries. Invasive malignant tumors in particular often have infiltrations and filaments containing malignant cells that penetrate into adjacent normal tissue. Some benign and some malignant tumors have tissue that superficially resembles tissue like that from which the tumor arose in both color and, to a certain extent, in texture. Some tumor types, including gliomas, have motile cells that may migrate a short distance away from the tumor into normal tissue; once these cells have found a hospitable location they may grow and form a new spinoff tumor. The new tumor may or may not become attached to the parent tumor, if it becomes attached it may resemble a filament of tumor. Either way, the tumor may develop a somewhat ragged edge with filaments and spots penetrating into adjacent tissue.
[0006] To reduce recurrence of many tumors after surgical treatment, including many malignancies, it is considered desirable to remove all detectable portions of the tumor.
[0007] While filaments of tumor, and motile cells, may stop extending for a time when they reach an organ capsule, resulting in tumor encapsulated in the organ, it is often undesirable to remove an entire organ or organ lobe— especially when an organ is critical for life and the tumor may not have invaded the entire organ. For example, removal of more brain tissue or spinal cord than necessary can cause life-altering neurological impairment. Similarly, it may be desirable to save as much as possible of a patient's only kidney. There are other organs and body structures where tumors may form but where it may be desirable to retain as much post-surgery organ structure and function as possible.
[0008] Invasive filaments and clones from formerly motile cell portions of tumors may not be readily visible to a surgeon - even under magnification. Other lesion types may also have portions that have color and structure that resemble nearby healthy tissue, making it difficult for the surgeon to distinguish the lesions from the healthy tissue.
[0009] A prior method of ensuring complete tumor removal while retaining as much organ as possible involves a pathologist cooperating with the surgeon. The surgeon removes the tumor and some adjacent tissue, while the pathologist immediately examines frozen sections to verify that the removed tissue includes a tumor-free margin. Should tumor portions be found to extend to boundaries of the removed tissue, extension of tumor beyond the removed tissue is assumed and more adjacent tissue is removed before closing the incision. This method tends to be slow, requiring extended anesthesia times and repeated frozen sections, and may require removal of more tissue than necessary because frozen sections can only be performed on tissue after the tissue is removed from the patient. Further, not all abnormal tissue types are readily distinguished in a frozen section. An alternative or supplemental method involves pathological examination of stained sections to verify complete tumor removal with removal of adequate margins of healthy tissue, however stained sections often take so much time to prepare that any further removal requires re-operation.
[0010] It is desirable to find improved ways of locating and identifying during surgery abnormal tissue, including tissue both abnormal for the organ and malignant tissue, including small invasive branches of tumors in tissue adjacent to tumors.
[0011] Generally, surgeons treat lesions that are visible to them during surgery. At times, lesions and tumors may lie under the surface of an organ, or under a visible and exposed surface of an operative site, where they may be obscured by overlying tissue and not readily visible, or may have poor contrast relative to surrounding stroma. It is desirable to make these lesions, including portions of malignant tumors, visible to a surgeon so that they can be more readily treated, with less normal overlying tissue damaged during treatment, than with current techniques.
[0012] It is known that some fluorescent compounds will accumulate in tumors and other abnormal tissues. Further, it is known that some prodrugs, such as 5- aminolevulinic acid (5-ALA) can be metabolized into fluorescent compounds to a greater extent in some tumor tissues than in surrounding normal stroma. Marking of tumors with 5-ALA metabolites and using resultant fluorescence at the surface of an operative site to guide surgery has been reported in the literature. For example Stummer, et al.,
Fluorescence-guided surgery with 5 -aminolevulinic acid for resection of malignant glioma: a randomized controlled multicentre phase III trial, Lancet Oncology, Lancet Oncology, Lancet Oncol., 2006. 7(5): p. 392-401, published online April 13, 2006 at oncology.thelancet.com, reports that removal of malignant glioma tumor tissue marked with fluorescent metabolites of 5-ALA and fluorescing in the visible spectrum at the surface of an operative site under violet-blue excitation light during surgical treatment of glioma improved extent of tumor resection and enhanced six month progression free survival in human subjects. Similar studies have also been performed in mice. It is expected that these results may apply for other lesion types. [0013] Experiments have been previously conducted with tomographic fluorescent imaging of concentrations of fluorescent compounds, or fluorophores, in biological tissues. Vasilis Ntziachristos and Ralph Weissleder, Charge-coupled-device based scanner for tomography of fluorescent near-infrared probes in turbid media, Medical Physics, Vol. 29, No. 5, May 2002, have reported a use of diffuse optical tomography in "small animal geometries". The device of Ntziachristos and Weissleder, however, operates in a transmission mode. In transmission mode, light is transmitted into a turbid medium, and emitted light is detected from several points on the surface, including points on an opposite side of the turbid medium from the points where light is applied. The turbid medium of Ntziachristos and Weissleder is about one inch thick, far thinner than many human organs and tissues, because it is used in small animals such as laboratory mice. The device of Ntziachristos and Weissleder applies light to the medium from a pulsed laser, and detects light from the medium, through an arrangement of optical fibers placed about the medium. The device of Ntziachristos and Weissleder uses an intensified charge-coupled device (ICCD) camera to time-resolve the detected fluorescence in a time-domain system. Additional devices for optical imaging of biological tissues have been reported in Hillman, E., Optical brain imaging in vivo:
techniques and applications from animal to man. Journal of Biomedical Optics, 2007. 12(5): p. 051402.
[0014] Frederic Leblond, et al, Diffuse optical fluorescence tomography using time-resolved data acquired in transmission, in Multimodal Biomedical Imaging II, vol. 6431. Proceedings of the International Society of Optical Imaging (2007) disclosed a time-dependent method for solving the diffusion equation (DE) for light propagation in tissues, and reconstruction algorithms for use therewith.
[0015] US Patent application 20080218727, to Djeziri, et al., entitled Method And Apparatus For Optical Image Reconstruction Using Contour Determination, 2008, describes the importance of determining tissue contours and the impact of tissue contour in diffuse optical tomography reconstruction algorithms in context of intact breast imaging. Djeziri proposes raster-scanning to determine an intensity profile, and using the intensity profile as a surface contour of the breast. He specifies using an optical fluid to fill space between the breast surface and the optical fibers of his diffuse optical tomography apparatus during diffuse optical imaging. [0016] During surgery, use of an optical fluid to fill space between transmit and receive optical fibers is often difficult because this fluid would need to fill the surgical wound, could infiltrate into the patient, and may require a dam around the wound. Further, operation of a diffuse optical imager in transmission mode may prove difficult if the body part being operated upon is thicker than an inch - as are the brain, kidneys, and many other organs.
[0017] Most tissues of the human body are soft tissues; these tissues are inherently flexible and readily deformable. Further, many of these soft tissues interface with other tissues along boundaries where considerable movement may take place.
During surgery, as adjacent structures such as skin, muscle, and bone are moved and pressure applied to soft tissues with instruments such as retractors, these tissues will deform and shift. Since these tissues may deform readily both between imaging and surgery, and during surgery, it is common for surgeons to find lesions, including tumors and foreign objects, and other surgical targets are no longer in the exact positions they occupied in preoperative images.
[0018] For a surgeon to properly treat these lesions, the surgeon must locate them during surgery. Further, for surgeons to avoid unintended damage to other nearby structures, it may also be necessary to locate particular portions of those other structures precisely during the surgery.
[0019] MRI and CT imaging are often used to provide high resolution preoperative images of surgical targets. The equipment required to make these images is bulky, expensive, and not easily incorporated into an operating-room environment.
Further, the intense magnetic fields required for MRI may be incompatible with other operating room instruments and equipment, and radiation emitted by CT machines may require surgeon and staff wear bulky and heavy lead-lined garments or leave the room during intraoperative imaging.
[0020] In Hartov, et al., Error Analysis for a Free-Hand Three Dimensional Ultrasound System for Neuronavigation, Neurosurgical Focus 6 (3), 5 Aug 1999, it was suggested that sensors, such as those produced by Ascension Technology Corporation, Milton, Vermont, be used to track a handheld ultrasound transducer in three dimensions. An alternative system uses a Stealthstation ® 3-D surgical navigation system produced by Medtronic, of Minneapolis, Minnesota, for tracking instruments in three dimensions relative to a patient during surgery. [0021] There are also chromophores naturally present in biological tissues, including human tissue. A leading such chromophore is the iron-containing heme group - as found in myoglobin and hemoglobin. Heme is generally found in both oxygenated and de-oxygenated forms in the body, and it is well known that absorption spectra of heme differs between the oxygenated and de-oxygenated forms; this difference in absorption may be used to identify tissues having different oxygen concentrations.
[0022] Many malignant tumor types have high metabolic activity due to rapid cell division and growth. These tumors often outgrow the local oxygen supply; some tumors stimulate rapid proliferation of blood vessels to overcome this, and some tumors develop core areas of low oxygen tension and may develop necrotic portions. Imaging of heme concentrations and oxygenation may assist in locating some types of malignant tumor tissue, as well as of imaging tissues such as muscle, bone marrow, liver, spleen, and blood vessels including arteriovenous malformations and aneurysms that naturally have high heme concentrations. Djeziri's diffuse optical imaging system of the breast described above is intended to visualize heme concentrations, such as those that result from rapid blood-vessel proliferation.
[0023] Muscle, including cardiac muscle, and brain activities are known to consume oxygen. A normal physiological response to this increase of oxygen consumption with activity is to dilate blood vessels to increase blood flow in affected tissue. In many diseases, including peripheral vascular disease, and cardiovascular disease, as well as cerebrovascular disease, ischemic bowel disease, the centers of some types of tumors, and other conditions, this physiological increase of flow is impaired resulting in a local decrease in oxygenation of heme. A significant decrease in oxygenation may produce pain or other signs and symptoms, as in intermittent claudication or angina. Further, mapping increases in blood flow due to brain activity can be of interest in monitoring activity in the brain.
[0024] For all these reasons, it is desirable to be able to map areas of heme concentration, to map areas of oxygenated heme and de-oxygenated heme, and to be able to view dynamic changes in oxygenation with tissue activity.
[0025] Other chromophores naturally present in some tissues, including some types of tumor tissues, are naturally fluorescent. Swartling, et al. Fluorescence spectra provide information on the depth of fluorescent lesions in tissue, Optics Letters, 2005, 44(10) ppl934-1941 found that emissions from fiuorophores have spectra that depend on the depth of the fluorophores.
[0026] The extent of resection for brain tumor procedures has been shown to correlate with patient survival and quality of life. Accurate tumor tissue identification with a surgical microscope alone can be challenging because of lack of visual or mechanical features in tissue to discriminate between normal tissue and tumor.
Fluorescence-guided neurosurgery marks tumor tissue with a fluorescent contrast agent, and uses fluorescence detection technologies to identify tumor tissue using the fluorescence signals emitted from tumor tissue. Current surgical microscopes enabled for fluorescence imaging typically perform single-band, single-spectral-wavelength, detection. Although useful, significant levels of tumor tissue can be left undetected using this fluorescence imaging approach.
[0027] Prior fluorescence detection technologies in the operating room display features and functionalities such as: i) surgical microscopes modified for broad-beam fluorescence imaging currently allow for wide-field, for example, up to 50 cm2, single- band, 620 - 720 nm, non-spectrally resolved fluorescence detection that assesses fluorescence qualitatively, without accounting for non- linear effects of tissue optical properties on the emitted fluorescence; and ii) surgical point-probe spectroscopy devices for fluorescence detection currently allow single-point, for example a 1 mm2, spectrally- resolved fluorescence detection that may or may not measure fluorescence quantitatively.
SUMMARY
[0028] According to a first aspect, an imaging system is provided. The imaging system includes an illumination device for illuminating a target and a surgical microscope for receiving light from the target. The surgical microscope includes at least one optical output port at which at least a portion of the received light is provided as an output from the surgical microscope. A tunable filter receives the portion of the received light provided as the output from the surgical microscope, the tunable filter being tunable to pass a filtered portion of the received light, the filtered portion of the received light having a plurality of wavelengths selected by the tunable filter and provided as output from the tunable filter. A high-resolution, broad-bandwidth electronic camera receives the light of a plurality of wavelengths selected by the tunable filter, the electronic camera converting the light of a plurality of wavelengths selected by the tunable filter to a plurality of electrical signals. A processor processes the plurality of electrical signals to form an image of the target.
[0029] According to another aspect, a surgical microscope system is provided. The surgical microscope includes an illumination device for illuminating a target and at least one optical output port at which at least a portion of received light received from the target is provided as an output. A tunable filter receives the portion of the received light provided as the output from the optical output port, the tunable filter being tunable to pass a filtered portion of the received light, the filtered portion of the received light having a plurality of wavelengths selected by the tunable filter and provided as output from the tunable filter. A high-resolution, broad-bandwidth electronic camera receives the light of a plurality of wavelengths selected by the tunable filter, the electronic camera converting the light of a plurality of wavelengths selected by the tunable filter to a plurality of electrical signals. A processor processes the plurality of electrical signals to form an image of the target.
[0030] A real-time hyperspectral fluorescence and reflectance imaging device is provided that may, in an embodiment, be integrated into a surgical microscope. In other embodiments, the device is used with another operating room instrument or used independently. The device provides:
i) reflectance hyperspectral imaging to quantify reflectance-derived optical
parameters, including but not limited to, oxygen saturation, oxy- and deoxy-hemoglobin, myoglobin, and bilirubin concentrations, and spectral distortions;
ii) reflectance hyperspectral imaging to provide information about the tissue and to provide information necessary to correct the fluorescence images for light absorption and scattering effects; and
iii) quantitative hyperspectral fluorescence imaging for intraoperative guidance of neurosurgical procedures, including but not limited to brain tumor resection procedures to more accurately and quantitatively detect tumor tissue across the whole field of view using spectrally resolved fluorescence imaging.
[0031] In another embodiment, a method of imaging includes: providing a white light to a target; taking a series of images through a tunable optical filter of the target while tuning the filter to a plurality of wavelengths, the series of images forming a spectral image; providing a stimulus light to the target; taking a fluorescence emission image of the target; processing the spectral image to determine an absorption and a scattering parameter at pixels of the fluorescence emission image; and correcting the fluorescence emission image using the absorption and scattering parameters to produce a corrected fluorescence emission image.
[0032] In another embodiment, a method of imaging includes providing a white light to a target; taking a series of images through a tunable optical filter of the target while tuning the filter to a plurality of wavelengths, the series of images forming a spectral image; processing the spectral image to determine an absorption and a scattering parameter at pixels of the images; and decomposing the spectral image to provide images of components of the target, the components including specific scattering and absorbing substances expected to be in the target.
[0033] Therefore, advantages of the present system and method include the ability to provide: i) wide-field and ii) spectrally-resolved fluorescence detection for iii) quantitative fluorescence and reflectance assessments in near real time. The present system combines the advantages of current wide-field fluorescence imaging approaches, which allow for a wide field of view of the surgical field, and the spectrally-resolved and quantitative capabilities of single-point spectroscopy devices, to provide the surgeon with quantitative fluorescence and reflectance images of the whole field of view.
BRIEF DESCRIPTION OF THE DRAWINGS
[0034] The foregoing and other features and advantages of the invention will be apparent from the more particular description of preferred aspects of the invention, as illustrated in the accompanying drawings in which like reference characters refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention.
[0035] FIG. 1 contains a schematic diagram of a through-microscope spectral resolved quantitative fluorescence imaging system and a surgical microscope enabled for fluorescence imaging, according to some embodiments.
[0036] FIG. 2 contains a schematic functional block diagram of a system, according to some embodiments. [0037] FIG. 3 contains a schematic functional block diagram of workflow of the quantitative fluorescence imaging according to some embodiments.
[0038] FIG. 4 is a schematic functional block diagram of the intraoperative standard probe system according to some embodiments.
[0039] FIG. 5 illustrates the PpIX fluorescence spectrum (left) and the diffuse reflectance spectrum (right) measured in a high-grade glioma following tissue excitation with blue and white light, respectively.
[0040] FIG. 6A-6F illustrate raw and corrected emission spectra measured for tissue phantoms, according to the some embodiments. Three different ΡρΓΧ
concentrations and a range of absorption and scattering values were used: (Fig. 6a, Fig. 6b, Fig. 6c) varying absorption with constant scattering; (Fig. 6d, Fig. 6e,Fig. 6f) varying scattering with constant absorption
[0041] FIG. 8 illustrates quantification of PpIX concentration in phantoms based on (a) raw fluorescence intensity (λ=635 nm), and (b) estimated Cppix levels derived following correction of the raw fluorescence, according to some embodiments.
[0042] FIG. 9 illustrates (a) empirical determination of the optimal a value for ΡρΓΧ quantification and (b) relationship between the estimated an actual PpIX concentration in phantoms following correction of the raw fluorescence, according to some embodiments.
[0043] FIG. 10 contains ROC curves quantifying the performance of three different fluorescence measurement schemes for optical datasets acquired in vivo during 14 brain tumor resection procedures.
[0044] FIG. 11 contains a schematic block diagram illustrating a system for supporting surgery, according to some embodiments.
[0045] FIG. 12 is a schematic block diagram of a hyperspectral imaging device, according to an alternative embodiment.
[0046] FIG. 13 is a flow chart of the method of depth resolution of fluorescence.
[0047] FIG. 14 illustrates spectra of fluorescent light emitted at several depths in tissue and measured above a surface of tissue.
[0048] FIG. 15 is a flow diagram of a method of imaging.
[0049] FIG. 16 is a flowchart of correction of a fluorescence image for absorption and scattering in a target, such as tissue. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0050] A hyperspectral imaging device is one that covers a broader range of the electromagnetic spectrum than visible light alone, such as visible and infrared light, and which is capable of resolving received light into more than the traditional three colors.
[0051] A hyperspectral imaging device is capable of acquire full field of view images of the region of interest, such as the surgical field of view, similar to broad beam fluorescence imaging devices used for wide field imaging.
[0052] A hyperspectral imaging device is one that is capable of selecting wavelengths of interest in the visible and infrared regions of the elelectromagnetic spectrum, and as such capable of acquiring multiple images at user-defined wavelengths of interest
[0053] A hypespectral imaging device is capable of pixel by pixel full spectra reconstruction using multiple images acquired at user-defined wavelengths of interest
[0054] Qualtiative, single-bandpass, visible fluorescence imaging (vis-FI) during surgery has demonstrated improved visualization of disease, bringing with it a growing interest in fluorescence imaging for surgical guidance, but not without limitations. The use of single bandpass detection in current vis-FI systems does not take full advantage of the spectral specificity of fluorophores. For example, spectrally resolved detection according to the present system enables un-mixing of non-specific
autofluorescence in detected tissue to improve detection accuracy. Further, with the advent of fluorescence for intraoperative use and associated novel fluorophores for surgical guidance (for example, protoporphyrin IX (PpIX), folate receptor targeting agents, blood pooling agents like fluorescein and indocyanine green), spectrally-resolved approaches enable simultaneous un-mixing of multiple fluorescent agents from the same fluorescent signal, thus enabling in vivo imaging of multiple reporter dyes.
[0055] We have shown that quantitative assessments of tissue fluorescence, decoupled from the effects of tissue optical properties, enable us to perform accurate quantification of absolute levels of fluorescent biomarkers in brain tissue. This quantitative approach significantly improves tumor detection, thus outperforming the diagnostic capabilities of standard vis-FI, and allowing us to detect tumor tissue that would have otherwise been left undetected. [0056] In an embodiment of an imaging system for use in surgery, quantitative hyperspectral fluorescence and reflectance imaging are provided for use during surgical procedures. The system detects cancer tissue by i) measuring a fluorescent contrast agent in a tumor and ii) measuring reflectance-derived endogenous optical markers to improve detection of diseased tissue using a quantitative hyperspectral fluorescence and reflectance imaging device that integrates with current surgical microscopes. In an alternative embodiment, quantitative hyperspectral fluorescence and reflectance imaging is provided for use independently or with another operating room instrument.
[0057] The system herein described provides near real-time quantitative hyperspectral fluorescence and reflectance imaging evaluation of tissue during surgical procedures to determine levels of fluorescent contrast agent and additional optical biomarkers in tissue in a field of view. Such information allows the surgeon to make decisions as surgery progresses and in conjunction with the surgical workflow, enabling a safer, more accurate and more effective surgical procedure through improved extent of tumor removal. A metabolic precursor to a fluorescent contrast agent— in an embodiment 5-aminolevulinic acid-induced protoporphyrin IX (PpIX) fluorescence, causes preferential accumulation of fluorophores in malignant and benign brain tumors. This system is applicable to other fluorescent contrast agents that may also mark tissue of different types, including tumor and stroma types, with high sensitivity and specificity. The present system uses a hyperspectral fluorescence and reflectance imaging device to determine fluorophore levels, such as PpIX levels, and fluorophore locations, throughout the field of view, giving the surgeon information on the levels of fluorescent agent which has accumulated in tissue and the locations of concentrations in tissue.
[0058] The additional optical biomarkers resolvable by the system, include, but are not limited to, hemoglobin concentration and oxygen saturation. These biomarkers are of use separately or in conjunction with fluorophore assessments to help inform surgical decisions.
[0059] In addition to hyperspectral imaging through the visual field of the system, the system has a probe-based hyperspectral contact point spectroscopy device, which interrogates 1mm2 of tissue at a time, and which also provides a significant improvement in tumor tissue detection. Some tumor tissue that does not display visible fluorescence using current surgical microscopes enabled for single-band fluorescence imaging, can be detected with a hyperspectral point process. The combined fluorescence and reflectance probe and method, according to embodiments described herein, measures optical biomarkers, including fluorophore concentrations, oxygen saturation, hemoglobin concentration, and scattering parameters, to improve tumor detection across a range of glioma histologies.
[0060] Fluorescence signals are strongly affected by variations in fluorophore depth, tissue absorption and transport scattering properties, i.e., tissue optical properties. A previous in vivo contact point spectroscopy technique quantifies fluorescence in tissue by compensating for the effects of tissue optical property variation and quantifies additional optical biomarkers such as hemoglobin concentration, oxygen saturation and scattering parameters. This technique was described in, for example, Valdes, P.A., Quantitative fluorescence in intracranial tumor: implications for ALA-induced PpIX as an intraoperative biomarker. Journal of Neurosurgery, 2011. 115: p. 11-17., which is incorporated herein in its entirety by reference.
[0061] Point spectroscopy quantitative fluorescence and reflectance results are extended to a broad-beam imaging platform in the present device.
[0062] The device sequentially acquires white light reflectance and fluorescence emissions over a broad range of wavelengths, in a particular embodiment in the range of 400-720 nm for white light reflectance and 550-720 nm for fluorescence when using, for example, PpIX. In alternative embodiments the wavelengths imaged by the hyperspectral imaging device may include additional wavelengths from 400 to 1100 nm. A light absorption-scattering correction applied across the whole imaged field of view uses reflectance data to correct the detected fluorescence spectrum. The device uses a spectrally-constrained normalization correction algorithm, in which the ratio of the reflectance near the region of the excitation wavelength and the reflectance near the region of the emission wavelength divides intensities of raw spectrally resolved fluorescence data to provide a corrected fluorescence data.
[0063] Spectral decomposition is then used to extract individual fluorophore contributions from the corrected fluorescence data, and quantify the concentration of fluorophores in tissue, and the depth of concentrations of fluorophores in tissue. In addition to the corrections required for fluorescence quantification, the reflectance data is used to retrieve additional optical parameters, including but not limited to oxygenation fraction, oxy- and deoxy-hemoglobin, myoglobin, and bilirubin concentrations, that may be of interest to a surgeon. [0064] The devices herein described provide:
i) reflectance hyperspectral imaging to quantify reflectance-derived optical parameters, including but not limited to, oxygen saturation, oxy- and deoxy-hemoglobin, myoglobin, and bilirubin concentrations, and spectral distortions;
ii) reflectance hyperspectral imaging to determine correction parameters for correcting fluorescence images for light absorption and scattering effects; and
iii) quantitative hyperspectral fluorescence imaging. The system provides near real-time, more accurate, quantitative, and spectrally resolved fluorescence and reflectance assessments of the whole field of view. The system provides combined use of spectrally resolved reflectance and fluorescence for more accurate tissue identification across the whole field of view.
[0065] We also show phantom, pre-clinical and clinical data where we acquire high resolution spectra for every pixel in the field of view in near-real time, resolve multiple dyes and other tissue-type information intraoperatively, and provide non- subjective, absolute estimation of fluorophore concentrations in tissue, improving detection limits by one to two orders of magnitude. Overall, our technique can have broad implications for fluorescence image guidance and improving extent of resection, by enabling clinically feasible in vivo fluorophore quantification.
[0066] We have shown that quantitative assessments of tissue fluorescence, decoupled from the effects of tissue optical properties, enable us to perform accurate quantification of absolute levels of fluorescent biomarkers in brain tissue. This quantitative approach significantly improves tumor detection, thus outperforming the diagnostic capabilities of standard vis-FI, and allowing us to detect tumor tissue that would have otherwise been left undetected.
[0067] FIG. 1 is a schematic diagram of a through-microscope spectral resolved quantitative fluorescence imaging system and a surgical microscope enabled for fluorescence imaging, according to some embodiments. The surgical microscope includes two optical ports (1 and 2) associated with two respective oculars for two surgeons and a third optical port (3) that is coupled to a RGB CCD camera for video rate and still image acquisition. In some embodiments, this CCD camera is a color or black-and-white stereo camera usable for image acquisition or for surface profile extraction using stereo-optic techniques. A fourth free optical port (4) allows for integration of the portable, quantitative fluorescence and reflectance imaging device (QFI) according to the present system.
[0068] In exemplary embodiments, the QFI includes: i) an optical component that fits onto a surgical microscope, or which can be used with another operating room instrument or used independently, ii) a high-speed, tunable, wavelength-selection device, for example, a liquid crystal tunable filter, iii) a camera, and iv) a digital image processor. In an embodiment, the tunable wavelength-selection device is a liquid crystal tunable filter such as described in Nourrit, V. et al., "High-resolution hyperspectral imaging of the retina with modified fundus camera". J Fr Opthalmol 33(10), 686-692 (2010). In some alternative embodiments, the tunable wavelength- selection device is an acousto-optic tunable filter such as described in Fellers, T., et al., "Acousto-Optic Tunable Filters (AOTFs)," National High Magnetic Field Laboratory, 1800 East Paul Dirac Dr., The Florida State University, Tallahassee, FL 32310 (2004). In some alternative
embodiments, the hyperspectral imaging device includes a scanning line
spectrophotometer which includes an entrance slit and a rotating scanning mirror. The mirror scans the image over the slit to a dispersive element, such as a prism or a diffraction grating. A two-dimensional image sensor receives a plurality of spectra along a first dimension of the image sensor, with each spectrum being spread along the second orthogonal dimension of the two-dimensional image sensor. Such scanning line spectrophotometers are described in Gao, et al., "Snapshot Image Mapping Spectrometer (IMS) with high sampling density for hyperspectral microscopy," Opt.Express 18, 14330- 14344 (2010).
[0069] The microscope has an illumination device adapted to provide illumination in both a blue or stimulus light mode for fluorescence imaging, and in a white light mode for white light reflectance imaging; wherein the stimulus light mode has a filter, or an emitter that provides specific wavelengths, such that light of fluorescence wavelength is excluded from the tissue. The QFI system connects to the free optical port of the surgical microscope.
[0070] In the method illustrated in FIG. 2, a hyperspectral fluorescence and reflectance imaging device is adapted to a surgical microscope. Hyperspectral fluorescence data are acquired, processed and displayed in near real-time during a surgical procedure, enabling a surgeon to view images representing quantitative levels of fluorescent agent and additional optical markers across the whole field of view. The quantitative information aids the surgeon in making surgical decisions given the levels of optical markers present, for example, PpIX, oxygen saturation, etc.
[0071] Referring to FIG. 2, the QFI provides two sets of 3-dimensional image cubes (χ,γ,λ) in memory of the digital image processor, each image cube having a sequence of two dimensional images recorded by the QFI, each image of the sequence recorded at different wavelengths. The image cubes include a white light reflectance image cube under white light mode and a fluorescence image cube under a selected fluorescence excitation or stimulus light mode, such that for each (x,y) coordinate of a 2- dimensional image of the image cubes, either a reflectance or fluorescence spectrum can be reconstructed.
[0072] FIG. 3 contains a functional block diagram of workflow of the quantitative fluorescence imaging of the present device, according to some embodiments. Referring to FIG. 3, both reflectance and fluorescence image cubes are acquired for each pixel coordinate (x,y). The reflectance spectra are used in a light attenuation correction algorithm to derive a correction map which accounts for the distorting effects of tissue optical properties on the measured fluorescence at each pixel coordinate (x,y). The correction map is applied to the fluorescence spectra at each pixel coordinate (x,y) to derive fluorescence spectra which have been corrected for the effects of tissue optical properties. Spectral un-mixing techniques are applied to decouple the individual contributions of each fluorophore of interest, for example, autofluorescence, ΡρΓΧ, to the corrected fluorescence spectra, and calculate a quantitative image of the absolute concentration of fluorescent biomarker in tissue, for example, Cppix.
[0073] The spectrally-resolved quantitative fluorescence imaging system and technique of the present system described herein were designed for seamless integration with the flow of surgery and compatibility with the state-of-the-art surgical microscopes. The imaging system is portable and affordable. In addition, the ability to spatially resolve down in the submillimeter range, enables both pre-clinical and clinical studies and ease of translation between each platform.
[0074] We explored the system's capabilities to quantify a commonly used clinical fluorescent biomarker, PpIX, in tissue simulating phantoms. We noted highly accurate and sensitive PpIX biomarker quantification in near-real time compared to the raw fluorescence, FRAW (0.89% vs. 0.27%, R2 linear regression coefficient), pointing to the ability of making fluorescence imaging in surgery quantitative and highly sensitive. Furthermore, quantitative fluorescence imaging demonstrated improved contrast compared to both vis-FI and FRAW , with approximately 2 and 1 orders of magnitude larger dynamical range than vis-FI and FRAW, respectively.
[0075] The imaging system provides significant improvements in
intraoperative diagnostic fluorescence imaging. Further, the imaging system provides an ideal pre-clinical-to-clinical testing platform for testing of novel imaging agents and preclinical models, with subsequent ease of translation from bench-to-bedside. The ease of application, seamless integration into the surgical workflow, portability, reasonable costs, and small footprints, allows for implementation across modern operating rooms and visualization of multiple fluorescent biomarkers.
RESULTS
Imaging System Design
[0076] The imaging system includes a set-up with a small footprint, which allows for ease of implementation in the operating room. The main components in a prototype embodiment include an optical adapter, a liquid crystal tunable filter (LCTF) (Cambridge Research Instruments), and a charge-coupled device (CCD) camera (Cooke Corp.) connected to a digital image processor and computer control system. The LCTF is a wavelength selection device to enable acquisition of images at each wavelength of interest and create the 3D spectrally resolved image cube. Other type of electronic camera, for example, EMCCD, ICCD, etc., or CMOS sensor cameras, which allow for collection of light and conversion to digital form, can also be used. The prototype embodiment liquid crystal tunable filter (LCTF) performs fast (50 ms) single band selection (7 nm FWHM) for filtration of incoming light in the range 400 - 720 nm. The LCTF displays a wavelength specific transmission response in the range λ = 400 - 720 nm, with a max transmittance of 64% at 710 nm. The CCD camera (1396 x 1024 pixels; 2 x 2 binning; 62% quantum efficiency at 580 nm.
[0077] A fully integrated, functional system requires only standard maintenance provided to conventional surgical microscopes. The system is portable, which allows for ease of transportation between operating rooms, and installation into a commercial surgical microscope is seamless. [0078] An optical adapter connects the main components (LCTF and camera) to an optical port of a surgical microscope, and between LCTF and a c-mount CCD camera. An LCTF performs fast (50 ms) single band selection (7 nm full width at half maximum) for filtration of incoming light in the range λ = 400 - 720 nm. The LCTF displays a wavelength specific transmission response at a selected wavelength in the visible light range (i.e., λ = 400 - 720 nm), with a maximum transmittance of 64% at 710 nm. The CCD camera (1396 x 1024 pixels; 2 x 2 binning; 62% quantum efficiency at 580 nm, PCO.Pixelfly, Cooke Corporation) senses the incoming light and transmits the digital counts to a computer control system. In a particular exemplary illustration, we operated in the visible region of the spectrum, given that two of the three major clinically used fluorescent imaging dyes operate in this region (i.e., fluorescein, PpIX). The system can be extended to the near infrared (650 - 1100 nm), given the advantages of working in this range. With the advent of novel imaging agents in the near infrared (NIR), this set up could readily accommodate either a LCTF operating in the NIR or a dual visible-NIR region LCTF.
Phantom Validation
[0079] We assessed the quantification capabilities of our QFI system. We developed a spectrally-constrained normalization algorithm that corrects the detected, raw fluorescence from the distorting effects of tissue optical properties. The varying attenuation due to tissue optical properties on both the excitation light and fluorescence emissions impact the detected fluorescence non-linearly. This varying attenuation was corrected, enabling absolute quantification of fluorophores in tissue independent of varying tissue specific attenuation. We used previously validated tissue simulating liquid phantoms of varying absorption (μ3) and reduced scattering (μ8') at the excitation and main emission peak of ΡρΓΧ (λ = 405 nm and λ = 635 nm, respectively) and with varying ΡρΓΧ concentrations in the range 20 - 5000 ng/ml - the commonly found range of ΡρΓΧ concentrations (lower to higher end) found in normal and pathological tissues.
[0080] Current state-of-the-art clinical systems use single band pass detection of the emitted fluorescence. We found from previous work and with our phantom studies, that vis-FI was able to visually detect concentrations no lower than -1000 ng/ml. Here we performed an area under the curve integration of the raw optical signal in the range λ = 610 - 720 nm, as a means to measure the raw, uncorrected fluorescence signal typically detected using contemporary clinical systems, FRAW- We then used our quantification technique on the spectrally-resolved wide filed data to calculate the pixel- specific PpIX concentrations, Cppix and thus estimate absolute PpIX levels across the full field of view. We demonstrate in our tissue simulating phantom studies that our quantitative imaging technique effectively deconvolves the non-linear distorting effects of varying optical properties on the emitted fluorescence (R2 = 0.89, vs. R2 = 0.27, linear regression analysis). Further, quantitative fluorescence imaging (i.e., QFI) demonstrated improved contrast compared to both vis-FI and FRAW , with approximately 2 and 1 orders of magnitude larger dynamical range than vis-FI and FRAW, respectively. These results demonstrate highly accurate and sensitive estimates of absolute PpIX levels in near-real time imaging mode, and that our technique improves the limit of detection of
fluorescence imaging by 1-2 orders of magnitude, pushing the lower limit of PpIX found in tissues. This latter point is of importance, since it indicates that our imaging system can quantify low, yet diagnostically significant PpIX levels currently found in low-grade gliomas, the tumor histology where the impact of fluorescence imaging on patient survival could be profound.
[0081] We tested the spectral-resolving power of our imaging system on tissue simulating phantoms with homogeneously distributed and varying levels of two commonly used clinical fluorophores, PpIX and fluorescein. Our system was able to simultaneously detect and spectrally un-mix the individual fluorescence emission spectra in imaging mode across the full field of view of both fluorophores at varying
concentrations. These results demonstrate the ability to do accurate simultaneous multiple-tracer imaging.
System Spatial Resolution
[0082] We first tested the spatial resolution of the system using a conventional method to measure the contrast transfer function (CTF) from a USAF 1951 standard contrast resolution target. The imaging system demonstrated submillimeter resolutions both in the vertical (214 μιη and 125 μιη) and horizontal directions (217 μιη and 125 μιη) (Rayleigh and Sparrow criteria, respectively), which are approximately 5 to 10 times smaller than the smallest length scale in which the surgeon operates. This indicates that the system demonstrates sufficient spatial resolution for both human as well as preclinical animal studies, which operate in the submillimeter to millimeter scales. Spectrally-resolved quantitative fluorescence imaging data.
[0083] Each quantitative fluorescence imaging sequence acquisition entails acquiring a 3-dimensional image cube at a 5 nm wavelength resolution (range λ = 450 - 720 nm) acquired under white light mode for reflectance followed by a 3-dimensional image cube at a 3 nm resolution (range λ = 600 - 720 nm) acquired under blue or excitation light mode for fluorescence.
[0084] Each 3-dimensional image cube (x, y, λ) includes a series of 2- dimensional images (x, y) (i.e., spatial dimensions) at a plurality N of specific wavelengths (λ) (i.e., spectral dimension) in the range of interest acquired under white light mode for reflectance or under blue light mode for fluorescence signals. That is, the image cube is a stack or series of 2-dimensional images taken at a plurality of wavelength settings of the tunable filter. Each individual pixel spatial coordinate (x, y) corresponds to a location on the field of view. We reconstructed high-resolution (i.e., 1 nm) full reflectance and fluorescence spectra for each pixel coordinate using a cubic spline interpolation technique for a total of 361,920 spectra per image cube (696 x 520 = 361,920).
Tissue attenuation correction algorithm.
[0085] The white light reflectance spectra are used to correct the detected fluorescence spectra for the attenuation caused by tissue absorption and scattering. We used a spectrally-constrained normalization technique to estimate the intrinsic fluorescence in tissue independent of the effects of tissue optical properties by calculating the intrinsic fluorescence value, Φ, with the relationshi in Eq. 1, ψ Corrected ~
Figure imgf000022_0001
where is the wavelength dependent, raw fluorescence intensity; Φχ 6 and Φπ m are the reflectance signals integrated over the ranges λ = 465-485 nm and λ = 625-645 nm, respectively. The range for Φχ was determined as close to the excitation wavelength band (i.e., 405 nm) to approximate light attenuation at excitation, and the range for <ϊ was determined at the main emission peak around λ = 635 nm. Our correction algorithm makes the assumption that most of the light attenuation is due to absorption at the excitation wavelength, given the high absorption due to hemoglobin, and that at the emission band, scattering dominates over absorption by 1-2 orders of magnitude more and can be corrected by an empirical power function of ^m .
[0086] We apply our tissue attenuation correction algorithm on each pair of reflectance and fluorescence image cubes. We integrate for each pixel coordinate (x;, y its corresponding high resolution reflectance spectrum to calculate Φ^εί (i) and (i) · We then estimated the corrected fluorescence spectrum for each (x;, y pixel coordinate,
^^orrected ® > by applying the relationship in Eq. 1 to each corresponding pixel coordinate (x;, y high-resolution raw fluorescence spectrum, Φ]¾ (i) . A non-negative least-squares routine is used on the corrected fluorescence spectra to spectrally un-mix the individual contributions of main fluorophores: PpIX, fluorescein, and tissue
autofluorescence using the following relationship, crel = (ΒτΒ)-1 ΒτΦ Cρ1oυrοrected (Eq.2) where B = [bib2. . .b ] is a matrix of basis spectra for N fluorophore components (for example, PpIX, fluorescein, autofluorescence), and Crel is a relative concentration fluorophore vector. A system-specific calibration factor derived from the least squares regression on phantoms of known PpIX concentrations is used to convert the spectrally un-mixed relative fluorophore concentration values into absolute biomarker(s) concentrations for each pixel, c abs = S c L a cU l crel (Eq. 3) where Scai is the system specific scalar calibration factor. We then are able to display a quantitative image map of PpIX concentrations.
Tissue phantoms.
[0087] Liquid tissue simulating phantoms that simulate the range of optical properties found in normal brain and brain tumor using an absorber dye (McCormick) as the main absorber and Intralipid as the scattering medium were fabricated for a total of nine (9) phantoms of three (3) different optical absorption (μ3 χ = 20, 40, and 60 cm"1) and scattering values (μ8 m = 8.7, 11.4, and 14.5 cm"1) for a total of nine (9) different phantoms. For each tissue phantom, nine (9) PpIX concentrations covering the range found in brain tumors (0.019, 0.039, 0.078, 0.156, 0.313, 0.625, 1.250, 2.500, and 5.000 μg/ml) were added, and a final set with non PpIX added for a total of 90 phantoms of varying absorption, scattering and PpIX.
[0088] We acquired a pair of reflectance and fluorescence image cubes for each phantom under typical low- light operating room conditions used for fluorescence guided surgery. Each pair was processed for quantitative fluorescence imaging to estimate a PpIX concentration image map. A region of interest analysis of the corresponding (x;, y pixel coordinates of the raw fluorescence image cube and on the calculated PpIX concentration image map for each phantom was used to (i) integrate the average area- under-the-curve intensity in the range 610 - 720 nm for the raw fluorescence spectra (i.e., f 120nm^F\uo ( J
J 610nm Raw \Λ ' αΛ), ^RAW and (ii) the average PpIX concentration, cppix, for the quantitative image map. The former corresponds to the raw fluorescence signal of PpIX, and is equivalent to using a bandpass filter to filter out only the raw fluorescence in that range of interest.
[0089] ^RAW and QFI derived cppix for each phantom were plotted versus the true known PpIX concentrations of each phantom. A linear regression analysis was then used to ascertain (i) the linearity of fluorescence with fluorophore concentration, which provides a measure of the non-linear effects of optical properties and the correction capabilities of our algorithm; and (ii) the limit of detection of both approaches, which provides a measure of the sensitivity of each approach to detect low levels of fluorophore.
[0090] A normalized contrast for the raw fluorescence and our quantitative approach was calculated by dividing RAW or cppjx at every concentration from i = 5.000 - 0.039 μg/ml by the RAW or cppix at the minimum concentration, 0.019 μg/ml, respectively,
Figure imgf000024_0001
System spatial resolution
[0091] A USAF 1951 (Edmund Optics) contrast resolution target was used to determine the spatial resolution of the imaging system. We imaged the USAF 1951 target under white light mode, which is made up of n = [0, 1 , 2, 3] groups, each with i =[1 , 2, 3, 4, 5, 6] horizontal and vertical elements consisting of black and white bar patterns. We extracted the cross-sectional intensity profiles for each element to calculate the element- specific contrast transfer function (CTF) using the following relationship,
CTF{i) = 7max ( - Aiiin (0 * 10Q
-^max ( ~*~ ^rnin ( ) where Imax and Imi„ are the maximum and minimum intensities of the bar pattern's cross- sectional profile. We matched the calculated element specific resolution using the following relationship
Re solution (i) =— ϊ— - * 1000
2n+ 6 (Eq. 6) where n is the group number, i is the element number, and resolution is given μιη. We calculated the Rayleigh and Sparrow horizontal and vertical resolutions by determining the point on the graph were the CTF = 26.4% or CTF = 0%, respectively.
Data Processing.
[0092] Data processing and analysis was done using custom MATLAB® software (Version R2010a, The Mathworks, Inc., Natick, MA, USA)
Statistical Analysis.
[0093] Statistical analyses were performed with MATLAB. Linear regression analysis was used to determine the coefficient of determination, R2. Non-parametric, Wilcoxon rank-sum (Mann- Witney) tests were use to compare differences. Two-sided P < 0.05 are described as statistically significant.
[0094] Experiments provided absolute quantification of tissue concentrations of the metabolic biomarker protoporphyrin IX (PpIX) following systemic administration of the prodrug 5 -aminolevulinic acid (ALA) using fiberoptic -based point fluorescence and diffuse reflectance spectroscopy in vivo. This technique significantly improves the clinical utility for detection of residual tumor during neurosurgical resection of various pathologies, including high-grade and low-grade gliomas.
[0095] In a point spectroscopy device referred to as the "Standard Probe", the full fluorescence spectrum is measured single points at the tissue surface, together with the full diffuse reflectance spectrum. A spectrally-constrained diffusion theory model is then used to determine the absorption and transport scattering spectra of the tissue (μ3 and μ'8, respectively) and these values are then used to correct the measured fluorescence spectrum yielding the true fluorescence signal. Data has been taken with the Standard Probe, this data has been used to confirm practicability of the hyperspectral imaging device described herein, and to test practicability of the spectrally -constrained normalization approach described herein. The local concentration of PpIX (Cppix) is then determined by spectral unmixing to eliminate contributions from PpIX photoproducts and tissue autofluorescence. A non model-based spectrally-constrained normalization approach is described herein. The potential advantages are that it simplifies the instrumentation, since only two reflectance detection channels are required. It is expected that the hyperspectral imaging device herein described will be able to provide similar information when used in vivo.
[0096] FIG. 4 is a schematic functional block diagram of the intraoperative standard probe system according to some embodiments. Referring to FIG. 4, measurements are made with three adjacent optical fibers making contact with the tissue at the distal end of a 1.1 mm diameter intraoperative probe. Each fiber has a core diameter of 200 μιη and the center-to-center separation between adjacent fibers is 260 μιη. One fiber is coupled to a broadband light-emitting diode (LED) delivering light from λ = 450 to 720 nm. Another fiber is coupled to a violet LED with center wavelength at λ = 405 nm. The third fiber is coupled to a miniature fiber optic spectrometer (USB2000+, Ocean Optics Inc., FL) with light detection capabilities between λ = 200 nm and 1100 nm at a spectral resolution < ~ 1 nm. The PpIX absorption spectrum exhibits a strong absorption band (Q-band) around λ = 405 nm, so that fluorescence excitation is efficiently achieved using the violet light source fiber. The effective tissue sampling depth is < ~ 1mm. White light reflectance and fluorescence emission spectra are sequentially measured using broadband and light excitation at λ = 405 nm, respectively. FIG. 5 shows typical spectra acquired during a surgical resection of a high-grade glioma. That is, FIG. 5 illustrates the PpIX fluorescence spectrum (left) and the diffuse reflectance spectrum (right) measured in a high-grade glioma following tissue excitation with blue and white light, respectively. The vertical bands in the diffuse reflectance spectrum (right) indicate the detection windows used in the normalization measurements.
[0097] Spectrally-constrained normalized values, φ1 3110, were computed using the formula ΦΗΒΟ (Α)
Figure imgf000027_0001
where <DHuo is the raw measured fluorescence spectrum, and Φ x and Φ m are the spectrally-integrated reflectance signals in the range λ = 465-485 nm and λ = 625-645 nm, respectively. The wavelength range for Φ x was chosen to be as close as possible to the fluorescence excitation band in order to correct for tissue attenuation of the excitation light. The second band samples the main PpIX fluorescence emission peak around λ = 635 nm. The underlying assumptions of Eq. (7) are: (1) that most of the spectral distortion effect is due to tissue absorption at the excitation wavelength, and (2) that the impact of light diffusion in the emission band, where scattering dominates over absorption, can be corrected by further dividing the raw fluorescence using an empirical power function of
0Ref
[0098] Three separate sets of measurements with PpIX were conducted to evaluate the accuracy of the spectrally-constrained normalization algorithm: (1) in tissue- simulating phantom experiments with a range of optical properties consistent with human brain tissue, (2) using ex vivo animal brain tumor tissue, and (3) using in vivo clinical data previously acquired with the Standard Probe during resection procedures in brain tumor patients. For each measurement, a background spectrum was acquired and subtracted from the raw fluorescence in order to account for the dark noise of the instrument, and the resulting spectrum was corrected using Eq. (7) for values of the empirical parameter a ranging from -2 to 2. The corrected spectra were then unmixed to separate the
contributions of the main fluorophores, namely PpIX, its photoproducts and tissue autofluorescence. The resulting pure PpIX spectra were then normalized to a calibration factor, which in turn was derived from the slope of the linear regression fits in phantoms with known dye concentrations, in order to extract an absolute PpIX concentration, Cppix.
[0099] The phantoms comprised Intralipid (Fresenius Kabi, Uppsala, Sweden) which provided the tissue-like background scattering, yellow food coloring (McCormick, London, Ontario) as the absorber and PpIX. Nine phantoms of differing optical properties were prepared, and a total of six concentrations of PpIX (0.15625, 0.3125, 0.625, 1.25, 2.5, and 5 μg/ml) for each phantom were mixed, giving a total of 54 separate phantoms with different absorption coefficient (μ3), reduced scattering coefficient (μ8') and fluorophore concentration. In order to mimic human brain tissue, the absorption and reduced scattering at the excitation wavelength ranged from μ3 = 20 to 60 cm"1 and μ8 ' = 15 to 25 cm"1 , respectively, and at the emission wavelength ranged from μ3 = 0.02 to 0.06 cm"1 and μ8 ' = 8.7 to 14.5 cm"1. Fluorescence and reflectance measurements were made for each phantom using the Standard Probe. FIG. 6A-6F illustrate raw and corrected emission spectra measured for tissue phantoms, according to the some embodiments. Three different ΡρΓΧ concentrations and a range of absorption and scattering values were used: (Fig. 6a, Fig. 6b, Fig. 6c) varying absorption with constant scattering; (Fig. 6d, Fig. 6e,Fig. 6f) varying scattering with constant absorption. The value of a= - 0.7 is used in Fig. 6c and Fig. 6f. Illustrated is the marked effects of varying absorption (FIG. 6(a) and scattering (FIG. 6(b)), on the measured fluorescence spectra, primarily in the intensity, with some distortion also of the spectral shape. In tissue, the latter is greatly due to the strong spectral dependence of hemoglobin absorption. Dividing the measured spectrum by Φ x (FIG. 7b) partially corrects for changes in absorption and, to a lesser extent, for changes in scattering (FIG. 7e). Normalization to a function of < Ref x and < Ref m as in Eq. (7) can further correct for changes in both absorption (FIG. 7c) as well as scattering (FIG. 7f), and yield quantitative fluorescence spectra that are largely independent of variations in the optical properties. As shown below, the optimal tissue-specific empirical parameter a is determined experimentally using the tissue- simulating phantoms (or, for example, Monte Carlo modeling) in order to achieve the highest accuracy in CppK . As in previous studies, the spectral unmixing algorithm was applied to the corrected fluorescence spectrum, using the known ΡρΓΧ emission spectrum as input. A calibration factor, derived from the linear regression fit of 0Ratl0( ) against the true concentration was then applied to the pure ΡρΓΧ spectrum to determine Cppix. FIG. 7 and FIG. 8 illustrates quantification of PpIX concentration in phantoms based on Fig. 7 raw fluorescence intensity (λ=635 nm), and FIG. 8 estimated CppK levels derived following correction of the raw fluorescence, according to some embodiments. FIG. 7 and Fig. 8 shows, as a function of the true ΡρΓΧ concentration, both the raw fluorescence intensity and the calculated Cppix following correction only for the strong absorption effects at the excitation wavelength. There is a significant improvement in the goodness-of-fit, from R2 = 0.639 (RMSE = 72%) for the detected fluorescence (<DFluo) to R2 = 0.944 (RMSE 24%) using a correction by the reflectance close to the excitation only (0Hu 0Ref x). Correction by the product of <DRef x<DRef m a provided the greatest improvement, with R2 = 0.984 (RMSE 13%), using a =
-0.7 (FIG. 9(a)). This algorithm is applied in the subsequent ex vivo tissue and clinical studies. FIG. 9 illustrates (a) empirical determination of the optimal a value for PpIX quantification and (b) relationship between the estimated an actual ΡρΓΧ concentration in phantoms following correction of the raw fluorescence, as described herein.
[0100] The spectrally constrained normalization method was then evaluated in terms of its efficacy to quantify Cppix in biological tissue. lxlO6 cells from the human glioma VX2 line were orthotopically implanted to a depth of 2 mm in the brains of five mice. Following three weeks of incubation, 100 mg/kg body weight of ALA (Sigma- Aldrich Inc., MO) was injected via the tail vein, and the animals were subsequently sacrificed by cervical dislocation under anesthesia at 0.5, 1, 2 , 3 and 4 h later, respectively. The brains were then removed intact and measurements made with the Standard Probe at three different locations for each sample. The probe measurements (fluorescence and reflectance) were used to compute the ratio in Eq. (7) for a = -0.7, from which the Cppix value was calculated, as above. For comparison, quantitative fluorimetric measurements of Cppix were made on ex vivo brain tissue samples using an established technique. Statistical analysis (paired Student' s t-test) showed no significant difference between the fluorimetric measurements and ratiometric estimates (p = 0.33). These results represent an initial demonstration that absolute quantification of ΡρΓΧ in tissue using the ratiometric approach according to some embodiments is feasible and accurate.
[0101] The Standard Probe has been used in ALA-induced PpIX fluorescence guided resection of human brain tumors. Here, we revisit the same clinical data and retrospectively apply the ratiometric algorithm in patients with a diagnosis either of glioma (N=5), meningioma (N=6) or metastatic brain tumors (N=3). Receiver operating characteristic (ROC) curves were generated to assess the performance of the
normalization approach for detecting abnormal tumor tissue. FIG. 10 contains ROC curves quantifying the performance of three different fluorescence measurement schemes for optical datasets acquired in vivo during 14 brain tumor resection procedures. FIG. 10 shows the ROC plots of the quantified Cppix using two ratiometric approaches compared with the raw measured fluorescence (< Fluo) intensity peak at λ = 635 nm for each tumor type. There was a highly statistically significant difference in the performance of the normalization approach, with the area under the curve (AUC) increasing to 0.92 (STD = 0.03) from the value of 0.60 (STD = 0.05) for the raw fluorescence signal.
[0102] The use of ALA-induced PpIX fluorescence guidance during tumor resection is gaining importance in the neurosurgical community, and there is increasing evidence that quantitative techniques that augment the visual fluorescence assessment of the surgeon yields clinical benefit. Our previous work demonstrated this using a full- spectrum device and algorithm for quantitative point measurements. According to some embodiments, an optimized non model-based spectrally-constrained normalization approach is used for PpIX quantification, applying a semi-empirical algorithm to correct for the distorting effects of optical attenuation in tissue. The advantage is that this is technically simpler in point-mode but is also translatable into wide-field quantitative fluorescence imaging mode more easily than the full-spectrum approach. A possible limitation compared to the Standard Probe method, which is, at least in principle, universal and does not require tissue-dependent calibration, is that it requires
experimental determination of the empirical optimization parameter, a, as well as a calibration factor derived from the linear regression fit using phantoms of known concentrations. Further detailed studies are required to establish any other limitations to the accuracy for absolute fluorophore quantification in vivo and this work is in progress, as is initial testing in wide-field quantitative imaging mode.
[0103] These approaches to determining Cppix in vivo (or the concentration of other exogenous molecules) may also be of value in applications such as photodynamic therapy in order to monitor and optimize the dose of photosensitizer and the drug-light time interval for treatment.
[0104] FIG. 11 is a schematic block diagram illustrating a system 100 for supporting surgery, according to some embodiments. The system of FIG. 11 includes a microscope body 102, which has multiple beam splitters 104 that permit light to be diverted to several optical ports simultaneously. Attached to a first optical port of body 102 is a tube 106 leading to a surgeon's binocular optical eyepieces 108.
[0105] Attached to a second optical port of body 102 are a first high definition electronic camera 120 and a second high definition electronic camera 122. Cameras 120, 122 are coupled to provide images to image capture interface 124 of a digital image processing system 126. Attached to a third optical port of body 102 is a hyperspectral imaging device 128 that in an embodiment has a tunable filter 130 adapted to receive light from body 102 and a high resolution broad-bandwidth electronic camera 132. The camera 132 of the hyperspectral imaging device 128 is also coupled to provide images to image capture interface 124 of the digital processing system 126. In an embodiment, tunable filter 130 is a liquid crystal tunable filter. In an alternative embodiment, tunable filter 130 is an acousto-optic tunable filter.
[0106] FIG. 12 is a schematic block diagram of a hyperspectral imaging device, according to an alternative embodiment. In an alternative embodiment, hyperspectral imaging device 128 of FIG. 11 is replaced with the hyperspectral imaging device of FIG. 12. In the hyperspectral imaging device of FIG. 12, light received through optical interface port 202 is scanned by a rotating mirror 204 across a spectrometer slit 206. Light passing through spectrometer slit 206 is dispersed according to wavelength by a dispersive device 208, such as a prism or a diffraction grating, and thence onto a planar image sensor 210 such that each column of planar image sensor 210 provides a spectrum of light received at a corresponding point in slit 206, and light received at slit 206 corresponds to light scanned from a row of a virtual image received through microscope body 102.
[0107] Referring again to FIG. 11, a tracker interface 140 of the image processing system 126 is coupled to use sensors 142 rigidly attached to a reference location within an operating room to track locations of microscope location sensors 144 and patient location sensors 146. Microscope location sensors 144 are rigidly attached to the microscope body 102, and patient location sensors 146 are attached to a frame 148 that may be attached to a patient while the patient is undergoing a surgical procedure. In a particular embodiment, frame 148 is adapted to be attached to a patient's skull 150 by screws (not shown) for the duration of a neurosurgical procedure during which the patient's brain 152 is exposed, and during which patient's brain 152 may be operated on with surgical instruments 154 to remove one or more lesions 156.
[0108] Microscope body 102 also has zoom optics 160, adapted for operation by a zoom motor/sensor 162, and a focus adjustment (not shown) adapted for operation by a focus motor (not shown). The microscope also has multiple illuminators 166, 168. In an embodiment, illuminators 166, 168 include white-light illuminators 166, and fluorescent stimulus illuminators 168, operating under control of an illumination interface 170 of the image processing system 126. The microscope body also has a heads-up display (HUD) projector 172 capable of providing graphical images through a combiner 174 of body 102 such that the graphical images are presented for viewing by a surgeon through surgeon's eyepieces 108 The surgeon's field of view through the operating microscope and its associated HUD is co-registered with that of the imaging system, allowing display of tissue classifications and hyperspectral imaging results superimposed on visible brain tissue, one-to-one comparisons, and intraoperative surgical decision making. At standard working distances between microscope and surgical cavity, surgical tools 154 fit between objective 160 and tissue 152.
[0109] Image processing system 126 also has a memory 178 into which image capture interface 124 saves images received from cameras 120, 122, 132; and at least one processor 180. Processor 180 is adapted for executing processing routines such as surface profile extraction routines 182, brain deformation modeling routines 184, fluorescence depth modeling routines 186, and hyperspectral image processing routines 188 stored in memory 178 and operable on images stored in memory 178. Processor 180 is also adapted for preparing images for display through display interface 190 onto monitor 192, and for communicating through network interface 194 to server 196; server 196 has a database containing information derived from preoperative MRI and CAT scans 198.
[0110] Among the image processing routines in memory of image processing subsystem 126 also has a classifier for classifying tissue types according to a combination of fluorescence and backscatter information. In a particular embodiment, the classifier is a K-nearest-neighbor classifier.
[0111] Image processing system 126 is also interfaced to a handheld backscatter fluorescence probe 143 that acts as a single-channel, single-point subset of the fluorescence imaging microscope. Probe 143 has white 145 and fluorescence stimulus 147 illuminators, and a spectrophometric detector 149, the illuminators 143, 145, and detector 149 coupled through an optical fiber and lens subsystem 151 to apply light to, and read light from, a single point on tissue surface. The probe couples into the image processing subsystem 126 through a probe interface 153.
[0112] Operation of the system 100 has several modes, and each mode may require execution of several phases of processing on processor 180, executing one or more of several routines, as mentioned above. For example, processor 180 can execute the hyperspectral image processing routine to perform the hyperspectral fluorescence and reflectance imaging of the system, as described above in detail. The hyperspectral fluorescence and reflectance imaging can also be performed in connection with stereo- optical extraction, using images captured by stereo cameras 120, 122, to perform tissue surface extraction and modeling, as described in U.S. Provisional Patent Application No. 61/583,092, filed in the United States Patent and Trademark Office on January 4, 2012, and incorporated herein in its entirety by reference. The tracking using tracking sensor 142, patient location sensors 146 and microscope location sensors 144 described above can be used in performing the brain deformation modeling routine. The hyperspectral fluorescence and reflectance imaging can also be performed in connection with fluorescence depth modeling, as described in U.S. Patent Application No. 13/145,505, filed in the United States Patent and Trademark Office on July 2, 2011, and incorporated herein in its entirety by reference, and described below in detail, where fluorescence and reflectance spectral information is derived from hyperspectral imaging device 128. It should be noted that in the referenced U.S. Patent Application No. 13/145,505 and described below in detail, the depth modeling is described as using a ratio of images at two wavelengths. Depth modeling using the hyperspectral fluorescence imaging of the present system can be improved with a modified process in which several ratios are used simultaneously using a fitting routine.
[0113] A phase-3 human trial of surface fluorescent microscopy during surgical removal of malignant tumors marked with 5 -ALA showed increased survival following surgery when all fluorescent tumor visible at excision cavity surfaces are removed; see Stummer, et al., Fluorescence- guided surgery with 5 -aminolevulinic acid for resection of malignant glioma: a randomized controlled multicentre phase III trial. Lancet Oncol., 2006. 7(5): p. 392-401, published online at oncology.thelancet.com.
[0114] 5-ALA may also be usable to locate tumors in other organs. In particular, 5-ALA has been shown to be preferentially metabolized into protoporphyrin IX in tumors of the prostate as compared to protoporphyrin IX production in surrounding normal prostate tissue. This may offer the opportunity to remove the prostate tumors with reduced risk of damaging nerves responsible for penile sensation and erection; thereby preserving functions thought important by many patients.
[0115] In yet another embodiment, dye administered to a patient to locate a tumor contains a fluorescent molecule or prodrug metabolized into fluorescent molecules in- vivo that is attached to a monoclonal antibody having specificity for tumor tissue.
[0116] In an embodiment tailored for use with tumors metastatic from a distant organ (not shown) such as lung, dye administered to a patient comprises a fluorescent molecule or prodrug attached to an antibody, such as a monoclonal antibody, having specificity towards a tissue type of the distant organ. Because many metastatic tumors retain some cell-surface markers similar to those of the parent tissue, dye preferentially delivers the fluorescent molecule or prodrug to the tumor tissue, while delivering a substantially lower concentration of the fluorescent molecule or prodrug to the organ.
[0117] A surgeon visualizes the tumor through the hyperspectral imaging device described herein.
[0118] Typically, tumors and organs are light propagation media where absorption and scattering is particularly important at short wavelengths, making them seem opaque to visible light. Consequently the white light images give good resolution but do not give good visibility of features beneath a surface of the tissue of tumor and organ. In particular, light at the blue end of the visible spectrum is often absorbed and scattered within less than a millimeter of the surface, rendering it difficult to see deeper structures. Further, because fine invasions of tumor may have color and structure superficially resembling that of surrounding tissues, these fine invasions of tumor at the surface may not be readily visible to the surgeon. However, the interaction of far-red and near-infrared light with biological tissue is such that propagation is inherently different when compared to that of visible light. For far-red and near-infrared light, scattering is still important but absorption by the main contributing chromophores such as hemoglobin and water are dramatically reduced (see FIG. 23). This provides an opportunity for imaging structures embedded deeper into the tissue using far-red and near-infrared light.
[0119] In an alternative embodiment, the monochromatic illuminator provides red or near infrared light between 600 and 1000 nanometers wavelengths, this light is capable of deeper penetration into organ and tumor than blue or green light and is within the range of wavelengths that can be detected by available electronic cameras. The wavelength used is chosen to be appropriate for the fluorophore used to label the organ or tumor.
[0120] The hyperspectral imaging device herein described may, in an embodiment, be operated to provide basic fluorescent imaging. This is done by illuminating the tissue with a light at a stimulus wavelength, in a particular embodiment by illuminating with a blue light-emitting diode (LED) having a filter to exclude illumination at a fluorescence emission wavelength. The tunable liquid-crystal filter is then set to pass only light at the fluorescence emission wavelength. This provides the surgeon with non depth-resolved information pertaining to fluorophore distribution beneath the surface; this information may be somewhat hazy due to scattering in the tissue.
[0121] In an embodiment, multiple monochromatic illuminators are provided, and the tunable liquid crystal filter adjusted appropriately, permitting rapid selection of surface and deep imaging wavelengths, and wavelengths suitable for different fluorophores.
[0122] The surgeon views the fluorescence image information on monitor 192 before, after, or at intervals while using surgical instruments such as surgical tools 154 to remove or destroy part or all of tumor 156.
[0123] When the surgeon wants to determine if any tumor tissue remains in the organ 152, the surgeon triggers the system to obtain new fluorescence tomographic image information, views that information of monitor 192, and may remove additional portions of tumor 156 and organ 152.
[0124] In an embodiment, blue or ultraviolet monochromatic stimulus- wavelength emitters have peak emission between three hundred fifty and four hundred five nanometers to induce visible fluorescence from protoporphyrin IX in tumor.
Alternative embodiments may replace blue or ultraviolet flood emitters with, or provide an additional set of monochromatic flood emitters, for providing stimulus specifically appropriate to a fluorophore other than protoporphyrin IX present in tumor. For example, near-infrared monochromatic emitters may be used with other dyes such as those based on indocyanine green. In an embodiment monochromatic emitters are LEDs; LEDs are available in a wide variety of spectral characteristics, can provide intense light, and are small and easy to use. In an alternative embodiment, monochromatic emitters are a tungsten halogen or high intensity discharge light source equipped with a monochromatic filter for transmitting a wavelength appropriate for inducing fluorescence in fluorophores of, or metabolically produced from, the dye in use. Such an alternative light source may also include optical fibers and lenses for directing light from the monochromatic filter to the tissue and tumor.
[0125] In some applications of the microscope other than open brain surgery, there may also be a matching liquid applied to the subject to give the area being interrogated a geometry that is simpler to model. [0126] In an embodiment, a left and right stereo image pair is captured 402. This image pair is used by the processor 180 to extract a surface profile of the tissue. Image cubes are then captured with the hyperspectral imaging device and placed in memory as previously described. A computer model of diffuse scattering in the tissue is then constructed.
[0127] The tomography reconstruction algorithm is based on the assumption that the received light from the tissue is diffused, resulting from scattering and absorption at each voxel in a model of the tissue. Each voxel is assigned a scattering parameter, absorption parameters for stimulus and emissions wavelengths, and fluorescence parameters, such as concentration and fluorophore lifetime. Further, it is known that an emissions spectra of light emitted from fluorophores in tissue is distorted by passage through tissue by absorption by chromophores present in that tissue, details of the distortions induced being correlated to depth of the fluorophores in tissue as illustrated in Fig. 14. Since this distortion due to depth effectively alters a "color" of received light at the QFI, this color for each pixel of the image cube may be determined from ratios of received light intensity in each of several of the 2-D images of the image cube captured at different wavelengths near peak fluorescent emissions. For example, should spectral distortions resemble those of FIG. 14, ratios, and thus "colors", may be computed from image data acquired at multiple wavelengths in the 680 to 710 nanometer range.
[0128] Voxels external to the modeled surface contour of the tissue have absorption, scattering, and fluorescence parameters assumed to be constant, known, and zero in most cases to set boundary conditions on the model of the tissue during the process of solving the differential equations associated with light transport. Voxels within the organ, tumor, or tissue are initially assumed to scatter light and have absorbance as per an average absorption and scattering of biological tissues of the tissue type being observed. In some applications the average values of these properties are chosen based on literature values.
[0129] The model is constrained to produce image data of "colors," or ratios between light received at each of several wavelengths, of intensity matching data at selected wavelengths in the fluorescent image cube.
[0130] In this embodiment, optical diffuse tomography images of absorbance may be displayed to the surgeon, showing concentrations of chromophores in the tissue. These images may be of use during surgery in locating hidden structures such as blood vessels, aneurysms, and other concentrations of heme.
[0131] Once the absorption and scattering parameters are refined, or in embodiments displaying fluorescent parameters only a default absorption and scattering parameters may be used; for voxels within the tissue an illumination at each voxel is computed for each position of the incident beam using a computer-based light propagation model of the tissue and organ.
[0132] Next, the fluorescence concentration parameters of each voxel are computed by using a least-squares fit of the fluorescence parameters to the captured fluorescence image data. The fluorescence parameters thus computed for each voxel form a three-dimensional model of fluorophore distribution in the tissue.
[0133] A sequence of tomographic images is constructed and displayed 414 by considering intersections of a plane with fluorescence distribution of the three- dimensional model of fluorescence distribution. In an embodiment, these tomographic images are displayed on monitor 192 and/or the HUD to the surgeon as a sequence of increasing depth, the sequence being preceded by a white light photographic image of the organ or cavity for reference.
[0134] In an alternative embodiment, the tomographic images of increasing depth are encoded by coloring each image with a false color ranging from blue for images at the organ surface to red for images more than a centimeter deep into the surface. These images are then summed. The sum image, having fluorescence intensity encoded as intensity and fluorescence depth encoded as color, is then displayed on display device. White light images and images of reflected and scattered laser light may also be displayed on display device to provide the surgeon with a visual reference.
[0135] Once the laser scanning is complete, and while image processing proceeds, white light illuminator is turned back on. Once the surgeon studies the tomographic images on display device, the surgeon may remove additional tumor or other organ tissue and repeat the process illustrated in FTG. 13.
[0136] In an embodiment, computer simulation of the microscope shows that fluorescence of protoporphyrin IX generated by 5-ALA tagging of tumor tissue can be resolved to about two and a half millimeter resolution to a depth of twelve millimeters in organ. Further, presence of strong concentrations of protoporphyrin IX can be detected to a greater depth than that at which this resolution can be obtained. [0137] Studies of a mouse model of glioma show that removal of
protoporphyrin-IX-tagged tumor tissue detected at the surface of an operative site with a surface fluorescence microscope enhance survival; this has also been shown to be true by Stummer for human glioma. It is expected that microscope will result in both improvements in survival beyond those obtained with surface fluorescent instruments and a reduction in post-surgical neurological impairment.
[0138] It is also expected that the microscope will be of use in treating other cranial tumors such as meningioma, pituitary tumors, acoustic neuroma, and some metastatic tumors. It is expected that the microscope will also be of use in surgical treatment of tumors in other sites, including skin, breast, liver, bowel, thyroid, eyes, pancreas, kidney, bladder, prostate and muscle, as well as some lesions of other types including vascular abnormalities.
[0139] The microscope is also of use with indocyanine green for visualization of aneurismal vasculature obscured by overlying vessels, aneurysms, or other tissue.
[0140] Light as the term used herein includes electromagnetic of the visible, near ultraviolet and near infrared portions of the spectrum that may be focused with lenses and imaged with readily available semiconductor imaging devices.
[0141] While the invention has been described with reference to fluorescent substances such as indocyanine green, protoporphyrin IX, and Fluorescein, the apparatus and methods herein described are applicable to other biocompatible fluorescent prodrugs, dyes and molecules such as are known or may be developed in the future, and which may tend to concentrate in tumor tissue to a different extent than in normal tissue.
[0142] An alternative embodiment of the method uses the fluorescent tomographic microscope heretofore described to visualize a fluorescent chromophore such as may occur naturally in some types of tumor tissues. In this embodiment, there is no need to administering a prodrug or a drug. The fluorescent tomographic microscope may also be used to visualize tumors or tissues having concentrations of such a naturally occurring fluorescent chromophore that differ from concentrations in nearby normal organ stroma. Nicotinamide Adenine Dinucleotide Hydride (NADH) is an example of a naturally-occurring fluorescent chromophore that is critical to function of mitochondria. As such NADH occurs in all eukaryotic tissues, including human organ stroma and tumor tissues, but may be present at different concentrations in tumor and nearby normal stroma. Metabolically inactive tissues, such as fatty tissue of breast, may have very low concentrations of NADH, while adjacent malignant tumor tissue may be metabolically very active and therefore possess large quantities of NADH; this difference in concentration permits using the fluorescent tomographic microscope to visualize the malignant tumor tissue.
[0143] In this embodiment, scattered, reflected, and fluorescent light 1336 from organ 1306 and tumor 1344 passes through a lens of lenses 1308, a collimating optic 1338, a filter system 1340, and into a high resolution CCD camera 1342. Filter system 1340 has transparent modes for use with multiple wavelengths for imaging heme, as well as excitation- wavelength blocking and fluorescent-light passing modes for use when imaging fluorescent chromophores. In an embodiment, CCD camera 1342 is actively cooled to provide reduced noise and enhanced sensitivity to infrared light.
[0144] In operation, in a heme imaging mode, images are captured by CCD camera 1342 at each of the three wavelengths separately; these images are passed to the image processing system 232 for processing. Image processing at the first and third wavelengths is as heretofore described with reference to FIG. 22; however absorbance and scattering parameters are refined as indicated in step 1006. Image processing at the second wavelength is performed by constructing the voxel-based model of scattering and absorbance 1002 at the second wavelength, but without fluorescence parameter. The absorbance parameters at each voxel are then ratioed to provide a measure of heme oxygenation at each voxel, and summed to provide a measure of heme concentration at each voxel.
[0145] The fluorescence, heme concentration, and heme oxygenation parameters at each voxel are then mapped into tomographic image planes and displayed as tomographic images for inspection by the surgeon. The surgeon can then use these images to distinguish tumor from adjacent and nearby normal organ stroma. This embodiment is useful during surgery for visualizing below- surface heme concentrations as well as fluorescence during a wide variety of surgical procedures; and is of particular utility for distinguishing malignant breast tissue from adjacent normal breast tissue. In addition to cancer surgery, this embodiment is also useful for intraoperative identification of berry aneurisms, arteriovenous malformations, hematomas, and blood vessels.
[0146] An embodiment as heretofore described with reference to FIG. 11 is capable of imaging heme and heme oxygenation below organ surfaces in real time, and is capable of generating an additional tomographic image indicating changes in heme concentration and / or heme oxygenation. These images indicating changes in heme concentration and oxygenation are useful for functional neuroimaging to confirm identification of foci of seizure activity because neural activity depletes oxygen and increases blood flow. Precise identification of foci of seizure activity is of importance during surgical procedures intended to alleviate epilepsy, and such surgery is occasionally done under local anesthesia to permit identification of such foci. Since the apparatus can identify changes in heme concentration and oxygenation over a centimeter below a surface, this embodiment may also be used for functional neuroimaging through the intact skull, although resolution is less than that available with the skull open.
[0147] In an alternative embodiment, fluorescence excitation of fluorophores in tissue is performed using a broad-beam source in the NIR at the excitation wavelength, λεχ. It has been found that fluorescent light emitted by concentrations of fluorophores like protoporphyrin IX is more strongly absorbed by tissue in the 650-670 nanometer wavelength range than in longer wavelengths such as near 700-710 nanometers, and that with proper stimulus radiation, protoporphyrin IX will emit radiation having wavelengths from 650 to 720 nanometers. Similar effects may also occur with fluorescent light emitted by other fluorophores at similar or other wavelengths. This effect is due to the presence in tissue of chromophores, substances - including hemoglobin— in tissue that absorb or scatter light more intensely at some wavelengths than others. Typically, though, while fluorescent radiation is emitted across this range, the emitted radiation is not uniform in intensity across the wavelength band.
[0148] FIG. 14 illustrates spectra of fluorescent radiation emitted by fluorophores at several depths in tissue as observed above the tissue. These spectra have been normalized to the same peak level. Spectra is that of fluorophores at the surface of the tissue, and is close to the spectra of light as emitted by the fluorophores themselves. Spectra is that of light from fluorophores four millimeters below the surface as seen from above the surface, spectra is that of light from fluorophores one centimeter below the surface as seen from above the surface, and spectra is that of light from fluorophores two centimeters below the surface as observed from above the surface.
[0149] In an embodiment for use with the prodrug 5 -ALA and the fluorophore protoporphyrin-IX, emitted fluorescence signals are measured at two wavelengths, λ1>2, each of which is longer than the excitation wavelength λεχ, using band-pass filters and a cooled CCD camera; in other embodiments emitted fluorescence signals are measured at multiple, meaning three or more, wavelengths using appropriate band-pass filters having center pass-band wavelengths at different wavelengths in the range of 650-720 nanometers. For example, if tissue has absorption and scattering characteristics similar to the tissue assumed in FIG. 27, and fluorescent light intensity II is measured through a filter having bandpass 1610 between 680 and 690 nanometers while a fluorescent light intensity 12 is measured through a filter having bandpass 1612 between 700 and 710, a ratio of II to 12 will be considerably smaller when the fluorophore is two centimeters below the surface than when the fluorophore is at the surface of the tissue.
[0150] Embodiments for use with other fluorophores than protoporphyrin ΓΧ will require different calibration tables and may operate at wavelengths other than those specified in the previous paragraph. Further, it is known that optical absorption and scattering characteristics differ from one type of tissue to another, and may also change somewhat with time as a subject's oxygenation levels change during surgery. In order to provide for this variability, an embodiment, has a library with normal optical
characteristics of a variety of tissue types that the fluorescent tomographic microscope is expected to encounter during surgery. For example, the library may contain optical characteristics for normal brain grey matter and for normal brain white matter when the microscope is used during removal of brain tumors such as gliomas. An operator then selects the appropriate tissue type for tissue in the field of view, and optical
characteristics of that tissue type are then used for computation of fluorophore depth and distribution.
[0151] In an alternative embodiment, library has optical characteristics of a variety of tissue types with each tissue type measured under different levels of oxygenation. In these embodiments, the operator measures tissue oxygenation with direct or optical techniques and optical characteristics of the tissue are determined based on both tissue oxygenation and tissue type.
[0152] Assuming that tissue absorption due to the fluorophore concentrations of tumor is much smaller than absorption by other chromophores of tissue, the light signals at each emission wavelength can be modeled using the expression
Figure imgf000041_0001
where R→ is the vector corresponding to the detection point on the tissue surface, which is mapped onto a pixel or sub-ensemble of pixels on the CCD chip. Qp is the quantum yield, the emission spectrum and CF(I"→) the concentration of fluorescent molecules at location r→, where r→ is an arbitrary vector corresponding to the location of a point inside the interrogated tissue. In Eq. (8), light transport is modeled as a diffusive process. The fluence function yfx is the excitation light field, and Gem is the diffusion equation Green's function, which corresponds to the radiant exposure in response to a light impulse at r→. For boundary conditions associated with an infinite homogenous medium, it is given by
G (R, r, S\ D*) = exp(-l*-I Si) where the diffusion constant is
Figure imgf000042_0001
and the penetration depth is
Figure imgf000042_0002
is tne absorption coefficient of the tissue while JJ,S' is the reduced scattering coefficient.
[0153] A closed form expression is derived from Eq. (8) and Eq. (9), from which a depth value can be estimated for each point imaged on the tissue surface, using an approximation that all fluorescence emitted from the surface is from a point-like distribution of fluorophore with molar concentration Cp location at position rs →. This is equivalent to setting in Eq. (8), which then takes the form
Figure imgf000042_0003
R - rs \, δλ, Όλ) (£q 1Q) where the depth of the distribution is I R - S \=d assuming that the detection point R→ is located directly above the source of fluorescence at rs.
[0154] To illustrate how fluorescence spectra are affected by varying the depth of the fluorophore distribution, see FIG. 27 for protoporphyrin- IX spectra computed with Eq. (10) for depths varying up to d=20 mm for an anticipated typical absorption spectrum labeled and a constant value of reduced scattering ( 8'=1 mm"1). As the object gets deeper in the tissue, the optical properties of the tissue demonstrate an increasing influence on shaping the observed spectrum because of the increasing lightpaths traversed by light. The information contained in the distorted spectra may be distilled into a single quantity by calculating the ratio of the signal at two emission wavelengths, r = /em {R, e ) eF em q2 ) _ Gem (R, rs , S , D )
y/em (R,rx , A2 ) X eF em { ) ~ Gem (R, rs , S^ , D^ )
(Eq. 11)
[0155] where the intensity values at each wavelength are normalized with the relative signal strength of an undistorted emission spectrum, implying that Γ should be equal to 1 for d=0 mm.
[0156] In Eq. (11), Γ is independent of the diffuse excitation field. This means that, in the point-source approximation limit, the ratio is independent of scattering and absorption of the stimulus light.
[0157] Inserting Eq. (9) into Eq. (11), we find that the logarithm of the fluorescence ratio for an infinite medium is linearly related to the depth in tissue of the fluorophores with a slope equal to the difference in penetration depth between wavelengths λ1 and λ2, This linear relationship can potentially be used in estimating depth from a simple measured ratio in the case of diffusive medium of infinite spatial extent. A more realistic model for epi-illumination imaging may also be obtained assuming there is an index of refraction mismatch boundary between tissue and air. The relationship obtained is then similar - up to a multiplicative factor - for depth values larger than approximately two millimeters. For smaller values than two millimeters, the relationship is non-linear.
[0158] An alternative embodiment superficially resembling that of FIG. 11 as heretofore described constructs a fluorescence image cube having images taken at a different center wavelength in the range of wavelengths emitted by a typical fluorescent material such as protoporphyrin IX, such as in the range 650-720 nanometers.
[0159] As described above with reference to Equations(8-l l), the spectra of fluorescent light passing through tissue is altered by absorption and scattering in the tissue, such that spectra of the received fluorescent light encodes depth information. In an embodiment, two bandpass elements are used to derive a spectral deformation and depth for the fluorescent light captured in each pixel; in an alternative embodiment three or more bandpass elements are used to more precisely estimate spectral deformation due to passage of emitted light through tissue and thereby estimate depth for each pixel of the image.
[0160] In an embodiment, the depth information determined as described above, using modeling of incident laser light and depth information derived from spectral information in received fluorescent light, are used to provide a refined depth information, and to thereby provide a refined model of the tumor. In an alternative mode of operating this embodiment, the scanning laser may be disabled and depth information derived solely from spectral deformation of fluorescent light stimulated by an appropriate flood illuminator.
[0161] In an embodiment using two image planes from the image cube in the wavelength range of fluorescence in the fluorophores in tumor, a two dimensional table is used. This table is generated according to the optical properties of the tissue at the wavelengths of the bandpass filters as determined by measurement through imaging of scattering of a beam, or from a library of optical properties as previously discussed herein. This table is indexed by intensity at the higher wavelength bandpass filter in X, and intensity in the lower wavelength bandpass filter in Y, the table having entries of approximate depth of tumor based upon differences between intensity of fluorescent light passing the higher wavelength filter and the lower wavelength filter. The table is precomputed and stored in memory of the image processor to permit fast access.
Intensity from each pixel in high wavelength and low wavelength images are used with a table interpolation algorithm as known in the art of computing to derive a depth associated with each pixel, and a three-dimensional model of fluorophore distribution is computed. Information from the model of fluorophore distribution is then displayed to the surgeon; this may be in the form of tomographic images, a topographic map, or by color coding of fluorescence magnitude with determined depth.
[0162] In another embodiment, the optical properties of the tissue as determined above at the wavelengths of the bandpass filter elements is used to compute the 'slope' for the log-linear relationship of the ratio of intensity at two wavelengths as described above with reference to substitution of Eq. (9) into Eq. (11). This 'slope' allows us relate the depth in tissue of the fluorophores to the difference in intensity between wavelengths λ1 and λ2,· Then, the depth can be estimated based on the measured
Figure imgf000045_0001
(Eq. 12)
[0163] Alternatively the more general expression associated with a semi- infinite diffusive medium may be used, as described above with reference to Eq.(8-10).
Hyperspectral imaging system look-up table (LUT) approach for estimation of tissue optical properties
[0164] The hyperspectral imaging system with white-light illumination can be used to estimate the tissue optical properties across the full field of view to create maps of tissue optical properties, such as absorption (μα) and reduced scattering (μ8 ') coefficients. An embodiment of this method uses a look-up table (LUT), which has of a set of basis functions of spectrally resolved diffuse reflectance including reflectance (Rd( )) at known absorption( μa(λ)) and scattering (μ8'(λ)). In a particular embodiment, the table is indexed by at least reflectance Rd, and wavelength λ, the components of reflectance spectra; and contains multiple spectra, each captured at different values of known absorption and scattering. The table has values recorded therein for absorption μa, and scattering μ8' at each table spectra. Captured spectra measured at each pixel, which includes reflectance at a number of different wavelengths, are fit to determine at least the closest table entries, and absorption and scattering parameters are read from the table. In order to conserve table space while providing adequate accuracy, lookups into the table for absorption and scattering parameters are interpolated using an interpolation method selected from the group of linear, cubic, or spline interpolation functions.
[0165] First, diffuse reflectance spectra, Rd( ), are measured in phantoms of known and varying optical absorption, μa, and reduced scattering, μ8' , across a range of values typically found in human tissues, including both normal tissues and pathological tissues such as tumor. The collected Rd k) at known μa(λ) and μ8'(λ) are used to create a 4-dimensional set of basis functions [Rd, λ, μa, μβ']. Wide-field spectrally-resolved reflectance images of tissue of unknown tissue optical properties, e.g., acquired during surgery, are then decomposed into Rd at each pixel. In an alternative embodiment, regularized minimization is applied to fit the basis functions to observed Rd k) at each pixel such that a best fit to a weighted sum of the basis functions matches Rd. The fitted basis functions yield estimates of μα(λ) and μβ'ίλ) at every pixel.
[0166] In particular embodiments, the estimated absorption and reduced scattering coefficients, μα(λ) and μβ'ίλ) are also used to determine additional quantitative biomarkers that include but are not limited to hemoglobin concentration and oxygen saturation.
[0167] This technique enables a hyperspectral imaging system to collect diffuse reflectance data, and apply a minimization algorithm unto a set of known basis functions, (in a particular example using the lookup table described above) to make explicit estimates of tissue optical properties across the full field of view.
Hyperspectral quantitative fluorescence imaging (qFI) using a look-up table (LUT) for estimation of tissue optical properties and a light transport model
[0168] The hyperspectral imaging system with white-light illumination and fluorescence excitation light (e.g., light at 405 nm for PpIX) can be used to perform quantitative fluorescence imaging using a LUT and light transport model approach. In this embodiment, the estimates of μα(λ) and μβ'ίλ) determined by fitting spectra to entries of the table, and looking up μα(λ) and μβ'ίλ) in the lookup table at every pixel m are used to create a 'correction image' . The 'correction image' is then used to correct the measured raw fluorescence based on a light transport model, such as the light transport model of Eq. 13 below, to produce corrected quantitative fluorescence images. This light transport model of Eq. 13 is derived from a model published as Kim, A., et al.,
Quantification of in vivo fluorescence decoupled from the effects of tissue optical properties using fiber-optic spectroscopy measurements, J Biomed Opt 15, 067006 (2010),
Figure imgf000046_0001
R (Eq. 14)
l+2/c(l-a'x) + [l- 2/c/3)]V3(l-a/x)
where Rtx is the total diffuse reflectance. This parameter depends on the internal reflectance parameter κ and the reduced albedo, a which in turn are defined as:
Figure imgf000047_0001
With
rid = -1.44nrei-2 + 0.7 lnrei-l +0.67 +0.0636nreU where nrei=ntissue/nexternai for matching internal and external refractive indexes κ =1.
μ3 Χ is the absorption coefficient at the excitation wavelength (e.g., if using PpIX as the fluorophore, λ=405 nm);
Fjcm is the raw (uncorrected) fluorescence emissions; and
Rm is the diffuse reflectance at the emission wavelength.
[0169] Spectral unmixing (e.g., least squares approach) is then used to separate the contributions from the fluorophore(s) being imaged, which may include PpIX and tissue autofluorescence, to construct a full-field of view image of fluorophore(s) concentrations, such as a PpIX concentration map. This method enables quantitative fluorescence imaging qFI by correcting the raw fluorescence using explicit estimates of the tissue optical properties at the fluorescence wavelength and stimulus wavelength to create a 'correction image' to apply a light transport model of quantitative fluorescence. The fluorescence model presented here assumes that the optical absorption at the excitation wavelength hx is high relative to that at the emission wavelength, and as such, most fluorophore absorption occurs close to the excitation light source. The model assumes that fluorescence migration paths are approximate by those of the reflectance photons at the emission wavelength.
[0170] In an embodiment, FIG. 15 is a flow diagram of a method of imaging.
[0171] FIG. 16 is a flowchart of obtaining and correcting a fluorescence image for absorption and scattering in a target, such as tissue, in order to quantify fluorescence and reflectance spectrally-derived images of the tissue. With reference to Fig. 15 and 16, and Fig. 11, white light is provided 502 to the target, or tissue, 152 by illuminators 166; and light from the target is imaged 504 by camera through a tunable optical filter of the target while tuning the filter to a plurality of wavelengths, the series of images forming a spectral image; providing a stimulus light to the target 506 using illuminators 168; and taking a fluorescence emission image of the target 508 through filters tuned to block stimulus light. In an optional step 509 the fluorescence emission images of target 508 are captured as fluorescence spectra by sweeping the tunable optical filter and capturing multiple images of the target, the fluorescence spectra are then decomposed 509 to provide individual fluorescence images related to each of several fluorophores expected to be present in the target.
[0172] The images are then processed to determine absorption and scattering spectral parameters; in an embodiment determining the absorption and scattering parameters is performed by interpolating in the table as previously described. The fluorescence emission image or images are then corrected 510 by using the absorption and scattering parameters to produce corrected fluorescence emission images giving a quantitative distribution of each fluorophore, such as PPIX 524 and other fluorophores 526 that may be native to the tissue or administered to a patient.
[0173] The absorption and scattering parameter spectra at each pixel are then decomposed 512 to provide tissue component images mapping concentrations of typical tissue constituents such as oxygenated and deoxygenated hemoglobin; these are then added and ratioed 514 to provide a total hemoglobin image 520 and a percent- oxygenation 522 map or image of hemoglobin oxygenation in the tissue. The images, including corrected, quantitative, fluorescence, hemoglobin, and percent oxygenation, are then displayed 516 for use in diagnosis.
Combinations
[0174] Various combinations of the elements may occur in a practical system. For example, in embodiments, optical system may be a surgical microscope, while in other embodiments other optical systems having the ability to illuminate with stimulus or white light, and to image in white or stimulus light and then in fluorescence emitted light may be used. In particular embodiments, the following combinations of features are anticipated:
[0175] An imaging system designated A, having an illumination device for illuminating a target, the illumination device adapted to selectively emit a white light and a fluorescent stimulus light; a device for receiving light from the target, the device for receiving light comprising at least one optical output port at which at least a portion of the received light is provided as an output; a tunable filter for receiving the portion of the received light provided as the output from the device for receiving light, the tunable filter being tunable to pass a filtered portion of the received light, the filtered portion of the received light having a plurality of wavelengths selected by the tunable filter; a high- resolution, broad-bandwidth electronic camera for receiving the light of a plurality of wavelengths selected by the tunable filter and encoding a plurality of electronic images therefrom, the electronic images representing spectra at pixels of the images; and a processor configured to process the plurality of electronic images to form an image of the target; wherein the image includes an image of fluorescent emissions by any fluorophore in the target.
[0176] It is understood that the system described in A is adapted for use with a target of human or other mammalian tissue.
[0177] An imaging system designated AB including the imaging system designated A wherein the device for receiving light is a surgical microscope.
[0178] An imaging system designated AC including the imaging system designated A or AB, wherein at least some of the electronic images the processor is configured to process are generated from fluorescence light received by the surgical microscope and wherein at least some of the electronic images processed by the processor are generated from reflectance light received by the surgical microscope.
[0179] An imaging system designated AD including the imaging system designated A, AB, or AC, wherein, in forming the image of the target, the processor is configured to use the electronic images generated from reflectance light to compensate the electronic images from the fluorescence light for absorption and scattering while generating the image of fluorescent emissions from the target.
[0180] An imaging system designated AE including the imaging system designated AD wherein the processor is configured to determine absorption and a scattering parameter at each pixel while compensating the electronic images from the fluorescence light for absorption and scattering.
[0181] An imaging system designated AF including the imaging system designated AE wherein a table interpolation is used to determine an optical absorption and a scattering parameter at each pixel. [0182] An imaging system designated AG including the imaging system designated A, AB, AC, AD, AE, or AF, wherein the tunable filter is a liquid crystal tunable filter.
[0183] The system designated AG or AH wherein a fluorophore in the target comprises protoporphyrin IX.
[0184] An imaging system designated AH including the imaging system designated A, AB, AC, AD, AE, or AF wherein the tunable filter is an acousto-optic tunable filter.
[0185] A surgical microscope system designated B, including: an illumination device for illuminating a target, the illumination device adapted for selectively emitting a fluorescent stimulus light and a white light; at least one optical output port at which at least a portion of received light received from the target is provided; a tunable filter for receiving the portion of the received light, the tunable filter being tunable to pass a filtered portion of the received light, the filtered portion of the received light having a plurality of wavelengths selected by the tunable filter and provided as output from the tunable filter; an electronic camera for receiving the light of a plurality of wavelengths selected by the tunable filter, the electronic camera adapted to convert the light selected by the tunable filter to a plurality of electronic images while the tunable filter is tuned; and a processor configured to process the plurality of electronic images to form images of the target, wherein the images of the target comprise images of fluorophores in the target.
[0186] A surgical microscope system designated BA including the system designated B, wherein the electronic images the processor is configured to process include images generated from fluorescence light received from the target and images generated from reflectance light received from the target.
[0187] A surgical microscope system designated BB including the system designated B or BA, wherein, in forming the images of the target, the processor uses the images generated from reflectance light to compensate the images generated from the fluorescence light for absorption and scattering in the target.
[0188] A surgical microscope system designated BC including the system designated B, BA, or BB, wherein the tunable filter is selected from the group consisting of a liquid crystal tunable filter and an acousto-optic tunable filter.
[0189] A method of imaging designated C including providing a white light to a target; taking a series of images through a tunable optical filter of the target while tuning the filter to a plurality of wavelengths, the series of images forming a spectral image; providing a stimulus light to the target; taking a fluorescence emission image of the target; processing the spectral image to determine an absorption and a scattering parameter at pixels of the images; and correcting the fluorescence emission image using the absorption and scattering parameters to produce a corrected fluorescence emission image.
[0190] A method designated CA including the method designated C further including decomposing the spectral image to provide images of components of the target, the components including specific scattering and absorbing substances expected to be in the target.
[0191] A method designated CB including the method designated CA wherein the target comprises tissue, and wherein the components of the target include oxygenated and deoxygenated hemoglobin.
[0192] The method designated CB wherein the images of components of the target are combined to form images of total hemoglobin and hemoglobin oxygen saturation.
[0193] A method of imaging designated D including providing a white light to a target; taking a series of images through a tunable optical filter of the target while tuning the filter to a plurality of wavelengths, the series of images forming a spectral image; processing the spectral image to determine an absorption and a scattering parameter at pixels of the images; and decomposing the spectral image to provide images of components of the target, the components including specific scattering and absorbing substances expected to be in the target.
[0194] A method designated DA including the method designated D wherein the target comprises tissue, and wherein the components of the target include oxygenated and deoxygenated hemoglobin.
[0195] The method of claim DA wherein the images of components of the target are combined to form images of total hemoglobin and hemoglobin oxygen saturation.
[0196] While the invention has been particularly shown and described with reference to a preferred embodiment thereof, it will be understood by those skilled in the art that various other changes in the form and details may be made without departing from the spirit and scope of the invention. It is to be understood that various changes may be made in adapting the invention to different embodiments without departing from the broader inventive concepts disclosed herein and comprehended by the claims that follow.

Claims

CLAIMS We claim:
1. An imaging system, comprising:
an illumination device for illuminating a target, the illumination device adapted to selectively emit a white light and a fluorescent stimulus light; a device for receiving light from the target, the device for receiving light
comprising at least one optical output port at which at least a portion of the received light is provided as an output;
a tunable filter for receiving the portion of received light provided as the output from the device for receiving light, the tunable filter being tunable to pass a filtered portion of the received light, the filtered portion of the received light having a plurality of wavelengths selected by the tunable filter; a high-resolution, broad-bandwidth electronic camera for receiving the light of a plurality of wavelengths selected by the tunable filter and encoding a plurality of electronic images therefrom, the electronic images representing spectra at pixels of the images; and
a processor configured to process the electronic images to form images of the target; wherein the images include an image of fluorescent emissions by at least one fluorophore in the target.
2. The imaging system of claim 1 wherein the device for receiving light is a surgical microscope
3. The imaging system of claim 2, wherein at least some of the electronic images the processor is configured to process are generated from fluorescence light received by the surgical microscope, and wherein at least some of the electronic images processed by the processor are generated from reflectance light received by the surgical microscope.
4. The imaging system of claim 3, wherein, in forming the image of the target, the processor is configured to use the electronic images generated from reflectance light to compensate the electronic images from the fluorescence light for absorption and scattering while generating the image of fluorescent emissions from the target.
5. The imaging system of claim 4 wherein the processor is configured to determine absorption and a scattering parameter at each pixel while compensating the electronic images from the fluorescence light for absorption and scattering.
6. The imaging system of claim 5 wherein a table interpolation is used to determine an optical absorption and a scattering parameter at each pixel.
7. The imaging system of claim 2, 3, 4, 5, or 6, wherein the tunable filter is a liquid crystal tunable filter.
8. The imaging system of claim 7 wherein the target is tissue containing protoporphyrin IX, and wherein at least one of the fluorophores in the target is the protoporphyrin IX.
9. The imaging system of claim 2, 3, 4, 5, or 6, wherein the tunable filter is an acousto-optic tunable filter.
10. The imaging system of claim 9 wherein the target is tissue containing protoporphyrin IX.
11. A surgical microscope system, comprising:
an illumination device for illuminating a target, the illumination device adapted for emitting a fluorescent stimulus light and a white light; at least one optical output port at which at least a portion of received light
received from the target is provided;
a tunable filter for receiving the portion of the received light, the tunable filter being tunable to pass a filtered portion of the received light, the filtered portion of the received light having a plurality of wavelengths selected by the tunable filter and provided as output from the tunable filter;
an electronic camera for receiving the light of a plurality of wavelengths selected by the tunable filter, the electronic camera adapted to convert the light selected by the tunable filter to a plurality of electronic images while the tunable filter is tuned; and
a processor configured to process the plurality of electronic images to form
images of the target, wherein the images of the target comprise images of fluorophores in the target.
12. The surgical microscope system of claim 11, wherein the electronic images the processor is configured to process include images generated from fluorescence light received from the target and images generated from reflectance light received from the target.
13. The surgical microscope system of claim 12, wherein, in forming the image of the target, the processor uses the images generated from reflectance light to compensate the images generated from the fluorescence light for absorption and scattering in the target.
14. The surgical microscope system of claim 11, 12, 13, wherein the tunable filter is selected from the group consisting of a liquid crystal tunable filter and an acousto- optic tunable filter.
15 A method of imaging comprising:
providing a white light to a target;
taking a series of images through a tunable optical filter of the target while tuning the filter to a plurality of wavelengths, the series of images forming a spectral image;
providing a stimulus light to the target;
taking a fluorescence emission image of the target;
processing the spectral image to determine an absorption and a scattering
parameter at pixels of the images; and
correcting the fluorescence emission image using the absorption and scattering parameters to produce a corrected fluorescence emission image.
16. The method of claim 15 further comprising decomposing the spectral image to provide images of components of the target.
17. The method of claim 15 or 16 wherein the target comprises tissue, and wherein the components of the target include oxygenated and deoxygenated hemoglobin.
18. The method of claim 17 wherein the images of components of the target are combined to form images of total hemoglobin and hemoglobin oxygen saturation.
19. A method of imaging comprising: providing a white light to a target;
taking a series of images through a tunable optical filter of the target while tuning the filter to a plurality of wavelengths, the series of images forming a spectral image;
processing the spectral image to determine an absorption and a scattering
parameter at pixels of the images; and
decomposing the spectral image to provide images of components of the target.
PCT/US2013/022266 2008-05-22 2013-01-18 Method and apparatus for quantitative hyperspectral fluorescence and reflectance imaging for surgical guidance WO2013109966A1 (en)

Priority Applications (11)

Application Number Priority Date Filing Date Title
US14/373,443 US20140378843A1 (en) 2012-01-20 2013-01-18 Method And Apparatus For Quantitative Hyperspectral Fluorescence And Reflectance Imaging For Surgical Guidance
US15/044,097 US11510600B2 (en) 2012-01-04 2016-02-15 Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
US15/367,243 US10568535B2 (en) 2008-05-22 2016-12-02 Surgical navigation with stereovision and associated methods
US15/402,077 US11813100B2 (en) 2012-01-04 2017-01-09 Methods for quantitative and enhanced-contrast molecular medical imaging using cross-modality correction for differing tracer kinetics
US16/443,760 US11656448B2 (en) 2012-01-20 2019-06-17 Method and apparatus for quantitative hyperspectral fluorescence and reflectance imaging for surgical guidance
US16/785,882 US11129562B2 (en) 2008-05-22 2020-02-10 Surgical navigation with stereovision and associated methods
US17/486,839 US11690558B2 (en) 2011-01-21 2021-09-27 Surgical navigation with stereovision and associated methods
US17/961,480 US11857317B2 (en) 2012-01-04 2022-10-06 Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
US18/144,767 US20230280577A1 (en) 2012-01-20 2023-05-08 Method and apparatus for quantitative hyperspectral fluorescence and reflectance imaging for surgical guidance
US18/204,091 US20230320649A1 (en) 2008-05-22 2023-05-31 Surgical navigation with stereovision and associated methods
US18/389,128 US20240081759A1 (en) 2012-01-04 2023-11-13 Methods for quantitative and enhanced-contrast molecular medical imaging using cross-modality correction for differing tracer kinetics

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201261588708P 2012-01-20 2012-01-20
US61/588,708 2012-01-20

Related Parent Applications (4)

Application Number Title Priority Date Filing Date
PCT/US2012/055755 Continuation-In-Part WO2013040555A2 (en) 2008-05-22 2012-09-17 Apparatus for measuring in-vivo mechanical properties of biological tissues
PCT/US2013/020352 Continuation-In-Part WO2013103870A1 (en) 2008-05-22 2013-01-04 Method and apparatus for calibration of stereo-optical three-dimensional surface-mapping system
PCT/US2013/024400 Continuation-In-Part WO2013116694A1 (en) 2008-05-22 2013-02-01 Method and apparatus for determining tumor shift during surgery using a stereo-optical three-dimensional surface-mapping system
US14/375,311 Continuation-In-Part US9336592B2 (en) 2012-02-03 2013-02-01 Method and apparatus for determining tumor shift during surgery using a stereo-optical three-dimensional surface-mapping system

Related Child Applications (7)

Application Number Title Priority Date Filing Date
US14/373,443 A-371-Of-International US20140378843A1 (en) 2012-01-20 2013-01-18 Method And Apparatus For Quantitative Hyperspectral Fluorescence And Reflectance Imaging For Surgical Guidance
PCT/US2013/024400 A-371-Of-International WO2013116694A1 (en) 2008-05-22 2013-02-01 Method and apparatus for determining tumor shift during surgery using a stereo-optical three-dimensional surface-mapping system
US201414345029A Continuation-In-Part 2008-05-22 2014-03-14
PCT/US2014/051356 Continuation-In-Part WO2015023990A1 (en) 2012-01-04 2014-08-15 Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
PCT/US2015/033672 Continuation-In-Part WO2015187620A1 (en) 2008-05-22 2015-06-02 Surgical navigation with stereovision and associated methods
US15/044,097 Continuation-In-Part US11510600B2 (en) 2012-01-04 2016-02-15 Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
US16/443,760 Continuation US11656448B2 (en) 2012-01-20 2019-06-17 Method and apparatus for quantitative hyperspectral fluorescence and reflectance imaging for surgical guidance

Publications (1)

Publication Number Publication Date
WO2013109966A1 true WO2013109966A1 (en) 2013-07-25

Family

ID=48799707

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2013/022266 WO2013109966A1 (en) 2008-05-22 2013-01-18 Method and apparatus for quantitative hyperspectral fluorescence and reflectance imaging for surgical guidance

Country Status (2)

Country Link
US (2) US20140378843A1 (en)
WO (1) WO2013109966A1 (en)

Cited By (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8882662B2 (en) 2012-06-27 2014-11-11 Camplex, Inc. Interface for viewing video from cameras on a surgical visualization system
WO2015023990A1 (en) * 2013-08-15 2015-02-19 The Trustees Of Dartmouth College Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
WO2015164774A1 (en) * 2014-04-25 2015-10-29 The Trustees Of Dartmouth College Fluorescence guided surgical systems and methods gated on ambient light
WO2015187620A1 (en) * 2014-06-02 2015-12-10 The Trustees Of Dartmouth College Surgical navigation with stereovision and associated methods
US9336592B2 (en) 2012-02-03 2016-05-10 The Trustees Of Dartmouth College Method and apparatus for determining tumor shift during surgery using a stereo-optical three-dimensional surface-mapping system
US9456200B2 (en) 2012-01-04 2016-09-27 The Trustees Of Dartmouth College Method and apparatus for calibration of stereo-optical three-dimensional surface-mapping system
EP3051275A4 (en) * 2013-09-27 2016-10-05 Fujifilm Corp Optical measurement device
US9642606B2 (en) 2012-06-27 2017-05-09 Camplex, Inc. Surgical visualization system
US9655545B2 (en) 2011-09-15 2017-05-23 The Trustees Of Dartmouth College Apparatus for measuring in-vivo mechanical properties of biological tissues
EP3128940A4 (en) * 2014-07-30 2017-05-31 Novartis AG Vital stain visualization in ophthalmic surgical procedures and associated devices, systems, and methods
EP3206061A1 (en) * 2016-02-15 2017-08-16 Leica Instruments (Singapore) Pte. Ltd. Illumination filter system and observation system for a multispectral fluorescence microscope
JP2017526899A (en) * 2014-06-05 2017-09-14 ウニベルジテート ハイデルベルク Methods and means for multispectral imaging
US9782159B2 (en) 2013-03-13 2017-10-10 Camplex, Inc. Surgical visualization systems
RU2641519C1 (en) * 2017-03-29 2018-01-17 Общество с ограниченной ответственностью "Медицинские видеосистемы" (ООО "МедВис") Method of quantitative estimation of photosensitizer concentration on video image in real time when conducting fluorescent research
CN108009550A (en) * 2017-11-09 2018-05-08 深圳大学 High spectrum image feature detection method and device based on curve of spectrum fitting
WO2018125940A1 (en) * 2016-12-27 2018-07-05 Urugus S.A. Dynamic hyper-spectral imaging of objects in apparent motion
US10028651B2 (en) 2013-09-20 2018-07-24 Camplex, Inc. Surgical visualization systems and displays
RU2672478C1 (en) * 2017-08-03 2018-11-15 Федеральное государственное бюджетное образовательное учреждение высшего образования "Оренбургский государственный университет" Method of optical diagnostics of pathologies in biological tissues
US10292771B2 (en) 2013-03-15 2019-05-21 Synaptive Medical (Barbados) Inc. Surgical imaging systems
WO2019164455A1 (en) * 2018-02-21 2019-08-29 Optotek D.O.O. Image-processing apparatus and image-processing method for detection of irregularities in tissue
US10568535B2 (en) 2008-05-22 2020-02-25 The Trustees Of Dartmouth College Surgical navigation with stereovision and associated methods
US10568499B2 (en) 2013-09-20 2020-02-25 Camplex, Inc. Surgical visualization systems and displays
EP3671306A1 (en) * 2018-12-21 2020-06-24 Leica Instruments (Singapore) Pte. Ltd. Image-processing device, fluorescence observation device and method for emulating a first type of fluorescence observation device on a second type of fluorescence observation device
US10702353B2 (en) 2014-12-05 2020-07-07 Camplex, Inc. Surgical visualizations systems and displays
US10852517B2 (en) 2015-06-02 2020-12-01 National University Corporation Asahikawa Medical University Observation assisting device, information processing method, and program
US10918455B2 (en) 2017-05-08 2021-02-16 Camplex, Inc. Variable light source
US10966798B2 (en) 2015-11-25 2021-04-06 Camplex, Inc. Surgical visualization systems and displays
US11154378B2 (en) 2015-03-25 2021-10-26 Camplex, Inc. Surgical visualization systems and displays
US11510600B2 (en) 2012-01-04 2022-11-29 The Trustees Of Dartmouth College Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
US11564639B2 (en) 2013-02-13 2023-01-31 The Trustees Of Dartmouth College Method and apparatus for medical imaging using differencing of multiple fluorophores
US11937951B2 (en) 2013-02-13 2024-03-26 The Trustees Of Dartmouth College Method and apparatus for medical imaging using differencing of multiple fluorophores

Families Citing this family (46)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11690558B2 (en) * 2011-01-21 2023-07-04 The Trustees Of Dartmouth College Surgical navigation with stereovision and associated methods
WO2013109966A1 (en) 2012-01-20 2013-07-25 The Trustees Of Dartmouth College Method and apparatus for quantitative hyperspectral fluorescence and reflectance imaging for surgical guidance
JP2014115151A (en) * 2012-12-07 2014-06-26 Shimadzu Corp Optical imaging device
KR101446173B1 (en) * 2013-02-21 2014-10-01 주식회사 고영테크놀러지 Tracking system and method for tracking using the same
WO2015073871A2 (en) * 2013-11-14 2015-05-21 The George Washington University Systems and methods for determining lesion depth using fluorescence imaging
CN107851176A (en) 2015-02-06 2018-03-27 阿克伦大学 Optical imaging system and its method
CA2977073A1 (en) * 2015-02-23 2016-09-01 Li-Cor, Inc. Fluorescence biopsy specimen imager and methods
JP6451494B2 (en) * 2015-05-19 2019-01-16 株式会社島津製作所 Imaging device
EP3314234B1 (en) 2015-06-26 2021-05-19 Li-Cor, Inc. Fluorescence biopsy specimen imager
EP3165153A1 (en) * 2015-11-05 2017-05-10 Deutsches Krebsforschungszentrum Stiftung des Öffentlichen Rechts System for fluorescence aided surgery
US20170224205A1 (en) * 2016-02-04 2017-08-10 Wright State University Light endoscope system for imaging, light delivery, and therapy response monitoring
WO2017184940A1 (en) 2016-04-21 2017-10-26 Li-Cor, Inc. Multimodality multi-axis 3-d imaging
WO2017223378A1 (en) 2016-06-23 2017-12-28 Li-Cor, Inc. Complementary color flashing for multichannel image presentation
US10228283B2 (en) * 2016-08-12 2019-03-12 Spectral Insights Private Limited Spectral imaging system
EP3545488A1 (en) 2016-11-23 2019-10-02 Li-Cor, Inc. Motion-adaptive interactive imaging method
DE102017203448B9 (en) * 2017-03-02 2021-12-23 Carl Zeiss Meditec Ag Microscopy system and microscopy method for quantifying fluorescence
WO2018200261A1 (en) 2017-04-25 2018-11-01 Li-Cor, Inc. Top-down and rotational side view biopsy specimen imager and methods
DE102018204426B4 (en) * 2017-05-11 2024-02-29 Carl Zeiss Meditec Ag Microscopy system
US10909723B2 (en) * 2017-06-13 2021-02-02 X-Rite, Incorporated Hyperspectral imaging spectrophotometer and system
TWI651519B (en) * 2017-12-26 2019-02-21 國家中山科學研究院 Temperature measurement correction method, electronic system and method for generating corrected regression coefficient table
US11131631B2 (en) * 2017-12-28 2021-09-28 University Of Notre Dame Du Lac Super-resolution fluorescence microscopy by stepwise optical saturation
SG11202006461VA (en) 2018-01-11 2020-08-28 Centre For Eye Res Australia Limited Method and system for quantifying biomarker of a tissue
SG11202003809UA (en) * 2018-01-12 2020-05-28 Agency Science Tech & Res Method and system for in vivo detection of adipose tissue browning
JP6911785B2 (en) * 2018-02-02 2021-07-28 日本電信電話株式会社 Judgment device, judgment method and judgment program
US10972643B2 (en) 2018-03-29 2021-04-06 Microsoft Technology Licensing, Llc Camera comprising an infrared illuminator and a liquid crystal optical filter switchable between a reflection state and a transmission state for infrared imaging and spectral imaging, and method thereof
US10924692B2 (en) 2018-05-08 2021-02-16 Microsoft Technology Licensing, Llc Depth and multi-spectral camera
US10477173B1 (en) 2018-05-23 2019-11-12 Microsoft Technology Licensing, Llc Camera with tunable filter and active illumination
US20200015907A1 (en) 2018-07-16 2020-01-16 Ethicon Llc Integration of imaging data
CN109363768A (en) * 2018-10-10 2019-02-22 南京诺源医疗器械有限公司 785nm wavelength light source near-infrared fluorescence imaging surgery guides system
US10931894B2 (en) 2018-10-31 2021-02-23 Microsoft Technology Licensing, Llc Tunable spectral illuminator for camera
TWI664582B (en) * 2018-11-28 2019-07-01 靜宜大學 Method, apparatus and system for cell detection
WO2020117982A1 (en) * 2018-12-05 2020-06-11 Intuitive Surgical Operations, Inc. Determining chromophore concentrations in tissues
US11245875B2 (en) 2019-01-15 2022-02-08 Microsoft Technology Licensing, Llc Monitoring activity with depth and multi-spectral camera
SG11202107280UA (en) * 2019-01-17 2021-08-30 Univ Health Network Systems, methods, and devices for three-dimensional imaging, measurement, and display of wounds and tissue specimens
US11896442B2 (en) 2019-12-30 2024-02-13 Cilag Gmbh International Surgical systems for proposing and corroborating organ portion removals
US11648060B2 (en) 2019-12-30 2023-05-16 Cilag Gmbh International Surgical system for overlaying surgical instrument data onto a virtual three dimensional construct of an organ
US11744667B2 (en) 2019-12-30 2023-09-05 Cilag Gmbh International Adaptive visualization by a surgical system
US11284963B2 (en) 2019-12-30 2022-03-29 Cilag Gmbh International Method of using imaging devices in surgery
US11776144B2 (en) 2019-12-30 2023-10-03 Cilag Gmbh International System and method for determining, adjusting, and managing resection margin about a subject tissue
US11219501B2 (en) 2019-12-30 2022-01-11 Cilag Gmbh International Visualization systems using structured light
US11759283B2 (en) 2019-12-30 2023-09-19 Cilag Gmbh International Surgical systems for generating three dimensional constructs of anatomical organs and coupling identified anatomical structures thereto
US11832996B2 (en) 2019-12-30 2023-12-05 Cilag Gmbh International Analyzing surgical trends by a surgical system
DE102021115587A1 (en) * 2021-06-16 2022-12-22 Carl Zeiss Meditec Ag CO-DESIGN OF OPTICAL FILTER AND FLUORESCENCE APPLICATIONS WITH ARTIFICIAL INTELLIGENCE
US20230319254A1 (en) * 2022-04-02 2023-10-05 Mantis Health, Inc. Stereoscopic camera adapter for enabling down-hole data capture and transmission
WO2024009204A1 (en) * 2022-07-04 2024-01-11 Leica Instruments (Singapore) Pte. Ltd. Controller for a surgical microscope and method
DE102022121505A1 (en) 2022-08-25 2024-03-07 Carl Zeiss Meditec Ag Method, computer program and data processing unit for preparing the observation of a fluorescence intensity, method for observing a fluorescence intensity and optical observation system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6678398B2 (en) * 2000-09-18 2004-01-13 Sti Medical Systems, Inc. Dual mode real-time screening and rapid full-area, selective-spectral, remote imaging and analysis device and process
US20050111758A1 (en) * 2003-08-15 2005-05-26 Holger Lange Systems and methods for registering reflectance and fluorescence hyperspectral imagery
US20100145416A1 (en) * 2008-12-10 2010-06-10 Uk Kang Apparatus for photodynamic therapy and photodetection
US7945077B2 (en) * 2005-11-30 2011-05-17 Lawrence Livermore National Security, Llc Hyperspectral microscope for in vivo imaging of microstructures and cells in tissues
US20120002012A1 (en) * 2010-07-02 2012-01-05 Intuitive Surgical Operations, Inc. Imaging mode blooming suppression

Family Cites Families (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5793350A (en) 1996-11-19 1998-08-11 Chips & Technologies, Inc. Apparatus and method for adaptive image stretching
US6734962B2 (en) * 2000-10-13 2004-05-11 Chemimage Corporation Near infrared chemical imaging microscope
AU1547101A (en) * 1999-11-26 2001-06-04 Applied Spectral Imaging Ltd. System and method for functional brain mapping and an oxygen saturation difference map algorithm for effecting same
US6748259B1 (en) * 2000-06-15 2004-06-08 Spectros Corporation Optical imaging of induced signals in vivo under ambient light conditions
US6811264B2 (en) 2003-03-21 2004-11-02 Mitsubishi Electric Research Laboratories, Inc. Geometrically aware projector
WO2005069887A2 (en) * 2004-01-16 2005-08-04 The City College Of The University Of New York Micro-scale compact device for in vivo medical diagnosis combining optical imaging and point fluorescence spectroscopy
WO2006062895A2 (en) 2004-12-06 2006-06-15 Cambridge Research And Instrumentation, Inc. Systems and methods for in-vivo optical imaging and measurement
US8398541B2 (en) 2006-06-06 2013-03-19 Intuitive Surgical Operations, Inc. Interactive user interfaces for robotic minimally invasive surgical systems
US20060287583A1 (en) 2005-06-17 2006-12-21 Pool Cover Corporation Surgical access instruments for use with delicate tissues
US20070167835A1 (en) 2005-07-25 2007-07-19 Massachusetts Institute Of Technology Tri modal spectroscopic imaging
CN101375315B (en) 2006-01-27 2015-03-18 图象公司 Methods and systems for digitally re-mastering of 2D and 3D motion pictures for exhibition with enhanced visual quality
TWI328709B (en) 2006-03-28 2010-08-11 Au Optronics Corp Liquid crystal display
US20070236514A1 (en) 2006-03-29 2007-10-11 Bracco Imaging Spa Methods and Apparatuses for Stereoscopic Image Guided Surgical Navigation
JP5208430B2 (en) 2007-01-31 2013-06-12 オリンパス株式会社 Fluorescence observation device for living tissue
US20080272312A1 (en) * 2007-05-04 2008-11-06 Chemimage Corporation Hyperspectral fluorescence and absorption bioimaging
WO2013109966A1 (en) 2012-01-20 2013-07-25 The Trustees Of Dartmouth College Method and apparatus for quantitative hyperspectral fluorescence and reflectance imaging for surgical guidance
CA2949031C (en) * 2008-07-24 2019-03-05 Massachusetts Institute Of Technology Systems and methods for imaging using absorption
WO2010019515A2 (en) 2008-08-10 2010-02-18 Board Of Regents, The University Of Texas System Digital light processing hyperspectral imaging apparatus
US8948851B2 (en) * 2009-01-20 2015-02-03 The Trustees Of Dartmouth College Method and apparatus for depth-resolved fluorescence, chromophore, and oximetry imaging for lesion identification during surgery
WO2011113162A1 (en) * 2010-03-17 2011-09-22 Haishan Zeng Rapid multi-spectral imaging methods and apparatus and applications for cancer detection and localization
US9699438B2 (en) 2010-07-02 2017-07-04 Disney Enterprises, Inc. 3D graphic insertion for live action stereoscopic video
JP5832537B2 (en) * 2010-08-05 2015-12-16 ケンブリッジ・リサーチ・アンド・インストルメンテーション・インコーポレーテッド Enhanced visual evaluation of samples
KR101750047B1 (en) 2010-10-11 2017-06-22 삼성전자주식회사 Method for providing and processing 3D image and apparatus for providing and processing 3D image
US8408569B2 (en) 2011-01-17 2013-04-02 Tenneco Automotive Operating Company Inc. Damper tube reinforcement sleeve
US9007430B2 (en) 2011-05-27 2015-04-14 Thomas Seidl System and method for creating a navigable, three-dimensional virtual reality environment having ultra-wide field of view
WO2012170963A1 (en) * 2011-06-08 2012-12-13 Digital Light Innovations System and method for hyperspectral imaging
US8784301B2 (en) * 2011-08-12 2014-07-22 Intuitive Surgical Operations, Inc. Image capture unit and method with an extended depth of field
US9655545B2 (en) 2011-09-15 2017-05-23 The Trustees Of Dartmouth College Apparatus for measuring in-vivo mechanical properties of biological tissues
US9766441B2 (en) 2011-09-22 2017-09-19 Digital Surgicals Pte. Ltd. Surgical stereo vision systems and methods for microsurgery
US9330477B2 (en) 2011-09-22 2016-05-03 Digital Surgicals Pte. Ltd. Surgical stereo vision systems and methods for microsurgery
US9712746B2 (en) 2013-03-14 2017-07-18 Microsoft Technology Licensing, Llc Image capture and ordering

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6678398B2 (en) * 2000-09-18 2004-01-13 Sti Medical Systems, Inc. Dual mode real-time screening and rapid full-area, selective-spectral, remote imaging and analysis device and process
US20050111758A1 (en) * 2003-08-15 2005-05-26 Holger Lange Systems and methods for registering reflectance and fluorescence hyperspectral imagery
US7945077B2 (en) * 2005-11-30 2011-05-17 Lawrence Livermore National Security, Llc Hyperspectral microscope for in vivo imaging of microstructures and cells in tissues
US20100145416A1 (en) * 2008-12-10 2010-06-10 Uk Kang Apparatus for photodynamic therapy and photodetection
US20120002012A1 (en) * 2010-07-02 2012-01-05 Intuitive Surgical Operations, Inc. Imaging mode blooming suppression

Cited By (65)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11129562B2 (en) 2008-05-22 2021-09-28 The Trustees Of Dartmouth College Surgical navigation with stereovision and associated methods
US10568535B2 (en) 2008-05-22 2020-02-25 The Trustees Of Dartmouth College Surgical navigation with stereovision and associated methods
US9655545B2 (en) 2011-09-15 2017-05-23 The Trustees Of Dartmouth College Apparatus for measuring in-vivo mechanical properties of biological tissues
US11510600B2 (en) 2012-01-04 2022-11-29 The Trustees Of Dartmouth College Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
US11857317B2 (en) 2012-01-04 2024-01-02 The Trustees Of Dartmouth College Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
US9456200B2 (en) 2012-01-04 2016-09-27 The Trustees Of Dartmouth College Method and apparatus for calibration of stereo-optical three-dimensional surface-mapping system
US9336592B2 (en) 2012-02-03 2016-05-10 The Trustees Of Dartmouth College Method and apparatus for determining tumor shift during surgery using a stereo-optical three-dimensional surface-mapping system
US11166706B2 (en) 2012-06-27 2021-11-09 Camplex, Inc. Surgical visualization systems
US11389146B2 (en) 2012-06-27 2022-07-19 Camplex, Inc. Surgical visualization system
US9615728B2 (en) 2012-06-27 2017-04-11 Camplex, Inc. Surgical visualization system with camera tracking
US9629523B2 (en) 2012-06-27 2017-04-25 Camplex, Inc. Binocular viewing assembly for a surgical visualization system
US9642606B2 (en) 2012-06-27 2017-05-09 Camplex, Inc. Surgical visualization system
US10925472B2 (en) 2012-06-27 2021-02-23 Camplex, Inc. Binocular viewing assembly for a surgical visualization system
US10925589B2 (en) 2012-06-27 2021-02-23 Camplex, Inc. Interface for viewing video from cameras on a surgical visualization system
US9681796B2 (en) 2012-06-27 2017-06-20 Camplex, Inc. Interface for viewing video from cameras on a surgical visualization system
US9723976B2 (en) 2012-06-27 2017-08-08 Camplex, Inc. Optics for video camera on a surgical visualization system
US11889976B2 (en) 2012-06-27 2024-02-06 Camplex, Inc. Surgical visualization systems
US8882662B2 (en) 2012-06-27 2014-11-11 Camplex, Inc. Interface for viewing video from cameras on a surgical visualization system
US9216068B2 (en) 2012-06-27 2015-12-22 Camplex, Inc. Optics for video cameras on a surgical visualization system
US10555728B2 (en) 2012-06-27 2020-02-11 Camplex, Inc. Surgical visualization system
US11129521B2 (en) 2012-06-27 2021-09-28 Camplex, Inc. Optics for video camera on a surgical visualization system
US10231607B2 (en) 2012-06-27 2019-03-19 Camplex, Inc. Surgical visualization systems
US9936863B2 (en) 2012-06-27 2018-04-10 Camplex, Inc. Optical assembly providing a surgical microscope view for a surgical visualization system
US10022041B2 (en) 2012-06-27 2018-07-17 Camplex, Inc. Hydraulic system for surgical applications
US9492065B2 (en) 2012-06-27 2016-11-15 Camplex, Inc. Surgical retractor with video cameras
US11937951B2 (en) 2013-02-13 2024-03-26 The Trustees Of Dartmouth College Method and apparatus for medical imaging using differencing of multiple fluorophores
US11564639B2 (en) 2013-02-13 2023-01-31 The Trustees Of Dartmouth College Method and apparatus for medical imaging using differencing of multiple fluorophores
US9782159B2 (en) 2013-03-13 2017-10-10 Camplex, Inc. Surgical visualization systems
US10292771B2 (en) 2013-03-15 2019-05-21 Synaptive Medical (Barbados) Inc. Surgical imaging systems
US10932766B2 (en) 2013-05-21 2021-03-02 Camplex, Inc. Surgical visualization systems
WO2015023990A1 (en) * 2013-08-15 2015-02-19 The Trustees Of Dartmouth College Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
US11147443B2 (en) 2013-09-20 2021-10-19 Camplex, Inc. Surgical visualization systems and displays
US10568499B2 (en) 2013-09-20 2020-02-25 Camplex, Inc. Surgical visualization systems and displays
US10881286B2 (en) 2013-09-20 2021-01-05 Camplex, Inc. Medical apparatus for use with a surgical tubular retractor
US10028651B2 (en) 2013-09-20 2018-07-24 Camplex, Inc. Surgical visualization systems and displays
EP3051275A4 (en) * 2013-09-27 2016-10-05 Fujifilm Corp Optical measurement device
WO2015164774A1 (en) * 2014-04-25 2015-10-29 The Trustees Of Dartmouth College Fluorescence guided surgical systems and methods gated on ambient light
WO2015187620A1 (en) * 2014-06-02 2015-12-10 The Trustees Of Dartmouth College Surgical navigation with stereovision and associated methods
JP2017526899A (en) * 2014-06-05 2017-09-14 ウニベルジテート ハイデルベルク Methods and means for multispectral imaging
EP3940371A1 (en) * 2014-06-05 2022-01-19 Universität Heidelberg Method and imaging apparatus for acquisition of fluorescence and reflectance images
JP2017522970A (en) * 2014-07-30 2017-08-17 ノバルティス アーゲー Biostain visualization in ophthalmic surgical procedures and related devices, systems, and methods
EP3128940A4 (en) * 2014-07-30 2017-05-31 Novartis AG Vital stain visualization in ophthalmic surgical procedures and associated devices, systems, and methods
CN107155302A (en) * 2014-07-30 2017-09-12 诺华股份有限公司 Vital stain visualization and relevant device, system and method in ophthalmic surgery
US10702353B2 (en) 2014-12-05 2020-07-07 Camplex, Inc. Surgical visualizations systems and displays
US11154378B2 (en) 2015-03-25 2021-10-26 Camplex, Inc. Surgical visualization systems and displays
US10852517B2 (en) 2015-06-02 2020-12-01 National University Corporation Asahikawa Medical University Observation assisting device, information processing method, and program
US11719920B2 (en) 2015-06-02 2023-08-08 Leica Microsystems (Schweiz) Ag Observation assisting device, information processing method, and program
US10966798B2 (en) 2015-11-25 2021-04-06 Camplex, Inc. Surgical visualization systems and displays
EP3940436A1 (en) * 2016-02-15 2022-01-19 Leica Instruments (Singapore) Pte. Ltd. Illumination filter system and observation system for a multispectral fluorescence microscope, multispectral fluorescence microscope, and microscopying method
US11921275B2 (en) 2016-02-15 2024-03-05 Leica Instruments (Singapore) Pte. Ltd. Illumination filter system and observation system for a multispectral fluorescence microscope, multispectral fluorescence microscope, and microscopying method
EP3206061A1 (en) * 2016-02-15 2017-08-16 Leica Instruments (Singapore) Pte. Ltd. Illumination filter system and observation system for a multispectral fluorescence microscope
US10823947B2 (en) 2016-02-15 2020-11-03 Leica Instruments (Singapore) Pte. Ltd. Illumination filter system and observation system for a multispectral fluorescence microscope, multispectral fluorescence microscope, and microscopying method
WO2018125940A1 (en) * 2016-12-27 2018-07-05 Urugus S.A. Dynamic hyper-spectral imaging of objects in apparent motion
CN110114646A (en) * 2016-12-27 2019-08-09 优鲁格斯股份有限公司 The dynamic Hyper spectral Imaging of object in apparent motion
JP7045379B2 (en) 2016-12-27 2022-03-31 ウルグス ソシエダード アノニマ Dynamic hyperspectral imaging of objects in phantom motion
JP2020506373A (en) * 2016-12-27 2020-02-27 ウルグス ソシエダード アノニマ Dynamic hyperspectral imaging of objects in apparent motion
US10502621B2 (en) 2016-12-27 2019-12-10 Urugus S.A. Dynamic hyper-spectral imaging of objects in apparent motion
RU2641519C1 (en) * 2017-03-29 2018-01-17 Общество с ограниченной ответственностью "Медицинские видеосистемы" (ООО "МедВис") Method of quantitative estimation of photosensitizer concentration on video image in real time when conducting fluorescent research
US10918455B2 (en) 2017-05-08 2021-02-16 Camplex, Inc. Variable light source
RU2672478C1 (en) * 2017-08-03 2018-11-15 Федеральное государственное бюджетное образовательное учреждение высшего образования "Оренбургский государственный университет" Method of optical diagnostics of pathologies in biological tissues
CN108009550A (en) * 2017-11-09 2018-05-08 深圳大学 High spectrum image feature detection method and device based on curve of spectrum fitting
CN108009550B (en) * 2017-11-09 2021-01-22 深圳大学 Hyperspectral image characteristic detection method and device based on spectral curve fitting
WO2019164455A1 (en) * 2018-02-21 2019-08-29 Optotek D.O.O. Image-processing apparatus and image-processing method for detection of irregularities in tissue
EP3671306A1 (en) * 2018-12-21 2020-06-24 Leica Instruments (Singapore) Pte. Ltd. Image-processing device, fluorescence observation device and method for emulating a first type of fluorescence observation device on a second type of fluorescence observation device
EP4239390A3 (en) * 2018-12-21 2023-11-15 Leica Instruments (Singapore) Pte Ltd Fluorescence observation device and method for emulating a first type of fluorescence observation device on a second type of fluorescence observation device

Also Published As

Publication number Publication date
US20140378843A1 (en) 2014-12-25
US11656448B2 (en) 2023-05-23
US20190384048A1 (en) 2019-12-19

Similar Documents

Publication Publication Date Title
US11656448B2 (en) Method and apparatus for quantitative hyperspectral fluorescence and reflectance imaging for surgical guidance
US11857317B2 (en) Method and apparatus for quantitative and depth resolved hyperspectral fluorescence and reflectance imaging for surgical guidance
US8948851B2 (en) Method and apparatus for depth-resolved fluorescence, chromophore, and oximetry imaging for lesion identification during surgery
US20180177399A1 (en) Method and device for multi-spectral photonic imaging
CA2658811C (en) Multi modal spectroscopy
BR112020015757A2 (en) DEVICES, SYSTEMS, AND METHODS FOR VIEWING AND REMOVING TUMOR
JP6247316B2 (en) Imaging system with hyperspectral camera guide probe
US20080306337A1 (en) Characterization of a Near-Infrared Laparoscopic Hyperspectral Imaging System for Minimally Invasive Surgery
US20120302892A1 (en) Portable optical fiber probe-based spectroscopic scanner for rapid cancer diagnosis
US20110042580A1 (en) Fluorescence quantification and image acquisition in highly turbid media
EP2583617A2 (en) Systems for generating fluorescent light images
WO2009052607A1 (en) Method and apparatus for microvascular oxygenation imaging
JP2008522761A (en) Systems and methods for normalized fluorescence or bioluminescence imaging
EP2661615B1 (en) An apparatus for optical analysis of an associated tissue
Turchin Methods of biomedical optical imaging: from subcellular structures to tissues and organs
WO2013103475A2 (en) Portable optical fiber probe-based spectroscopic scanner for rapid cancer diagnosis
Pratavieira et al. Optical imaging as auxiliary tool in skin cancer diagnosis
US20230280577A1 (en) Method and apparatus for quantitative hyperspectral fluorescence and reflectance imaging for surgical guidance
RU2497558C1 (en) Intraoperative combined spectroscopic diagnostic technique for cerebral and spinal tumours
KR102521793B1 (en) Apparatus and method for determining the depth of an epifluorescent object in an optical absorption and scattering medium and for determining the fluorescence concentration of an object
Carver et al. Real‐time detection of breast cancer at the cellular level
RU2528087C1 (en) Device for measuring haemoglobin concentration and degree of blood oxygenation in mucous membranes
Savelieva et al. Combined spectroscopic and video fluorescent instrument for intraoperative navigation when removing a glial tumor
WO2012127378A1 (en) An apparatus for optical analysis of an associated tissue sample
Savelieva et al. Combined Video Analysis of ICG and 5-ALA Induced Protoporphyrin IX and Hemoglobin Oxygen Saturation in near Infrared.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13738517

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 14373443

Country of ref document: US

122 Ep: pct application non-entry in european phase

Ref document number: 13738517

Country of ref document: EP

Kind code of ref document: A1