WO2016020684A1

WO2016020684A1 - Multiplexed optical tomography

Info

Publication number: WO2016020684A1
Application number: PCT/GB2015/052272
Authority: WO
Inventors: Paul French; James Mcginty
Original assignee: Imperial Innovations Limited
Priority date: 2014-08-05
Filing date: 2015-08-05
Publication date: 2016-02-11
Also published as: GB201413872D0

Abstract

An optical tomography system comprising a support arranged to support an object, and an imaging system comprising an objective lens (70) arranged to receive light travelling from the object (62) over a range of directions, detection means (66) arranged to detect the light and means (68 ) defining at least one aperture arranged to define a plurality of optical paths between the objective lens and the detection means whereby the detection means can detect image s of the object from a plurality of different viewing directions.

Description

Multiplexed Optical Tom ography

Field of the Invention

The present invention relates to three-dimensional imaging systems, and in particular to optical tomography systems, for example for imaging mesoscopic biological samples .

Backgroun d to the Invention

As biological research progresses from studies of mono-layers of cells on glass to in situ measurements of both ex vivo and in vivo biological systems , it becomes necessary to apply three-dimensional (3 -D) imaging techniques in order to map structure and function throughout a sample . Confocal/multiphoton/harmonic generation laser scanning microscope s provide optical sectioning to p ermit the acquisition of 3 -D z-stacks (stacks of planar images) and also offer improved contrast compared to wide-field imaging but they typically suffer from limited ( 100 ' s μιη) penetration depth and fields of view ( l O ' s μιη) and exhibit anisotropic resolution. Thus, while they are widely used to image microscopic specimens, they are less suitable for larger samples for which the acquisition of 3 -D data sets can be very time consuming . To addre ss this challenge, a number of imaging technique s have been developed for relatively transparent sample s in the "meso scopic" regime ( 1 - 10 mm) , including optical proj ection tomography (OPT), selective plane illumination microscopy (SPIM) and ultramicro scopy . Optical tomographic imaging can also be applied to more opaque scattering samples, albeit with reduced image resolution, for which spectroscopic techniques, structured illumination and/or appropriate mathematical tools can be employed to addre ss the issue of optical scattering and scattered light can also be used to provide information about the sample for the reconstruction of tomographic images . OPT is the optical equivalent of X-ray computed tomography (CT), in which the 3 -D structure (a stack of X-Z slices) of a rotating sample is reconstructed from a serie s of wide-field 2-D proj ections (X-Y images) obtained at different proj ection angles . Typically, digital image s are acquired throughout a full rotation (360°) and a filtered back-proj ection (FBP) algorithm is used for image reconstruction . This approach assume s "parallel proj ection" corresponding to parallel ray (or plane wave) propagation of the signal with negligible scattering in the sample, which is appropriate for X-ray CT, but optical scattering can be a significant issue when imaging in biological tissue . Reconstructed OPT images can be degraded by a scattered light background unless the sample s are inherently transparent or have been rendered transparent by a chemical clearing process .

OPT has been widely applied to anatomical studies of fixed, cleared sample s such as mouse embryos for research into developmental biology . However it would potentially be beneficial to apply it to histopathology and the study of disease mechanisms and potential therapies in disease models . OPT image s can be formed using transmitted light, e . g . to map absorption coefficients, or using fluorescence radiation . Figure 1 represents a transmission OPT system in which an optical light source 10 is located on one side of a sample chamber 12 and arranged to direct light towards the chamber 12, and a detector array 14, such as a CCD detector array, is located on the opposite side of the chamber to the source 10 and arranged to detect light from the source that is transmitted through the sample imaging chamber 12 and through the sample 13 located in the chamber. Figure 2 represents a fluorescence OPT system in which the source 20 is located on one side of the sample chamber 22 and a detector array such as a CCD array 24 is located away from the axi s along which light is transmitted through the source, and arranged to detect light emitted as fluorescence from the sample chamber 22. In each case the system include s a means, such as a rotating sample holder, to rotate the sample inside the imaging chamber 12, 22, so as to rotate the sample 13 , 23 between a number of orientations to allow images to be acquired by the detector array 14, 24, for each of a number of proj ections . The transmitted light or fluorescence radiation can be characterised to provide spectroscopic information, e . g . spectrally resolving the transmitted light or resolving fluore scence radiation with respect to excitation and emission spectra, fluore scence lifetime and/or polarisation . One possible application is to utilise fluorescence lifetime imaging (FLIM) to provide a spectroscopic readout for OPT. For histopathology, OPT offers the opportunity to directly obtain 3-D images of intact "volumetric" samples rather than the standard approach of mechanically slicing the samples and combining digital images of each section to reconstruct 3-D images. This is important because mechanical "sectioning" can damage fragile samples. Absorption contrast can arise from endogenous chromophores, including blood, and from exogenous labels or stains, e.g. the standard H&E stains. Fluorescence contrast, including spectral or fluorescence lifetime contrast, can arise from endogenous fluorophores, such as elastin, collagen, NADH, flavoproteins etc., or from exogenous labels including dyes or genetically expressed fluorescent proteins - although the fluorescence properties of the latter can be degraded when certain chemical clearing processes are employed. The autofluorescence can sometimes be used, e.g. by using spectroscopic parameters such as fluorescence lifetime, to provide a label-free readout of the state of biological tissue, e.g. to indicate disease or damage, or to contrast different types of tissue.

For studying disease and for drug discovery, there is an increasing interest in translating studies of biological processes at the cellular level from monolayers (or very thin layers a few cells thick) of cell cultures on coverslips that are easily imaged using established microscopy techniques to 3-D cell or tissue cultures or to live organisms that are more suitable for mesoscopic imaging approaches. The chemical clearing process is inherently fatal to live organisms and so it is interesting to apply OPT and other optical imaging techniques to inherently transparent live organisms - particularly those that can be genetically manipulated to serve as disease models. To date OPT has been applied to D. melanogaster [C. Vinegoni, C. Pitsouli, D. Razansky, N. Perrimon, V. Ntziachristos, "In vivo imaging of Drosophila melanogaster pupae with mesoscopic fluorescence tomography ," Nat. Meth.5, 45-47 (2008)], C. elegans [U. J. Birk, M. Rieckher, N. Konstantinides, A. Darrell, A. Sarasa-Renedo, H. Meyer, N. Tavernarakis, J. Ripoll, "Correction for specimen movement and rotation errors for in-vivo optical projection tomography," Biomed. Opt. Exp. 1, 87-96 (2010] and D. rerio (zebrafish) embryos [J. McGinty, H. B. Taylor, L. Chen, L. Bugeon, J. R. Lamb, M. J. Dallman, P. M. W. French, "In vivo fluorescence lifetime optical projection tomography," Biomed. Opt. Express 2, 1340-1350 (2011)]. As well as imaging the spatial-temporal distribution of fluorescent labels (e .g . fluore scent proteins that are labelling specific proteins of intere st), it is als o possible to study the interactions of biomolecules and this can be done using Forster resonant energy transfer (FRET), which can be read out using FLIM [S . Kumar et al . FLIM FRET Technology for Drug Discovery : Automated Multiwell-Plate High-Content Analysis, Multiplexed Readouts and Application in Situ . ChemPhysChem 12 : 609-626 (20 1 1 )] .

The potential to apply OPT to "meso scopic" sample s (i . e . mm-cm scale) for biomedical research has prompted significant interest in optimizing the image quality and re solution and minimizing the image data acquisition time . Image quality can be degraded by artifacts resulting from system misalignment, intensity-based signal variations and system aberrations and methods have been described to correct or suppress such artifacts . Two fundamental limits that can restrict the application of OPT are imaging speed and spatial resolution . As has been established with x-ray computed tomography, a minimum number of angular proj ections are required to adequately sample the subj ect for tomographic reconstruction . For OPT of mm-cm sample s, this is in some cases approximately 360 proj ections (i . e . angularly spaced by one degree), which implies a total image acquisition time of 360 x the time for a single image acquisition, which can vary from ms to seconds .

The image acquisition time can be particularly extended for FLIM OPT where a series of time-gated fluorescence intensity images may be acquired at each angular proj ection as shown in Figure 3 . In such a system a light source in the form of an ultrashort pulsed laser 30 is arranged to direct pulses of light towards a sample chamber 22, in this case via a mirror 34. The image acquisition system comprises a gated optical intensifier (GOI) 38 located in the imaging path in front of the detector array 36 with a filter 32 between the GOI and the chamber 22. This arrangement forms time-gated fluore scence images from a sample supported on a rotatable holder within the sample chamber 22. The GOI is arranged to relay the fluorescence image to the detector array 36 only during short imaging periods . A delay generator 40 , controlled by a computer 42, is arranged to switch the GOI 38 in synchronism with the light source 30 to generate a serie s time-gated images acquired at specific delays after the excitation pulse s reach the sample . For each of the imaging periods the computer 42, which is connected to the output of the CCD array 36, is arranged to store a s et of image data. Therefore a series of image data sets is built up corre sponding to the fluore scent images recorded at different delay times after the laser excitation pulse . This data can be used to generate FLIM OPT images as is well known .

For spectrally resolved OPT, it is usually necessary to acquire multiple spectrally-resolved images for each angular proj ection, e . g . by imaging through different spectral filters . Thus spectral OPT can increase the total image acquisition time by the number of spectral channels de sired .

It is possible to reduce image acquisition time for a FLIM OPT systems and for other implementations of OPT by reducing the number of angular proj ections but this compromises the reconstructed tomographic image quality and the distortion can become s significant for data sets with less than about 90 proj ections . As has been established with x-ray computed tomography, it is sometime s possible to reconstruct reasonable tomographic images with fewer proj ections using iterative algorithms but still tens of angular proj ections are typically required . In general it is desirable to minimize the image acquisition time for experimental convenience, to b e able to resolve dynamics and also to minimize the exposure of the sample to optical radiation, which can result in photobleaching of fluorophore s and phototoxicity .

Ultimately it would be desirable for many applications to acquire all the optical proj ection data simultaneously, i . e . in a "single-shot" acquisition with no rotation of the sample required . This could be important to image live samples such as growing plants or organisms that are not convenient to rotate . Conventionally this is difficult to achieve since it would require tens of parallel imaging channels . While this could conceivably be realised with a corresponding number of imaging paths and cameras, it would be extremely complex and expensive and technically challenging to implement. To some extent this could be mitigated by using image splitters in front of the cameras to record multiple images on each camera and thereby reduce the number of cameras required for a given number of imaging channels but the cost and complexity of such an implementation would remain significant. If one could reduce the complexity and number of required imaging paths and cameras to a practical level, one could envisage acquiring 3 -D volumetric OPT images of stationary sample s such as growing plants or animals or one could envisage rotating the imaging paths and cameras around a stationary sample .

Image quality can also be degraded by deviations from the parallel ray assumption that underlies the standard FBP algorithm . These arise when OPT is implemented with relatively high numerical aperture (NA) optics, for which rays at a relatively large range of angles with respect to the optical axis are collected. Figure 4 shows the approximate relationship between the depth of field (D OF) and the numerical aperture NA; in particular a higher NA results in a lower D OF . Relatively high NA optics are necessary for producing high resolution images of small sample s and are generally desirable for fluore scence imaging because the light collection efficiency increases with numerical aperture . There is a trade-off between increasing the NA - to improve the in-focus lateral resolution and light collection efficiency - and reducing the NA to increase the depth of field (D OF) in order to ensure that the whole sample is in reasonable focus (i . e . that the lateral resolution doe s not vary significantly along the optical axis) . Figure 4 shows the limiting case (sketched for a single resolution element) when the depth of field of the imaging system is comparable to the diameter of the sample . In this case the tomographic image of the whole sample is reconstructed from approximately plane wavefronts as expected for back proj ection.

When OPT is undertaken with sample s that extend beyond the depth of field of the obj ective lens - as is often the case - the tangential resolution of the reconstructed images typically decreases radially away from the axis of rotation . Figure 5 (a) however shows the case when the DOF is matched to the radius of the sample - in this case all of the sample will be "in focus" for part of its revolution and an image of approximately uniform spatial re solution can still be reconstructed . For the case illustrated in Figure 5 (b), however, the DOF is less than the sample radius and the reconstructed spatial resolution will be decreased (and the image degraded) away from the focal plane . This situation is typical for many biomedical applications where high re solution (from relatively high NA optics) is required but the sample size (e . g . a zebrafish embryo) is much greater than the DOF . For imaging zebrafish in an OPT microscope with a NA of -0.07, which corre sponds to a depth of field of -540 λ in water (-270 μιη for a wavelength of 500 nm) the spatial resolution in the focal plane is -4.4 μιη but this is degraded away from the focal plane . Since a zebrafish embryo is typically - 1 mm in diameter, the spatial resolution therefore varie s significantly across the sample .

One way to address this issue and achieve a uniform image resolution throughout a sample that is larger than the DOF of the imaging system is to translate the sample with re spect to the focal plane such that different portions of the sample are sequentially imaged "in focus" . In some implementations, however, this approach can increase the total image acquisition time and it generally increases the total light exposure for each tomographic image acquisition . It also adds expense and complexity because of the additional moving parts compared to the single axis rotation of OPT.

In a previous international patent application published as WO20 1 3/ 132257 we proposed that the issues of spatial resolution and imaging speed could b e ameliorated by appropriately multiplexing the imaging readouts in a fluorescence-based OPT system . Specifically, we proposed that angularly multiplexed imaging channels could reduce the data acquisition time of OPT for a given number of optical proj ections by acquiring the image data for more than one proj ection simultaneously and that, if different imaging channels were focused to different depths in a sample , the trade-off between resolution and sample thickne ss could be overcome . We also proposed that image s acquired simultaneously at different proj ections could be used to locate specific feature s in 3 -D space and therefore the position of such features could be tracked with a time resolution limited by the frame rate of the cameras rather than the acquisition time for the OPT data set. We further proposed that the image data from different angular proj ections could be simultaneously acquired on the same camera using mirror assemblie s or optical fibre bundle s to convey the image information from different imaging channels, defined by different imaging obj ective lense s, to different regions of the imaging sensor. While the ideas proposed in WO20 13/ 132257 do addre ss the issues of spatial resolution and image acquisition time for OPT, they still require rotation of the sample and the multiple imaging channels each require separate obj ective lenses and separate imaging paths . The relaying of multiple image s to a single camera sensor is possible but cumbersome and it would be difficult to implement more than a few imaging channels because of the difficulty in packing multiple obj ective lenses around the sample .

Sum m ary of the Invention

The present invention provides an optical tomography system, which may b e an optical proj ection tomography system, comprising a support arranged to support an obj ect, and an imaging system comprising an obj ective lens arranged to receive light emitted from the obj ect over a range of directions, detection means arranged to detect the light, and means defining at least one aperture arranged to define a plurality of optical paths between the obj ective lens and the detection means whereby the detection means can detect image s of the obj ect from a plurality of different viewing directions . The at least one aperture may be located between the obj ective lens and the detection means, and may be in the back focal plane of the obj ective lens . Alternatively the system may comprise a condenser lens and the at least one aperture may be located on the back focal plane of the condenser lens . The imaging system may further comprise at least one optical component, for example a tube lens or lenses, arranged to focus light from the obj ective lens, for example onto the detection means . The obj ective lens may be arranged to focus the light from the obj ect at infinity . The detection means may comprise a single imaging detector or an array of detectors . The detection means may have a plurality of areas, which in an array of detectors may comprise a plurality of sub-arrays, each arranged to receive light transmitted along a respective one of the optical paths . The detected images may be formed by a single tube lens or by a number of lense s or other optical components . The imaging system may comprise a plurality of imaging or 'tube ' lenses one in each optical path forming a 'multiplexed imaging channel ' . The image s recorded may corre spond to different viewing directions . Each of the tube lenses may be arranged to focus an image onto the detection means, for example onto a respective one of the sub -arrays . Alternatively the system may further comprise an image relay lens or lenses arranged to relay the images formed by the tube lense s on to the detection means . Each of the optical paths and the lenses through which the optical paths pass, can b e considered as forming respective imaging sub-channels, each arranged to image the obj ect from a re spective viewing direction .

Therefore the imaging system may comprise a first imaging system or lens or set of lenses arranged to image the obj ect from a plurality of viewing directions to an intermediate image plane , and a second imaging system or lens or set of lenses arranged to magnify or de-magnify this plurality of images from different viewing directions onto the imaging detector.

The support may be arranged to rotate the obj ect about an axis, or it may b e stationary .

The viewing directions may be angularly spaced around the axis .

The realisation of angularly resolved imaging channels using a common obj ective lens is reminiscent of common-main-obj ective typ e stereomicroscopes although that application is quite distinct from what is proposed here . In stereomicroscopy, the goal is to acquire two angularly resolved images to provide depth cues when viewing a 3 -D scene, mimicking human vision . The present invention concerns optical proj ection tomography which requires the acquisition of many angularly resolved 2-D image s sampling throughout 1 80 or 360 rotation in a plane around a sample to provide the set of angular proj ections to required reconstruct a complete 3 -D map of optical properties of a sample using filtered back proj ection or another tomographic reconstruction algorithm . The system may further comprise data acquisition means arranged to acquire a plurality of sets of angularly-resolved image data from each of the imaging systems . Where there are a plurality of multiplexed imaging channels spaced around the sample, the data acquisition means may be arranged to acquire image s from each of the viewing directions of each of the multiplexed imaging channels simultaneously . If there is only one multiplexed imaging channel, the support means may be arranged to rotate the obj ect between a plurality of orientations and the data acquisition means may be arranged to acquire at least one data set, or one data set from each multiplexed imaging channel, for each of the orientations . The data acquisition means may be arranged to acquire a data set from each of the viewing directions simultaneously, or in succes sion, for each of the orientations . The angular offset or spacing between the viewing directions about the axis of rotation may be an integer multiple of the angular spacing between the orientations, so that as the obj ect is rotated each of the optical paths can be used to generate image data sets from the same direction relative to the obj ect. Alternatively the angular offset or spacing between the viewing directions about the axis may be an integer multiple of the angular spacing between the orientations plus a fraction, such as a half, of that angular spacing , so that as the obj ect is rotated the different optical paths can be used to generate image data sets from directions which are angularly spaced relative to the obj ect more closely than, for example at half of, the angular spacing between the viewing directions .

The plurality of imaging channels may be focussed at respective focal points or planes which are equidistant from the axis of rotation of the obj ect. However, the focal points or plane s may be at different distances from the axis of rotation . This means that as the obj ect is rotated, different parts of it will be imaged in focus by the two (or more) imaging channels or subchannels . This could be implemented, for example, by arranging that the tube lens in each imaging sub -channel may be adj ustably mounted so that its position can be adj usted along its optical axis .

The system may further comprise proce ssing means, such as a processor, arranged to receive the image data sets and process them to generate a further image data set, which may be a tomographic or three-dimensional image data set.

The system may be an optical proj ection tomography system, and for example may be a fluore scent imaging system . However it may be transmission imaging system, or even a scattering imaging system. In each case the system may further comprise a source of radiation which may be detected after transmission through, or scattering in, the obj ect, or which may cause the fluorescence which is then detected .

The system may further comprise a sample chamber. The support means may be arranged to support the sample within the chamber. The chamber may have a wall, part of which may be formed by the obj ective lens which also forms part of a plurality of the optical multiplexed imaging channel . Indeed each of the optical multiplexed imaging channels may include a lens which forms part of the wall of the chamber.

The chamber may be filled with an index matching fluid having a refractive index similar to that of the sample .

The invention further provides an optical tomography system comprising a support arranged to support an obj ect, and an imaging system comprising an obj ective lens arranged to receive light travelling from the obj ect over a range of directions, detection means arranged to detect the light and means defining at least one aperture or localised light source in the back focal plane of the condenser lens arranged to define a plurality of optical paths between the condenser lens and the sample whereby the sample can be illuminated from a plurality of different viewing directions . Some embodiments of the invention may permit the use of multiple simultaneous imaging directions using a single obj ective lens, so that the directions can be more closely spaced than would be possible using multiple conventional obj ective lenses . The system may further comprise any one or more features, in any combination, of the embodiments of the invention that will now be de scribed by way of example only with reference to the accompanying drawings .

Brief Description of the Drawin gs

Figure 1 is a schematic view of a transmission optical tomographyimaging system forming part of an embodiment of the invention ;

Figure 2 is a schematic view of a fluorescence optical tomography system forming part of an embodiment of the invention;

Figu re 3 is a schematic view of a fluorescent lifetime imaging system forming part of an embodiment of the invention;

Figu re 4 is a diagram showing the depth of field and numerical aperture in an optical tomography system ;

Figures 5a and 5b are diagrams of different sized sample s in an optical tomography system ;

Figure 6 is a diagram of an optical tomography system according to an embodiment of the invention;

Figure 7 is a diagram of an optical tomography system according to a further embodiment of the invention;

Figure 8 is a diagram of an optical tomography system according to a further embodiment of the invention;

Figure 9 is a diagram of an optical tomography system according to a further embodiment of the invention;

Figu re 10 is a diagram of an optical tomography system according to a further embodiment of the invention;

Figu re 11 is a diagram of an optical tomography system according to a further embodiment of the invention;

Figu re 12 is a diagram of an optical tomography system according to a further embodiment of the invention;

Figu re 13 is a diagram of an optical tomography system according to a further embodiment of the invention;

Figu re 14 is a diagram of an optical tomography system according to a further embodiment of the invention; Figu re 15 is a diagram of an optical tomography system according to a further embodiment of the invention;

Figure 16 is a diagram of an optical tomography system forming the basis of further embodiments of the invention;

Figu re 17 is a diagram of an optical tomography system according to a further embodiment of the invention;

Figu re 18 is a diagram of an optical tomography system according to a further embodiment of the invention;

Figu re 19 is a diagram of an optical tomography system according to a further embodiment of the invention;

Figu re 19a is a top view of part of the system of Figure 19 ; and

Figu re 20 is a diagram of an optical tomography system according to a further embodiment of the invention . Description of Em bodiments of the Invention

Referring to Figure 6 an imaging system according to an embodiment of the invention comprises a light source (not shown) and a sample chamber (not shown for clarity in this and subsequent figure s) having a rotatable sample holder for supporting a sample 62 and arranged to rotate the sample about an axis of rotation Z . A detector array 66 in the form of a camera (which could be a CCD camera or another type of camera) is arranged to detect fluore scence emitted from the sample 62. The array can be considered as made up of a number of sub-arrays 67, in this case three, each of which is arranged to detect emissions travelling to the array 66 from different angularly separated optical paths . This multiplexed imaging channel forms part of an OPT system which is otherwise the same as that of Figures 1 to 3 and will not be described in detail here . It will be appreciated that the same imaging system could b e incorporated into other type s of OPT system .

Each of the detector sub-arrays 67 records an image from its own optical system defining a respective optical path or imaging sub -channel, in each case comprising a respective imaging lens or 'tube lens ' 63 and a respective aperture 68 in this case formed as an opening in an opaque aperture member. A single obj ective lens 70, having a focal length fj forms part of all three imaging sub -channels and is located between the apertures and the sample chamber so that the aperture s 68 all lie in its back focal plane, which is als o known as the Fourier plane . The tube lense s 63 , which have a focal length f₂ are also all located in a common plane which is parallel to the Fourier plane of the obj ective lens 70 and to the plane of the detector array 66. They are located so that the apertures 68 also lie on the focal plane of the tube lenses, and so that the detector array lies on the back focal plane of the tube lense s . For example , light emitted from the on-axis focal point of the obj ective lens over a range of angles will pass through the obj ective lens, and be refracted onto parallel paths perpendicular to the Fourier plane and through the apertures 68 , and then be imaged onto a respective detector sub-array to contribute to the images of the obj ect. Each of the imaging sub-channels will transmit light emitted from the obj ect at a respective angle (or in fact range of angles) , centred on a respective viewing direction . In this embodiment, the three detector sub -arrays 67 will therefore receive light emitted from the obj ect in three respective different viewing directions . Light travelling along each of the imaging sub-channels is focused on to a respective one of the detector sub-arrays 67 to form a respective image . Therefore the detector array 66 can detect images from three different viewing angle s simultaneously .

The tube lense s 63 in this embodiment can be separate lenses or formed as an integrated array of lenses parallel to the focal plane . The detector array is then also divided into a corresponding array of sub-arrays . This could provide a set of imaging directions which are angularly spaced apart in two different directions, one parallel and one perpendicular to the axis of rotation of the sample Z . This could provide additional information that could be utilised using more sophisticated reconstruction algorithms than simple FBP, including iterative algorithms . This approach could be applied to aid tomographic reconstruction using scattered light.

The use of a microlens array near the back focal plane of the obj ective has previously been proposed for light field microscopy where the goal is to provide 3 -D information over a single field of view of a sample [T. G . Georgiev, US patent US 7, 872, 796 B2] and indeed the use of multiple angularly resolved imaging channels is reminiscent of some 3 -D imaging approaches described as "light field imaging" or "integral photography" or "integral imaging", particularly using a "plenoptic camera" incorporating an array of microlense s . However, these technique s concern depth-resolved imaging of surface s in a scene from a single viewpoint, e . g . to enable retrospective focussing, unlike the present invention which concerns acquiring 2-D images of the sample from many directions spanning 360 degrees rotation followed by reconstruction of the complete 3 -D image (i . e . including internal optical properties) using filtered back proj ection or another tomographic image reconstruction technique . Thus the image acquisition and data processing of the present invention, which is an extension of OPT, are quite distinct from light field imaging or integral photography .

In a further modification to the embodiment of Figure 6 , the tube lense s 63 are located on the back focal plane of the obj ective lens 70. The tube lenses 63 can then themselve s be integrated with the apertures that define the separate imaging channels .

The imaging system of Figure 6 can be de scribed as a multiplexed imaging channel and multiple such imaging channels, each with their own obj ective lens and detector array, can be combined in an OPT system to provide multiple sets of angularly resolved imaging sub-channels, as represented in Figure 7. If each imaging channel is focused at a different depth in the obj ect, this can be used to increase the numbers of imaging depths in one system . Alternatively the imaging channels or sub -channels can be arranged to image the same depth in the sample but could select light of different wavelengths (e . g . by using a filter) to provide spectrally resolved imaging channels . This can be used to decrease the required time to acquire spectrally resolved OPT data.

Referring to Figure 7, in a further embodiment of the invention, three multiplexed imaging channels 160, each identical to that of Figure 6 with corresponding parts indicated by the same reference numerals increased by 100, are spaced, in this case equally, around a sample chamber. As each multiplexed imaging channel 160 can image the obj ect from three different viewing directions, the whole system can image it from nine different directions simultaneously . Obviously the exact number of imaging directions in each system and the number of imaging systems can be varied as appropriate . This approach can be used to realise a One-shot' OPT imaging system in which the obj ect does not need to be rotated as it can be imaged from a large number of directions, preferably simultaneously, but optionally sequentially by moving the imaging channels . This system can be used for sample s for which rotation is impractical, such as growing plants or live animals or intact tissues . The system can also be used to capture 3 -D fluorescence distributions at the frame rate of the cameras and be applied to dynamic samples, including specimens passing through a channel in the (vertical) z axis perpendicular to the plane of the diagram in Figure 7. Thus this can be used to implement 3 -D imaging in flow cytometry or for high throughput 3 -D imaging of sample s such as tumour spheroids or rapid imaging of small organisms such as nematode worms or drosophila. The entire OPT configuration can be scanned vertically to map the 3 -D distribution of a long obj ect such as a plant stem .

If the arrangement of Figure 7 does not provide sufficient angular proj ections for the desired OPT image quality, it is possible to combine this approach with some rotation of either the sample with respect to the OPT set-up or vice versa. This can still enable relatively rapid OPT.

The optical configuration in Figure 6 is one possible configuration to provide multiple angularly separated imaging sub -channels using a single obj ective lens but it may not be the most convenient, for example in terms of magnification . Figures 8 to 10 show three further embodiments, each having multiple angularly multiplexed imaging sub-channels, using a single obj ective lens, which can provide flexibility in terms of magnification and other imaging properties . In each case a single multiplexed imaging channel is shown, and the systems are the same as that described in Figure 6 unle ss otherwise described, and corre sponding components are indicated by the same reference numbers increased by 200, 300 and 400 respectively . There are clearly other variations and combinations of these approache s that could also be used .

In the embodiment of Figure 8 , the obj ective lens 270 , aperture s 268 , and tube lenses 263 are arranged as in the embodiment of Figure 6. However the CCD camera including the detector array 266 is further away from the tube lense s 263 . A further image relay lens 280 is located between the back focal plane 282 of the tube lense s and the detector array 266, and is arranged to focus images from all of the three imaging sub -channels onto the plane of the detector array 266. The arrangement in Figure 8 can be adapted to provide different imaging channels focusing at different depths in the sample by translating the tube lenses along their optical axe s so that they are at different distances from the focal plane of the obj ective lens 270. To achieve this, the tube lenses 263 can be adjustably mounted so that their positions can each b e adjusted independently along their optical axes .

Referring to Figure 9, in a further embodiment, the configuration of Figure 8 can be further extended by replacing the image relay lens 380 with a pair of lenses 380 , 384, one behind the other. The lens 380 closer to the tube lense s is adj ustably mounted so that its position can be adj usted along its optical axi s . The arrangement in Figure 9 can also be adapted to realise different imaging sub-channels focusing at different depths in the sample by translating the tube lenses , which are also adj ustably mounted, along their optical axe s as with the embodiment of Figure 8. By translating lens 380 along its optical axis, all the imaging channels can be adjusted together to focus to the same adj ustable depth in the sample .

The arrangements in Figure s 6-9 relay all the multiplexed imaging subchannels associated with a specific obj ective lens to the same detector array i . e . the same camera sensor. It is also possible to use multiple camera sensors in a common image plane, one for each imaging channel . However, referring to Figure 10, in a further embodiment a separate camera is used for each imaging sub -channel and the imaging plane s, i . e . the planes of the detector arrays 466 of the three cameras, are not coincident, but facing in different directions, in this case being at right angle s to each other. This is achieved by providing one camera detector array 466 and a pair of imaging lenses 480, 484 for the central imaging channel that are on the optical axis of the obj ective lens, as in the system of Figure 9. However, in the two outer imaging channels, a mirror 486 is placed in the optical path between the tube lens 463 and the two imaging lense s 480, 484 so that the optical path is turned through 90° . This approach has the advantage that the OPT image s can be acquired with a large number of camera pixels, thereby potentially increasing the achievable spatial resolution of the OPT reconstructed 3 -D image . All of the optical arrangements of Figure s 6 to 1 0 can be made more complex to achieve similar imaging functionality but minimise optical aberrations such as field curvature , spherical aberration and chromatic aberration . To deal with optical aberrations, these multiplexed OPT set-ups can also incorporate adaptive optics elements such as spatial light modulators (SLM) or segmented mirrors (SM) . This might be conveniently implemented by locating the se adaptive optics elements in the regions of collimated rays ("infinity space "), e . g . between the obj ective and tube lense s 370, 363 or between the two imaging lense s 380, 384 in Figure 9. The se adaptive optics elements can als o be used to realise focusing of different angularly multiplexed imaging sub - channels to different depths in the sample .

It is also possible for some implementations to replace the array of tube lenses by an adaptive optics element such as a segmented mirror to simulate the function of the array of lense s . Such an element can also incorporate compensation of optical aberrations in the system and provide focusing of different imaging channels to different depths in the sample .

In Figure 1 1 a further embodiment is shown . Features corresponding to thos e in Figure 10 are indicated by the same reference numerals increased by 100. This numbering convention is used for the further embodiments de scribed below with the numbers increasing by a further 100 for each embodiment. In each case the features are the same unle ss the de scription or drawings indicate otherwise . Referring to Figure 1 1 , the systems of Figure s 6 and 7, and indeed those of Figure s 8 , 9 and 10, can also be used for imaging with transmitted light. In this case the obj ective lens 570, apertures 568 , tube lenses 563 and detector array 566 are the same as in the embodiment of Figure 6 , but the light source 5 10 is located on the optical axis of the obj ective lens 570 on the oppo site side of the sample chamber 522, and a condenser lens 526 is provided between the light source and the sample to relay light from the source 5 10 onto the sample from a range of directions . This form of transmitted light OPT can be implemented with incoherent light sources although it can also b e undertaken with laser illumination . In the latter case it may be necessary to addre ss the issue of speckle , in some embodiments using a rotating diffuser to time average the phase of the laser radiation across the field of view .

The use of multiple tube lense s, in the embodiments described above, to realise the simultaneous angularly separated imaging sub -channels require s the use of two or more relatively small lense s . This entails some complexity and presents optical de sign challenge s in terms of mounting them appropriately and minimising optical aberrations . An alternative approach is to provide multiple apertures after the obj ective and to use a single lens to form the corresponding multiple images on the detector array in conj unction with optical elements to provide angular separation of the simultaneous imaging sub-channels so that the images are formed on different parts of the detector array . In one embodiment (not shown) an array of prismatic elements can provide the required separation as has previously been discussed for integral photography [Georgiev, T . , Zheng, K. C , Curle ss, B . , Sale sin, D . , Nayar, S . , And Intwala, C . (2006) . Spatio-angular resolution tradeoff in integral photography . In EGRW ' 06 : Proc . the 17th Eurographics workshop on Rendering] . This can be applied to fluorescence OPT or to transmission OPT or to a combination of both . For example , the central image channel could utilise transmitted light while the two other channels could provide fluorescence images in different spectral windows selected by appropriate filters 688 inserted between the obj ective and tube lenses . Alternatively, the required separation of the simultaneous imaging sub-channels could b e provided by using mirrors at different angles of incidence to "fold" the optical imaging arm and reflect the different imaging channels at different angle s such that they do not overlap on the camera sensor. The use of prisms and other optical components to separate angularly resolved imaging channels to realise a plenoptic camera was recently described in an MSc thesis at Imperial College London [H. D. Young, MSc thesis : "Plenoptic Cameras " (201 3) Imperial College London] . However, this prior work concerned plenoptic depth-resolved imaging of 3 -D obj ects, rather than OPT with reconstruction of the complete 3 -D map of optical propertie s throughout the sample using filtered back proj ection or an alternative tomographic image reconstruction technique . The author does suggest applying the plenoptic camera to a rotating sample to increase the light detection efficiency by collecting light through multiple apertures but does not identify the different sub-image channels with different angular proj ections for OPT.

It is also possible to implement sequential acquisition of angularly multiplexed imaging sub-channels by scanning one or more apertures in the BFP of the obj ective lens . As illustrated in Figure 12, if a single aperture 668 is used and the aperture member is movably mounted so that it can be scanned across the BFP, the camera 666 can be arranged to sequentially acquire different angular proj ections without sample rotation . Alternatively, a spatial light modulator could be used to provide an electronically adj ustable aperture or multiple apertures that could be scanned to acquire different angular proj ections . The sequential acquisition of the angular proj ections enable s each of the images to be captured with the full pixel count of the camera sensor, i . e . using the same full area of the detector array for each proj ection, and so this approach can lead to higher resolution of the reconstructed OPT image than configurations where multiple angular proj ections are recorded on different areas of the same camera sensor.

Following the concept illustrated in Figure 7, this arrangement can b e implemented with a number of obj ective lense s 770 angularly spaced around the sample, each with its own set of imaging channels and detector array, as illustrated in Figure 13 . Thus it may be possible to acquire sufficient angular proj ections to reconstruct the OPT image without having to rotate the sample . This can be applied to samples for which rotation is impractical, such as growing plants or live animals or intact tissues . In a further embodiment (not shown) several imaging systems as shown in Figure 13 are angularly spaced around the axis of a tube , which extends along the z axis perpendicular to the plane of the page of Figure 13 , but also spaced axially along the tube . The aperture could be implemented by one or more spiralling slits that effectively present an aperture or apertures corresponding to varying angular proj ections along the z axi s . This arrangement can image a sample travelling through the tube through apertures in the tube wall at different angles as a function of distance along the tube . Thus, as the sample travelled along the tube, a sequence of images corre sponding to different angular proj ections can be acquired with no moving parts . The proce ssor which generates the image from the detector signals could be arranged to us e helical tomographic reconstruction techniques, such as those described in [Alicia Arranz, Di Dong, Shouping Zhu, Markus Rudin, Christos Tsatsanis,Jie Tianand Jorge Ripoll, Opt Exp 2 1 (20 13 ) 259 12] .

Referring to Figures 14- 16, in further embodiments, illumination from a source 8 10 can be used to image with scattered light. For coherent illumination of weakly scattering sample s, imaging channels can collect the scattered light detected at different angle s with respect to the illumination direction and determine the structure of the sample by reconstructing the amplitude and phase of the scattered light, for example using phase retrieval technique s in a manner analogous to ptychography or to Fourier ptychography, which is an extension of ptychography but utilise s illumination of the sample at different angles . Rotation of the sample in such a system can provide a wide range of illumination angles that can lead to a high resolution in the reconstructed image . This can be realised in a system based on the simple geometry shown in Figure 14, with multiple apertures or a single movable aperture, for example between the obj ective and tube lenses 870, 863 , with coherent plane wave illumination implemented in a conventional transmitted light OPT set-up . In this case the different intensity images required for the Fourier ptychographic reconstruction are acquired by rotating the sample .

More strongly scattering sample s can also be imaged by collecting scattered radiation and/or fluore scence at a range of illumination angles for tomographic reconstruction without phase information . When imaging in scattering sample s, including biological sample s such as live zebrafish, it is possible to use more sophisticated algorithms than simple filtered back proj ection to reconstruct tomographic image s . Some of these algorithms can include models of light scattering and it may b e useful to acquire information about the scattering of the illumination radiation in order, for example, to use inverse scattering approache s to reconstruct tomographic images . This approach can make use of measurements analysing the transmitted and/or fluore scence light with respect to spectroscopic and other parameters, which can include wavelength, polarisation, scattering angle and/or photon arrival time or path length in the sample and can be applied to obtain improved transmitted light tomographic image s and also to inform the reconstruction of fluorescence tomographic image s .

Angularly resolved detection of transmitted and scattered light can b e implemented by incorporating an aperture 968 scanning in the BFP of the obj ective lens 970 , as illustrated in Figure 1 5 . This scan is undertaken at each rotational position of the sample, so that the full CCD pixel count of the detector array 966 can be used to acquire the information from angular subsets of the scattered light and using this angularly resolved information can improve the precision with which the light scattering is characterised . Alternatively, a spatial light modulator could be used to provide an electronically adj ustable aperture or multiple aperture s in the BFP of the obj ective lens that could be employed to provide different angular proj ections . Alternatively the Fourier multiplexed imaging approache s to simultaneously acquire images at different angular proj ections de scribed above can be used to provide angularly resolved information concerning the scattering of the illumination light. This can be realised with modifications to the arrangements shown in Figures 6- 1 1 that incorporate angularly defined illumination and appropriate filters to collect scattered light or fluore scence or both . For example Figure 16 shows a further embodiment in which transmitted light and scattered light image s are acquired by respective imaging systems, each the same as that in Figure 6. It would be possible to acquire transmitted light and scattered light images in one multiplexed imaging channel and spectrally resolved fluore scence images at different angular proj ections in a different multiplexed imaging channel . Alternatively, all the angularly resolved image channels can be used to record transmitted or scattered illumination light or the image channels could all acquire fluorescence images in the same spectral band . Clearly other configurations are pos sible including the use of time- resolved and or polarisation-resolved detectors .

The Fourier multiplexing approach can also be extended to illumination of the sample where, in further embodiments, aperture s or point illumination sources can be deployed in the back focal plane of the condenser to realise angularly resolved illumination .

For example the arrangement shown in Figure 17 can be employed where an aperture 1 168 is scanned in the BFP of the condenser lens 1 124 to provide illumination at different angles while the sample is stationary . Similarly, a spatial light modulator could be used to provide an electronically adj ustable aperture or multiple apertures in the BFP of the condenser lens 1 124 that could be employed to provide different angular proj ections . Alternatively a focussed laser beam waist could be translated in the back focal plane of the condenser lens 1 124 to provide illumination at different angular proj ections with greater efficiency .

The set-up depicted in Figure 17 could be modified to replace the single moving aperture 1 168 in the BFP of the condenser lens with multiple aperture s 1268 , as indicated in Figure 1 8. In some implementations it is po ssible to use the same lens to serve as condenser and obj ective lens . This would entail illuminating and imaging from the same side of the sample with apertures in the back focal plane of the lens arranged to transmit illumination radiation and/or backscattered light and/or fluorescence . The illumination systems of Figure s 17 and 1 8 can in some case s be combined with multiple or moving apertures between the obj ective lens and the CCD camera. For example , the interference of coherent laser emanating from two or more points in the back focal plane of the condenser, using moving or multiple apertures such as those in Figure s 17 and 1 8 , could be used to realis e structured illumination of the sample during the OPT acquisition as is shown in Figure 19. In this embodiment the two illuminating beams 13 1 0a, 13 10b are displaced in the vertical plane as shown in Figure 19a so as to produce horizontal fringes in the sample such that the structured illumination would look similar for different angularly multiplexed imaging channels, i . e . channels angularly spaced around the vertical rotational axis of the sample . This could be applied with appropriate data proce ssing algorithms to improve the image quality of the sample region within the depth of focus . It is well known that structured illumination techniques can be used to form optically sectioned images in microscopy (i . e . form images of the region of the sample within the depth of field of the obj ective lens with contributions from "out of focus" light being suppre ssed) as de scribed by [M. A. A. Neil, R. Juskaitis, and T. Wilson, Optics Letters, Vol. 22, pp. 1905-1907 (1997) ] and [M.A .A. Neil, R. Juskaitis, T. Wilson, Optics Communications 153, 1-4 (1998) ] . It is also well known that structured illumination and related technique s such as "HiLo imaging" [D. Lim, K. K. Chu, and J. Mertz, Optics Letters, Vol. 33, (2008) 1819-1821 ] can be applied in scattering sample s to obtain improved image quality of the sample region in the focal volume , as has been demonstrated for example in light sheet microscopy [T. Breuninger, K. Greger, and E. H. K. Stelzer, Optics Letters, Vol. 32, (2007) 1938-194] , and [J. Mertz and J. Kim, Journal of Biomedical Optics Vol 15, p01 6027, (2010) ] and in multiphoton microscopy of thick samples [ V. Andresen et al. , PLoS ONE 7(12) : e50915 (2012) ] . This approach has also been demonstrated with optical computed tomography when imaging with transmitted light with a single imaging detector [E. Kristensson, E. Berrocal, and M. Alden, Optics Express Vol 20, 14437-14450 (2012) ] but not with fluorescence imaging or with multiple angularly resolved detectors as is proposed here .

In further implementations the excitation beams may be periodically modulated in amplitude or phase such that fluorescence emitted from the focal volume where two or more excitation beams overlap will be temporally modulated . If the excitation beams are modulated at different frequencies, then the fluore scence from the focal volume will include a contribution that is modulated at the difference frequency . Alternatively, if the phase between the excitation beams is modulated at a single frequency, the fluorescence collected from the focal volume may be modulated at the same frequency . Using appropriate demodulation techniques, optically sectioned images of fluorescence from the focal volume can be acquired .

Thus the modification of the OPT set-up shown in Figures 1 8 and 19 to include two or more apertures in the BFP of the condenser to realis e structured illumination in the sample could be used to improve image quality by supres sing contributions from out of focus and scattered light. Structured illumination can be conveniently implemented by imaging the light diffracted off a grating structure onto the sample . This approach effectively provide s the required point source s of illumination in the back focal plane of the condenser for laser illumination and can also be employed with incoherent illumination . Figure 20 illustrate s this approach for the case when the same lens serve s as both obj ective and condenser and the grating is orientated so as to produce horizontal fringes .

The set-ups for OPT with laser illumination could be modified to arrange that the spectral components of incident laser radiation are spatially dispersed in the BFP of the condenser lens . This could be used to implement 'temporal focusing ' as has been de scribed to provide wide-field optical section in combination with multiphoton excitation, e . g . [D. Oron, E. Tal, and Y. Silberberg, Optics Express vol 13 (2005), 1468-1476] . The set-ups depicted in Figures 20-22 could be further modified to combine the approach of temporal focussing with the technique of structured illumination, as discussed in [H. Choi e t al, Biomedical Optics Express, vol 4 (2013) 995-100] , to enable clearer images of the sample within the depth of field of the imaging system to be acquired, particularly in the presence of scattered light.

The OPT techniques of the various embodiments described above can all b e combined with spectroscopic contrast such as excitation or emission wavelength, polarisation or fluorescence lifetime .

The pre sent invention can be applied to any current application of OPT including developmental biology of both animals and plants, volumetric histopathology of ex vivo sample s, in vivo imaging of live organisms including live disease models such as zebrafish for drug discovery and studie s of disease mechanisms . For imaging live samples, it is important to minimis e the image acquisition time and the light dose in order to maximise the survival chances of the samples, e . g . to minimise the time they are maintained anaesthetized . Some embodiments of the present invention can address this critical issue by reducing the image acquisition time to acquire high resolution images and increasing the light collection efficiency by enabling the use of higher NA imaging systems .

Claims

Claim s

1 . An optical tomography system comprising a support arranged to support an obj ect, and an imaging system comprising an obj ective lens arranged to receive light travelling from the obj ect over a range of directions, detection means arranged to detect the light and means defining at least one aperture arranged to define a plurality of optical paths between the obj ective lens and the detection means whereby the detection means can detect images of the obj ect from a plurality of different viewing directions .

2. A system according to claim 1 wherein the detection means has a plurality of areas each arranged to receive light transmitted along a re spective one of the optical paths .

3 . A system according to claim 2 wherein the detection means comprise s an array of detectors, and each of the areas comprise s a sub-array of the detectors .

4. A system according to any foregoing claim wherein the imaging system further comprises at least one imaging lens, wherein the at least one imaging lens is arranged to focus an image from each of the optical paths .

5 . A system according to claim 4 wherein the imaging system comprises a plurality of imaging lenses, one in each of the optical paths .

6. A system according to claim 5 wherein the imaging lenses are integrated into the means defining the at least one aperture .

7. A system according to claim 4 wherein the at least one imaging lens comprise s one imaging lens or sequence of lenses arranged to focus an image from each of the optical paths .

8. A system according to claim 7 further comprising one or more optical elements arranged to separate spatially from each other the images from the optical paths .

9. A system according to any one of claims 4 to 8 wherein the at least one imaging lens is arranged to focus an image onto the detection means .

10. A system according to any one of claims 4 to 8 further comprising an image relay lens or other optical system arranged to relay the image formed by the imaging lens onto the detection means .

1 1 . A system according to any foregoing claim wherein the imaging system is a first imaging system arranged to image the obj ect from a first plurality of viewing directions, and the tomography system further comprises a second imaging system arranged to image the obj ect from a second plurality of viewing directions .

12. A system according to any one of claims 4 to 9 wherein the at least one imaging lens is arranged to image parts of the obj ect which are at different respective distance s from the obj ective lens .

13 . A system according to claim 12 wherein the distances are adj ustable .

14. A system according to any foregoing claim wherein the sample is illuminated by an illumination system comprising a condenser lens arranged to direct light to the obj ect over a range of directions and means defining at least one aperture or localised light source in the back focal plane of the condenser lens arranged to define a plurality of optical paths between the condenser lens and the sample whereby the sample can be illuminated from a plurality of different viewing directions .

15 . An optical tomography system comprising a support arranged to support an obj ect, and an imaging system comprising an obj ective lens arranged to receive light travelling from the obj ect over a range of directions, detection means arranged to detect the light and means defining at least one aperture or localised light source in the back focal plane of the condenser lens arranged to define a plurality of optical paths between the condenser lens and the sample whereby the sample can be illuminated from a plurality of different viewing directions .

16. A system according to claim 14 where the same lens serves as the condenser lens for the illumination and obj ective lens arranged to receive light travelling from the obj ect over a range of directions .

17. A system according to any one of claims 14 to 16 wherein the means defining the at least one aperture in the back focal plane of the obj ective lens or the condenser lens is formed of an opaque material having at least one opening therein forming the at least one aperture .

1 8. A system according to claim 17 wherein the means defining the at least one aperture has a plurality of openings therein forming a plurality of apertures .

19. A system according to claim 17 wherein the means defining the at least one aperture has an opening therein and is movable so that the opening can sequentially define more than one of the optical paths .

20. A system according to any foregoing claim wherein structured illumination is implemented to suppress contributions from out of focus and scattered light to the images .

2 1 . A system according to any foregoing claim wherein temporal focussing is implemented to suppre ss contributions from out of focus and scattered light to the images .

22. A system according to any foregoing claim wherein the obj ect is imaged using at least one of: transmitted light; scattered light; and fluore scence .

23 . A system according to any foregoing claim wherein the different optical paths are spectrally resolved .

24. A system according to any foregoing claim wherein fluorescence lifetime imaging is applied in one or more different optical paths .

25. A system according to any of the above claims wherein the different optical paths are resolved with respect to polarisation.