WO2012039597A2

WO2012039597A2 - Fruit ripeness grading system

Info

Publication number: WO2012039597A2
Application number: PCT/MY2011/000202
Authority: WO
Inventors: Abdul Rashid Mohamed Shariff; Mohd Zaid Abdullah; Mohd Hamiruce Marhaban; Suhaidi Shafie; Meftah Salem M.Alfatni; Mohd Din Amiruddin
Original assignee: Universiti Putra Malaysia
Priority date: 2010-09-23
Filing date: 2011-09-14
Publication date: 2012-03-29
Also published as: MY157647A; WO2012039597A3; BR112013006568A2

Abstract

A fruit ripeness grading system (10) which used computer vision application in agricultural quality inspection to ensure ripeness category of fruit characterized in that, the fruit ripeness grading system (10) includes: a) a housing (12) having an enclosure (12a) for scanning process; b) an illumination means (14) with optical lens illumination filter provided at the enclosure (12a) of the housing (12); c) a camera (16) provided at top portion of the enclosure (12a) of the housing (12); d) a feeding device (18) for conveying fruit samples to the housing (12); e) conveyer speed inverter (19); f) a processing unit (20) process and analyze the fruit sample image; g) a data acquisition interface (22) provided in between the camera (16) and the processing unit (20); wherein the feeding device (18) further provided with separator (18a) controlled by USB controller (23) and supported by compressor (17) for controlling and separating the fruit samples; wherein the processing unit (20) further provided with a disk top computational unit serves to transfer data to a computer.

Description

FRUIT RIPENESS GRADING SYSTEM FIELD OF INVENTION

The present invention generally relates to a fruit ripeness grading system, and more . particularly to an oil palm fruit ripeness grading system which used a computer vision application in agricultural quality inspection to ensure ripeness category of the oil palm fruit.

BACKGROUND OF THE INVENTION

The traditional method of agricultural product quality assessment is tedious and costly. It is easily influenced by physiological factors, inducing subjective and inconsistent evaluation results. Agricultural product quality conventionally plays a fundamental role in nearly all food industry quality assessments. Innovative computer technologies have been utilized in numerous research projects around the world in order to construct new machines for agricultural product quality assessment. Some prior arts have brought out some inventions that disclosed quality inspection with different methods and device systems.

For example, WO 99/60353 discloses a method of detecting colors to allow detecting and sorting objects, by determine the hue, intensity and saturation (HIS) color space for each RGB received pixel to be used for quality inspection system.

US 5,589,209 disclosed a method for non-destructive determination of quality parameters in fresh produce, by transmitting ultrasound waves to the produce from transducer, and received it by a nearby transducer in order to analyze the maturity of the sample.

US 2009/0226584 Al and WO 01/69191 A disclosed a fruit ripening display based on the apparatuses and methods of employing multiple platform, and other features for ripening fruit. Further, apparatuses and methods for measuring and correlating characteristics of fruit with visible/near infra-red spectrum are disclosed in US 2009/0226584 Al and WO 01/69191 Al .

Furthermore, WO 96/03226 disclosed an optical inspection system for edible natural products. Accordingly, the invention deeply focused on the illumination part. Nevertheless, a new method and apparatus for monitoring fruit quality and ripeness using light induced luminescence is disclosed in WO 2006/135267 A2.

In the other hand, WO 03/051540 Al shows a device for classifying products invention which relates to the field of product sorting devices for sorting products such as vegetable and fruit by radiating means for irradiating the products.

However, none of the above-proposed techniques suggest a suitable approach for oil palm fruit ripeness grading. Since oil palm fruits have specific features and the ripeness classes such as under ripe, ripe and over rip are close to each other, thus the specification of illumination system is highly considered.

It will be appreciated that different grading systems can be used with different techniques to classify different types of fruits. However, the system of invention is focusing on specific techniques to work with the parameters and properties of oil palm Fresh Fruit Bunch (FFB) in order to obtain a fast, accurate and objective ripeness classification of oil palm fruit. Accordingly, features of constructions, combination of elements and arrangement of parts of the preferred embodiments will be exemplified in the detailed description. SUMMARY OF THE INVENTION

The present invention relates to a fruit ripeness grading system. Accordingly, the fruit ripeness grading system used computer vision application in agricultural quality inspection to ensure ripeness category of fruit. Said fruit ripeness grading system includes: a) a housing having an enclosure for scanning process; b) an illumination means with optical lens illumination filter provided at the enclosure of the housing; c) a camera provided at top portion of the enclosure of the housing; d) a feeding device for conveying fruit samples to the housing; e) conveyer speed inverter; f) a processing unit process and analyze the fruit sample image; g) a data acquisition interface provided in between the camera and the processing unit; wherein the feeding device further provided with separator supported by compressor for controlling and separating the fruit samples; and wherein the processing unit further provided with a disk top computational unit serves to transfer data to a computer.

Accordingly, the housing of the system provides sufficient dark enclosed space and environment for scanning process. In accordance with the preferred embodiment, the enclosure of the housing is coated by non-reflective dark colour material. Said enclosure further provided with non-reflective dark colour flexible doors for controlling the system scanning environment. Preferably, the non-reflective dark colour flexible door is flexible rubber strap material.

Accordingly, the illumination means of the system includes defused tubes of LED light disposed along the enclosure of the housing. It will be appreciated that the illumination means is constant, stable, defused and with white LED light tubes lighting based on the fruit feature that produce a field of illumination. The optical lens illumination filter is provided in path of light beam selectively controllable to pass by a predetermined visible wavelength band of the entire light beam.

Preferably, the camera of the system is a suitable charge coupled device (CCD) camera used to capture fruit sample's image. Said camera is positioned up-right on top of the enclosure of housing to capture image information of the fruit sample as illuminated by the predetermined visible band of the light beam. P T/MY2011/000202

4

It will be appreciated that the feeding device further includes programmable motor to generate power and conveyor speed inverter for moving a conveyor continuously under a certain size and weight of fruit samples. In accordance with the present system, the separator of the feeding device is moved by a hydraulic arm which controlled by USB controller (23) and supported by a compressor.

Accordingly, the processing unit is a computer. Said processing unit used programmable software such as Mablab and/or Labview to obtain the desired data and result of the image.

Preferably, the data acquisition interface of the system includes RGB camera cable and frame grabbers. It will be appreciated that system is portable, adopting in industrial chain framework, and enable to use in similar application with different agriculture fruits.

The present invention also provide a method for fruit ripeness grading which used computer vision application in agricultural quality inspection to ensure ripeness category of fruit. Accordingly, the method includes image processing and analysis for fruit ripeness grading and classification of fruit sample based on colour feature, texture feature, empty sockets and thorns feature.

Accordingly, the image processing and analysis for fruit ripeness grading and classification are done by analysis the fruit sample of three different ripeness categories, i.e. under ripe, ripe and over ripe based on training model.

In accordance to the present invention, the image processing and analysis of fruit sample is based on different steps, wherein the steps includes fruit image acquisition, fruit image pre-processing, fruit image segmentation and fruit image feature extraction. It will be appreciated that all said images are linked to training mode, where fruit classification and retrieval are integrated for decision making based on the training model and similarity. Accordingly, the fruit image acquisition is the step which used for capture the fruit image. The fruit image pre-processing enhances the fruit image in order to obtain desired valuable informatioh during image processing. The fruit image segmentation is used to segment the fruit image based on its original parts in order to implement specific image processing on return on investment (ROI) for desired target; and the image feature extraction is a technique used to extract and measure the feature of fruit such as colour and texture for system training and testing.

It will be appreciated that the fruit ripeness grading system provides non-destructive measurement method that does not require laboratory examination for determining the ripeness of the fruits. Accordingly, the fruit ripeness grading system design is taking consideration of fruit size, weight, and shape that make the system multipurpose for use in similar application with different agriculture fruits.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. shows a physical setup of a fruit ripeness grading system in accordance with preferred embodiment of present invention;

FIGS. 2 and 3 illustrating layout details of the fruit ripeness grading system in accordance with preferred embodiment of the present invention;

FIG. 4 shows an overall integration flow chat of the fruit ripeness grading system;

FIG. 5 illustrating a real time fruit grading system software for image path;

FIG. 6 shows concept details of image process steps for the fruit ripeness grading system;

FIG. 7 shows image acquisition steps of the system equipments in accordance with preferred embodiment of the present invention;

FIG. 8 shows image segmentations of the fruit ripeness grading system in accordance with different image investigating techniques.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A detailed description of preferred embodiments of the invention is disclosed herein. It should be understood, however, the disclosed preferred embodiments are merely exemplary of the invention, which may be embodied in various forms. Therefore, the details disclosed herein are not to be interpreted as limiting, but merely as the basis for the claims and for teaching one skilled in the art of the invention.

A fruit ripeness grading system and apparatus for used with computer vision application in agricultural quality inspection to ensure ripeness category of the oil palm fruit according to the preferred embodiments of the present invention will now be described in accordance to the accompanying drawings FIGS. 1 to 8, both individually and in any combination thereof.

In accordance with preferred embodiments, the fruit ripeness grading system (10) for quality inspection, particularly for oil palm fruit ripeness grading generally includes a housing (12), an illumination means (14), camera (16), feeding device (18), conveyer speed inverter (19), processing unit (20), and data acquisition interface (22).

Accordingly, the housing (12) provides an enclosure (12a) for scanning process wherein the enclosure (12a) preferably provided with darken finishing layers with black colour painting thereof. Particularly, the enclosure (12a) of the housing (12) is coated by non-reflective dark colour material. The enclosure (12a) further provided with non-reflective dark colour flexible doors (12b) for controlling the system scanning environment. Preferably, the non-reflective dark colour flexible doors (12b) can be of, but not limited to a flexible rubber strap material. It will be appreciated that the housing (12) provides sufficient dark enclosed space and environment for scanning process.

Accordingly, the illumination means (14) of the fruit ripeness grading system (10) is provided at the enclosure (12a) of the housing (12). Preferably, defused tubes of LED light are used as shown in FIG. 7. Said defused tubes of LED light are preferably disposed along the enclosure (12a) of the housing (12). It will be appreciated that the illumination means (14) is constant, stable, defused and with white LED light tubes lighting supported by lighting control (21) based on the fruit feature that produce a field of illumination. An optical lens illumination filter (not shown) is further provided in the path of light beam selectively controllable to pass by a predetermined visible wavelength band of the entire light beam. By way of example but not limitation, the optical lens illumination filter is of 25mm, F1.4 with high megapixel.

The camera (16) is provided at top portion of the housing (12). Preferably, a suitable charge coupled device (CCD) camera is used to capture fruit sample's image. By way of example but not of limitation, a CCD digital cameral DFK 41BF02.H Fire Wire CCD colour Camera is used. Particularly, the camera (16) is positioned up-right on top of the enclosure (12a) of housing (12) to capture image information of the fruit sample as illuminated by the predetermined visible band of the light beam. It will be appreciated that the camera (16) captures the fruit sample image one time when it is conveyed exactly in the enclosure (12a) of the housing (12), otherwise the system will stopped.

The feeding device (18), supported by the conveyer speed inverter (19) is utilized for conveying the fruit samples to the housing (12). Said feeding device (18) also used for controlling and separating the fruit samples after grading or classifying. Preferably, the feeding device (18) is provided with separator (18a) moved by hydraulic arm (18b) supported by the compressor (17). The USB controller (23) is fixed between the computer (20) and separator (18a) in order to control movement of the separator (18a) based on the fruit ripeness result of the testing sample. It will be appreciated that a programmable motor (18c) is provided to generate power for moving a conveyor (18d) continuously under a certain size and weight of fruit samples. The conveyor (18d) is preferably made of, but not limited to non-reflective rubber belt (15) to carry the fruit samples to pass through camera field so that the camera (16) captures the image before conveyed to the separator (18a). Fruit containers (25) are provided for temporally storage of the fruit samples.

The processing unit (20) such as a computer is provided to process and analyze the fruit sample image. Accordingly, the processing unit (20) used programmable software such as Mablab and/or Labview to obtain the desired data and result of the image. It will be appreciated the processing unit (20) is further provided with a disk top computational unit which serves to transfer data to the computer.

It will be appreciated that the data acquisition interface (22) is provided in between the camera (16) and the processing unit (20). Preferably, RGB camera cable and frame grabbers are used (not shown). For example and not a limitation, the frame grabber of NIPCI-8285, IEEE 1395 board with vision is used to grab the RGB image to processing unit (20), i.e. the computer.

It will be appreciated that the fruit ripeness grading system (10) provides nondestructive measurement method that does not require laboratory examination for determining the ripeness of the fruits. It will also be appreciated that the fruit ripeness grading system (10) can be portable and adopting in industrial chain framework. Accordingly, the system enables to classify large numbers or quantities of fruits with high speed, accurate and time saving. Said fruit ripeness grading system design is taking consideration of fruit size, weight, and shape that make the system multipurpose for use in similar application with different agriculture fruits.

With reference now to FIG. 5, the fruit ripeness grading system (10) contents two general methodological stages, i.e. a training stage (30) and a testing stage (40). The training stage (30) includes data collection, data analysis and training model. Accordingly, technique of. ripeness classification of fruit image based on training model is done by analysis the fruit sample of three different ripeness categories, i.e. under ripe, ripe and over ripe.

The testing stage includes testing the grading system internally in the lab; testing the grading system through the field in order to make sure the system is enable to provide high percentage of internal validity finding for the system design; and generalization steps that generating the system setting for used with other different fruits sample of similar application. For example, as shown in FIG. 4, the fruit ripeness grading system can be used for other different fruits sample of similar application. Accordingly, the processing unit (20) of the system enables to provide analysis of different fruit type classifications, preset from type 1 to type N. As for the oil palm fruit ripeness grading and classification, the system further provides predetermined classes from class 1 to class N, wherein the classes will then be integrated and analyzed based on colour, texture, fresh fruit bunch (FFB) empty sockets and fresh fruit bunch (FFB) thorns.

It will be appreciated that image processing and analysis of the system is executed through general and standard steps. Accordingly, the oil palm fruit bunch passed through fruit image processing stages based on different steps as shown in FIG. 6. The steps includes fruit image acquisition, fruit image pre-processing, fruit image segmentation and fruit image feature extraction, wherein all images will then be linked to training model and the fresh fruit bunch for classification and retrieval are integrated for decision making based on the training model and similarity.

Accordingly, the fruit image acquisition is the step which used for capture the fruit image by the CCD camera (16) wherein the fruit samples are conveying by the feeding device (18) through the housing (12) so that the image is captured and send to the processing unit (20) through the data acquisition interface (22) for further analysis. The housing (12) provides the enclosure of controlled environmental by constant, stable, defused tubes of LED light based on fruit features, parameters and properties, so that provide the system with fixed image environmental.

The fruit image pre-processing is to enhance the fruit image in order to obtain desired valuable information during the image processing. For example, image crop and resize are used to reduce the image size so that to fit the image with the programmed software format and to speed up the processing time.

The fruit image segmentation is used to segment the fruit image based on its original parts in order to implement specific image processing on return on investment (ROI) for desired target. Accordingly, there are three different techniques and algorithms types are investigated as shown in FIG. 8, i.e. the Illumination, camera and Y2011/000202

1 1 processing unit. In order to obtain tremendous segmentation based on high performance and speed, the Modified Excess Red (MExR) technique is used wherein the MExR to be attuned, preferably at 2* Red - Green - Blue (FIG. 8 d). For Colour- Based Segmentation, L*a*b* or Lab colour space is used (FIG. 8 c), and for the segmentation image masking technique is used. Said techniques maneuver with morphological segmentation operation (FIG. 8 b). As for FIG. 8, it shows an original image.

The fruit image feature extraction is the technique used to extract and measure the feature of fruit such as colour and texture for system training and testing. For example, a colour features extraction and a texture feature extraction are used. Accordingly, oil palm fruit ripeness grading system includes different techniques for colour feature extraction, such as a statistical measurement which includes mean, standard deviation and median, a colour intensity tresholding technique, a gabor wavelet transform technique, and a colour histogram technique. The texture feature extraction may include wavelet transform technique, Grey Level Co-occurrence Matrix (GLCM) technique and Basic Gray Level Aura Matrix (BGLAM) technique.

In accordance with the preferred mode, the oil palm fruit ripeness grading system decision making can be divided into two types. Accordingly, a classification based on training and testing, and retrieval based on similarly. In particular, the classification based on training and testing using different techniques such as Artificial Neural Network (ANN) technique, K Nearest Neighborhood (KNN) technique, and Support Vector Machines (SVM) technique. Meanwhile, the retrieval based on similarity is attained by calculating the distance between the inquiry image and image database by the use of, for example, Quadratic distance technique and Euclidean distance technique.

While embodiments of the invention have been illustrated and described, it is not intended that these embodiments illustrate and describe all possible forms of the invention. Rather, the words used in the specification are words of description rather than limitation and various changes may be made without departing from the scope of the invention.

Claims

12 CLAIMS

1. A fruit ripeness grading system (10) which used computer vision application in agricultural quality inspection to ensure ripeness category of fruit characterized in that, the fruit ripeness grading system (10) includes:

a) a housing (12) having an enclosure (12a) for scanning process;

b) an illumination means (14) with optica] lens illumination filter provided at the enclosure (12a) of the housing (12);

c) a camera (16) provided at top portion of the enclosure (12a) of the housing (12);

d) a feeding device (18) for conveying fruit samples to the housing (12); e) conveyer speed inverter (19);

f) a processing unit (20) process and analyze the fruit sample image; g) a data acquisition interface (22) provided in between the camera (16) and the processing unit (20);

wherein the feeding device (18) further provided with separator (18a), supported by compressor (17) for controlling and separating the fruit samples;

wherein the processing unit (20) further provided with a disk top computational unit serves to transfer data to a computer.

2. A fruit ripeness grading system (10) according to Claim 1, wherein the housing (12) provides sufficient dark enclosed space and environment for scanning process.

3. A fruit ripeness grading system (10) according to Claim 1, wherein the enclosure (12a) of the housing (12) is coated by non-reflective dark colour material.

4. A fruit ripeness grading system (10) according to Claim 1, wherein the enclosure (12a) further provided with non-reflective dark colour flexible doors (12b) for controlling the system scanning environment.

5. A fruit ripeness grading system (10) according to Claim 3, wherein the non-reflective dark colour flexible door (12b) is flexible rubber strap material.

6. A fruit ripeness grading system (10) according to Claim 1, wherein the illumination means (14) includes defused tubes of LED light disposed along the enclosure (12a) of the housing (12), said illumination means (14) is constant, stable, defused and with white LED light tubes lighting based on the fruit feature that produce a field of illumination.

7. A fruit ripeness grading system (10) according to Claim 1, wherein the optical lens illumination filter is provided in path of light beam selectively controllable to pass by a predetermined visible wavelength band of the entire light beam.

8. A fruit ripeness grading system (10) according to Claim 1, wherein the camera (16) is a suitable charge coupled device (CCD) camera used to capture fruit sample's image.

9. A fruit ripeness grading system (10) according to Claim 1, wherein the camera (16) is positioned up-right on top of the enclosure (12a) of housing (12) to capture image information of the fruit sample as illuminated by the predetermined visible band of the light beam.

10. A fruit ripeness grading system (10) according to Claim 1, wherein the feeding device (18) further includes programmable motor to generate power and conveyer speed inverter (19) for moving a conveyor (18d) continuously under a certain size and weight of fruit samples.

1 1. A fruit ripeness grading system (10) according to Claim 1, wherein the separator (18a) is moved by hydraulic arm (18b), which controlled by USB controller (23) and supported by compressor (17).

12. A fruit ripeness grading system (10) according to Claim 1, wherein the processing unit (20) is a computer.

13. A fruit ripeness grading system (10) according to Claim 1, wherein the processing unit (20) used programmable software such as Mablab and/or Labview to obtain the desired data and result of the image and the data acquisition interface (22) includes RGB camera cable and frame grabbers.

14. A fruit ripeness grading system (10) according to Claim 1, wherein the system is portable, adopting in industrial chain framework, and enable to use in similar application with different agriculture fruits.

5. A method for fruit ripeness grading which used computer vision application in agricultural quality inspection to ensure ripeness category of fruit characterized in that, the method includes image processing and analysis for fruit ripeness grading and classification of fruit sample based on colour feature, texture feature, empty sockets and thorns feature.

6. A method for fruit ripeness grading according to Claim 17, wherein the image processing and analysis for fruit ripeness grading and classification are done by analysis the fruit sample of three different ripeness categories, i.e. under ripe, ripe and over ripe based on training model.

7. A method for fruit ripeness grading according to Claims 18, wherein the image processing and analysis of fruit sample is based on different steps, wherein the steps includes fruit image acquisition, fruit image pre-processing, fruit image segmentation and fruit image feature extraction.

18. A method for fruit ripeness grading according to Claims 18 and 19, wherein all images are linked to training mode, and fruit classification and retrieval are integrated for decision making based on the training model and similarity.

19. A method for fruit ripeness grading according to Claim 19, wherein the fruit image acquisition is the step which used for capture the fruit image.

20. A method for fruit ripeness grading according to Claim 19, wherein the fruit image preprocessing enhances the fruit image in order to obtain desired valuable information during image processing.

21. A method for fruit ripeness grading according to Claim 19, wherein the fruit image segmentation is used to segment the fruit image based on its original parts in order to implement specific image processing on return on investment (ROI) for desired target.

22. A method for fruit ripeness grading according to Claim 19, wherein the image feature extraction is a technique used to extract and measure the feature of fruit such as colour and texture for system training and testing.