This Small Business Innovation Research (SBIR) Phase I project investigates the feasibility of employing high?resolution, colorized, 3D models of food produce captured with structured light technology to identify (ID) and sort produce by quality. The feasibility of accomplishing this within the real-time constraints of in-field harvesting will be determined. The PROBLEM being addressed is that high fidelity modeling currently requires controlled calibration and acquisition processes, expensive sensor hardware and time consuming global optimization algorithms taking minutes, or hours, to complete. The proposed APPROACH is to research novel processing algorithms to extend 2D super-resolution principles and exploit new probabilistic motion and sensor error models to achieve precise multi-frame registration and fast global optimization. This research will exploit new methods which operate on smaller clusters of ?similar? pixels and leverage 3D probabilistic occupancy mappings, taking advantage of imaging and geometry features simultaneously to reduce computation time and noise. If achievable, the BENEFITS include development of a cost-effective and feature-rich advanced data modeling technology that can be integrated into produce collection machinery and used to cost-effectively segment individual food items based on cosmetic imperfections. This is expected to provide both reduced costs and increased revenues to small and medium farm enterprises.
The broader impact/commercial potential of this project addresses two areas: 1) a critical gap within domestic produce farming which prevents small farmers from competing with large, corporate enterprise farms in terms of efficiency, quality control and product pricing and 2) a very real advance in the broader area of sensor systems and technology. Although a distributed, low cost modeling and sorting application for fruits and vegetables is targeted in Phase I, the technology should provide similar benefits to the broader consumer foods market by improving the distribution and sorting of meats, seafood, cheeses or even baked goods, and the like, while also advancing many other areas that can benefit from good ID and sorting technology. At the very least, this cost effective, automated, sorting capability is expected to increase small farm revenues by at least 25% and provide the offeror with a burgeoning, worldwide business through its strategic partners. In addition, such an affordable, and fast, 3D data modeling and inspection technology for accurately categorizing geometric defects can be expected to have a much broader impact within the field of data modeling and inspection.
The majority of produce quality-grading systems today use color and other multi-spectral 2D image features as criteria by which to sort food items. They set experimentally defined thresholds on features such as color, color-based textures and intensities, to classify per-pixel segments of the produce as good or bad. While this technique has shown success when tailored to a specific type of produce, classification accuracy suffers when the system is presented with a wider variety of fruits and vegetables that vary in size, shape, roughness and topology (stem, pulp, skin, calyx). However, trained harvesters typically use shape as an important consideration when evaluating produce to detect cosmetic abnormalities which can be unappealing to consumer markets and can indicate disease. In some cases, processing facilities can require produce to meet specific geometric criteria to efficiently be processed by their machinery. Understanding the shape of the produce will assist farmers in identifying the appropriate market into which the food item should be sold. Understanding and leveraging 3D shape and geometry of produce will allow sorting algorithms to better detect and classify these cosmetic abnormalities. To address this challenge, Neya Systems, LLC has developed a software process that extends 2D super-resolution principles and generates accurate high-resolution 3D color models of produce using inexpensive commercial off-the-shelf (COTS) structured light sensing hardware. The feature-rich data models resulting from this process can be integrated into turnkey portable solutions, commoditizing inspection processes and improving quality control for small local farms and collectives. Based on this Phase I research and analysis, we have shown that individual high resolution splices of the models generated using this software can be created and classified within real-time constraints of standard sorting processes, averaging 10-15 frames per second for our smallest food item on a standard Intel core i7 processor. This includes the ability to accurately track motion precisely over a half meter scanning trajectory. The resulting models are not only much higher resolution (with an average of 0.5 mm spacing between pixels) but also show a 1.9x improvement in the average geometric accuracy of the model when compared to a micron-precision baseline model. Based on our preliminary analysis of using the high resolution model splices for quality classification, we found that shape features not only add to the overall accuracy of classification, but for this specific set of features actually outperforms the color-based classification in every case. While the training data sets are too small to provide conclusive results at this stage, this certainly motivates the case for continued research with additional features (both shape and color) as well as more sophisticated classification techniques as applied to high resolution color 3D data models.
