How do you use deep learning to determine the quality of apples?
Every year, millions of apples are harvested in the Netherlands, which have to be sorted before they go to the consumer. This is done partly by the sorting machine and partly by hand.Two Fontys students argue, based on their research findings, that it should be possible to use deep learning to determine the quality of apples and then separate apples based on these factors.With a roller conveyor and sorting cameras, the machine ensures that apples of the right sizes come out through the right exits. It is then human work to check the apples for quality.
Felix van der Heijden and Jelle Stappers, then fourth-year students of mechatronics at Fontys Hogeschool Techniek & Logistiek in Venlo, were commissioned by Fruitteeltbedrijf Stappers Baarlo to investigate whether the sorting process for quality apples could be automated. In doing so, they wanted to make use of the technique of deep learning. Deep learning is a form of artificial intelligence that can detect objects automatically. This is achieved by placing a camera with a deep learning application in the sorting machine, so that the machine itself can distinguish the good apples from the bad ones. After this, it can perform actions to sort by quality.
Labels and weights
In order to detect objects, the software must first know what the object in question looks like. To do this, objects are labeled. In this case it was apples and quality factors, the labels of which are then given to the deep learning algorithm so that the system can 'train'. During this training, images are compared with each other in order to assign weights to various properties, such as color and shape. The final result is a "classes" and a "weights" file that allow the algorithm to recognize objects.
The software / deep learning application Yolo (you only look once) was chosen, because knowledge of it had already been gained. In combination with Qt Creator (an Integrated Development Environment) it should be possible to detect the quality of apples.
From Yolo to Tensorflow
It was decided to switch from Yolo to Tensorflow, as this system offers more support and is more widely used. Multiple neural networks are available in Tensorflow. For testing, Mobilenet was used. Detecting objects in an image takes 20 ms in this network. Two datasets were then realized: one to detect apples and one to detect quality. Tensorflow proved to be better at detecting individualapples, but due to lack of data from rejected apples, it was not possible to properly detect all quality factors.
Opportunities for the Future
From the study, the students conclude that Tensorflow is more suitable for use in business. The system is better supported and more robust than Yolo and it works faster and more accurately. Due to the lack of a large and evenly distributed data set, damage prediction was not successful within this study. The students indicate that this can be improved by creating a dataset where each class consists of the same number of labels. They did not get back to this themselves.
The conclusion is hopeful: based on the findings of the study, it should be possible to use deep learning to determine the quality of apples and to separate apples based on these factors.