Every year, millions of apples are harvested in the Netherlands, which have to be sorted before they go to the consumer. This is done partly by the sorting machine and partly by hand.Two Fontys students argue, based on their research findings, that it should be possible to use deep learning to determine the quality of apples and then separate apples based on these factors.With a roller conveyor and sorting cameras, the machine ensures that apples of the right sizes come out through the right exits. It is then human work to check the apples for quality.
IN SHORT
This is the headline
Every year, millions of apples are harvested in the Netherlands, which have to be sorted before they go to the consumer. This is done partly by the sorting machine and partly by hand.
This is the headline
Two fourth-year mechatronics students from Fontys University of Applied Sciences were commissioned by Fruitteeltbedrijf Stappers Baarlo to investigate whether the sorting process for quality apples could be automated using deep learning.
This is the headline
In order to detect objects, the software must first know what the object in question looks like. To do this, objects are labeled. In this case, it was apples and quality factors, whose labels are then given to the deep learning algorithm so that the system can "train
Felix van der Heijden and Jelle Stappers, then fourth-year students of mechatronics at Fontys Hogeschool Techniek & Logistiek in Venlo, were commissioned by Fruitteeltbedrijf Stappers Baarlo to investigate whether the sorting process for quality apples could be automated. In doing so, they wanted to make use of the technique of deep learning. Deep learning is a form of artificial intelligence that can detect objects automatically. This is achieved by placing a camera with a deep learning application in the sorting machine, so that the machine itself can distinguish the good apples from the bad ones. After this, it can perform actions to sort by quality.
Labels and weights
In order to detect objects, the software must first know what the object in question looks like. To do this, objects are labeled. In this case it was apples and quality factors, the labels of which are then given to the deep learning algorithm so that the system can 'train'. During this training, images are compared with each other in order to assign weights to various properties, such as color and shape. The final result is a "classes" and a "weights" file that allow the algorithm to recognize objects.
The software / deep learning application Yolo (you only look once) was chosen, because knowledge of it had already been gained. In combination with Qt Creator (an Integrated Development Environment) it should be possible to detect the quality of apples.
"To be able to recognize apples, a dataset had to be created first. In order to do this as accurately as possible, the practice situation with camera was used."
From Yolo to Tensorflow
It was decided to switch from Yolo to Tensorflow, as this system offers more support and is more widely used. Multiple neural networks are available in Tensorflow. For testing, Mobilenet was used. Detecting objects in an image takes 20 ms in this network. Two datasets were then realized: one to detect apples and one to detect quality. Tensorflow proved to be better at detecting individualapples, but due to lack of data from rejected apples, it was not possible to properly detect all quality factors.
Opportunities for the Future
From the study, the students conclude that Tensorflow is more suitable for use in business. The system is better supported and more robust than Yolo and it works faster and more accurately. Due to the lack of a large and evenly distributed data set, damage prediction was not successful within this study. The students indicate that this can be improved by creating a dataset where each class consists of the same number of labels. They did not get back to this themselves.
The conclusion is hopeful: based on the findings of the study, it should be possible to use deep learning to determine the quality of apples and to separate apples based on these factors.
TIPS
This is the headline
By placing a camera with deep learning in a sorting machine, the machine itself can "learn" to distinguish the good apples from the bad ones.
This is the headline
Proper preparation and "training" will give you a better chance of getting your desired outcome. The final result of this project was a "classes" and a "weight" file that allows the algorithm to recognize objects.
This is the headline
From the study, the students conclude that Tensorflow is more suitable for use in business. The system is better supported, is more robust than Yolo, it works faster and is more accurate.
Fontys
Fontys Expertise Center High Tech Systems and Materials (HTSM) connects Fontys students and researchers with high-tech companies in the Brainport area.