Skip to content

Murder or accident: image recognition helps Shanghai determine cause of death

Labworld asked Sioux for image analysis technology to make its forensic analysis products more effective. The technology house discovered in a preliminary study that the real problems and gains were in improved workflow. With the use of Machine Learning technology, bottlenecks were then eliminated, resulting in significant time savings and efficiencies.
Shanghai

IN SHORT

This is the headline

Fresh look from external parties at product or process creates new opportunities

This is the headline

Put your supplier in touch with your customers to uncover problems or bottlenecks faster

This is the headline

When digital experts have identified bottlenecks for you, they are then also better able to develop the right solution

This is the headline

An independent external party also has a good overview and insight into the available technologies resulting in a better fitting solution

During 2018, Shanghai-based Labworld is knocking on Sioux Technologies' door for image recognition technology. With this, they hope to improve an analysis process that will be conclusive about accident or murder in drowning deaths. These types of murders are relatively common in China due to the lack of firearms. To clarify the cause of death in drowning victims, forensic experts use electron microscopes to look for diatoms (diatoms) in samples of lung, kidney or liver tissue. These are single-celled algae found throughout natural waters. There are as many as ten thousand different species, all different in shape and size. Due to all kinds of influences, each surface water has its own specific populations. These can therefore serve as a fingerprint of the crime scene. If the seaweed in the surface water at the deceased's location does not match the composition of the water in the lungs, something is wrong.

Important to speak the language of the end customer

Labworld knows Sioux from Phenom-World, now part of Thermo Fisher Scientific. The Chinese company supplies Thermo Fisher's table electron microscopes with its self-developed analysis tools. These are aimed at CSI-like investigations into murders, among other things. Labworld saw opportunities for improvement with image recognition. Martijn Kabel, innovation manager at Sioux Technologies: "Labworld thought that automatic image recognition would speed things up considerably. We thought so too at first." However, Sioux started by mapping out the entire workflow. To do so, an employee of their Chinese branch visited forensic experts in Guangzhou. The provincial police station there is known as an authority in the field of diatom-based forensics. Kabel: "It was very important that our colleague spoke the language and at the same time knew what was possible within Sioux".

The Sioux employee saw experts taking images manually. Then they identified and counted the diatoms. In this way, it takes several days to investigate a single drowning case. "Because we mapped the workflow very precisely, we saw that with one-by-one scanning, and analysis, there was far too much human-machine interaction required. Scaling up was almost impossible. Image recognition would not change that much either," says Kabel. It became clear that the real gain was in reducing the scanning time.

By doing so, they hope to improve an analysis process that will conclude accident or murder in drowning deaths. These types of murders are relatively common in China due to the lack of firearms.

Object recognition and not image recognition turns out to be key to saving time

The workflow analysis revealed that the assessment process would be greatly accelerated if the forensic expert were assisted with an initial rough assessment: whether there were any diatoms visible in the image at all. Sioux thought this could be done automatically. The computer would then have to make a pre-selection of the images on which diatoms might be visible. Experts would then only be needed for the final assessment: whether the images are indeed diatoms and which ones they are.

For the automatic pre-selection of diatoms, object recognition algorithms based on Machine Learning technology from the world of autonomous driving turned out to be useful. Rob Knoops, involved in the Labworld project as a mathematical engineer: "We put a signal square around potential diatoms in the microscope image, just like the object recognition algorithm in autonomous driving puts a square around potential road signs and people. As a next step, you can create a higher resolution image of these potential hits. The forensic expert can then assess whether this is indeed one, or a miss. With this rough pre-selection, you do get misses, but they are acceptable."

In this way, time was saved in this particular workflow by detecting potential hits at the lowest possible resolution. To then take higher resolution images only of potential hits. With this workflow, fewer higher-resolution images are needed, which reduces the time demand on the electron microscope.

Optimal user interface provides another improvement

Kabel: "When optimizing the entire analysis process, we put ourselves in the shoes of the end user and did a user experience analysis. On the basis of this, we developed a mockup and discussed it with the customer. This showed different samples with an indicator that shows the progress of the automated scanning process. The concept also showed which subareas had already been analyzed and where potential diatoms were present. We asked the people at Labworld, "Suppose we set it up this way, does that fit with what you expect? As we talked it through, what works well for them naturally came to the surface. They also came up with new ideas themselves. This is how we fine-tuned the design of the workflow application until we arrived at a concept that worked well."

"Therefore, look primarily at what is important to the customer, or to your product, and try to determine within the time and budget available how you can achieve maximum results through smart data collection."

Focus on workflow and value more important than technology

Kabel emphasizes that it is important to focus on the workflow and the value stream itself. Technicians often tend to think in terms of technical possibilities, but often these are not the limiting factors. Rather, those are time and budget. "Therefore, look primarily at what is important to the customer, or to your product, and try to determine within the time and budget available how you can achieve maximum results through smart data collection."

Knoops also emphasizes the end-user perspective: "It is not at all necessary for the forensic expert to make a nice picture of a diatom - although this is technically possible and it produces wonderful pictures. It is not even important to recognize diatoms perfectly automatically. Experts mainly want to know that it is a diatom and what kind. With the method, forensic experts save considerable time, so drowning cases can now be handled much faster."

TIPS

This is the headline

By putting technology vendors in touch with your customers, they are able to deliver the best results. After customer research, the solution in this case was found to be not in enhanced images, but in automatic recognition of raw scans.

This is the headline

When you give suppliers insight into how your customers work and use your products, they are also able to think with you and make more improvements than you initially asked for. In this case, adaptations by Sioux made the interface of the analysis tools significantly more user-friendly.

Picture of Sioux

Sioux

We bring high-tech to life. Impact with Smart Modules and Software.

View Business
Sign up for our newsletter and receive updates.