Is it a human or a shelf? Helping AGV's to 'see' better

What sounds like a philosophical question – what is the difference between a human and an object? – is really a very practical one. At least for an autonomous forklift whose artificial intelligence needs to be taught to understand the structure and elements of a typical warehouse so that it can find its own way through the facility without injuring anyone or damaging anything. At the very least, it must be able to identify shadowy outlines and not be misled by optical illusions.

"Deep PTL" is the name of the join research effort, a combination of deep learning and the acronym for production, transport and logistics. The cooperation got underway in September 2018 and is anticipated to last through early 2022. Together with University of Freiburg scientists and sensor specialists from Sick AG, engineers from KION’s own CTO organization are looking into artificial intelligence models which require as few parameters as possible to accomplish the recognition of objects.

Currently, the research team is focusing on generating the necessary data, which is necessary for training the neural network and developing it further. A mobile platform equipped with the latest sensor technology has been specially developed for this purpose. It is driven manually through various storage and production halls to "collect" high-quality data records. Initial trials are planned at STILL and other locations. In the future, the platform will also be used by customers to obtain as diverse data as possible.

"We use deep learning methods, which are very powerful in the recognition of objects and are also the reason why there is currently so much hype about artificial intelligence in so many industries," says Patrick Erbts, project manager for the project. Deep learning – or hierarchical learning – is part of a broader family of machine learning methods based on artificial neural networks, which can mathematically solve tasks that humans "intuitively" solve.

Best results despite limited resources

Autonomous industrial trucks present another challenge: "All the deep learning methods currently known are computationally intensive," states Erbts. "However, our vehicles are not equipped with high-performance computers and our resources are limited. Even if it is assumed that computers will continue to gain computing power in the future, it makes sense to think about solutions that work with the computer chips installed on board.

This is one advantage the intralogistics industry has over automotive companies, which are also eager to improve object recognition: A warehouse has a large stock of recurring elements such as shelving units, employees and warehouse equipment. "Yet, there are often very diverse lighting conditions in warehouses," adds Erbts. This can be a challenge for neural networks.

Current systems can barely make a distinction

When the camera position changes or a dark corner comes into view, it can trigger the neural network to recognize objects incorrectly. "So, we need to merge data from different sensors such as laser scanners and cameras," explains Erbts, In addition to the question of resources and the assorted visibility conditions, the sensor combination is the project’s third major research area. Currently, self-driving warehouse vehicles normally put on the brakes when an object moves into the scanner area, but the software cannot tell whether it is a human being or a fluttering plastic tarpaulin. However, thanks to the project, the ability to differentiate is expected to improve considerably.

One of the questions that will need to be clarified is how thoroughly must an industrial truck need to determine what is in front of it? Is it enough if it recognizes that it is a moving object does it need to be able to identify that it is a human being or another forklift? Against the background of the limited computing capacity of vehicle computers, this becomes an exciting task. "Research always involves entering unknown territory," says Erbts. And artificial intelligence is currently a trending research topic. "With this project, we are laying the basic groundwork with the KION Group and we are always trying to stay one step ahead of the game.”