Report

Teaser, summary, work performed and final results

Periodic Reporting for period 2 - SafeLog (Safe human-robot interaction in logistic applications for highly flexible warehouses)

Teaser

\"The European market for e-commerce is growing rapidly, thus the need for larger warehouses and distribution centers increases. In such facilities goods for the end-users or products in the B2B sector are stored, commissioned and shipped. In logistics, the concept of full...

Summary

\"The European market for e-commerce is growing rapidly, thus the need for larger warehouses and distribution centers increases. In such facilities goods for the end-users or products in the B2B sector are stored, commissioned and shipped. In logistics, the concept of full automation is not desirable or even feasible. Thus the vision of \"\"Warehouse Co-workers\"\" becomes more and more important to advance automation as an assistant system to the human worker. As a consequence there is a dire need for new technologies of realizing safety for warehouses, where human and robots co-work.

Therefore the overall objectives of SafeLog is the conception and implementation of a large-scale flexible warehouse system which enables a safe and efficient collaboration of human and robots in the same area at the same time. The objectives are:

1. Safety concept: Design of a holistic safety concept, which allows the collaboration of humans for a flexible warehouse system.
2. Safety vest: Hard- and software design for a new kind of safety vest.
3. Planning and scheduling: Development of planning and scheduling algorithms for a heterogeneous fleet manager.
4. Assisting technologies: AR based interaction strategies to support workers in a robotised warehouse system.
5. Assessment and integration: Integration and Assessment of Use-cases and Scenarios.
6. Dissemination and information: The proposed concept requires a strong dissemination and information concept to inform warehouse-operating companies.

\"

Work performed

\"From the very beginning, SafeLog laid emphasis on a clear scenario definition which served to derive the essential use cases for demonstrating the envisioned functionalities. Based on this information a comprehensive set of requirements where extracted which lead to a specification. These scenarios were further refined into component test cases.Regarding component integration, emphasis was given to perform tests as soon as possible. Thus already in the second repoting period, system wide integration was started and interactions between components were tested. Furthermore a new, bigger test cell was constructed in Augsburg by SLA allowing larger scale tests.

A crucial part in SafeLog is the safety concept. Thus an initial safety risk assessment was done, the functional safety plan elaborated, documented, and in multiple iteration discussed with a subcontracted certification body - TÃœV Nord. The sensor and transmitter technology was chosen and experimentally verified. A workshop with TÃœV Nord was held and an agreement regarding further cooperation was settled. Constant communication with TÃœV Nord up to the first prototype of the safety vest ensured that all safety requirements were followed, and communication will continue until the final product and it\'s official certification. For further elaboration three document were prepared which specify requirements, architecture and integration tests. From there test case specifications were derived. Based on that, tests are being conducted.

Regarding non-safety critical Safety vest architecture, essential building blocks for the robust localisation of the human in the warehouse were researched and developed. These include: a method for detection and tracking of multiple moving objects from stereo images, an algorithm for detecting and identifying racks in camera images, and a ground node detection and identification algorithm using monocular camera images. Tests were performed in a relevant environment, IML\'s \"\"Living Labs\"\", showing good results in the tracking of the user wearing a safety vest with back mounted stereo and monocular cameras.

For the heterogeneous fleet manager an initial simulation environment was implemented which allows the testing of planning algorithms. Based on this simulation the design and development of coordinated multi-entity path planning algorithms as well as a behaviour-based approach were investigated. In this regard several centralized planning algorithms for coordination of a mutli-robot teams were developed and tested as well as a decentralized, multi-agent based fleet manager and a deep reinforcement learning navigation algorithm. The algorithms were extended with a human-aware route planning algorithm, able to deal with sometimes unpredictable human movement.

In SafeLog the HoloLens was chosen as AR-interaction device. The localisation capabilities were tested at SLA\'s test cell in Ettlingen and proven to be robust. The human intention recognition system was also tested in the same environment. It was proven to be effective at predicting the intentions of the human workers, the results being published in a journal paper. A Virtual Reality test environment was constructed to enable large-scale warehouse tests as well as testing the communication withother components. A demo showing the main workflow of the worker wearing the AR glasses, including all the interaction paradigms, was successfully presented at IROS 2018.
To support the visibility of SafeLog a web site was launched (safelog-project.eu) and continuously updated with news, publications, test sets etc. A flyer, a roll-up and posters were designed. About 65 result presentations at conferences, workshops and other events have been done as well as 5 workshops (co-)organised. A highlight was the participation in the IROS 2018 with a project booth and a large demonstration of the project concept and prototypes. A total of 9 press articles were published addressing the general public to cr\"

Final results

In the first period of SafeLog special focus was laid on:

* The safety concept to achieve a certifiable system. Due to the close interaction with the certification body TÃœV Nord a promising concept could already be defined.
* Localisation and planning algorithms, which already lead to several promising prototype implementations.
* Evaluating existing AR-Devices, which lead already to first AR-based interaction metaphors.

In the second period of SafeLog new advancements were made:

* The safety vest entered the prototype stage, with constant interaction with TÃœV Nord ensuring that the safety system is developed in compliance with all the safety norms and therefore being eligible for certification
* State of the art planning algorithms were extended for more scalable approaches for large fleets as well as the capability to deal with sometimes unpredictable human agents
* The localisation system of the safety vest was further developed to include stereo odometry, ground node and rack identification as well as moving object identification. With visual odometry and ground node identification working together good results have been achieved in a relevant environment with a motion capture sensor providing ground truth.
* A human intention recognition algorithm was developed and successfully tested in real world and virtual reality environment proving the feasibility of the approach.
* AR interaction paradigms were extended and a system developed to be mobile and go beyond the state of the art HUD-based and pick-by-light based approaches.

With in total 23 scientific publications, SafeLog has already contributed to the current state of the art.
As for now it is expected that SafeLog will achieve all its objectives in the last project year with all implications regarding foreseen impacts.

Website & more info

More info: http://safelog-project.eu/.