Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - PICKPLACE (Flexible, safe and dependable robotic part handling in industrial environments)

Teaser

Pick and place are basic operations in most robotic applications, whether in industrial setups (e.g., machine tending,assembling or bin picking) or in a service robotic domain (e.g., agriculture or at home). In some structured scenarios andwith certain types of parts, picking...

Summary

Pick and place are basic operations in most robotic applications, whether in industrial setups (e.g., machine tending,assembling or bin picking) or in a service robotic domain (e.g., agriculture or at home). In some structured scenarios and
with certain types of parts, picking and placing is a mature process. However, that is not the case when it comes to manipulating parts with high variability or in less structured environments.
Handling systems are present in any logistics system to interface between the storage and the transportation systems. For non-structured scenarios, picking, packing and unpacking systems do exist at laboratory level. However, they have not reached the market yet due to factors like the lack of efficiency, robustness and flexibility of currently available manipulation and perception technologies.
The market demands systems that allow for a reduction of costs in the supply chain, increasing the competitiveness for manufacturers and bringing a cost reduction for consumers. Handling systems represent the highest impact in the shorttomidterm in warehouse-based systems (mainly at order picking and distribution centres) and in intra-logistics operations in factories and retail.
The technology gap is the lack of flexible solutions that can handle objects of variable size, shape and weight as well as different surface properties and stiffness.
PICKPLACE proposes combining human and robot capabilities to achieve a safe, flexible, dependable and efficient hybrid pick-and-package (PAK) solution. It includes dynamic package configuration planning, flexible grasping strategies using an innovative multifunctional gripper, robust environment perception and mechanisms and strategies for safe human-robot collaboration.
The Technological and Industrial objectives (TO and IO) identified are:
TO 1: To develop a new generation of multifunctional grippers to handle products of different morphology, weight and rigidity and that are able to reach difficult to access target positions
TO 2: To develop reactive grasp-planning algorithm based on cognitive capabilities and allows the robot to effectively grasp different objects
TO 3: Robust and efficient bin-picking solution based on object pose identification and fast and safe robot path planning
TO 4: Human and robot affordance aware dynamic package planning for mono and multireference configurations.
TO 5: Dynamic robot planning based on cognitive capabilities exploiting perceived monitoring and human activity.
TO 6: Reliable environment perception system and strategies for safe collaborative scenarios based on Speed and Separation Monitoring combined with Power and Force Limiting.
IO 1: To increase the pick-and-package global performance in terms of flexibility, dependability and error reduction.
IO 2: Improvement of the working conditions of operators by a proper layout design and task allocation between worker and robot.

Work performed

The activity done in this first period has result in the following results:

A design of the initial prototype of the multifunctional gripper, including 3 different end effectors (suction, magnetic and grasping) has been done and two physical prototypes have been built. Using the three effectors it would be possible to handle the huge variety of products included in both end-users’ scenarios. After a first validation process a new gripper design is foreseen for the second period.
The robot motion planning is done using MoveIT! that has been adapted and validated in simulation as well as realistic environment. Reactive grasp-planning is understood as the adaptation of an initial grasp of an object in response to sensory feedback (in our case haptic feedback) to achieve a more stable and reliable grasping. The algorithms developed allow adapting the robot motion planning in accordance to the pressure feedback from tactile sensors embedded in the grippers. It has been validated for different sensor configurations in suction cups and finger grippers’. Cognitive capabilities of learning from grasping failure will be reported in the second report period.
Different algorithms for monitoring and adapting the grasp (Gripper force control) measuring the occurrence of collisions have been implemented and tested.
Two different deep learning approaches have been developed for object identification: the first one aims at finding grasping areas without relying on the identification of the objects. The second approach relies on the identification and segmentation of the objects. Based on them, it is possible to handle the picking problem of multiple (sometimes unknown) parts.
Both identification modules will be fused in the second period of the project in order to make the system more robust and efficient.
The mosaic creation algorithms have been designed. They handle the problem of stackable and non-stackable parts packaging configuration. The final implementation will be done in the coming months.
An algorithm to optimize a cost function that takes into account the human 3D occupancy map to off-line plan paths that reduce the probably of interference with the operator has been developed.
For online modification of the velocity to ensure safety and the redundancy an algorithm based on Model Predictive Control (MPC) is proposed.
The planner takes into account the human factor analysis done in two experiments to study of how motion planning parameters affect the human perception in collaborative task.
It has been developed a system that combines machine vision and deep learning to monitor the shared human – robot working environment. The system uses a previous work (proximity detection) that allows monitoring a volume around a device and exploits SSD architecture for human detection. The system will be finished in the second period of the project.

Final results

Potential impact:
-Increasing the market-readiness of robotics applications including in terms of technological validation outside the laboratory and of sound operational and cost-benefit models.
UHS has already presented the initial prototype in a fair and it is part of a permanent demonstrator at their facilities in Oñati.
-Lowering of market entry barriers of a business or regulatory nature and increasing industrial and commercial investment in Europe at a rate comparable with other global regions
-Contributing to the faster growth of competitive small and mid-scale robotics companies in Europe
The Business Model and Plan are in line with this objective. Some results, such as the multifunctional gripper can open the possibility to transfer the technology to SMEs or even the creation of new ones

The more specific impacts for the final end-users (Error reduction, Cost reduction and Space reduction) will be measured at the end of the project once the final systems are deployed and demonstrated.

By the end of the Project, the expected results are:
1-Complete solution for collaborative pick-and-packing applications
2- Multi-modal, reactive grasping system
3-Human-aware motion planning software
4-Stereo-reconstruction, detection, tracking and classification software
5-Vision based object pose recognition
6-Real time task allocation and scheduling planning
7-Real time mosaic planning
8-Multifunctional gripper
9-Software tool for semi-automatic risk identification

Website & more info

More info: http://pick-place.eu/.