#	Pagina
attuale pagina	/open-h2020/projects/206244/results.html

Report

Teaser, summary, work performed and final results

Periodic Reporting for period 2 - REFILLS (Robotics Enabling Fully-Integrated Logistics Lines for Supermarkets)

Teaser

Summary

\"While online grocery stores are expanding, supermarkets continue to provide customers with the sensory experience of choosing goods while walking between display shelves. Therefore, retail and logistics companies are concerned with making the shopping experience more comfortable and exciting while, at the same time, using technology to reduce costs and improve efficiency. The REFILLS project aims at developing robotic systems able to address the in-store logistics needs of the retail market.

Three scenarios building on top of each other are considered:
- in Scenario #1, mobile robots inspect shelves and generate semantic environment maps for layout identification and store monitoring
- Scenario #2 employs a team of robots: a robotic arm for automatic picking of the cartons from a standard mixed pallet in the back room, pre-sorting and distributing them in trolleys; a fleet of mobile robots transporting the filled trolleys to the shop floor in optimal positions; a handling robotic unit and a pointing robotic unit assisting the clerk in shelf refilling
- in Scenario #3, the autonomy of the robot is strengthened, resulting in a robotic clerk capable of manipulating articles varying in shape, surface, fragility, stiffness and weight, and refill shelves without human intervention.

The REFILLS scenarios trigger a number of research and technology challenges that are tackled within the project. Information on the supermarket articles is exploited to create powerful knowledge bases, which are used by the robots to identify shelves, recognize missing or misplaced articles, handling them and navigate the shop. Reasoning allows robots to cope with changing task requirements and contexts, and perception-guided reactive control makes them robust to execution errors and uncertainty. A modular approach is adopted for the design of cost-efficient robotic units.

The work plan will generate exploitable results through three integration and evaluation phases. A final demonstration will take place at a real retail store.

In sum, REFILLS is committed to generating wide impact in the retail market domain and beyond through the development of efficient logistics solutions for professional use.

\"

Work performed

\"During the first year, all the requirement specifications for the scenarios of REFILLS project were worked out. Moreover, a detailed plan for experiments and assessment of performance, user-friendliness, human robot collaboration factor and cost-to-benefit ratio was specified.

From the hardware side, the first version of the robotic modules was released and tested, and a new gripper for the execution of the de-palletizing task was designed. Force/tactile sensors have been integrated into a standard industrial gripper to be used for grasping single items of different shapes, dimensions, materials, weight.

Concerning research and software development, a first prototype of the planogram editor was released, a product knowledge base was built with scanned CAD models, and a virtual reality environment for observing human reaching and placing trajectories was developed.

Moreover, a list of perception tasks required in Scenario #1 for autonomous semantic mapping of the furniture inside a store was assembled. A first prototype of system employing available third-party bar code detection software was developed and evaluated. The problem of classify the boxes on a mixed palled in Scenario #2 using RGB-D data was addressed as well.

For what concerns Scenario #3, reactive control strategies at gripper level were designed and tested to avoid slippage of the grasped object in the presence of external disturbances, including gravity and inertial forces to which the object is subject to during the robot motion. These control algorithms can be also applied to perform in-hand manipulation actions, such as pivoting, i.e., rotate the object about an axis normal to the contact area while avoiding translational slippage. Such manipulation abilities are necessary to allow the robot to place the items on the shelf facings when other products are present in the neighboring facings.

The requirement specifications for a software layer called Store Management System (SMS) have been set. This software ensures communication and coordination between the different automated components used in Scenario #1 and #2 and with the Enterprise Resource Planning (ERP) system of the end customer.

\"

Final results

\"To fulfil the requirements of the proposed challenging scenarios, the robotic units to be developed in REFILLS require enhancements of the system abilities and relevant step changes in the following scientific and technological (S/T) domains:

Cognition-enabled control for shelf management - Autonomous mobile robots with visual sensors generate large amounts of valuable data. REFILLS will develop semantic mapping technology which fuses collected data with knowledge about store layouts to provide a powerful intelligent analysis tool. Furthermore, REFILLS will build manipulation knowledge bases to perform combined task and motion planning for in-store manipulation activities. To that end, REFILLS will extend an existing robotic knowledge base with semantic representations of sensor-based reactive strategies and motion planning.

Perception for interacting with objects - Multimodal sensing capability (e.g., visual, force, haptic and proximity) has to be exploited in order to allow the robot to recognize objects and find their pose, up to safely grasping a large variety of objects (different in terms of size, weight, shape, stiffness, fragility, texture), as well as avoiding collisions in the close vicinity of objects and shelves where occlusions of visual sensors are likely to occur.

Sensor-based reactive control - Sensor-based reactive control strategies allow safe and reliable object manipulation even in presence of uncertainties on object locations and scene mapping. REFILLS has to achieve a closer integration of the reactive (feedback) and deliberative (planning) parts by developing a set of reactive control algorithms endowing the robot with \"\"instinctive\"\" reaction capabilities, which will provide continuous feedback to the planning level about the status of the task.

Design of a unique set of robotic modules and integration for in-store logistics application - The project can build for algorithmic developments on existing, commercially available platforms and controllers of the participant KUKA, but for a successful application in in-store logistics, a new set of robotic modules has to be developed that can flexibly form robotic units needed at different times and places as required by professional logistics applications.

The application scenarios of REFILLS concern with the four main processes of grocery in-store logistics. In-store logistics is a domain where technologies for automatic shelf replenishment processes are strongly needed for cost and clerk health/ergonomic reasons and for being able to offer better services to customers. By providing innovative robotic solutions for the new domain of in-store logistics, Europe has the challenge to increase, maintain and strengthen its market share of 50% in professional service robotics by 2020. The industrial participants of the REFILLS Consortium: KUKA, Swisslog, INTEL and DM (end user) are committed to this European challenge and want to lead the process and contribute to fill the gap with USA and China.\"