Report

Teaser, summary, work performed and final results

Periodic Reporting for period 1 - SPIRIT (A software framework for the efficient setup of industrial inspection robots)

Teaser

Inspection robots are more and more often used in industry for quality control of parts with complex shape. The main concept is to move a sensor over the surface of the part so that all relevant areas are inspected. The deployment of such inspection robots proved to be...

Summary

Inspection robots are more and more often used in industry for quality control of parts with complex shape. The main concept is to move a sensor over the surface of the part so that all relevant areas are inspected. The deployment of such inspection robots proved to be difficult, time-consuming and – consequently – expensive. SPIRIT aims at simplifying the deployment of inspection robots by developing a framework that enables the transition from programming of a robotic inspection task to configuring the task. This requires generic solutions for the following key challenges:
• Motion programs need to be generated automatically to ensure full coverage of the part while avoiding collisions. Manual or semi-automatic motion planning for the robot proved to be excessively difficult and time-consuming.
• Small deviations of the part shape invalidate previously planned inspection paths and motion programs, because full coverage can no longer be ensured. Robots need to adapt to the actual shape of the part.
• To fully assess the quality of the part, back-projection is needed that maps the single measurements seamlessly to the 3D model of the part. Only a seamless mapping allows proper data analysis.
• The performance of inspection robots in terms of cycle time and accuracy is not well known. There are very little operational data available that would allow e.g. a system integrator to assess whether a certain application will meet its target values.

The SPIRIT project will consider the following class of robotic inspection tasks:
• The task requires that the sensor is moving continuously (“scanning”) over the surface.
• The sensor acquires 2D or 3D patches of data (“images”, “point clouds”) from the part.
• The sensor is handled by a stationary multi-axis handling system (“robot”).

This class of tasks covers important inspection tasks such as 2D vision, 3D vision, thermography, ultra-sound, profile scanners or X-ray that are widely used in the manufacturing industry.

SPIRIT will take the step from programming of the robotic inspection task towards configuration of a task by importing a new 3D CAD model of the part, selecting the inspection technology and updating the model of the robotic work-cell. This will reduce the specific engineering costs of an inspection robot by 80% and increase the return on investment, thus creating a total market potential of 600-1.000 robotic installations per year mainly in Europe and Asia. A particular impact will be on SMEs, that can reduce the risk of deploying inspection robots and can thus more easily access a market that goes significantly beyond a regional one.

Work performed

\"In the first stages of the project the definition of the single use cases was completed, including key performance indicators. Each of the inspection methods (3D vision, X-Ray, thermography, profile scanners) was analyzed in detail. From the single technologies an abstract model of an image-based inspection process was developed. This single model just uses a set of parameters to describe all the inspection processes and does not require programming.
The generic process model was then integrated into a coverage planning algorithm that generates a set of inspection points that ensure full coverage of the part, that are free of collisions and that are reachable for the robot. The coverage planning was tested on a wide range of parts coming from 4 different use cases, and for the full range of inspection technologies. Using the inspection points, path planning needs to generate a path that passes through all the points as quickly as possible. This is achieved by finding a balance between the total length of the path and the need to avoid tight turns, which usually slows down the robot. Robot programs were generated for the first test case (3D vision, engine inspection) in accordance with the project plan.
In order to setup the inspection process, calibration procedures are needed to establish the relative position of all relevant coordinate systems. While conventional methods were sufficient for part and workcell calibration, a newly developed method was used for calibrating the sensor relative to the robot (hand-eye calibration). The calibration procedure was applicable to all inspection technologies, so that only the calibration artifacts (checkerboard) had to be designed for each type of sensor. An evaluation of the method was done by assessing the contribution of various calibration errors (sensor internal; sensor vs. robot; robot vs. part).
Finally initial work was done to start the activities related to data mapping. The accurate calibration - as described previously - built the basis for this activity, but additional methods are needed for image registration and stitching. In accordance with the project plan the first test case included mainly 3D data, so that 3D point cloud matching was used. Frist tests were also done to record a set of standard 2D images and mapping them to the CAD model of the part.

All of the developments were integrated into two components of the overall software framework. The \"\"offline framework\"\" includes the offline path planning, the modelling of the robot, the workcell and the process model of the inspection process. The \"\"inline framework\"\" includes the calibration procedures, the execution of the motion program on the robot and the data mapping. A first demonstration was made on the example of the inspection of an assembled engine, where a 3D sensor acquired sets of point clouds that were then used to e.g. assess whether the correct types of components have been mounted on the engine. As a particular challenge the engine was moving continuously during the inspection and the path was adapted in real-time to account for deviations in the predicted position of the engine.
\"

Final results

The SPIRIT project will advance the state of the art in three key topics that are urgently needed to make the implementation of inspection robots more efficient:
• Coverage planning for complex parts and a variety of inspection processes. This will be achieved through a generic process model that covers a wide range of image-based inspection processes.
• Reactive path planning to enable inline adaptation to changes of the part. By using sensor information (e.g. coming directly from the inspection sensor) local changes to the orobt’s path can be determined and a local re-planning can be made to ensure full coverage of the part while avoiding collisions.
• Seamless mapping of 2D images to a 3D object. This requires the accurate calibration of the whole robotic system, the precied synchronization of image acquisition and robot movements and advance methods for image stitching, in particular when mapping the images to parts of complex 3D shape.

These developments will be integrated in a two different frameworks:
• An offline software framework that is used for planning the inspection task. This will include automatic coverage and robot motion planning in a 3D model of the robotic workcell. It will also enable the parameterization of the inspection task itself and the modelling of the robot.
• An inline software framework that is used for the actual execution of the inspection tasks on the robot. It will include calibration procedures, the synchronisation of the data acquisition and robot motion, the reactive path planning to adapt to deviation of the part and the seamless data mapping.

Website & more info

More info: http://www.spirit-h2020.eu.