Process demonstration, machine learning and process database

The central element of the Festo System Laboratory is the evaluation of process signals using machine learning, see Fig. 1. If a new process is demonstrated, numerous signals are recorded as part of the process demonstration using so-called instrumented tools. The resulting signals, as well as the geometry of the processed object, are evaluated using machine learning and are stored in a process database. This process database is the basis for the process execution on new, previously unseen objects.

Process demonstration, machine learning and process database.

Process demonstration

Process demonstration and recording of the process signals is performed using instrumented tools. These are tools that are equipped with additional sensors in order to record all relevant process signals, see Fig. 2 and 3. Examples of employed sensors are force/torque sensors, inertial measurement units (IMU) and reflective markers for the pose measurement. The use of instrumented tools allows to directly record the actual process performance during the demonstration without having to interpret the entire human motion and posture throughout the demonstration. At the same time, a similar tool can be used on the robot end-effector to execute the demonstrated process in new situations.

Instrumented tool for a packaging process.

Instrumented tool for a polishing or cleaning process.

Machine learning and process database

With the help of sensor fusion, the recorded process signals from the demonstration are combined, geometrically aligned with the digital workpiece and the actual movement of the instrumented tool is calculated. Furthermore, the contact forces applied at the contact point between the workpiece and the tool are estimated. The visualization of a single demonstration is shown in Fig. 4.

This processed data is split into appropriate sequences and is related to the underlying workpiece geometry, on which a machining path is calculated on the surface. With the help of probabilistic methods in machine learning, the process parameters of the human demonstration are learned and then stored in a process database. This process database is the basis for generating tool motions on new, previously unseen objects with similar geometry.

Visualization of a demonstration using the instrumented tool.